An intelligent system that optimizes cloud provider commitment coverage for maximum efficiency.

Abstract

The method includes receiving the customer's BDEs through a facilitating system ("FS"), processing the BDEs to determine the customer workload coverage needs, determining the optimal blend of required commitments, combining the accounts owned by the FS into the customer's organization accordingly, so that at least one commitment is held within each FS's account, combining the accounts, and monitoring the customer's workload coverage needs to detect changes. Depending on the changes, the method further includes adding accounts to or removing accounts (in whole or in part) from the customer's organization. The system includes instructions stored in memory, which, when executed, cause one or more processors to receive the customer's N-day on-demand workload usage, calculate a stable usage baseline based on that, calculate the customer's target coverage, and allocate a set of customer usage discounts ("CUDs") to cover the target coverage.

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 63 / 464,078, filed on May 4, 2023, titled "INTELLIGENT SYSTEMS TO OPTIMIZE CLOUD PROVIDER COMMITMENT COVERAGE FOR MAXIMUM EFFICIENCY", which is hereby incorporated by reference in its entirety (including the appendix).

[0002] Technical Field The present invention relates to machine - learning - based optimization, and more particularly to systems and methods for optimally distributing a blend of various cloud provider commitment types across a set of customers without such customers having to purchase their own commitments.

Background Art

[0003] Background Businesses and entities are increasingly required to store data and applications in the cloud. Any enterprise with an online sales platform or other important types of online customer interactions (such as not only insurance companies, banks, and brokers, but also educational institutions and healthcare providers, etc.) all rely extremely on cloud - based systems to provide not only their respective services but also online customer - facing interactions. To facilitate their online presence, such entities utilize cloud provider services such as Amazon Web Services (known as "AWS" for example), Microsoft's "Azure", Google Cloud Platform ("GCP: Google Cloud Platform"), and IBM Cloud.

[0004] Cloud providers have various pricing structures, including, for example, dollars per hour. They also typically offer commitments, which are a kind of "bulk purchase." A commitment is a contractual obligation between the user and the cloud provider to spend a certain amount of resources at either dollars per hour or dollars per specific SKU for each hour of the commitment period in exchange for a discount. The metric for how much of a commitment is actually used is its utilization, which measures the portion of the commitment used by the user or customer.

[0005] Commitments typically require a commitment to stable spending over a period of 1 to 3 years. If the workload decreases over time, customers risk losing money by having to pay for those commitments even if they are not used. However, if customers do not purchase commitments, they are forced to pay higher on-demand prices. Commitments may be called "Committed Use Discounts" or CUDs.

[0006] However, many cloud users do not fully utilize their commitments. In fact, it is understood that typical utilization of such commitments averages significantly below 100% over the commitment's time interval. As a result, customers tend not to fully cover their on-demand workloads with commitments.

[0007] Therefore, what is needed in cloud computing technology is not only a way for cloud provider customers to reap the benefits of their commitments, but also a way to optimize their use to achieve virtually full utilization. [Overview of the project] [Means for solving the problem]

[0008] Brief explanation of the drawing Several embodiments will be readily apparent from the following detailed description in conjunction with the accompanying drawings. For the sake of this description, similar reference figures designate similar structural elements. Various embodiments are shown in the figures of the accompanying drawings not as limitations, but as examples.

[0009] This patent or application file includes at least one drawing performed in color. A copy of this patent or patent application publication, including the color drawing, will be provided by the Patent and Trademark Office upon request and payment of the required fees. [Brief explanation of the drawing]

[0010] [Figure 1A] Describe an exemplary AWS organization using exemplary embodiments. [Figure 1B] This document illustrates process flowcharts of exemplary bottom-up optimization processes in various embodiments. [Figure 2] This depicts the first part of the process flowchart of a detailed bottom-up optimization process according to various embodiments. [Figure 3] This depicts the second part of the process flow diagram shown in Figure 2. [Figure 4] This diagram illustrates process flow charts of top-down optimization processes in various embodiments. [Figure 5] This describes exemplary historical data for on-demand use and exemplary predictive stable use baselines under various embodiments. [Figure 6] The calculation of the potential availability of all workloads is shown using a baseline as depicted in Figure 5. [Figure 7] This is an enhanced version of the process flowchart in Figure 1A, which includes the use of real-time data. [Figure 8A] This describes a comparison between on-demand use and a committed resource optimized using data acquired at predetermined intervals as input. [Figure 8B] This paper describes a comparison between on-demand use and committed resources optimized for real-time data input. [Figure 9] Describe an example of on-demand use of VCPUs for a customer over a two-week period. [Figure 10] Describe the utilization of the same customer after optimization by Flexsave according to various embodiments. [Figure 11] Show the transfer of a DoiT-owned GCP project to a customer's billing account according to a Flexsave embodiment of the Google Cloud Platform ("GCP"). [Figure 12] Show the constraints regarding the transfer of CUDs based on which SKU a given CUD was purchased for. [Figure 13] Describe an exemplary system architecture of a GCP Flexsave embodiment including an API backend and an AI optimizer. [Figure 14] Show a steady usage baseline using 30 days of data. [Figure 15] Show some examples of determining a new 30-day baseline from total time on-demand data. [Figure 16] Show a 24-hour verification process according to various embodiments. [Figure 17] Show the purchase of CUDs within a given safety margin according to various embodiments. [Figure 18] Show the calculation of the availability of all workloads by using a target baseline and a steady baseline. [Figure 19] Describe an example of coverage according to various embodiments. [Figure 20] Describe an exemplary optimization process according to various embodiments. [Figure 21] Same as Figure 6 above and show the distribution of CUDs among BAs according to various embodiments. [Figure 22] Show an example where over-provisioned CUDs for a given SKU are transferred to an under-provisioned billing account. [Figure 23]Describe an exemplary "spiky" workload with significant fluctuations in all-on-demand workloads. [Figure 24] Describe an exemplary first optimization run to obtain each workload for its specified target coverage. [Figure 25] Describe an exemplary second optimization run to spread any excess supply among customers. [Figure 26] Show how excess supply can still persist after 100% coverage has been achieved. [Figure 27] Show an exemplary mobile optimization approach for optimizing coverage according to one or more embodiments, where the usage of an exemplary customer varies significantly over a 24-hour period. [Figure 28] Show the exemplary mobile optimization approach of FIG. 27 for more stable / predictable customers, where the 24-hour values and the values for each 6-hour window are very close. [Figure 29A] It is the first section of a process flow diagram for optimizing the distribution of committed inventory. [Figure 29B] It is the second section of the process flow diagram shown in FIG. 29A.

Mode for Carrying Out the Invention

[0011] Detailed Description Systems and methods for facilitating and managing the collective use of one or more commitments by multiple entities are presented herein. In one or more embodiments, the benefits of commitments can be enjoyed while maintaining very high utilization. In some embodiments, an intermediary or facilitation system may be used in which a customer of a cloud service provider (who is also a customer of the facilitation system) may be involved. In some of the descriptions provided below, various embodiments of such facilitation systems will be referred to as “Flexsave”. In some of the descriptions herein, the applicant’s trademark “DoiT” will also be used to designate an exemplary intermediary or facilitation system or its owner / provider. DoiT is a trademark used by the applicant in relation thereto, and Flexsave is one of its services.

[0012] Flexsave enables customers to receive discounts on qualifying workloads by leveraging DoiT-owned inventory of commitments. This inventory can be dynamically moved between DoiT customers, thus allowing DoiT customers to either reduce or increase their usage over a period of time. This allows customers to benefit from commitment discounts without the risk of still having to pay for commitments later after their usage has decreased.

[0013] In the following detailed description, reference is made to the accompanying drawings that form part of this specification and illustrate exemplary embodiments that may be performed, where similar numbers throughout specify similar parts. It should be understood that other embodiments may be used without departing from the scope of this disclosure and that structural or logical modifications may be made. Accordingly, the following detailed description should not be taken as restrictive, and the scope of these embodiments is defined by the appended claims and their equivalent forms.

[0014] Several aspects of this disclosure are disclosed in the attached specification. Alternative embodiments and equivalents thereof of this disclosure can be devised without departing from the spirit or scope of this disclosure. It should be noted that the elements disclosed below are indicated by similar reference numerals in the attached drawings.

[0015] Various operations may be described as a series of discrete actions or, by extension, operations in the manner that is most helpful in understanding the subject matter of the claim. However, the order of description should not be interpreted as meaning that these operations are necessarily order-dependent. In particular, these operations may not be performed in the order presented. The operations described may be performed in a different order than in the embodiments described. Various additional operations may be performed and / or the operations described may be omitted in additional embodiments.

[0016] For the purposes of this disclosure, the phrase "A and / or B" means (A), (B), or (A and B). For the purposes of this disclosure, the phrase "A, B, and / or C" means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B and / or C). 70670

[0017] This specification may use the phrases "in one embodiment" or "in several embodiments," each of which may refer to one or more of the same or different embodiments. Furthermore, terms used in reference to some embodiments of this disclosure, such as "including," "having," etc., are synonymous.

[0018] In one or more embodiments, Flexsave is an automated system that uses AI and machine learning to distribute an optimal blend of various commitment types across a set of customers to maximize savings without the need for customer commitments (i.e., without customers having to buy their own commitments directly from the cloud service provider).

[0019] Depending on the cloud service provider, Flexsave may use various commitment types that can be purchased within an account or project (e.g., varying by purchase type, duration, or other criteria). Flexsave commitments can then be moved between customer-owned organizations or billing accounts to achieve best or optimal coverage, taking into account changes to customer workloads, existing commitments, and projected usage. In one or more embodiments, Flexsave may generate savings on cloud computing workloads without requiring any changes to either the customer infrastructure or the customer's existing workloads. In short, the described system is named "Flexsave" because it flexibly generates customer savings.

[0020] In various embodiments, Flexsave technology can be applied to any cloud service provider. For the sake of ease of illustrating the various Flexsave functionalities that follow, two specific exemplary embodiments are described (one, for example, an Amazon Web Services ("AWS") customer and the other, for example, a Google Cloud Platform ("GCP") customer). Nevertheless, it should be understood that these examples are merely illustrative, and Flexsave can be applied to any cloud provider that offers a commitment with associated discounts on on-demand cloud services directly.

[0021] The following section first explains Flexsave using AWS as an example, followed by an explanation of Flexsave using GCP. As noted above, neither should be understood as a limitation; each is simply an example.

[0022] AWS Flexsave Introduction AWS Flexsave requires customers to have an AWS Organization consisting of both a configured bulk billing feature and enabled discount sharing. In various embodiments, an AWS Organization can be understood as follows:

[0023] AWS Organizations is an account management service that enables you to create and centrally manage multiple AWS accounts into a single organization. AWS Organizations includes account management capabilities and unified billing capabilities that better suit your business budget, security, and compliance needs. As an administrator of an organization, you can create accounts within the organization and invite existing accounts to join it.

[0024] For example, see the following URL (after removing XXX): https: / / docs.aws.amazon.com / organizations / latest / XXXuserguide / orgs_introduction.html.

[0025] In addition, AWS features consolidated billing, which allows spending from all accounts under the same organization to be processed as if it originated from a single account. Therefore, any quantity discounts, tiered pricing, or commitment discounts can be applied to all aggregated spending across all accounts, regardless of where the spending originated (i.e., which account it originated in). See, for example, the following URL: https: / / docs.aws.amazon.com / awsaccountbilling / latest / XXX aboutv2 / consolidated-billing.html

[0026] Next, we will explain some background details of AWS commitments that are useful for understanding the optimizations described below in various embodiments.

[0027] AWS offers two types of commitments: Savings Plans and Reserved Instances. Each of these is explained in detail at https: / / aws.amazon.com and therefore does not need to be repeated here. In exchange for a customer commitment to spend or use a certain amount of resources at each hour during the duration, AWS offers a discount. Importantly, there is a required hourly expenditure, and therefore the customer cannot terminate earlier by using more resources earlier. Both AWS commitment types have the following common features.

[0028] It can be purchased for either 3 years (higher savings) or 1 year (lower savings); and It can be purchased from one of the following: All prepayment required - full payment at the time of purchase (greater savings); Partial upfront payment - 50% is paid at the time of purchase, and the remaining 50% is spread out over time over the purchase period (medium-level savings); or No upfront payment is required; all fees here are charged hourly, proportional to the period fee.

[0029] In addition, the following features are specific to AWS Reserved Instance Type commitments: It can only be purchased for one of the AWS support services (i.e., EC2 or RDS); It may be purchased for the use of specific resources (i.e., 30 instances of an m5.xlarge machine with Linux); and, It may be purchased in relation to a specific region or area.

[0030] There is also a variation of this commitment type known as Convertible Reserved Instance, which is only for EC2 services and allows customers to change the type of purchase reservation in the parameters to a different one. For more information, see Exchange Convertible Reserved Instance - Amazon Elastic Compute Cloud.

[0031] Finally, the following features are specific to the savings plan type commitment: The customer commits a certain amount of dollars / hour at a discount rate specific to the savings plan rate; the savings plan may apply to two or more services or may be limited to selected SKUs.

[0032] Flexsave AWS in operation In some embodiments, Flexsave dynamically grants an account to an AWS organization that includes various commitments. These commitments are then shared with all customer-owned accounts under the same organization, thus generating savings across all workloads.

[0033] Figure 1 depicts exemplary AWS organizations 101 in various embodiments. Referring to Figure 1, a customer-owned account 110 is shown, which is enabled for both unified billing and discount sharing, as indicated by arrow 115. The customer account 110 runs various workloads. Several DoiT (System Provider)-owned accounts 120, including Flexsave commitments, are also shown, but the DoiT accounts are not assigned any workloads.

[0034] In some embodiments, each AWS account used by Flexsave contains a single commitment with different time-based granularity. In some embodiments, as shown in 125, Flexsave may dynamically adjust the commitments required by the customer by moving enough accounts into or out of the AWS organization 101 to achieve optimal coverage.

[0035] The optimal coverage required by an organization is highly dependent on workload execution and is typically influenced by: regional workload execution, workload computation specifications, workload operating system, commitment type, commitment length, type of commitment purchased from DoiT, contractual discounts offered to the organization under the agreement with the cloud provider (either a fixed-rate discount or an SKU-level discount), and existing commitments purchased by customers.

[0036] In some embodiments, AWS Flexsave can operate by using all AWS-provided commitment mechanisms, including not only standard or convertible reservations but also savings plans. Standard / convertible reservations are historically older commitment types that require the specification of several parameters of the covered workload and apply only to the workload. Some commitments allow for changes to the attributes of the covered workload, but these are not done dynamically based on the actual workload being covered.

[0037] Savings plans are a newer form of commitment that requires the commitment to be defined in dollars per hour at a discount rate, but depending on the type of savings plan, this commitment type can be allocated more dynamically to generate savings.

[0038] In some embodiments, once a workload is covered, Flexsave uses the AWS Cost and Usage Report to calculate the charges. Depending on the need, the workload covered by Flexsave may be converted to a new rate specific to what the customer will pay, or a new charge may be added to the report representing, for example, a DoiT margin or cost that generates the coverage (i.e., if the savings are covered by a commitment paid outside the AWS organization that generated them). In some embodiments, additional processing may remove any underutilization caused by Flexsave for either customer-owned commitments or Flexsave-owned commitments.

[0039] Therefore, for example, as shown in Figure 1A, DoiT may purchase commitments from a cloud vendor (AWS) in its own account, where each account is responsible for a certain amount of commitment / hour of a specific type. In some embodiments, DoiT may own thousands of accounts holding various types of commitments ranging from $0.5 / h to $25 / h, for example, both expenditure-based commitments and a wide variety of resource-based commitments. However, no workloads run in the accounts owned by DoiT that hold those commitments.

[0040] In this case, for example, DoiT can determine the coverage needs within the customer organization. Similar to the one-stop billing feature mentioned above, DoiT looks at all spending within the organization and determines the optimal blend of commitments required by the customer, available inventory, and risks associated with the workload.

[0041] In this embodiment, for example, DoiT joins its owned accounts to the customer organization. Due to the enabled bulk billing feature, commitments in the DoiT account (without workloads) are shared with the workloads running in the customer account. Since DoiT owned commitments can cover workloads within the customer account, there is no need to change how the workloads function.

[0042] In some embodiments, DoiT continuously monitors the need to adjust customer coverage—either up or down—and, accordingly, adds or removes accounts from the customer organization as usage changes.

[0043] Flexsave's advantages for customers who want to acquire their own commitments Commitment management using a DoiT-type promotion system offers several improvements and efficiencies that are not simply available at the individual customer scale. Therefore, while customers can purchase their own commitments, these commitments must be purchased for either a one-year or three-year period. If a customer's usage declines at some point after purchase (e.g., one or two months later), the customer loses money because they committed to a certain hourly expenditure / usage that was not fully utilized. It should be noted that commitments are based on hourly expenditure / usage over a fixed period, and the commitment holder cannot use the total usage covered by the commitment initially. Therefore, a customer cannot balance their total usage over the entire commitment period they deem appropriate or desired by increasing usage during busy months and decreasing usage during less busy months. Flexsave becomes more flexible and therefore better in this static aspect of commitments in the cloud services industry. In various embodiments, DoiT allows a customer to receive a given commitment (e.g., commitment X) for a certain period, and if a customer's usage declines, DoiT may move commitment X or a portion thereof to a different customer whose usage has increased. In practice, no customer will achieve 100% coverage by default (Flexsave aims for 85% coverage of the total workload), so any temporary surplus in inventory can be dynamically redistributed among existing customers within its reserve 15%.

[0044] Due to DoiT, Flexsave's operators, or any equivalent entities having a large customer base, in some embodiments, Flexsave can easily add or remove commitments as spending on individual customers changes. Thus, Flexsave enables its customers (who are also customers of cloud service providers) to benefit from the scale that only a facilitation system can provide.

[0045] Flexsave AWS System and Optimization Overview Therefore, in some embodiments, a larger Flexsave of the AWS system includes multiple AWS organizations belonging to a customer, each of which may own any number of AWS organizations. In some embodiments, AWS Flexsave attempts to find the right balance of coverage across the entire system while factoring existing constraints to maximize both customer coverage and DoiT revenue: for example: -Existing Inventory-DoiT possesses a certain amount of commitment of various types that can be used with the customer. Depending on the load on the system, there may be too much or too little inventory at any given time; - Customer coverage ensures that waste is not covered by DoiT; - Various SLAs defined by AWS organizations, such as minimum or maximum coverage; and - Determining the best inventory type for each AWS organization - For example, any upfront inventory is not very suitable for customers who have a commitment to improving DoiT revenue due to discounts on recurring charges.

[0046] It should be noted that a given customer typically has one AWS Organization. However, under certain circumstances (for example, when an acquisition or separation occurs by a business unit, with the newly acquired company retaining its own organization), a given customer may have multiple organizations, or one organization for outgoing applications (to maintain more restricted access and improved governance), and another for back-office applications such as R&D with looser access.

[0047] In some embodiments, each AWS organization is considered separately for optimization purposes if there is a contractual service level agreement ("SLA") for a customer, which may be multiple separate SLAs that affect multiple organizations but are considered equally. Except for the fact that an SLA override may occur from the same customer to multiple organizations, in some embodiments, the exemplary system considers each organization separately and has no connection to the customer.

[0048] The more organizations a system has, the better it works due to the power of scale, as the decrease / increase in needs within individual organizations becomes more easily equalized as the number of organizations increases. Note that "AWS Organization" is a specific term and is often capitalized in this disclosure to refer to that specific use in the AWS world. However, AWS Organization has equivalent components within each CSP and is also understood to be a general and generic "organization" of a CSP's customers in a general sense and is sometimes spelled lowercase even when referring to AWS customer entities.

[0049] DoiT recommends that each customer have only one organization, as this helps them take advantage of quantity discounts, etc. (which do not apply across the entire organization) by having all of the customer's spending within a single organization, thus enabling the customer to negotiate better discounts with their cloud provider. However, even if a customer has business needs for more organizations, DoiT does not advise them to do so, and then all of that customer's organizations are included in DoiT's optimization process.

[0050] In some embodiments, the exemplary Flexsave system achieves best performance by performing a two-step optimization process. The first optimization step involves a bottom-up optimization computed for each isolated AWS organization to determine a possible coverage model for that AWS organization. The second optimization step then involves a top-down optimization computed for the entire Flexsave system (i.e., all organizations served by Flexsave) to achieve best overall system performance given constraints. These two optimization steps are described below with reference to Figure 1B and a more detailed set of process flowcharts shown in Figures 2-4.

[0051] Figure 1B shows all the optimizations used by Flexsave in various embodiments. Figure 1B applies to both this Flexsave AWS example and the Flexsave GCP example described below; for other cloud providers, the CUR in block 150 will be replaced with similar or equivalent billing and usage data output or provided by that specific other cloud provider. Blocks 150 and 155 of Figure 1B describe the bottom-up optimization process, and boxes 165 and 170 cover the top-down optimization process. Each of these processes is described below.

[0052] Bottom-up optimization In some embodiments, each AWS organization is considered separately for the purpose of determining the optimal coverage for that AWS organization before any constraints of the system or SLA are applied. This includes multiple organizations owned by the same single customer, as mentioned above. In some embodiments, the basis (or input to) the Flexsave optimization is the AWS Cost and Usage Report ("CUR") at a time-by-time granularity with resource IDs (see https: / / docs.aws.amazon.com / cur / latest / userguide / what-is-cur.html). This is shown, for example, at 150 in Figure 1B.

[0053] A Customer Account Report (CUR) is an AWS-issued statement of account that provides detailed information about all resources for which a customer is being billed. A CUR includes information about the resource, its associated SKU (stock keeping unit), the time for which it is billed, any resource-specific metadata, and pricing and cost details for that resource.

[0054] In some embodiments, based on the CUR, a series of three recalculations may be performed to determine the optimal coverage of the various commitments the customer may take.

[0055] (1) Initially, decisions regarding stable and eligible spending may be made based on the potential available inventory over the qualifying period, not only with respect to both already covered and uncovered workloads, but also with respect to any historically low utilization of any of DoiT's customer commitment mechanisms.

[0056] The following are definitions of technical terms used in relation to the above recalculation (1): Covered workloads refer to any resources that may be eligible for discounts under a commitment already covered by either a customer-owned commitment or a DoiT-owned commitment (such workloads cannot be covered twice by any commitment). Uncovered workloads refer to resources that are eligible to be covered by a commitment but are currently running under an on-demand pricing scheme. Inventory refers to commitments that have already been purchased by DoiT and are available for use to cover workloads. Eligibility period refers to the minimum period of information available regarding workloads running within the organization that allows us to determine what constitutes a stable expenditure. In some embodiments, this may be taken as 7 to 30 days. Finally, historical underutilization refers to historical data regarding underutilized commitments within the organization. Since each commitment represents dollars paid per certain amount of resources or time, AWS will bill for it regardless of whether it was used or not. Note that this information indicates that "all commitments granted to an organization (or possibly a customer) have not been fully utilized and therefore potentially result in a loss of some money."

[0057] (2) Depending on the commitment type, the order of application may vary according to the cloud provider's rules. Customers may have existing commitments that may be applied in a different order, so in some embodiments, Flexsave needs to simulate the exact behavior of a customer's system if Flexsave were to add its inventory to that system. Once Flexsave determines the maximum level of workload coverage it can cover, it simulates the impact that the Flexsave inventory might have had on an AWS organization in the past.

[0058] The following is a specific example of the order in which application changes are made, and assumes the following facts: - The customer owns a 1-Year No Upfront Compute Savings Plan at $1.00 / hour; - The customer will run an m5.8xlarge Linux instance and a p3.2xlarge Linux instance in US East 1; and - A customer-owned SP of $1.00 will initially cover 88.6% of m5.8xlarge utilization, as it has a higher savings rate (27%) than p3.3xlarge (21%). Uncovered workloads will consist of on-demand tasks - a total of $3.235 on-demand tasks run by p3.2xlarge: $0.175 (11.4% of $1.536 at the on-demand rate) and $3.06.

[0059] Now, by granting a $1 Flexsave inventory as a 3-Year No Upfront Compute Savings Plan, the order will be changed as follows: - The 3-year savings plan will cover the m5.8xlarge by prioritizing and fully utilizing $0.779 of the allocated $1.00, leaving $0.201 for the next workload; - The remaining DoiT commitment of $0.201 will cover 11.2% of the p3.2xlarge instance (as the associated 3-year rate is $1.795); and - The $1.00 in the Customer Ownership Savings Plan ("SP: Savings Plan") now covers the remaining p3.2xlarge instances in place of the previously used m5.8xlarge instances - now covering 41.6% of the remaining p3.2xlarge utilization (we have $1.00 and the rate associated with this SP for those instances is $2.403). This leaves 47.2% of the p3.2xlarge instances as uncovered workloads (charged at $1.444 on demand) (the associated rate on demand is $3.06).

[0060] Therefore, as described above, by granting DoiT inventory, customer-owned commitments were moved to cover instances with lower savings rates.

[0061] (3) Following the recalculation described above, the optimal coverage for this account is determined based on past usage by using predictive data (e.g., trends, seasonality, etc.) based on past usage to build a predictive model for the organization that shows the impact of each added commitment and the customer returns and DoiT returns. In some embodiments, such a model, as shown in Figure 1B, 155, may be built using an ML model that is trained to best predict future usage based on existing data.

[0062] In one or more embodiments, the following algorithms for bottom-up optimization may be implemented: - Begin with a Career Scale (CUR) showing all resources with related workloads; - Eliminate all commitments and create a system where all workloads are priced on demand (i.e., everything is considered uncovered); - Determine all commitments owned by the customer and order them in the manner in which they will be applied; - Order workloads in the order they will be applied; - Apply all customer-owned commitments that have a higher priority than the priority DoiT would want to assign to the best-saving workload until the commitment is gone; - Apply (reversely apply) all customer-owned commitments that have a lower priority than the priority DoiT would want to assign to the lowest-cost workload, and ensure that they are used correctly; - The space between high-rate and low-rate commitments, where nothing was applied before, is now where DoiT commitments can be applied. Those workloads are then assigned to a rate associated with DoiT commitments, and it is determined how much can be granted.

[0063] Figures 2 and 3 together show detailed process flowcharts of bottom-up optimization processes that implement the algorithm described above, according to various embodiments of the bottom-up optimization process. Note that due to the horizontal size, the overall bottom-up optimization process is divided into two figures labeled "Bottom-up A" and "Bottom-up B". Therefore, referring to the right side of Figure 2 (the process moves from left to right in the figure), the output 240 of block 220 (recalculation) is supplied to block 320 (organizational coverage) in Figure 3 on the left side. Similarly, if selected to be generated, the optional output 250 (real-time usage data) of block 230 in Figure 2 is also supplied to block 320 (organizational coverage) in Figure 3, as shown on the left side of Figure 3. Next, the final part of the AWS Flexsave process (top-down optimization) will be described.

[0064] Top-down optimization In some embodiments, information obtained from the bottom-up optimization phase can be used to generate a desired state for the entire system. This is shown in blocks 165 and 170 in Figure 1B, and in more detail in Figure 4, where output 350 of the final block of bottom-up B (block 320 in Figure 3): organizational coverage is supplied to commitment distribution as input to block 430 on the right side of Figure 4. Note that another input to block 430 in Figure 4: the DoiT inventory of commitment 410 (continuously stored values ​​and managed by the Flexsave system) is supplied to block 430 from a different source and is not an output of the bottom-up optimization process.

[0065] In some embodiments, a top-down optimization algorithm may be used to determine the distribution of commitments by using an organization-by-organization model and to apply the following rules: - We guarantee that any minimum or maximum contractual requirements for each account will be met in accordance with the SLA established for that account. - Allocate commitments to all accounts to satisfy the desired default level of coverage. - If the default is unsatisfactory due to a lack of inventory, prioritize the workload with the highest ROI followed by the highest stability. - For any excess inventory across the entire default, prioritize workloads that have the highest revenue followed by stability. - If the existing inventory exceeds the overall system's capacity to handle it, prioritize minimizing waste (the workload with the highest utilization).

[0066] As used in top-down optimization, the following technical terms have the following meanings: The default coverage level specifies, as a percentage, how much of your regular usage / spending will be covered by Flexsave. Inventory refers to all purchased commitments available for use by Flexsave to cover customer workloads. Workload revenue refers to the revenue generated by covering a specific workload after paying the commitment fee, and any savings passed on to the customer. Finally, workload stability is defined as running a workload that is stable in terms of spending / utilization and is eligible for discounts from qualified commitments, which, if covered, will generate positive savings over the entire period.

[0067] Next, we will describe the second Flexsave example discussed above, in which the Flexsave technology is implemented for the customer using GCP.

[0068] Flexsave GCP Overview GCP's Flexsave works by leveraging the commitment sharing functionality of billing accounts. Resource-based commitments purchased within a GCP project (which is part of a DoiT GCP organization) (see https: / / cloud.google.com / resource-manager / docs / cloud-platform-resource-hierarchy#organizations) can be dynamically granted to a customer's billing account and can cover all project workloads granted to that customer's billing account, regardless of the workload's GCP organizational configuration. A billing account is a logical grouping of billing for many cloud services. A customer may have one or more billing accounts within their organizations, but in most cases, they will only have one.

[0069] In some embodiments, each project owned by GCP Flexsave contains only a small commitment of a specific type (single SKU). In some embodiments, Flexsave can dynamically adjust the required coverage by assigning selected SKUs to many projects that match the desired coverage. This is because Flexsave may obtain permission to move GCP projects into and out of the customer's billing account. Thus, in some embodiments, Flexsave creates a project and adds commitments to it. The process then moves the project into and out of the customer's billing account.

[0070] In one or more embodiments, the optimal coverage required for a billing account is highly dependent on workload execution and is influenced by the following: Regional workload execution; Computing specifications for the workload; Contractual discounts (either a fixed-rate discount or an SKU-level discount) provided to the billing account based on the agreement with the cloud provider; and Existing commitments purchased by the customer.

[0071] In some embodiments, the Flexsave system in the GCP includes two key components: a Purchase Recommendation Engine (PRE) and an Optimization Engine (OE). In some embodiments, the PRE may determine optimal coverage by using, for example, 30 days of historical data. In such embodiments, the OE may manage daily changes in usage to prevent waste and optimize coverage.

[0072] Recommended engine to purchase In some embodiments, the PRE (also known as the “Recommender”) is responsible for generating recommendations regarding the number of commitments to make a purchase. In such embodiments, the PRE may do this by examining historical usage data both at the individual billing account level and at the system level. In addition, the PRE may include risk models and system inventory data for generating recommendations. In some embodiments, the algorithms performed by the PRE may employ probabilistic and machine learning techniques to provide the desired output.

[0073] In some embodiments, a historical stable usage model is required to recommend commitment purchases. The recommender may, for example, look at historical data of on-demand usage and employ machine learning techniques to predict a stable usage baseline for the workload. This is shown in Figure 5, described below, which is a plot of on-demand usage 510 over an entire 30-day period. Also shown in Figure 5 is a new 30-day baseline 520 generated by PRE from data 510. Note that "in some embodiments, the stable usage baseline points to the lower limit of usage over a given period and ignores dips that occur within less than 20% of the time interval." Other versions of the stable usage baseline may use various metrics for on-demand data.

[0074] Optimization engine In some embodiments, the OE ("Optimizer") is responsible for distributing the commitment inventory from oversupplied billing accounts to undersupplied billing accounts. Each billing account has a predetermined target coverage that the Optimizer attempts to satisfy. In some embodiments, the following steps may be taken to perform the optimization: (1) Determine the recent stable usage baseline; and (2) Allocate inventory.

[0075] (1) Determine the recent stable usage baseline For this task, the optimal model looks at historical data of on-demand usage and employs machine learning techniques related to time series forecasting to predict a stable usage baseline for the workload over recent time intervals. Exemplary inputs 510 and outputs 520 of this process are shown in Figure 5.

[0076] (2) Allocate inventory In some embodiments, once a stable utilization baseline is determined, the optimizer may calculate the potential availability of all workloads. The optimizer may then use this availability to make commitment allocations that satisfy multiple optimization targets. An exemplary calculation of this nature is shown in Figure 6, where the capacity 610 of each project for several projects is used as input to generate the potential 620 of all workloads. This process is also detailed in Appendix A, slides 18–26. Note that Figure 6 is actually taken from part of slide 20 of Appendix A.

[0077] Real-time strategies for Flexsave optimization It should be noted that the above-described embodiment utilizes billing data exports (AWS CUR and GCP billing exports) as the primary information source. However, these data sources suffer from inherent delays in generating the necessary data. In practice, the typical delay for billing data ranges from 12 to 36 hours. Using this older and less up-to-date data can introduce prediction errors and distort optimization.

[0078] In one or more embodiments, to remedy this problem, the exemplary Flexsave system may be enhanced to use a quasi-real-time data source. Note that additional permission from the customer may be required to use a quasi-real-time data source. Typical latency for these quasi-real-time data sources is usually in the range of 0 to 1 hour, excluding the time required to process the data.

[0079] In this context, Figure 7 illustrates how a real-time data source is integrated into the flow. Referring to this, Figure 7 shows that a new block, "Real-time Data 153," is added to the processing flow in Figure 1B. Thus, the recalculated real-time data is an additional input to block 155, where the predictive model for the organization is calculated.

[0080] These real-time data sources include information about the use of cloud resources and commitments. Examples of relevant data sources include GCP Cloud Audit Log, AWS CloudTrail, AWS CloudWatch, GCP Cloud Asset Inventory, and AWS Config.

[0081] In some embodiments, such enhanced data sources enable the system to respond to changes in cloud workloads in near real-time and thus achieve improved performance.

[0082] An example demonstrating the benefits of real-time data for optimization. If the data source is a billing data export as described above, the presence of inherent delays means that the system may not be able to respond immediately to changes in workload. The case shown in Figure 8A is illustrative and shows a plot of usage 810 versus time (days) 820. Referring to Figure 8A, on-demand usage 833 (shown in black) decreases at noon on day 6 at time 850, but this decrease is only reflected in billing data within 24-48 hours. Therefore, Flexsave cannot move commitment 835 (shown in green) until day 8. This results in waste 840, which is the difference between commitment 835 and actual on-demand usage 833. Thus, from time 850 at noon on day 6 to day 8, the commitment is oversupplied.

[0083] For comparison, real-time data would allow changes to be made within 1-2 hours, and the waste component would be significantly reduced. This is shown in Figure 8B, where the commitment is moved shortly after real-time usage declined at midnight on day 6 (shown as time point 851), thus tracking real-time usage almost accurately. Therefore, in the situation shown in Figure 8B, there is no waste at all.

[0084] Exemplary Flexsave scenarios for GCP The following are real-world examples of Flexsave functionality used by GCP customers.

[0085] We consider GCP customer A, who uses E2 compute workloads in the US East 1 region and utilizes e2-standard-8 instances consisting of 8 VCPUs and 32GB of memory per instance.

[0086] Figure 9 shows the on-demand utilization of E2 VCPUs for an exemplary two-week period in January 2023. As shown in Figure 9, on-demand utilization fluctuates between 48 and 128 units within the two-week period.

[0087] The daily cost for this workload is $6.432 per instance, and the total cost for two weeks is $1,008.41, as shown in Table A provided on the following page:

[0088] [Table 1]

[0089] However, if this customer was using Flexsave, the optimizer could apply a daily committed usage discount ("CUD") (see slides 4-5 in Appendix A). Assuming sufficient inventory is available, an exemplary applicable daily CUD is shown in Figure 10. This example assumes that the optimizer targets 85% coverage of on-demand usage. Thus, referring to Figure 10, we see on-demand usage 1010 and DoiT CUD 1020. Since DoiT CUD utilizes DoiT commitments, DoiT CUD (and therefore the value when using commitments) is much cheaper, and the only difference between on-demand usage 1010 and DoiT CUD 1020 is the on-demand fee that the customer would pay. The total cost to the customer is (i) CUD (shown in purple) + (ii) the difference between on-demand usage and DoiT CUD (the orange box percentage shown above the purple box in Figure 10). Therefore, by owning a large number of Commitments (CUDs) that can be freely shared across its entire customer base, Flexsave enjoys the economic benefits of CUDs at a scale sufficient to avoid wasting CUDs during periods of low usage.

[0090] The total costs for this exemplary GCP user, including on-demand usage fees, DoiT CUD, and operational Flexsave, are all shown in Table B below.

[0091] [Table 2]

[0092] Comparing Table A and Table B, we can see how Flexsave technology can achieve a 50% saving when used by this exemplary user.

[0093] AI components used in Flexsave GCP processing Figures 11–26 show the AI ​​component of an exemplary Google Cloud Platform ("GCP") embodiment of Flexsave technology. AI is used not only to calculate which CUDs should be purchased, but also to optimize coverage to prevent / eliminate both under- and over-supply.

[0094] Please note the following: While this disclosure describes exemplary AWS and GCP embodiments, it should be understood that the systems, methods, and techniques described herein apply to any available workload service provider (cloud or otherwise) that a customer's ability to purchase commitments with various types of CUDs may have.

[0095] Figure 11 illustrates the movement of DoiT-owned GCP projects to a customer's billing account using an exemplary Flexsave embodiment of GCP. As noted above regarding the exemplary AWS embodiment, customers may offer discounts on workloads by applying CUDs. In some embodiments, the exemplary mechanism is as follows: customers enable discount sharing in their billing accounts ("BAs"). DoiT then generates GCP projects 1110 with CUDs and moves those projects 1110 to customer BA1150. For example, there may be four DoiT projects held in the DoiT inventory, as shown. Based on the compatibility between the DoiT inventory and a given DoiT customer's billing account, for example, projects 1 1115 and 2 1120 may be moved to customer BA1 1155, project 3 1125 may be moved to customer BA2 1160, and project 4 1130 may be moved to customer BA3 1165.

[0096] As shown in Figure 12, CUDs must be purchased for a specific SKU. A SKU is defined by region, family type (e.g., N1), and hardware (e.g., VCPU). Thus, as shown in Figure 12, SKU1 can be moved from Project 1 1115 of DoiT BA1 1155 to Customer BA2 1160, but since both BAs contain SKU1, SKU2 of Project 2 1120 is not compatible with Customer BA2 1160, and therefore SKU2 cannot be moved from DoiT BA1 1155 to Customer BA2 1160. However, since Project 2 1120 is compatible with BA3 1165, SKU2 can be moved from Project 2 1120 of DoiT BA1 1155 to Customer BA3 1165. Therefore, DoiT preferably maintains an inventory of various projects with various SKUs in each project so that SKUs can be moved to compatible customer BAs as needed. As used herein, an SKU is a unique combination of region, family type, and hardware. SKUs must be consistent for a CUD to move from one billing account to another. Also as used herein, the term “workload” refers to an SKU instance. Thus, for example, SKU US East 1, N1, VCPU in billing account A is called a “workload.” As shown in Figure 12, DoiT BA1 1155 includes projects 1 1115 and 2 1120, which carry CUDs compatible with SKUs 1 and 2, respectively, while customer BA2 1160 has workloads running SKUs 1 and 3, and BA3 1165 has workloads running SKU 2. Thus, as mentioned above, project 1 1115 can be moved from DoiT BA1 1155 to customer BA2 1160, for example, because customer BA2 1160 runs a workload compatible with that SKU. However, as mentioned above, since DoiT Project 2 1120 is only present in BA3 1165 where the compatible workload runs, only this can be moved to customer BA3 1165.

[0097] In some embodiments, the AI ​​component can perform various functions. For example, the AI ​​component may be used to generate CUD purchase recommendations. In addition, for example, once CUD is purchased, the AI ​​component may be used to move the CUD from customers who are oversupplied to customers who are undersupplied. These AI functions are described below.

[0098] Figure 13 depicts an exemplary system architecture of a GCP Flexsave embodiment. This exemplary system architecture includes both an API (or backend) 1310 and an AI optimizer 1320. In some embodiments, the API 1310 handles source billing data, such as generating projects to be used for purchase and verifying that the projects remain within the attached billing account, maintains all data sources, and performs CUD purchase and CUD move, as well as other functions. In some embodiments, the AI ​​optimizer 1320 may generate optimization recommendations as well as purchase recommendations, as shown.

[0099] Figures 14–19, described below, illustrate how a stable usage baseline can be calculated according to various embodiments and then used to determine coverage. As shown in Figure 14, 30 days of data can be used, for example, to determine a stable usage baseline. The upper plot 1410 (February 21–March 19) shows the customer's total on-demand hourly usage. In this example, this fluctuates regularly over each 24-hour period. As shown, this plot 1410 can be used to set a new baseline 1420, which is the lower horizontal line, above which the on-demand hourly plot does not fall below from February 21 onwards.

[0100] In some embodiments, the stable usage baseline 1420 may be modeled based on, for example, the Google recommendation “Maximize Savings Choice.” This refers to Google-provided choices (known as “Google Recommendations”) that recommend purchasing CUDs. These recommendations include several optional selectors (“Maximize Savings Choice” is one of them) regarding how to determine the optimal value of the recommendation. Once selected, “Maximize Savings Choice” calculates the number of CUDs to purchase for SKUs that maximize savings, including the cost of low usage (since paying the cost of 2-3 hours of low usage may still be worthwhile if this results in greater savings overall).

[0101] Figure 15 shows two examples for determining a new 30-day baseline: one is a regularly fluctuating on-demand curve, similar to the example in Figure 14, and the other is a nearly constant on-demand curve (actually a horizontal line). In the upper plot 1510, the on-demand plot 1511 fluctuates significantly with a periodic waveform, similar to the example in Figure 14. However, in this example, the pattern repeats every 7 days (with some variation), fluctuating between 1.6-1.7K and over 3K, and has a period of several days at the end of the pattern where utilization is lowest at 1.4K or slightly below. Thus, a new baseline 1513 can be constructed from the on-demand plot 1511, which is shown as the thick horizontal line 1513 below it (with a value of approximately 1.4K). In the lower plot of Figure 15 (i.e., plot 1520), the total on-demand hourly plot is constant and only has a value of 2. Thus, the new baseline is also set to 2.

[0102] Figure 16 illustrates an exemplary technique for addressing the capture of changes in a stable baseline by using what is called a 24-hour validation process (but it does not have to be exactly 24 hours; this is merely illustrative) in various embodiments. Referring to plot 1610 of the upper set depicting 30-day on-demand data 1611 and the corresponding stable baseline 1613, as shown in Figures 14, 15, the on-demand plot 1611 on March 17 is below the 30-day stable baseline 1613 generated as described above with reference to Figures 14, 15. The 30-day baseline 1613 has a value of 100, as shown in plot 1610. However, Figure 16 also shows the generation of a baseline with a 24-48 hour window to validate the current 30-day baseline. The 24-48 hour window can be used as a safety check for declining workloads. Thus, as shown in plot 1620, if the 24-hour baseline 1623 is less than the then-prevailing 30-day baseline 1613, the lower 24-hour baseline 1623 is used instead. In 1620, the 24-hour baseline 1623, calculated from March 17th to March 19th as shown, has a value of zero, so this value is used instead of the value for the 30-day baseline 1613 in plot 1610.

[0103] Please note the following: In Figures 14–16, the stable baseline is chosen so that the on-demand plot is virtually always above it, as it fluctuates. In an alternative embodiment, it may be optimal to shift the baseline slightly upward so that the on-demand plot has a large dynamic range (it is at high values ​​for a considerable period). Thus, at some point in time, the baseline is well below the values ​​of the on-demand plot, indicating overcoverage for most of the time, and may be a better approximation for calculating the target baseline, as will be explained below.

[0104] Figure 17 illustrates the purchase of CUD within a predetermined safety margin in various embodiments. In some embodiments, for targeted coverage, the exemplary Flexsave system does not purchase on-demand hourly activity at the baseline, but rather may purchase CUD at 85% of that baseline (baseline calculated as described above with reference to Figures 14-16). Thus, as shown in Figure 17, on March 24, 2023, the total hourly on-demand activity is 414.0 (shown on the upper curve 1711), but the purchased targeted coverage is, for example, 85% of that, i.e., 351.9 (shown on the lower curve 1713). In other embodiments, different values ​​for the target baseline may be selected.

[0105] Figure 18 shows the calculation of the availability of all workloads using the stable baseline 1811 and the associated target baseline 1815. Here, availability is the difference between the target baseline 1815 and the actual value of the DoiT CUD now owned. In this example, the target baseline 1815 is 32 / 36, or 8 / 9 (=88.88%), of the stable baseline 1811. Therefore, in this example, the target baseline is 32, and the DoiT CUD 1850 now in the existing set has a value of 4, and the available CUD 1830 to purchase is equal to 28 (i.e., the difference between the target baseline and the CUD already on hand). "Total CUD" and "DoiT CUD" are identical in this example and therefore overlap completely, and are represented by the set of CUD 1850 having a value equal to 4.

[0106] Figure 19 illustrates an example of coverage under various embodiments. Here, the total CUD (shown in dark gray (teal)) 1950 and the DoiT CUD (shown in pink) 1951 have the same value and therefore overlap, making them difficult to distinguish in Figure 19. As shown, both are within a predetermined safety margin, being less than the total on-demand plot 1911 (blue line at the top of the figure). Also as seen in the figure, in the far right bin of Figure 19, the total number of CUDs increased even if the total on-demand usage for this customer had not changed, but in fact, it decreases at the far right of the on-demand plot 1911. This is because there is a surplus in another customer. Rather than generating waste, the surplus CUD was moved to this customer shown in Figure 19 (where it goes above the 85% target baseline), but still within 100% of the stable baseline limit.

[0107] It should be noted that, "Generally, Google recommendations are often inaccurate because they lack recommendations for some workloads, and even when provided, they are often slow to respond to changes in workloads." Since GCP algorithms appear to assume that CUD remains stable, those algorithms do not address on-demand workload motion as shown above with respect to the fluctuating on-demand plots in Figures 14, 15, 16, and 19.

[0108] Figures 20-26, described below, illustrate optimization in various embodiments. Constant workload changes can typically result in some customers being oversupplied and others undersupplied. In some embodiments, system optimization rebalancing may be performed daily to shift oversupplied CUD to undersupplied customers.

[0109] Figure 20 reflects probability calculations corresponding to various exemplary optimization processes in various embodiments. Initially, a 24-hour stable utilization baseline is determined as described above. Next, the probability (the number of CUDs needed or surplus) is calculated using the target coverage of the stable utilization baseline, as described above. The target coverage can be some percentage less than 100% of the stable utilization baseline, as described above. This probability is shown in Figure 20 for each BA, where a negative probability indicates a surplus of CUDs, as seen in BA-1. Finally, surplus CUDs can be distributed to workloads that require them, for example. As seen in Figure 20, BA-1 has a large negative probability (surplus CUDs - oversupply), while BA-5 has a large probability (undersupply).

[0110] Figure 21, which is the same as Figure 6 above and presented again here for convenience, shows exemplary distributions of CUDs in various embodiments. Note that CUD allocation is not a trivial computation problem. CUD allocation is an example of a classic bin packing problem, and in some embodiments, Google's OR tools, such as the solver in Google's OR Tools package designed to tackle bin packing, can be used for efficient allocation. For example, in some embodiments, the solver described below can be used: https: / / developers.google.com / optimization / pack / bin packing#:~:text=Each%20item%20must%20be%20placed,items%20have%20to%20be%20packed.

[0111] Figure 22 shows an example where excess CUD for a given SKU is moved to an undersupplied billing account. Thus, CUD is removed from BA1-3 as shown, and CUD is added to BA4-6 as also shown. In some embodiments, the exemplary optimizer may move surplus CUD to a BA where space is found. With respect to CUD that cannot be moved (for example, if there is no place to move the CUD as all other customers already have sufficient CUD), the surplus CUD may be left where it is so that if the workload is spiked, for example as shown in Figures 5, 14, or Figure 23 described below, at least most or a significant portion of the surplus CUD may be used. Similarly, if all customers already have sufficient CUD, the surplus CUD may be moved from a BA with a stable workload, as shown in Figure 15, to a BA with a spiked workload, as shown in Figure 15, 1520, since there is usually a greater demand for the *portion* of the spiked workload above its stable baseline.

[0112] Figure 23 depicts an exemplary extreme “spike” workload that exhibits not only significant fluctuations in the total on-demand workload but also a wild dynamic range (and thus often far beyond its stable baseline). In this approach, the customer is not charged for surplus CUD unless it is actually used, but as mentioned above, this type of workload is a good candidate for oversupply due to surplus CUD. In the example in Figure 23, as in Figure 19 above, here total CUD is equal to DoiT CUD because the customer does not have any of their own CUD. Also as in Figure 19, there is a complete overlap between these two categories, so it is somewhat difficult to distinguish the dark gray “Total CUD” from the pink DoiT CUD, as the customer previously did not have any of their own CUD.

[0113] Figure 24 depicts an exemplary first run of an exemplary optimizer to obtain each workload to its specified target coverage. The input to the first run of the exemplary optimizer is the value in the "~Previous Coverage" column 2410, and the output from the optimizer is the value in the "~Subsequent Coverage" column 2420. In this example, the target coverage is 85%, which is now satisfied by customers A, B, and C after the first optimization run, but customer D remains oversupplied. The CUD, which has changed as indicated by the arrows (up or down) in column 2410, is for each customer's different workload size. Therefore, the corresponding rate of change reflected in column 2420 for each change in CUD will be the rate of change of various actual numbers of CUD. Thus, for customer A, an additional 20 CUD results in a change of 78% to 85%, while an additional 30 CUD for customer B results in a change of 68% to 85%, and so on.

[0114] Figure 25 depicts an exemplary second run of the exemplary optimizer to distribute any oversupply among those customers who are not oversupplied. Here, as shown in Figure 24 in column 2420, after the first optimizer run, instead of one oversupplied workload (e.g., customer D) and all other workloads below 100% supply, the oversupply of customer D is distributed among the other workloads, and as a result, customers A, B, and D are all at 100% of their respective stable baselines and therefore now exceed the target coverage. However, given this second optimization run change, no oversupplied workloads remain.

[0115] Alternatively, Figure 26 illustrates another case where oversupply may still remain for one or more workloads after 100% coverage has been achieved for all (most) other workloads. Here, in this example, all customers have oversupply. Note here that the percentage is relative to the size of the workload, as mentioned above. Consider workloads A, B, and C, each with 100 CUD, and 15% is added to each, so each becomes size 115, and now has 100% coverage. Workload D may be a larger workload with an on-demand value of 375, and has 450 CUD, and is therefore 120% supplied. In the example in Figure 26, 45 CUD (i.e., 10% of CUD) may be removed from customer D, reducing the CUD by 10%, and its coverage becomes 405 / 375 (=108%). This is shown as 100% in column 2620, using approximations.

[0116] Exemplary optimization process for providing coverage In one or more embodiments, various optimization processes may be used to generate a set of recommendations for a given payer. In some embodiments, the best approach for a given customer may differ from that of another customer, because each customer will have different parameters (e.g., 15th percentile vs. 5th percentile) based on their own usage patterns.

[0117] In some embodiments, various methods may be used to find the optimal-best possible (fixed) commitment (with respect to customer type) for a given timeframe.

[0118] One technique called "Moving Weekly Percentile" can utilize moving weekly percentiles. Here, an n-day window function can be used to obtain the moving x-th percentile (x=5). Various experiments have shown that combining a 24-hour window and a 7-day window can be very effective.

[0119] Another technique, known as "Moving Optimum," can calculate the optimal (straight) line of coverage for a selected (windowed) timeframe. In some embodiments, 1-day, 3-day, and 7-day windows may be used to obtain both aggressive and conservative estimates, and the decision on which to use may then be based on the customer's predictability feature. For stable and predictable customers, it has been found that the 1-day and 7-day recommendations should not differ significantly. However, whenever there is a large difference between the 1-day and 7-day windows, in some embodiments, a more conservative value may be selected, for example.

[0120] In some embodiments, these windowed values ​​can be combined as follows: Simple average - average estimate of all windows; and Over-averaging – Here, weighting is used to control which estimate should be trusted most. For example, for predictable customers whose usage changes can be accurately foreseen, the optimal daily weighting can be increased up to 100%, and therefore relies almost entirely on the predicted / enhanced data.

[0121] In some embodiments, a function may also be used to calculate the maximum likelihood savings rate for each applicable unit increment. This can be done, for example, by averaging the performance of applicable units at the same coverage level as previous ones.

[0122] Therefore, for example, when adding $1 to a desired commitment of $1.5, the average savings rate in applicable cost units between, for example, ($0.5 to $1) and ($1 to $1.5) will be calculated.

[0123] The example shown in Figure 27 illustrates the thinking behind the exemplary “moving optimal” method. In this example, 24-hour data with a moving 6-hour long window is used. Thus, there are four distinct windows over the 24-hour period. Here, we can see that the optimal values ​​for each of the four 6-hour windows 2710, 2711, 2712, and 2713 (shown as straight horizontal lines in each 6-hour window) are different from those of the other three 6-hour windows. The optimal values ​​for each 6-hour window also deviate (and most of them significantly) from the 24-hour optimal commitment 2720, which is shown as a dashed line running across the entire plot.

[0124] On the other hand, Figure 28 shows an exemplary customer that is stable / predictable. Thus, as shown in Figure 28, the optimal values ​​for the four 6-hour periods (i.e., 2810, 2811, 2812, and 2813) are very close to those for the 24-hour optimal commitment 2820.

[0125] Figures 29A and 29B together show a process flowchart for allocating commitment inventory by a facilitation system such as DoiT using technology such as Flexsave or its equivalent. Starting from the left side of Figure 29A, the input data includes not only the DoiT inventory of commitments 2901 but also allowable commitments with metrics that can be assigned to each customer organization 2903. These inputs are fed to processing block 2910, where the commitment inventory is allocated to satisfy any explicit coverage requirements. From block 2910, processing continues to block 2920, where the remaining inventory is allocated to satisfy default coverage across the remaining organization. Default coverage here represents how much (85%, as shown above, unless otherwise configured) the facilitation system wants to cover by default. This is done both to leave space for unexpected changes in workload and to maintain some space across the organization in case some inventory needs to be moved from reducing workloads of other customers.

[0126] From block 2920, the processing path depends on whether residual inventory still exists. Therefore, although not shown, it is understood that block 2920 is immediately followed by a query block that determines whether there is residual inventory after the inventory has been allocated to satisfy default coverage across the entire residual organization, as done in block 2920. If the response to this query is "no", then at 2923, processing moves to block 2990 and terminates. However, if the response to the query is "yes", and therefore, if commitment inventory still remains after the allocation in block 2920, then data regarding the residual inventory is provided via 2925 to block 2930, shown in Figure 29B, which is described next, and processing continues there.

[0127] Next, referring to Figure 29B, in block 2930, the data on remaining inventory received from block 2920 via 2925 is allocated up to full coverage based on stability and savings. Following block 2930, it is again determined (not shown, but via a query block immediately following block 2930) whether there is any remaining inventory after the allocation of commitment inventory shown in block 2930. If the response to the query is "yes", in 2933, the process moves to block 2940, where the remaining inventory is allocated to minimize waste. However, if there is no remaining inventory following the allocation shown in block 2930, as shown in 2935, the processing flow moves to block 2995, where the process ends. Returning to block 2940, an exemplary procedure to minimize waste would consider using commitments beyond the stable baseline. This will be a combination of (i) how much commitment can be utilized and (ii) what the savings rate will be - the product of these values ​​or data will inform the exemplary system how much waste will be generated, and the allocation of block 2940 will be prioritized based on minimum waste. In this regard, it should be noted that the stable baseline only determines "what is considered possible to cover", but this does not mean that there will be no spending above the stable baseline, as seen in the various plots of on-demand usage discussed above. This is especially true for the various "spiking" workload request plots discussed above.

[0128] Flexsave system implementation In some embodiments, the entire Flexsave system may be hosted on a CSP server, such as one on the Google Cloud platform within a DoiT organization.

[0129] In some embodiments, all associated computing systems may be packaged within containers and hosted, for example, in the GCP serverless offering, where almost all of these may run on a service called, for example, CloudRun, where some systems may use AppEngine. The fact that these workloads are containerized means that they may run other services (such as Google Kubernetes Engine, either as a Google-managed Kubernetes cluster or as a GKE-hosted CloudRun) on regular virtual machines that can host containers, as needed. Technically, these workloads and computing systems may run on any cloud provider that helps run containerized workloads.

[0130] In some embodiments, a variety of smaller services within GCP, for example, may be used to facilitate the organization of tasks. For example, in some embodiments, Cloud Composer may be used to organize billing import and recalculation jobs, Cloud Scheduler may be used for periodic jobs, Cloud Task may be used for reliable job execution scheduling, and Firestore may be used for a configuration database, and so on.

[0131] In some embodiments, the second major component may be Google BigQuery, a petabyte-scale data warehouse solution. In such embodiments, it may be used for storing and processing billing information, all recalculated data, as well as built-in ML capabilities.

[0132] For example, all of the above software can be adequately hosted by DoiT on GCP with respect to both Flexsave for AWS and Flexsave for GCP, and any other implementations of Flexsave from other cloud service providers.

[0133] Typically, with respect to Flexsave versions, customers do not host any infrastructure or perform any operations themselves. Flexsave only requires customer permission to access that infrastructure to make necessary changes and read data (i.e., (i) download billing information data (CUR in the AWS example), and (ii) grant / redeem Flexsave / DoiT commitments to / from the customer's respective organizations).

[0134] With regard to real-time embodiments, depending on the source of the information, customers may be required to configure something within their environment to forward the relevant real-time data stream to DoiT for further processing. Even in such a scenario, the data would still be processed on the DoiT hardware.

[0135] In some embodiments, the DoiT interface with the cloud provider always goes through the public API of the relevant cloud. In some embodiments, the facilitation system, such as Flexsave / DoiT, does not need to own or operate any hardware because all functionality is hosted by the cloud provider. However, it is understood that such Flexsave embodiments are not bound to the cloud provider in any way, and Flexsave can run perfectly well on its own hardware within the data center if needed (only requiring the same public API access).

[0136] Various implementations of the above-described systems and technologies may be implemented by digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on a chip (SOC), load programmable logic devices (CPLDs), and computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be implemented in one or more computer programs, one or more computer programs may be executed and / or interpreted on a programmable system including at least one programmable processor, the programmable system may be a dedicated or general-purpose programmable processor for receiving data and commands from a storage system, at least one input device and at least one output device, and for transmitting data and commands to the storage system, at least one input device and at least one output device.

[0137] Program code configured to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, a dedicated computer, or other programmable data processing device so that, when the program code is executed by the processor or controller, it enables the functions / operations defined in the flowcharts and / or block diagrams. The program code may be fully executed on a machine, partially executed on a machine, partially executed on a machine, partially executed on a remote machine as a standalone software package, or fully executed on a remote machine or server.

[0138] In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in association with a command execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More specific examples of machine-readable storage media include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM), flash memory, optical fibers, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0139] To provide user interaction, the systems and technologies described herein may be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) and a keyboard and pointing device (such as a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide user interaction. For example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback or tactile feedback), and input from the user may be received in any form (including acoustic input, voice input or tactile input).

[0140] The systems and technologies described herein may be implemented in computing systems including background components (e.g., data servers); or computing systems including middleware components (e.g., application servers); or computing systems including front-end components (e.g., user computers having a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein); or computing systems including any combination of such background components, intermediate computing components, or front-end components. The components of the system may be interconnected by digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks may include, for example, local area networks (LANs), wide area networks (WANs), and the Internet.

[0141] A computer system may include clients and servers. Clients and servers are typically far apart from each other and therefore interact via a communication network. The client-server relationship is generated by computer programs running on each computer and has a client / server relationship with each other. A server may be, for example, a cloud server, a server in a distributed system, or a server connected to a blockchain.

[0142] According to some embodiments of the present disclosure, the disclosure also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of sharing resources or generating services as described in the aforementioned embodiments of the present disclosure are performed.

[0143] It should be understood that the various forms of the processes described above can be used to rearrange, add, or remove steps. For example, the steps described herein may be performed in parallel, in series, or in different orders, as long as the desired results of the technical solutions disclosed herein are achieved, and are not limited herein.

[0144] The above-described embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, subcombinations, and substitutions may be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be within the scope of protection of this application.

Claims

1. The system facilitates the receipt of billing data exports ("BDE") from cloud service providers' customers. To process the BDE in order to determine the workload coverage needs within the customer's organization, To determine the optimal blend of commitments required by the aforementioned customer, The combination of accounts owned by the aforementioned promotion system and the customer's organization in accordance with the required commitments, wherein at least one commitment is held within the account of each promotion system, and A method comprising monitoring the customer's workload coverage needs at predetermined time intervals to detect changes, and accordingly adding or subtracting an account or a portion thereof from the customer's organization.

2. The method according to claim 1, wherein the determination further includes looking at all expenditures within the customer organization and determining the optimal blend of commitments, available inventory, and risks associated with the workload required by the customer.

3. The method according to claim 1, wherein the merging of the accounts further includes enabling pre-billing of the customer organization in a single transaction.

4. The method according to claim 1, wherein no workload is executed in the account owned by the facilitation system that is responsible for the commitment.

5. The method according to claim 1, wherein the combining, adding, and subtracting further includes obtaining permission from the customer to move the project into and out of the customer's organization.

6. A computer program product for managing cloud service provider commitments, wherein the computer program product is This includes a computer-readable storage medium having computer-readable program code embodied therein, The aforementioned computer-readable program code is: Receiving BDE from a cloud service provider's customer via a facilitation system. To process the BDE in order to determine the workload coverage needs within the customer's organization, To determine the optimal blend of commitments required by the aforementioned customer, The combination of accounts owned by the aforementioned promotion system and the customer's organization in accordance with the required commitments, wherein at least one commitment is held within the account of each promotion system, and A computer program product that can be executed by one or more computer processors to monitor the customer's workload coverage needs at predetermined time intervals in order to detect changes, and to add or subtract an account or a portion thereof to the customer's organization accordingly.

7. The computer program product according to claim 6, wherein the determination further includes looking at all expenditures within the customer organization and determining the optimal blend of commitments, available inventory, and risks associated with the workload required by the customer.

8. The computer program product according to claim 6, wherein the merging of the accounts further includes enabling pre-payment of the customer organization in a single transaction.

9. The computer program product according to claim 6, wherein no workload is executed in the account owned by the facilitating system that is responsible for the commitment.

10. The computer program product according to claim 6, wherein the combining, adding, and subtracting further includes obtaining permission from the customer to move the project into and out of the customer's organization.

11. A system for optimizing coverage for one or more customers of a workload service provider, wherein the system is At least one processor, and Includes memory containing instructions, When the aforementioned instruction is executed, it will cause at least one processor to, for each customer, To receive the on-demand workload usage of the aforementioned customer for N days, Calculate a stable usage baseline based on the data for the aforementioned N days. Calculating the target coverage of the customer, which is a predetermined percentage of the stable usage baseline. A system that allocates a set of commit usage discounts ("CUDs") to cover the aforementioned target coverage.

12. The system according to claim 11, wherein the workload service provider is a cloud service provider.

13. The aforementioned predetermined percentage is, A number between 0.75 and 0.90, or ０．８５ The system according to claim 11, wherein at least one of the above.

14. The system according to claim 11, wherein the data for the N days is data for either 30 days or 31 days.

15. The system according to claim 11, wherein, when the instruction is executed, it further causes the at least one processor to transfer the CUD from the facilitating system to the customer in order to satisfy the target coverage.

16. The system according to claim 11, wherein, when the instruction is executed, it further causes the at least one processor to perform a recent time baseline verification process to determine whether the stable usage baseline has changed.

17. The stable usage baseline is a first stable usage baseline, and the recent time verification process is To obtain a window of on-demand workload usage data for the most recent M hours, To determine whether the on-demand workload usage data for the most recent M hours falls below the stable usage baseline, and If the answer is "yes", To generate a second stable utilization baseline of on-demand workload utilization for the most recent M hours, and The system according to claim 16, comprising using the second stable usage baseline to calculate the target coverage of the customer.

18. The one or more customers are multiple customers, and when the instruction is executed, it further causes at least one processor to: Determine whether any customer's CUD exceeds those target coverages. The system according to claim 11, wherein in the first optimization, CUDs are moved between oversupplied customer billing accounts and undersupplied customer billing accounts.

19. The system according to claim 18, wherein in the first optimization, if all customer billing accounts are supplied beyond their respective target coverage, the CUD may be moved to the billing account up to the then operational stable usage baseline of the customer billing account.

20. When the aforementioned instruction is executed, it further causes at least one processor to: Determine whether any customer remains oversupplied after the first optimization. If the answer is "yes", The system according to claim 18, wherein the excess coverage is reallocated to full coverage, firstly based on stability and savings rate, and secondly based on minimizing waste.

Images