Method and apparatus for running cloud application and for selecting scaling policy

By introducing scaling strategies based on service quality measurement and resource consumption in the cloud system, dynamically adjusting resource allocation is solved, and the problem that cloud application scalers cannot flexibly adjust resource allocation in the existing technology is solved, and the maximum between service quality and cost is achieved Good trade-offs to ensure efficient operation of cloud applications under different load scenarios.

CN119032344APending Publication Date: 2024-11-26ROBERT BOSCH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380020732.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-02-08
Filing Date
2023-02-01
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Existing cloud application scalers cannot flexibly adjust resource allocation when handling load changes, resulting in difficulty in achieving optimal trade-offs between service quality and cost, and cannot effectively meet service quality requirements in different load scenarios.

Method used

By introducing scaling strategies based on service quality measurement and resource consumption in the cloud system, using the scaler to dynamically allocate processing resources, combined with timing simulation software for load simulation, selecting the optimal scaling strategy to meet service quality requirements and minimize resource consumption.

Benefits of technology

It realizes dynamic adjustment of resource allocation under different load scenarios, maximize service quality measurement, and minimize cloud costs, ensuring efficient operation of cloud applications under various load situations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119032344A_ABST
    Figure CN119032344A_ABST
Patent Text Reader

Abstract

The invention relates to a computer-implemented method for operating a cloud application having a plurality of interacting microservices (2) in a cloud system, comprising the following steps:-assigning (S3, S5) processing resources to the microservices (2) by means of a scaler according to a scaling policy; running (S1) the cloud application according to the assigned processing resources; wherein the scaling policy determines processing resources to be dispatched to the microservice in accordance with a quality of service metric that accounts for performance of the cloud application and in accordance with a resource metric that accounts for expenditure of the processing resources that have been dispatched to the microservice.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a cloud application which is created as a distributed application by means of microservices and also to measures for resource management of such a cloud application. Background Art

[0002] Software applications executed in the cloud, i.e. in a central location remote from the end user, are often implemented as distributed applications with so-called microservices. Microservices are individually running software modules that communicate with each other via appropriate interfaces, such as so-called REST interfaces and message queues, and thus interact to provide software tasks.

[0003] Microservices of a software application can be scaled, i.e. copied or moved, to changed processing resources or hardware platforms, for example more powerful or less powerful hardware, by so-called scalers. Known scalers are inflexible and can only accommodate a wide range of load requirements to a limited extent. Summary of the invention

[0004] According to the invention, a method for running a software application in a cloud system is provided according to claim 1 and a method for providing a scaling policy for a cloud application according to the parallel claim. A corresponding computer program product and a corresponding machine-readable storage medium are also subject matter of the present application.

[0005] Further refinements are described in the dependent claims.

[0006] According to a first aspect, a method for running a cloud application in a cloud system is provided, wherein the cloud application has a plurality of interacting microservices, and the method comprises the following steps:

[0007] - Dispatching processing resources to microservices based on scaling policies with the help of scalers;

[0008] -Run cloud applications based on the allocated processing resources;

[0009] The scaling policy determines processing resources to be allocated to the microservice based on a quality of service metric describing the performance of the cloud application and based on a resource metric describing the consumption of processing resources already allocated to the microservice.

[0010] In order to provide cloud applications, the cloud applications are usually implemented in the form of a system composed of micro services that communicate with each other. The micro services are run in a computing environment that is usually remote from the user and is in a preferably wireless communication connection with the user.

[0011] Microservices are individually running software modules that communicate with each other via suitable interfaces, such as so-called REST interfaces and message queues. Thus, microservices can interact to provide software tasks.

[0012] The communication between microservices and between a microservice and one or more databases and / or users is carried out by communication notifications, for example in the form of query or response messages, either via HTTP requests or via message queues. The latter represent message services that allow asynchronous processing of information. The message services operate according to the FIFO (First In First Out) principle and enable parallel processing of information in the case of high load or high communication load.

[0013] Microservices architecture is more heterogeneous than unified software. Microservices can also be developed in different programming languages ​​for cloud applications and run on different platforms, such as VM, Azure, ABS, local cloud. As a result, microservices architecture provides much greater design freedom than unified software applications. Microservices can be executed and scalable on processing resources, such as one or more computing devices, such as processors, computing cores, virtual machines, etc.

[0014] One, more or all microservices of a software application are typically scaled by an implemented scaler, which may be part of an operating system such as K8S or implemented as an autoscaler. The scaler allows microservices to be copied or moved to changed processing resources or hardware platforms, such as more powerful or less powerful hardware. The scaler allows communication notifications to be provided to one or more parallelized similar microservices. The scaler typically operates based on rules, wherein the scaling rules, i.e., the allocation of computing power to the individual microservices, are performed based on rules, such as according to load requirements.

[0015] The scaling policy is configured to determine processing resources to allocate to the microservice based on at least one current value of the quality of service metric and at least one current value of a resource metric for processing resources currently allocated to the microservice.

[0016] Thus, the above method provides a method for operating a scaler for distributed applications, which operates based on a dominant parameter, namely a quality of service metric that quantifies the quality of service (the higher the better the requirements are met). The scaler is thus designed to maximize the quality of service metric so that resource allocation can always be performed in an optimal manner.

[0017] In other words, the scaling strategy determines, in particular, the processing resources to be allocated to the microservices in such a way that the expenditure of processing resources required to achieve or meet a predefined or predefinable quality of service measure is minimized.

[0018] Essentially, a regulator design is proposed for this purpose for a scaler of distributed applications. In this case, resource allocation is performed according to a dominant variable. In the case of a microservice architecture, resource allocation by the scaler can be regarded as a regulator, the distributed software application can be regarded as a regulation segment, and the dominant variable can be regarded as a measure of the quality of service (quality of service measure).

[0019] Furthermore, the quality of service metric may be related to a maximum or average response time of the cloud application and / or depend on an average information content of the responses of the cloud application.

[0020] In order to determine the quality of service metric, the performance of the cloud application is taken into account over a predetermined time period. For example, the response time can be determined over a predetermined time period and the quality of service metric can be determined therefrom, for example as an average response time. Alternatively, system states, such as the filling state of a message queue, etc., can also be taken into account in the quality of service metric. In addition, the quality of service metric can also be indirectly determined based on the hardware state, CPU load, memory utilization, or a combination of two hardware states for each microservice and / or the system state of one or more databases, such as write and read rates. The hardware state and system state have the advantage that they can be detected very simply compared to, for example, the response time for each user.

[0021] It may be provided that the scaling strategy has a controller structure for regulating to one or more desired quality of service metrics, in particular to one or more values ​​or value ranges of one or more quality of service metrics.

[0022] The control, i.e. "regulation", by the scaler is designed so that the total cost is minimized as much as possible according to the cost or cost function. For example, the cost function can be based on one or more resource metrics, such as CPU load, working memory used, length of message queues, response time of microservices or a combination of one or more of these resource metrics. In this case, the more technical resources are allocated and / or used, the greater the value of the cost function for a pre-given allocation of resources. The cost function can be a combination, such as a weighted combination, of assigning a dimensionless number to one or more of the above resource metrics, which represents the cost associated with the provision or allocation of resources.

[0023] To this end, the scaler is run so as to achieve a trade-off between maximizing the quality of service metric and minimizing the cost represented, for example, by the cloud cost, which depends on the resource metric. The cost may correspond to the cost of the available or provided processing resources for executing the microservices and their communications.

[0024] For example, the quality of service can be predetermined by a maximum response time of, for example, 100 ms for a website that processes information via a microservice backend as a cloud application. If the number of users of the website increases, the cloud application can be overloaded and the response time increases. As a result, the quality of service decreases and additional processing resources are assigned to the re-queried microservices until the response time of the website is acceptable again. Correspondingly, if the number of users of the website decreases and thus the load requirements on the website decrease and the consumption of the allocated processing resources is therefore too high according to the consumption or cost function, the microservices that are too strongly parallelized can be scaled down.

[0025] To evaluate the processing resources, resource metrics can be used, which quantitatively evaluate the provided processing resources. According to one embodiment, the processing resources can be adapted according to the quality of service metric, the rated preset for the quality of service metric, and according to the cost, for example, the cloud cost realized by one or more resource metrics, such as CPU load, used working memory, length of message queues, response time of microservices, and a combination of one or more of these resource metrics.

[0026] According to another aspect, a method for providing a scaling policy for a specific cloud application in a cloud system is provided, wherein the cloud application has a plurality of interacting microservices, the method comprising the following steps:

[0027] - Use timing simulation software to determine the runtime, processor load, and memory requirements of microservices;

[0028] - providing a plurality of different usage scenarios, each of which prescribes a temporal course of the load requirement on the cloud application for an evaluation period;

[0029] - providing a plurality of different scaling strategies, the scaling strategies determining the processing resources to be allocated to the microservices in particular based on a quality of service metric describing the performance of the cloud application and based on a resource metric describing the cost of the processing resources already allocated to the microservices;

[0030] - performing a load simulation in order to determine, for each of the scaling strategies, an aggregated quality of service metric consisting of the quality of service metrics of all usage scenarios and an aggregated resource metric consisting of the resource metrics of all usage scenarios;

[0031] - Selecting a scaling strategy from a plurality of scaling strategies based on a corresponding aggregated quality of service metric and a corresponding aggregated resource metric.

[0032] A scaling strategy may be selected from a plurality of scaling strategies using one or more predetermined or predeterminable criteria, such as thresholds for a corresponding aggregated quality of service metric and a corresponding aggregated resource metric. For example, a scaling strategy may be selected for which the corresponding aggregated quality of service metric exceeds a first threshold and for which the corresponding aggregated resource metric is below a second threshold.

[0033] The above method accordingly provides the following possibilities: designing a scaler for a cloud application in an appropriate manner. For this purpose, a distributed software application can be modeled using timing simulation software in order to determine the runtime and memory requirements for the included microservices. Based on a plurality of pre-given different usage scenarios, a load simulation is performed in order to determine the expected quality of service metric, which usage scenarios can be distinguished, for example, by the time course of the usage or load, in particular in the form of the number of users, the requested tasks, etc. The load simulation is performed based on different scaling strategies, which are characterized by different strategies for allocating resources to each microservice according to the load state of the usage scenario, so that a quality of service metric and resource consumption are obtained for each combination of different scaling strategies and different usage scenarios. The load simulation is used to determine the quality of service metric and consumption of the allocation of processing resources so that one of the scaling strategies can be appropriately selected based on the Pareto front obtained therefrom. This enables the selection of an optimized scaling strategy for the distributed application.

[0034] It may be provided that the aggregated quality of service metric corresponds to an average value or a maximum value of the quality of service metric for the evaluation duration and / or that the aggregated resource metric corresponds to an average value or a maximum value of the resource metric for the evaluation duration.

[0035] Furthermore, the resource metric may indicate resource consumption, in particular resource costs, in particular with respect to the number of required cores, virtual machines and / or processors, network bandwidth and / or the need for working memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The following is a more detailed explanation of the embodiments based on the attached drawings, wherein:

[0037] Figure 1 A schematic diagram showing a cloud application composed of microservices;

[0038] Figure 2 Shown to illustrate the operation Figure 1 A flow chart of a method of a cloud application scaler; and

[0039] Figure 3 A flow chart illustrating a method for selecting a scaling policy for a scaler used with a particular cloud application is shown. DETAILED DESCRIPTION

[0040] Figure 1 The structure of a cloud application in a cloud system 1 as a distributed application with a large number of microservices 2 is schematically shown. The cloud system represents a computing environment for processing and executing microservices 2. The microservices 2 can be developed in different programming languages ​​and can run on different platforms of the cloud system 1, such as VM, Azure, AWS, local cloud, etc.

[0041] The microservices 2 are connected to each other via a communication interface 3. One or more databases 4 may also be provided, which are connected to one or more microservices 2 via corresponding communication interfaces 3. One or more of the microservices 2 communicate with each other.

[0042] The communication interface 3 can be designed, for example, in the form of a REST interface and is used to perform communication between the corresponding microservice 2 and a database 4 or other microservices 2. In addition, the cloud application can be used by a large number of users via a web-accessible user interface 5, which can also be designed as a microservice.

[0043] Communication can be done either through HTTP requests or via message queues. Message queues enable asynchronous processing and work on the FIFO principle. This enables short-term data backlogs to be handled without affecting other system components, such as other microservices, and without overloading other microservices.

[0044] A scaler 6 is provided, which predetermines specific processing resources for each microservice 2 according to a predetermined scaling strategy, for example in the form of a number of cores and / or processor performance, etc., and can change the specific processing resources. In addition, the scaler 6 can also copy microservices 2 and run them in parallel as separate microservices 2, wherein corresponding tasks can be assigned in parallel to similar microservices 2 for load distribution.

[0045] Now according to Figure 2 The flowchart of describes in more detail a method for operating a scaler 6, in particular in a cloud system 1. Here, the proposed scaling strategy corresponds to a "two-point regulation" by which processing resources are allocated.

[0046] In step S1 , the sealer 6 is operated according to a predefined allocation of processing resources and continuously detects a quality of service measure.

[0047] The service quality metric represents a metric describing the performance, such as the usability of the cloud application, such as the average response time of a website, with information about the total number of users and / or a predetermined evaluation period determined and provided by the microservice backend in the cloud application. A further possibility of evaluating the service quality metric in another way or in combination with the response time is to evaluate a metric of the completeness of the information transmitted to the user.

[0048] In order to determine the quality of service metric, system states, such as the filling state of a message queue, etc., can also be taken into account in the quality of service metric. In addition, the quality of service metric can also be determined indirectly based on the hardware state, CPU load, memory utilization or a combination of both hardware states for each microservice and / or system states of one or more databases, such as write and read rates. The hardware state and system state have the advantage that they can be detected very simply compared to, for example, the response time for each user. The quality of service metric quantifies the overall performance of the cloud system 1 during the evaluation period.

[0049] In step S2, it is checked whether the quality of service metric meets a predetermined first QoS criterion. If the quality of service metric is specified quantitatively, this can be done, for example, by means of a threshold comparison with a QoS threshold as the first QoS criterion. If it falls below the QoS threshold, this indicates a reduced quality of service (option: No), the quality of service metric does not meet the predetermined first QoS criterion, and the method is continued with step S3, otherwise (option: Yes) the method is continued with step S4.

[0050] In step S3, additional processing resources are allocated to the cloud application. The resource allocation of additional processing resources, such as the number of computing nodes reserved at the cloud provider (horizontal scaling method), or the CPU power and / or working memory allocated to each computing node (vertical scaling method) can be performed for one or more or all microservices, and in particular according to the CPU load caused by them, the working memory required by them respectively, or the filling state of the message queue connected thereto, or a combination of multiple of these descriptions. The determination of the CPU load, the working memory required by the microservices respectively, or the filling state of the message queue connected thereto is performed by a cloud operating system, such as K8S. The CPU and RAM utilization is detected for each container (the smallest unit of a microservice). The method is then continued with step S1.

[0051] In step S4, it is checked whether the quality of service measure meets a predetermined second QoS criterion. If the quality of service measure is specified quantitatively, this can be done, for example, by means of a further threshold comparison with a further QoS threshold as the second QoS criterion. If the further QoS threshold is exceeded, this indicates an increased quality of service (option: no), the quality of service measure does not meet the predetermined second QoS criterion, and the method is continued with step S1, otherwise (option: yes), the method is continued with step S5.

[0052] In step S5, the allocation of processing resources is deleted from the cloud application. The resource allocation of the processing resources to be removed, for example the number of computing nodes reserved at the cloud provider (horizontal scaling method) or the CPU power and / or working memory allocated to each computing node (vertical scaling method) can be performed for one or more or all microservices and in particular as a function of the CPU load caused by them, the working memory required by them respectively or the filling state of the message queues connected thereto or a combination of a plurality of these specifications.

[0053] The above scaling strategy corresponds to a two-point scaler operating in a two-point regulation form. Alternatively, a proportional scaler or the like may also be provided, which determines resource allocation and performs allocation directly based on the quality of service metric.

[0054] Figure 3 A method is shown for determining a suitable scaling strategy with which the sealer 6 can be operated.

[0055] In step S11 , for this purpose, the distributed application is analyzed with the aid of a timing simulation software and the microservices are parameterized with corresponding values ​​for runtime, processing load and memory requirement.

[0056] Furthermore, in step S12, various scaling strategies are provided or designed, which in particular determine the allocation of processing resources in each case depending on a quality of service metric.

[0057] In step S13, usage scenarios are pre-given for various load situations. A load simulation is performed for each load scenario and for each scaling strategy, and service quality metrics and resource metrics are detected. As described above, the service quality metrics can be described based on usage characteristics, such as the maximum response time of a website and / or the information content of the response of a cloud application. In addition, the state of each microservice: CPU load, working memory utilization or a combination thereof can be used to determine the service quality metrics. In addition, system states of the database such as write rate and read rate are system states suitable for determining the service quality metrics. Here, the service quality metrics are determined for a time length facing the usage scenario or an evaluation time period corresponding to the time length.

[0058] Resource metrics describe resource consumption, for example, represented by the number of computing nodes ordered from a cloud provider or the network bandwidth reserved or available for a cloud application. The resource metrics can be determined by the cloud provider by the number of required cores, virtual machines or processors, network bandwidth and working memory requirements.

[0059] like Figure 4As shown in , a Pareto front can now be derived, which is detected by the aggregated service quality metric QoS (e.g., the service quality metric averaged over the usage scenarios) and the resource metric K (e.g., the resource metric averaged over the usage scenarios) for different scaling strategies.

[0060] In particular, the quality of service metrics are averaged for different N usage scenarios in order to be able to optimally select a scaling strategy for different loads, such as low, medium or high loads.

[0061] In step S14, a scaling strategy is selected from the Pareto front, which scaling strategy describes a good trade-off between the quality of service metric and the resource metric for the scaling strategy under consideration. The scaling strategy can be selected using one or more predetermined or predeterminable criteria, such as threshold values ​​for the quality of service metric and the resource metric.

Claims

1. A computer-implemented method for running a cloud application in a cloud system, the cloud application having a plurality of interacting microservices (2), the method comprising the following steps: - allocating (S3, S5) processing resources to microservices (2) according to a scaling strategy with the help of a scaler; - running (S1) cloud applications according to the allocated processing resources; The scaling policy determines processing resources to be allocated to the microservice based on a quality of service metric describing the performance of the cloud application and based on a resource metric describing the consumption of processing resources already allocated to the microservice.

2. The method of claim 1, wherein the scaling policy has a regulator structure for regulating to one or more desired quality of service metrics.

3. The method according to claim 1 or 2, wherein the quality of service metric is related to a maximum or average response time of the cloud application and / or depends on an average information content of the responses of the cloud application.

4. A device for running a cloud application in a cloud system, the cloud application having a plurality of interacting microservices (2), wherein the device is configured to: - using the scaler to allocate processing resources to microservices according to the scaling policy (2); -Run cloud applications based on the allocated processing resources; The scaling policy determines processing resources to be allocated to the microservice based on a quality of service metric describing the performance of the cloud application and based on a resource metric describing the consumption of processing resources already allocated to the microservice.

5. A computer-implemented method for providing a scaling policy for a specific cloud application in a cloud system, the cloud application having a plurality of interacting microservices (2), the method comprising the following steps: - using timing simulation software to determine (S11) the running time, processor load and memory requirements of the microservice (2); - providing ( S13 ) a plurality of different usage scenarios, each of which predetermines a time course of a load requirement on the cloud application for an evaluation period; - providing (S12) a plurality of different scaling strategies, said scaling strategies determining processing resources to be allocated to the microservice (2) based on a quality of service metric describing the performance of the cloud application and based on a resource metric describing the consumption of processing resources already allocated to the microservice; - performing (S13) a load simulation in order to determine, for each of the scaling strategies, an aggregated quality of service metric consisting of the quality of service metrics of all usage scenarios and an aggregated resource metric consisting of the resource metrics of all usage scenarios; - selecting (S14) a scaling strategy from a plurality of scaling strategies based on the corresponding aggregated quality of service metric and the corresponding aggregated resource metric.

6. The method according to claim 5, wherein the aggregated service quality metric corresponds to an average or maximum value of the service quality metrics for the evaluation duration, and / or wherein the aggregated resource metric corresponds to an average or maximum value of the resource metrics for the evaluation duration.

7. The method according to claim 5 or 6, wherein the resource metric describes resource consumption, in particular with respect to the number of required cores, virtual machines and / or processors, network bandwidth and / or working memory requirements.

8. A device for providing a scaling policy for a specific cloud application in a cloud system, the cloud application having a plurality of interacting microservices (2), wherein the device is configured to: - using timing simulation software to determine the runtime, processor load and memory requirements of the microservice (2); - providing a plurality of different usage scenarios, each of which prescribes a temporal course of the load requirement on the cloud application for an evaluation period; - providing a plurality of different scaling strategies, said scaling strategies determining processing resources to be allocated to a microservice (2) based on a quality of service metric describing the performance of the cloud application and based on a resource metric describing the consumption of the allocated processing resources; - performing a load simulation in order to determine, for each of the scaling strategies, an aggregated quality of service metric consisting of the quality of service metrics of all usage scenarios and an aggregated resource metric consisting of the resource metrics of all usage scenarios; - Selecting a scaling strategy from a plurality of scaling strategies based on a corresponding aggregated quality of service metric and a corresponding aggregated resource metric.

9. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to perform the steps of the method according to any one of claims 1 to 3, 5 to 7.

10. A machine-readable storage medium comprising instructions, which, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 1 to 3, 5 to 7.