Amplifying the scalability of microservice meshes

Abstract

A grid-type scaler is selected for performing overall grid-width scaling of the microservice grid. The microservice grid is scaled using the grid-type scaler and one or more grid scaling rules to generate a predicted value of a monitoring parameter in one or more monitoring parameters. A single-service scaler is selected. The monitoring parameter is measured using the single-service scaler to generate a measured value. For a separate service associated with the microservice grid, a target value of the monitoring parameter is calculated using the predicted value and the measured value. The target value is compared with a defined threshold value to generate a scaling value for determining whether scaling is required for the separate service.

Description

Background Art

[0001] An application in a microservices architecture (e.g., in a large-scale cloud computing system deployment) will consume a set of services, which in turn may consume another set of services (i.e., the services form a "service mesh"). Applications and services can be scaled out by adding more instances (and scaled in by removing instances). A load balancer typically distributes calls between application and service instances, so that instances of each service typically have similar operating system usage.

[0002] A service (or microservice) grid can be monitored using a monitoring infrastructure that will periodically read operating system parameters (e.g., CPU load, memory usage, network input / output, disk latency, and block paging). As usage load changes, the number of instances of a service in the service grid can be adjusted by the scaling component based on the information provided by the monitoring infrastructure. In this case: 1) if the usage load increases, instances need to be added, otherwise response times increase and service requests may be delayed or even rejected, which results in a poor user experience, or 2) if the usage load decreases, instances should be removed, otherwise operational resources are wasted and overall resource consumption is higher than required, resulting in increased operational costs. These two requirements are in tension: 1) ensure that there are enough instances to ensure efficient operation and user satisfaction, but 2) only as many instances as possible can operate at the lowest possible cost. Summary of the invention

[0003] This disclosure describes scaling up the scaling resiliency of a microservices mesh.

[0004] In an implementation, a grid-type scaler is selected for performing overall grid width scaling of a microservice grid. The microservice grid is scaled using the grid-type scaler and one or more grid scaling rules to generate a predicted value of a monitoring parameter in one or more monitoring parameters. A single service-type scaler is selected. The monitoring parameter is measured using the single service-type scaler to generate a measured value. For a separate service associated with the microservice grid, a target value of the monitoring parameter is calculated using the predicted value and the measured value. The target value is compared with a defined threshold value to generate a scaling value for determining whether scaling is required for the separate service.

[0005] The described subject matter may be implemented using a computer-implemented method; a non-transitory computer-readable medium storing computer-readable instructions to perform the computer-implemented method; and a computer-implemented system comprising one or more computer memory devices cooperatively coupled to one or more computers and having a tangible non-transitory machine-readable medium storing instructions that, when executed by one or more computers, perform the computer-implemented method / computer-readable instructions stored on the non-transitory computer-readable medium.

[0006] The subject matter described in this specification can be implemented to achieve one or more of the following advantages. First, the autoscaling functionality is extended to be aware of the service (or microservice) grid and the dependencies between the service grids. Second, additional operating system parameters (e.g., CPU load, memory usage, network input / output, disk latency, and block paging) are monitored and evaluated in the calculation of the service grid target size. The grid target size is used to automatically accelerate scaling out or scaling in to react faster to changing demand situations to avoid slow response times. Third, the described method allows for a balance between ensuring that there are enough instances to ensure efficient operation and user satisfaction, but only as many instances as possible to operate at the lowest possible cost.

[0007] The details of one or more implementations of the subject matter of this specification are set forth in the detailed description, claims, and drawings. Based on the detailed description, claims, and drawings, other features, aspects, and advantages of the subject matter will become apparent to those of ordinary skill in the art. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram illustrating an example application with three services, application / service calls, and OS monitoring infrastructure in a microservices architecture according to implementations of the present disclosure.

[0009] Figure 2 is a diagram showing an implementation method according to the present disclosure Figure 1 A block diagram of the number of instances of each service at different points in time in a microservice architecture.

[0010] Figure 3 FIG. 1 is a diagram showing a conventional scaler and OS monitoring infrastructure according to an implementation of the present disclosure relative to FIG. Figure 1 and 2 A block diagram of the use of microservice architecture.

[0011] Figure 4 is a flow chart illustrating an example of a computer-implemented method for amplifying scaling elasticity of a microservices mesh according to implementations of the present disclosure.

[0012] Figure 5 is a block diagram illustrating an example of a computer-implemented system for providing computing functions associated with the described algorithms, methods, functions, processes, procedures, and programs according to implementations of the present disclosure.

[0013] Like reference numbers and designations in the various drawings represent like elements. DETAILED DESCRIPTION

[0014] The following detailed description describes scaling elasticity of amplifying a microservices mesh, and is presented to enable those skilled in the art to implement and use the disclosed subject matter in the context of one or more specific implementations. Various modifications, changes, and permutations may be made to the disclosed implementations, and these modifications, changes, and permutations will be apparent to those of ordinary skill in the art, and the general principles defined may be applied to other implementations and applications without departing from the scope of the present disclosure. In some instances, one or more technical details that are not necessary to gain an understanding of the described subject matter and are within the skill of those of ordinary skill in the art may be omitted so as not to obscure one or more described implementations. The present disclosure is not intended to be limited to the implementations described or shown, but is consistent with the widest scope consistent with the described principles and features.

[0015] Microservices architecture is a software development technique based on a variation of the service-oriented architecture (SOA) architectural style that builds applications as a collection of loosely coupled services. In a microservices architecture, services are fine-grained and protocols are lightweight. A service (or microservices) grid (or "grid") is a paradigm that provides integrated services for containers and microservices-based applications directly from within a computing cluster. A service grid provides monitoring, scalability, and high availability services through an application programming interface (API) without using discrete computing devices.

[0016] An application in a microservices architecture (e.g., in a large-scale cloud computing system deployment) will consume a set of services, which in turn may consume another set of services (i.e., the services form a "service mesh"). Applications and services can be scaled out by adding more instances (and scaled in by removing instances). A load balancer typically distributes calls between application and service instances, so that instances of each service typically have similar operating system usage.

[0017] The service mesh can be monitored using a monitoring infrastructure that will periodically read operating system (OS) parameters (e.g., CPU load, memory usage, network input / output, disk latency, and block paging out). As usage load changes, the number of instances of a service in the service mesh can be adjusted by the scaling component based on the information provided by the monitoring infrastructure. In this case: 1) if the usage load increases, instances need to be added, otherwise response times increase and service requests may be delayed or even rejected, which results in a poor user experience, or 2) if the usage load decreases, instances should be removed, otherwise operational resources are wasted and overall resource consumption is higher than required, resulting in increased operational costs. These two requirements are in tension: 1) ensure that there are enough instances to ensure efficient operation and user satisfaction, but 2) only as many instances as possible can be operated at the lowest possible cost. The question is how to operate the service mesh using as few hardware resources as possible and still maintain fast response times and high user satisfaction. This is because the fewer the number of instances for a typical usage load (the lower the operating cost), the faster autoscaling is required to react to scaling out to meet higher demand. Similarly, the faster autoscaling can identify lower usage and scale-down, the lower the resource usage and operating costs.

[0018] An automatic scaling method for a microservices architecture for large cloud computing deployments is described. For example, in a single-application client-server architecture, if more requests need to be handled, scaling with additional instances immediately results in higher throughput. Scaling is typically triggered by a higher load on a set of instances of a service type, and then more instances of that service type are started. However, in a microservices architecture, there may be a time lag between the initial scaling out of application instances and the actual higher throughput because not only the application instances need to be scaled out, but also instances of services used by the application and other services used by the services used by the application.

[0019] Figure 11 is a block diagram 100 of an implementation according to the present disclosure, which shows an example application (App) 102 with three services (services 104, 106, and 108) in a microservice architecture, application / service calls (calls 109 / 110, 112, and 114, respectively), and an OS monitoring infrastructure 116. For example, once more application instances (App 102) are instantiated by call 109, the additional application instances send more requests to the services (e.g., service A 104 and service B 106), which increases the overall load of the services. The OS monitoring infrastructure 116 detects this situation at a defined time interval (e.g., once every minute). At the next time point for measuring OS parameters, the scaler component triggers scaling of the service (here, for example, service A 104) that is detected to be under high load. Once the service is scaled, the service (service A 108) sends the service request (i.e., call 112) to the required service at the next level (here, for example, service C 108, because service B 106 does not need to call additional services). The OS monitoring infrastructure 116 detects again at defined time intervals that a service request is sent to the next level of required services. Likewise, at the next time point for measuring OS parameters, the scaler component will trigger scaling of the next level service (Service C 108) that is now detected to be under high load.

[0020] Go to Figure 2 , Figure 2 is a diagram showing an implementation method according to the present disclosure Figure 1 2 . A block diagram 200 of the number of instances of each service at different points in time in a microservices architecture of FIG. 2 . For example, at time T2 202, the number of instances 204 of App 102 increases to 4, but it is not until time T2' 206 that the number of instances 208 of Service A 104 increases to 4 and the number of instances 210 of Service B 106 increases to 7. In some implementations, with a constant defined time interval (e.g., once per minute), the iterative adjustments 212 each take 1 minute, and in order to reach the desired scale of App 210, Service A 104, Service B 106, and Service C 108 scaling shown at T3 214 (4, 4, 7, and 6, respectively), up to five minutes of scaler iterations will be required. As a result, the associated service mesh reacts slowly to changes in load, and users may experience poor response times for relatively long periods of time. Likewise, once the load decreases, resources remain over-provisioned for unnecessary times, thereby incurring avoidable costs. In contrast, the improved autoscaling method detects the change at T1 and computes the final required scaling without intermediate iterations—in effect, jumping from T1 to T3.

[0021] Figure 3FIG. 1 is a diagram showing a conventional scaler and OS monitoring infrastructure (OS monitor 116) according to an implementation of the present disclosure relative to FIG. Figure 1 and 2 A block diagram 300 of the use of a microservice architecture. For example, Figure 3 As shown, when calls (e.g., calls 109, 110, 112, and 114) are made to instantiate an instance of App 102, Service A 104, Service B 106, or Service C 108, the scaler 302 is configured to work with the OS monitor 116 and one or more monitoring rules (e.g., monitoring rule 304 or 306) to scale App 102, Service A 104, Service B 106, and Service C 108.

[0022] Because regular scaling out in a mesh has latency, and you need to ensure fast response times, you typically have two options:

[0023] a) over-provisioning to ensure fast response times, or

[0024] b) Tighten supply to ensure low costs.

[0025] The improved monitoring and scaling infrastructure described herein both reduces costs (relative to over-provisioning) and improves response time (relative to under-provisioning). Thus, the improved monitoring and scaling infrastructure does not impose additional burden on the system.

[0026] To address the inefficiencies with respect to time and CPU load / memory usage, at a high level, two improvements to the previously described autoscaling process are proposed. First, the scaler can be configured to: 1) compute the change in the number of instances as a function of the current number of instances, rather than just as a percentage or fixed number (single service, quantity-aware scaler), and 2) be configured to compute the change in the number of instances as a function of the change in the load parameter, rather than just the load parameter value (extended single service, single service quantity-aware scaler, derivative change-aware scaler). Second, the scaler can be configured to: 1) monitor the application and scale all related services (mesh scaler), and 2) monitor requests to an application and the required mesh to scale related services based on request type by computing the change in the number of instances of the service based on the request type (request-aware mesh scaler).

[0027] At a high level, the improved autoscaling functionality can provide the following advantages / features over the regular scaler functionality:

[0028] Single-server scaler

[0029] 1. Single service, automatic service scaler

[0030] ●Call the monitoring infrastructure every x seconds for new values ​​for all containers, identify groups of containers running the same service or application, and for each group of containers running the same service or application, trigger the creation / destruction of application / service instances based on the calculations performed on the monitoring parameters.

[0031] 2. Single-service, volume-aware scaler

[0032] ●If there are non-linear effects when scaling the service, a quantity-aware scaler can be used to adjust for the non-linear effects.

[0033] 3. Single-serving, derivative-change-aware scaler:

[0034] ● If the number of instances of a service varies up and down and "oscillates" around a certain value, this scaler can be used to dampen the oscillation effect, and

[0035] ● If the load changes rapidly and at a large scale, accelerated scaling is required. If large changes are detected, the scaler can detect the rapid changes and trigger accelerated scaling.

[0036] Grid zoomer

[0037] 1. Grid Zoomer

[0038] ● For “deep” call stacks of services, scaling lag can become increasingly problematic. This scaler can be used to eliminate lag.

[0039] 2. Request-aware grid scaler:

[0040] ●If call requests trigger very different actions and call different related services, the scaler can be used to react specifically to different request types.

[0041] =Identify the call distribution and whether the call distribution is statistically distributed across all usage types or systematically favors one or other usage type.

[0042] ● Large numbers of users are usually represented statistically.

[0043] ●Users triggered by events (e.g., Monday morning, Black Friday, an email campaign, or a Twitter post) behave systematically similarly.

[0044] • If the identified distributions are systematically similar (rather than random), use this scaler because it can be parameterized to reflect systematic differences.

[0045] In more detail, with respect to a single service scaler, for the automatic service scaler, "container" is used as a synonym for host, VM, container, depending on the infrastructure configuration. "Load" is defined as the relative consumption of available resource capacity (e.g., CPU or memory). In some implementations, the load of a service instance is considered to be between 0 and 1.

[0046] For the purposes of this disclosure, “Autoscaler Infrastructure” (ASI) is defined as:

[0047] ●ASI can read monitoring parameters from the monitoring infrastructure of the container that runs the instance of the service.

[0048] ○ Aggregate the deployment width (e.g., datacenter (DC) and tenants in the infrastructure) of the containers in which the service is running system load.

[0049] ○ For each instance, collect the load.

[0050] ○ Sum the load of all available instances and calculate the average load.

[0051] ●ASI centralized storage:

[0052] ○ A list of services and the number of instances of each service.

[0053] ○ Mapping of service instances to containers (allows mapping of monitoring parameters to service instances).

[0054] ○ For each service, monitoring rules depend on the monitoring parameters.

[0055] ASI trigger:

[0056] o If the monitoring rules of the service applied to the measured monitoring parameters return a scale-out event, then create (n) instances of some service.

[0057] o If the monitoring rule of the service applied to the measured monitoring parameter returns a scale-in event, then stop the instance(s) of a certain service.

[0058] In some implementations, the ASI process includes:

[0059] • ASI calls the monitoring infrastructure every x seconds to get new values ​​for all containers.

[0060] ●ASI identifies a collection of containers running the same service or application.

[0061] For each collection of containers running the same service or application, ASI:

[0062] ○ Calculate the average value (or median, or maximum value or other function) of the monitored parameters.

[0063] o Calculate monitoring rule values, depending on monitoring parameters (previous step).

[0064] ○ Trigger the creation / destruction of instances based on the results of the previous step.

[0065] ●ASI reads the monitoring parameters again according to specific intervals.

[0066] In some implementations, the ASI is a function of c, where c = the load of a resource (eg, CPU or memory) used by the instance, where ASI = f(c).

[0067] Number-aware scaler

[0068] Number-aware scaling is an extension of the automatic service scaler. The function that calculates the number of new instances also depends on the number of examples already running (i s = # of instances of the service), where ASI Q =f(c,i s ). Using this additional parameter, other effects can be taken into account, such as:

[0069] ● A minimum number of instances is required to guarantee high availability SLA,

[0070] • Scaling for a small number of instances follows different rules than for a very large number of instances.

[0071] In some implementations, the number-aware scaler can account for nonlinear effects in scaling. Nonlinear effects can be determined by measuring service scaling. The cause may be an architecture that cannot scale linearly (e.g., requiring twice as many instances to achieve twice the load).

[0072] Most importantly, the population-aware scaler is the first iteration to address scaling latency: if a large number of instances reach a load threshold, then adding just one additional instance will have little effect on reducing the overall load on the service. As scaling increases, one instance becomes laggy, and for services with many instances, scaling one by one can be too slow. Therefore, the population-aware scaler increases the number of instances of a service in larger increments when there are already more instances running that are experiencing higher load.

[0073] Derivative Change-Aware Scaler

[0074] A scaler, which takes into account not only the load but also the speed at which the load changes (i.e. the derivative of the load with respect to time), can be used to accelerate or dampen scaling depending on the speed of change:

[0075] ASI D =f(c,c'), where c'=dc / dt

[0076] The number of new instances is calculated not only using the OS monitoring infrastructure parameter values, but also using the derivatives of the monitoring parameters (i.e., their rates of change). For example, a rapid increase in CPU load indicates a rapid increase in usage and requires faster scaling out. Scaling out can be configured using a derivative change aware scaler.

[0077] In some implementations, the CPU load used as the parameter evaluated by monitoring is typically the load average (meaning it is a moving average) of the last x seconds (or minutes). As a result, changes in CPU load are measured in a "moving average CPU load" with latency. Using CPU load without averaging triggers scaling events for short peak usage changes. To compensate for the latency due to the moving average, a derivative change aware scaler can be used to speed up changes during the initial load increase.

[0078] By being parameterized differently, the derivative change-aware scaler can suppress changes. A collection of instances has a certain "inertia": instances take time to start up, so there is a delay between a scaling action and the resulting higher capacity. Instances may also only be running effectively after some time (e.g. to fill a cache or use a service that has scaled up). If during this period - because the capacity is still insufficient - the load remains above the threshold that triggers scaling out, the derivative change-aware scaler will start more instances and the service mesh will (after some time) have too much capacity. The derivative change-aware scaler will then stop the instances, but this action may also be delayed because instances will often shut down "gracefully" to allow running actions to complete (that is, not all executed actions will be terminated after a few milliseconds). As a result, scaling inwards acts with "inertia".

[0079] Additional instances also typically add capacity for more than just "one more call", but rather the instance can handle multiple calls in parallel (depending on the service type). In these cases, adding an instance provides capacity overhead, and the CPU load remains low afterwards.

[0080] The depth of the service mesh (i.e., the call stack of services that call services) can also add "inertia" to the system. The deeper the mesh, the longer it takes to react to changing request volumes.

[0081] For a single service scaler, scaling is:

[0082] ASI DQ =f(c,c',i s ),

[0083] ● The target number of instances required to deliver (I s ),

[0084] The target number is provided to the microservice infrastructure to adjust the number of running instances to the target number, and

[0085] in:

[0086] f(c,c',i s )=k1*f1(c,i s )+k2*f2(c',i s ),and

[0087] For k2<0, the change is "restricted", for k2>0, it is accelerated.

[0088] Grid Scaler

[0089] For the purposes of this disclosure, a "Grid Scaler Infrastructure" (MSI) is defined as: (where modifications compared to ASI are indicated by '*'):

[0090] - The MSI can read monitoring parameters from the monitoring infrastructure of the container running the instance (* of the application).

[0091] MSI centralized storage:

[0092] ○ A list of services and the number of instances of each service.

[0093] ○ Mapping of service instances to containers.

[0094] ○* A list of applications and the number of instances of each application.

[0095] ○*For each application and service, a list of other services used.

[0096] ○* For each application, the monitoring rules of the application and all directly or indirectly used services (ie, services called by called services) that depend on the monitoring parameters.

[0097] ■*Monitoring rules are defined by application developers.

[0098] ■* Monitoring rules are based on load testing, which provides a static picture of the relationship between the application and multiple instances of each type of service.

[0099] MSI trigger

[0100] ○*If the monitoring rules of the application applied to the measured monitoring parameters return scale-out / scale-in events, creation / stop of (n) instances of the application, including creation / stop of (n) instances of each called service (and services called by the services).

[0101] Therefore, in some implementations:

[0102] MSI = f m (c a ), where c a is the load of the application.

[0103] ●MSI delivery vector: V = (I1, I2...I n ), where I i is the number of instances required for each service type.

[0104] * or *

[0105] ISI delivery vector: V = (D1, D2...D n ), where D i is a demand for service type i. The demand is passed to the single service scaler ASI i .

[0106] ○ Demand is a relative measure (e.g., a service mesh is measured and sized for some request frequency, such as 100 / s), so an application has a demand of 100. Depending on the services used, 100 incoming requests are triggering a request to service s i Request Ri.

[0107] ■For example, 100 calls to application, 80 calls to s1, and 200 calls to 2.

[0108] ■Therefore: D1=80, D2=200.

[0109] ■ The scaler for s1 is initially sized to handle 80 requests using I1 (e.g. 4) instances.

[0110] ■If the demand reaches 160, the scaler will calculate, for example, 8 instances.

[0111] ○ Therefore, the grid scaler uses the collection s of services it calls i and the service weight w i to configure.

[0112] ○ The grid scaler is initially sized to handle x requests / second. The actual requests are y requests / second. Therefore, the demand on the application D a Yes: y / x = instance request / initial size.

[0113] ○ About Services i The need is: D i =w i *D a .

[0114] In some implementations, the definition of an application and the services used by the application are defined in a "multi-target application" definition - a concept describing an application consisting of a collection of services, where:

[0115] ●A j =V(s1,s2..s n ).

[0116] Therefore, in some implementations:

[0117] MSI(a) = f m (c a ),in:

[0118] ○ a is the application.

[0119] ○ c a is the load of the application.

[0120] ●MSI output V=(D1,D2…D n )

[0121] ●ASI(s)=f(c s , c s ',i s , D s ),in:

[0122] ○ s is service,

[0123] ○ c s is the load of the service,

[0124] ○ c s ' is the derivative of the service with respect to time,

[0125] ○ i s is the number of instances of the service, and

[0126] ○ D s This is an MSI-specific requirement.

[0127] ●ASI(s) output I s , the number of instances required for the service.

[0128] Request-aware grid scaler

[0129] The request-aware grid scaler limits the usage of related services based on the application's request type. For example:

[0130] ●The app has "Create Account / Address", "Create Order", "Print Return Form" app service orders.

[0131] ●If the user calls "Create Account", the Address Service will be called, but the PDF Generator will not be called.

[0132] ●If the user calls "Print Return Form", the PDF generator is called.

[0133] ●Thus, knowing the request made to the application (e.g., create an account vs. print a return form) allows scaling the actual service called rather than all services (e.g., PDF generator service).

[0134] In some implementations, a "Request-Aware Mesh Scaler Infrastructure" (RMSI) is defined as: (modifications compared to MSI are indicated by '*')

[0135] ●RMSI can read monitoring parameters from the monitoring infrastructure of the container that runs the instance of the service.

[0136] ●RMSI centralized storage:

[0137] ○ A list of services and the number of instances of each service.

[0138] ○ Mapping of service instances to containers.

[0139] ○ A list of applications and the number of instances of each application.

[0140] ○ For each application and service, a list of other services called, (*a list of usage types to which each called service belongs).

[0141] o For each application, based on the monitoring parameters s (* and usage type), the monitoring rules of the application and all called services (and services called by called services, and so on).

[0142] ■*Monitoring rules are initially defined by application developers.

[0143] ■It is calculated based on the monitoring rules of the service type.

[0144] ■It can be replaced by a trained model (machine learning).

[0145] *RMSI can:

[0146] ○ Read monitoring parameters from incoming requests,

[0147] ○Determine the dosage type

[0148] o Store the number of calls for each usage type over a period of time (e.g., last 5 minutes).

[0149] *RMSI can:

[0150] ○ Calculate weighted scaling events,

[0151] ○ Scaling rules are applied to each usage type and the result is multiplied by the number of stored calls.

[0152] ○ Divide the result by the total number of stored calls.

[0153] ●RMSI:

[0154] ○ If the monitoring rules of the application applied to the measured monitoring parameters return a scale-out / scale-in event, trigger the creation / stop of (n) instances of the application, including the creation / stop of (n) instances of each called service (and services called by the service).

[0155] In some implementations, the RMSI process includes:

[0156] • RMSI calls the monitoring infrastructure every x seconds to get new values ​​for all containers.

[0157] =RMSI calls the monitoring infrastructure to obtain a list of new calls and calculate their usage type.

[0158] ●RMSI identifies a collection of containers running the same application.

[0159] ●RMSI for a collection of containers running the same application:

[0160] ○ Calculate the average value (or median, or maximum value or other function) of the monitored parameters.

[0161] o Calculate monitoring rule value, depending on monitoring parameters (previous step), usage type information and weight the scaling according to usage type weight.

[0162] ○ Trigger creation / destruction of instances of the application and called services based on the results of the previous step.

[0163] ●RMSI reads the monitoring parameters again.

[0164] Therefore, in some implementations:

[0165] MSI(a) = f m (c a , R a ),in:

[0166] ○a is an application.

[0167] ○c a is the load of the application.

[0168] ○R ais a vector (R1, R2, ... R m ), which has the number of request types per second R j .

[0169] ○For each request type R j , can calculate / define requirements :D ji

[0170] ■D 1i =(D 11 , …D 1n ), demand for service of request type 1.

[0171] ■D 2i =(D 21 , …D 2n ), demand for service of request type 2.

[0172] ■…

[0173] ■D i1 =(D 11 , …D n1 ) is the demand for service I of different request types.

[0174] The demand for a service is then passed to the ASI: j (D ij ).

[0175] In some implementations, the grid scaler is used in conjunction with an automatic service scaler. MSI will scale the entire grid with a single trigger action, but the load on individual services may depend on parameters that are not reflected in the grid scaling rules actually used. Therefore, ASI is used to fine-tune the capacity of individual services. In some cases, the orchestration of MSI and ASI can be implemented as a closed-loop control circuit.

[0176] Table 1 shows an example of the calculations performed by the improved method:

[0177] CPU Load 50％ 60％ 70％ 80％ 90％ MSI: +20％ +30％ +0％ -10％ -20％ ＝： 60％ 78％ 70％ 72％ 72％

[0178] Table 1

[0179] CPU load multiplied by MSI prediction:

[0180] ●If Load > Threshold → Scale,

[0181] ●If load ~ threshold → stay, or

[0182] ●If load < threshold → reduce.

[0183] In the example, the grid scaler performs overall (coarse-grained / low-granularity) grid width scaling, while the single service scaler is used to fine-tune the scaling of each individual service to improve the grid scaler scaling. As a specific example, one possible way to combine the two types of scalers is as follows:

[0184] ● The load change predicted by the grid scaler is multiplied by the actual load measured by the single service scaler to determine the target load used to determine scaling needs. For example, if the grid scaler predicts a +20% increase in load for a specific service in the grid, and the single service scaler for the service measures a current load of 50%, then the target load is calculated as 50%*120%=60% (120, because you started at 100 and increased by 20, so it is 120).

[0185] ●The target value of 60% can now be compared to the defined threshold to determine if any scaling is required. In the example case where the threshold is 70%, no scaling is required using the described calculation. However, if the calculation is not performed as described, and the grid scaler prediction is simply added, then the threshold will be reached (i.e., 50% + 20% = 70%) - triggering scaling up. A more complex calculation is recommended because the grid scaler measurements may be too coarse compared to the actual measured load. Therefore, the recommended calculation compensates for the "ignorance" of the grid scaler by taking the current load (i.e., 50%) into account.

[0186] ● The other quantities are similar: 60% load measured by a single service scaler and a +30% increase predicted by the grid scaler means 60%*130%=78%. Or the last quantity with a negative prediction: 90% measured and -20% predicted means 90%*80%=72% (80,

[0187] Because 100-200).

[0188] Figure 4 is a flow chart illustrating an example of a computer-implemented method 400 for amplifying scaling elasticity of a microservices mesh according to an implementation of the present disclosure. For clarity of presentation, the following description generally describes the method 400 in the context of other figures in this specification. However, it should be understood that the method 400 can be performed, for example, by any system, environment, software, and hardware that is suitable, or a combination of systems, environments, software, and hardware. In some implementations, the various steps of the method 400 can be run in parallel, combined, looped, or in any order.

[0189] At 402, a mesh scaler is selected for performing overall mesh width scaling of a microservice mesh. In some implementations, the mesh scaler includes a mesh scaler and a request-aware mesh scaler. From 402, method 400 proceeds to 404.

[0190] At 404, the microservices mesh is scaled using a mesh scaler and one or more mesh scaling rules to generate a predicted value of a monitoring parameter in one or more monitoring parameters. In some implementations, scaling the microservices mesh using the mesh scaler is performed in a single trigger action. From 404, method 400 proceeds to 406.

[0191] At 406, a single service scaler is selected. In some implementations, the single service scaler includes an automatic service scaler, a quantity-aware scaler, and a derivative change-aware scaler. In some implementations, the grid scaler passes the predicted value to the single service scaler. From 406, method 400 proceeds to 408.

[0192] At 408 , the monitoring parameter is measured using the single service scaler to generate a measurement value. From 408 , method 400 proceeds to 410 .

[0193] At 410 , for individual services associated with the microservices mesh, target values ​​for monitoring parameters are calculated using the predicted values ​​and the measured values. From 410 , method 400 proceeds to 412 .

[0194] At 412, the target value is compared to a defined threshold to generate a scaling value for determining whether scaling is required for the individual service. In some implementations, the single-service scaler: reads monitoring parameters from a monitoring infrastructure of a container executing an instance of the service; aggregates the system load of the container in which the instance of the service is executed; collects the load of each service instance; and calculates the average load by summing the load of all available service instances. In some implementations, the single-service scaler triggers the creation or destruction of an instance of the individual service based on the scaling value. After 412, method 400 can stop.

[0195] Figure 5 500 is a block diagram illustrating an example of a computer-implemented system 500 for providing computing functions associated with the described algorithms, methods, functions, processes, procedures, and programs according to an implementation of the present disclosure. In the illustrated implementation, the system 500 includes a computer 502 and a network 530.

[0196] The computer 502 shown is intended to include any computing device, such as a server, a desktop computer, a laptop / notebook computer, a wireless data port, a smart phone, a personal digital assistant (PDA), a tablet computer, one or more processors within these devices, or a combination of computing devices, including physical or virtual instances of computing devices, or a combination of physical or virtual instances of computing devices. In addition, the computer 502 may include an input device, such as a keypad, a keyboard, or a touch screen, or a combination of input devices that can accept user information, and an output device that communicates information related to the operation of the computer 502 on a graphical user interface (UI) (or GUI) or other UI, including digital data, visuals, audio, another type of information, or a combination of information types.

[0197] The computer 502 can function in a distributed computing system, for example, as a client, a network component, a server or a database or another persistence, or a combination of roles for performing the subject matter described in the present disclosure. The computer 502 shown is communicatively coupled to a network 530. In some implementations, one or more components of the computer 502 can be configured to operate within an environment or combination of environments, including cloud computing, local, or global.

[0198] At a high level, the computer 502 is an electronic computing device that is operable to receive, transmit, process, store, or manage data and information associated with the described subject matter. According to some implementations, the computer 502 may also include or be communicatively coupled to a server, such as an application server, an email server, a web server, a cache server, or a streaming data server, or a combination of servers.

[0199] Computer 502 may receive requests (e.g., from a client software application executing on another computer 502) over network 530 and respond to the received requests by processing the received requests using the software application or a combination of software applications. Requests may also be sent to computer 502 from an internal user (e.g., from a command console or through another internal access method), an external or third party, or other entity, person, system, or computer.

[0200] Each component of the computer 502 can communicate using the system bus 503. In some implementations, any or all components of the computer 502, including hardware, software, or a combination of hardware and software, can be interfaced over the system bus 503 using an application programming interface (API) 512, a service layer 513, or a combination of an API 512 and a service layer 513. The API 512 can include specifications for routines, data structures, and object classes. The API 512 can be independent of or dependent on a computer language and refer to a complete interface, a single function, or even a set of APIs. The service layer 513 provides software services to the computer 502 or other components (whether or not shown) that are communicatively coupled to the computer 502. Using the service layer 513, all service consumers can access the functionality of the computer 502. Software services, such as those provided by the service layer 513, provide reusable, defined functionality through a defined interface. For example, the interface can be software written in a computing language (e.g., JAVA or C++) or a combination of computing languages, and provides data in a specific format (e.g., Extensible Markup Language (XML)) or a combination of formats. Although shown as an integrated component of computer 502, alternative implementations may show API 512 or service layer 513 as a separate component associated with other components of computer 502 or communicatively coupled to other components of computer 502 (whether shown or not). In addition, any or all portions of API 512 or service layer 513 may be implemented as a sub-module of another software module, enterprise application, or hardware module without departing from the scope of the present disclosure.

[0201] The computer 502 includes an interface 504. Although shown as a single interface 504, two or more interfaces 504 may be used according to the specific needs, expectations, or specific implementations of the computer 502. The computer 502 uses the interface 504 to communicate with another computing system (whether or not shown) that is communicatively linked to the network 530 in a distributed environment. Generally, the interface 504 is operable to communicate with the network 530 and includes logic encoded in software, hardware, or a combination of software and hardware. More specifically, the interface 504 may include software that supports one or more communication protocols associated with the communication, so that the network 530 or the hardware of the interface 504 is operable to communicate physical signals within and outside the computer 502 shown.

[0202] The computer 502 includes a processor 505. Although shown as a single processor 505, two or more processors 505 may be used depending on the particular needs, desires, or particular implementations of the computer 502. In general, the processor 505 executes instructions and manipulates data to perform the operations of the computer 502 and any algorithms, methods, functions, processes, procedures, and programs described in this disclosure.

[0203] The computer 502 also includes a database 506, which can store data of the computer 502, another component (whether or not shown) that is communicatively linked to the network 530, or a combination of the computer 502 and another component. For example, the database 506 can be an in-memory or conventional database that stores data consistent with the present disclosure. In some implementations, the database 506 can be a combination of two or more different database types (e.g., a hybrid in-memory and conventional database) according to the specific needs, expectations, or specific implementations of the computer 502 and the functions described. Although shown as a single database 506, two or more similar or different types of databases can be used according to the specific needs, expectations, or specific implementations of the computer 502 and the functions described. Although the database 506 is shown as a component of the computer 502, in alternative implementations, the database 506 can be external to the computer 502.

[0204] The computer 502 also includes a memory 507, which can store data of the computer 502, another component or components (whether or not shown) that are communicatively linked to the network 530, or a combination of the computer 502 and another component. The memory 507 can store any data consistent with the present disclosure. In some implementations, the memory 507 can be a combination of two or more different types of memory (e.g., a combination of semiconductor and magnetic storage) depending on the specific needs, expectations, or specific implementation of the computer 502 and the functions described. Although shown as a single memory 507, two or more similar or different types of memory 507 can be used depending on the specific needs, expectations, or specific implementation of the computer 502 and the functions described. Although the memory 507 is shown as an integral part of the computer 502, in alternative implementations, the memory 507 can be external to the computer 502.

[0205] Application 508 is an algorithmic software engine that provides functionality according to the specific needs, expectations, or specific implementation of computer 502, particularly with respect to the functionality described in the present disclosure. For example, application 508 can be implemented as one or more components, modules, or applications. In addition, although shown as a single application 508, application 508 can be implemented as multiple applications 508 on computer 502. In addition, although shown as integrated with computer 502, in alternative implementations, application 508 can be external to computer 502.

[0206] The computer 502 may also include a power supply 514. The power supply 514 may include a rechargeable or non-rechargeable battery, which may be configured to be user-replaceable or non-user-replaceable. In some implementations, the power supply 514 may include power conversion or management circuitry (including recharging, standby, or other power management functions). In some implementations, the power supply 514 may include a power plug to allow the computer 502 to be plugged into a wall outlet or another power source, for example, to power the computer 502 or to charge a rechargeable battery.

[0207] There may be any number of computers 502 associated with or external to the computer system containing the computer 502, each of which communicates via the network 530. Furthermore, the terms "client," "user," or other appropriate terms may be used interchangeably as appropriate without departing from the scope of the present disclosure. Furthermore, the present disclosure contemplates that multiple users may use one computer 502, or that one user may use multiple computers 502.

[0208] Implementations of the described subject matter may include one or more features alone or in combination.

[0209] For example, in a first implementation, a computer-implemented method includes: selecting a grid-type scaler to perform overall grid width scaling of a microservice grid; scaling the microservice grid using the grid-type scaler and one or more grid scaling rules to generate predicted values ​​of monitoring parameters in one or more monitoring parameters; selecting a single service-type scaler; measuring the monitoring parameters using the single service-type scaler to generate measured values; for individual services associated with the microservice grid, calculating target values ​​of the monitoring parameters using the predicted values ​​and the measured values; comparing the target values ​​with a defined threshold to generate a scaling value for determining whether scaling is required for the individual services.

[0210] The foregoing and other described implementations may each optionally include one or more of the following features:

[0211] The first feature can be combined with any of the following features, wherein the grid type scaler includes a grid scaler and a request-aware grid scaler.

[0212] A second feature, which may be combined with any previous or following feature, wherein scaling of the microservice mesh using the mesh scaler is performed in a single trigger action.

[0213] A third feature, which may be combined with any previous or following feature, wherein the single service type scaler includes an automatic service scaler, a quantity-aware scaler, and a derivative change-aware scaler.

[0214] A fourth feature, which may be combined with any of the previous or following features, wherein the grid-type scaler passes the prediction value to a single serving scaler.

[0215] A fifth feature, which may be combined with any previous or following feature, wherein the single service scaler: reads monitoring parameters from a monitoring infrastructure of containers executing instances of a service; aggregates system loads of the containers in which instances of the service are executed; collects the load of each service instance; and calculates an average load by summing the loads of all available service instances.

[0216] A sixth feature, which may be combined with any previous or following feature, is wherein a single service scaler triggers creation or destruction of instances of separate services based on a scaling value.

[0217] In a second implementation, a non-transitory computer-readable medium stores one or more instructions executable by a computer system to perform the following operations, including: selecting a grid-type scaler to perform overall grid width scaling of a microservice grid; scaling the microservice grid using the grid-type scaler and one or more grid scaling rules to generate predicted values ​​of monitoring parameters in one or more monitoring parameters; selecting a single service-type scaler; measuring the monitoring parameters using the single service-type scaler to generate measured values; for individual services associated with the microservice grid, calculating target values ​​of the monitoring parameters using the predicted values ​​and the measured values; comparing the target values ​​with a defined threshold to generate a scaling value for determining whether scaling is required for the individual services.

[0218] The foregoing and other described implementations may each optionally include one or more of the following features:

[0219] The first feature can be combined with any of the following features, wherein the grid type scaler includes a grid scaler and a request-aware grid scaler.

[0220] A second feature, which may be combined with any previous or following feature, wherein scaling of the microservice mesh using the mesh scaler is performed in a single trigger action.

[0221] A third feature, which may be combined with any previous or following feature, wherein the single service type scaler includes an automatic service scaler, a quantity-aware scaler, and a derivative change-aware scaler.

[0222] A fourth feature, which may be combined with any of the previous or following features, wherein the grid-type scaler passes the prediction value to a single serving scaler.

[0223] A fifth feature, which may be combined with any previous or following feature, wherein the single service scaler: reads monitoring parameters from a monitoring infrastructure of containers executing instances of a service; aggregates system loads of the containers in which instances of the service are executed; collects the load of each service instance; and calculates an average load by summing the loads of all available service instances.

[0224] A sixth feature, which may be combined with any previous or following feature, is wherein a single service scaler triggers creation or destruction of instances of separate services based on a scaling value.

[0225] In a third implementation, a computer-implemented system includes: one or more computers; one or more computer memory devices, which are cooperatively coupled to the one or more computers and have a tangible, non-transitory machine-readable medium storing one or more instructions. When executed by the one or more computers, the one or more instructions perform one or more operations, including: selecting a grid-type scaler to perform overall grid width scaling of a microservice grid; scaling the microservice grid using the grid-type scaler and one or more grid scaling rules to generate predicted values ​​of monitoring parameters in one or more monitoring parameters; selecting a single service-type scaler; measuring the monitoring parameters using the single service-type scaler to generate measured values; for individual services associated with the microservice grid, calculating target values ​​of the monitoring parameters using the predicted values ​​and the measured values; comparing the target values ​​with a defined threshold to generate a scaling value for determining whether scaling is required for the individual services.

[0226] The foregoing and other described implementations may each optionally include one or more of the following features:

[0227] The first feature can be combined with any of the following features, wherein the grid type scaler includes a grid scaler and a request-aware grid scaler.

[0228] A second feature, which may be combined with any previous or following feature, wherein scaling of the microservice mesh using the mesh scaler is performed in a single trigger action.

[0229] A third feature, which may be combined with any previous or following feature, wherein the single service type scaler includes an automatic service scaler, a quantity-aware scaler, and a derivative change-aware scaler.

[0230] A fourth feature, which may be combined with any of the previous or following features, wherein the grid-type scaler passes the prediction value to a single serving scaler.

[0231] A fifth feature, which may be combined with any previous or following feature, wherein the single service scaler: reads monitoring parameters from a monitoring infrastructure of containers executing instances of a service; aggregates system loads of the containers in which instances of the service are executed; collects the load of each service instance; and calculates an average load by summing the loads of all available service instances.

[0232] A sixth feature, which may be combined with any previous or following feature, is wherein a single service scaler triggers creation or destruction of instances of separate services based on a scaling value.

[0233] The implementation of the subject matter and functional operations described in this specification can be implemented in digital electronic circuits, in tangibly implemented computer software or firmware, in computer hardware including the structures disclosed in this specification and their structural equivalents, or in a combination of one or more of them. The software implementation of the described subject matter can be implemented as one or more computer programs, that is, one or more computer program instruction modules encoded on a tangible, non-temporary computer-readable medium, for the operation of the computer or the operation of the computer-implemented system. Alternatively or additionally, the program instructions can be encoded in / on artificially generated propagation signals, for example, machine-generated electrical, optical or electromagnetic signals, which are generated to encode information for transmission to a receiver device for execution. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or one or more combinations of computer storage media. Configuring one or more computers means that one or more computers have installed hardware, firmware or software (or a combination of hardware, firmware and software) so that when one or more computers execute the software, specific computing operations are performed.

[0234] The terms "real time", "(rapid) real time (RFT)", "near (real time) (NRT)", "quasi real time" or similar terms (as understood by one of ordinary skill in the art) mean that the action and response are close in time such that an individual perceives the action and response essentially simultaneously. For example, the time difference in response to the display of data (or the initiation of display) following the individual's action to access the data may be less than 1 millisecond (ms), less than 1 second (s), or less than 5 seconds. Although the requested data need not be displayed (or initiated for display) immediately, it is displayed (or initiated for display) without any intentional delay, given the processing limitations of the described computing systems and, for example, the time required to collect, accurately measure, analyze, process, store or transmit the data.

[0235] The term "data processing equipment", "computer" or "electronic computer device" (or equivalent terms understood by a person of ordinary skill in the art) refers to data processing hardware and includes all types of devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. A computer may also be or also include a dedicated logic circuit, such as a central processing unit (CPU), a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). In some implementations, a computer or a computer-implemented system or a dedicated logic circuit (or a computer, or a combination of a computer-implemented system and a dedicated logic circuit) may be hardware-based or software-based (or a combination of hardware and software). A computer may optionally include code that creates an execution environment for a computer program, for example, code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or a combination of execution environments. The present disclosure contemplates the use of a computer or a computer-implemented system with an operating system, such as LINUX, UNIX, WINDOWS, MAC OS, ANDROID or IOS, or a combination of operating systems.

[0236] A computer program, which may also be referred to or described as a program, software, software application, unit, module, software module, script or code, or other component, may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it may be deployed in any form, including, for example, as a standalone program, module, component, or subroutine, for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store portions of one or more modules, subroutines, or codes). A computer program may be deployed to run on one computer or on multiple computers that are located at one site, or distributed across multiple sites and interconnected by a communication network.

[0237] Although portions of the programs shown in the various figures may be shown as separate components, such as units or modules, which use various objects, methods, or other processes to implement the described features and functions, the programs may appropriately include many sub-units, sub-modules, third-party services, components, libraries, and other components. Conversely, the features and functions of the various components may be appropriately combined into a single component. The thresholds used to make computational determinations may be static, dynamic, or both statically and dynamically determined.

[0238] The described methods, processes, or logic flows represent one or more examples of functionality consistent with the present disclosure, and are not intended to limit the present disclosure to the implementations described or shown, but rather to conform to the widest scope consistent with the described principles and features. The described methods, processes, or logic flows may be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output data. The methods, processes, or logic flows may also be performed by special purpose logic circuits, and computers may also be implemented as special purpose logic circuits, such as CPUs, FPGAs, or ASICs.

[0239] The computer for executing a computer program can be based on a general or special microprocessor, or two types of CPU. Usually, the CPU will receive instructions and data from a memory and write instructions and data to the memory. The basic element of a computer is a CPU for executing or executing instructions, and one or more memory devices for storing instructions and data. Usually, a computer will also include or be operably connected to receive and / or transmit data from and / or to one or more mass storage devices for storing data, such as a disk, a magneto-optical disk or an optical disk. However, a computer does not need to have such a device. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game controller, a global positioning system (GPS) receiver or a portable storage device.

[0240] Non-transitory computer-readable media for storing computer program instructions and data may include all forms of permanent / non-permanent or volatile / non-volatile memory, media and memory devices, including, for example, semiconductor memory devices, such as random access memory (RAM), read-only memory (ROM), phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) and flash memory devices; magnetic devices, such as magnetic tapes, cassettes, cartridges, internal / removable disks; magneto-optical disks; and optical storage devices, such as digital versatile / video disks (DVD), compact disk (CD) ROM, DVD+ / -R, DVD-RAM, DVD-ROM, high definition / density (HD)-DVD and BLU-RAY / BLU-RAY DISC (BD) and other optical storage technologies. The memory may store various objects or data, including caches, classes, frameworks, applications, modules, backup data, jobs, web pages, web page templates, data structures, database tables, repositories storing dynamic information, or other appropriate information including any parameters, variables, algorithms, instructions, rules, constraints, or references. In addition, the memory may include other appropriate data, such as logs, policies, security or access data, or report files. The processor and memory may be supplemented by or incorporated into dedicated logic circuits.

[0241] In order to provide interaction with a user, the implementation of the subject matter described in this specification can be implemented on a computer having a display device, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a light emitting diode (LED), or a plasma monitor to display information to the user, and the computer also having a keyboard and a pointing device, such as a mouse, a trackball, or a trackpad, through which the user can provide input to the computer. Input can also be provided to the computer using a touch screen, such as a tablet computer surface with pressure sensitivity or a multi-point touch screen using capacitance or electrical sensing. Other types of devices can be used to interact with the user. For example, the feedback provided to the user can be any form of sensory feedback (e.g., visual, auditory, tactile, or a combination of feedback types). Input from the user can be received in any form, including acoustic, voice, or tactile input. In addition, the computer can interact with the user by sending documents to and receiving documents from a client computing device used by the user (e.g., by sending a web page to a web browser on a user's mobile computing device in response to a request received from a web browser).

[0242] The term "graphical user interface" or "GUI" may be used in the singular or plural to describe one or more graphical user interfaces and each display of a particular graphical user interface. Thus, a GUI may represent any graphical user interface, including but not limited to a web browser, a touch screen, or a command line interface (CLI), that processes information and efficiently presents the information results to a user. Typically, a GUI may include a plurality of user interface (UI) elements, some or all of which are associated with a web browser, such as interactive fields, drop-down lists, and buttons. These and other UI elements may be related to or represent the functionality of a web browser.

[0243] The implementation of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, such as a data server, or includes a middleware component, such as an application server, or includes a front-end component, such as a client computer with a graphical user interface or a web browser, through which a user can interact with the implementation of the subject matter described in this specification, or any combination of one or more such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of wired or wireless digital data communication (or a combination of data communications), such as a communication network. Examples of communication networks include local area networks (LANs), radio access networks (RANs), metropolitan area networks (MANs), wide area networks (WANs), Worldwide Interoperability for Microwave Access (WIMAX), wireless local area networks (WLANs), for example, using 802.11a / b / g / n or 802.20 (or a combination of 802.11x and 802.20 or other protocols consistent with the present disclosure), all or part of the Internet, other communication networks, or combinations of communication networks. The communication network may communicate, for example, Internet Protocol (IP) packets, Frame Relay frames, Asynchronous Transfer Mode (ATM) cells, voice, video, data, or other information between network nodes.

[0244] A computing system may include clients and servers. Clients and servers are generally remote from each other and generally interact through a communication network. The relationship between a client and a server is generated by computer programs running on the respective computers and have a client-server relationship with each other.

[0245] Although this specification contains many specific implementation details, these should not be interpreted as limitations on the scope of any creative concept or the scope that can be claimed, but as descriptions of features of specific implementations for specific creative concepts. Certain features described in this specification in the context of separate implementations may also be combined or implemented in a single implementation. On the contrary, the various features described in the context of a single embodiment may also be implemented individually or in any sub-combination in multiple embodiments. In addition, although the previously described features may be described as working in certain combinations and even initially stated as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may involve a sub-combination or a variation of the sub-combination.

[0246] Specific implementations of the subject matter have been described. Other implementations, variations, and permutations of the described embodiments are within the scope of the appended claims, which will be apparent to those skilled in the art. Although operations are depicted in the drawings or claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequence, or that all of the operations shown (some operations may be considered optional) be performed to achieve the desired effect. In some cases, multitasking or parallel processing (or a combination of multitasking and parallel processing) may be advantageous and performed when deemed appropriate.

[0247] In addition, the separation or integration of various system modules and components in the previously described implementations should not be understood as requiring such separation or integration in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0248] Therefore, the previously described example embodiments do not define or constrain the present disclosure. Other changes, substitutions, and alterations are also possible without departing from the spirit and scope of the present disclosure.

[0249] Furthermore, any claimed implementation is considered applicable to at least one computer-implemented method; a non-transitory computer-readable medium storing computer-readable instructions to perform the computer-implemented method; and a computer system comprising a computer memory cooperatively coupled to a hardware processor configured to perform the computer-implemented method or the instructions stored on the non-transitory, computer-readable medium.

Claims

1. A computer-implemented method comprising: selecting a mesh scaler for performing overall mesh-width scaling of a microservices mesh that provides services for containers and microservices-based applications, wherein the applications and services are scaled outward by adding instances to the services and scaled inward by removing instances from the services; scaling the microservices grid using the grid scaler and one or more grid scaling rules to generate predicted values ​​of monitoring parameters; selecting a single-service scaler configured to fine-tune scaling of each individual service; measuring the monitoring parameter using the single serving scaler to generate a measurement value; For a single service associated with the microservice grid, calculating a target value of the monitoring parameter using the predicted value and the measured value; The target value is compared with a defined threshold value to generate a scaling value for determining whether scaling is required for the individual service.

2. The computer-implemented method of claim 1 , wherein the grid-type scaler comprises: a grid scaler configured to monitor the application and scale all related services, and A request-type aware mesh scaler is configured to monitor requests to an application and, based on the request type, scale a microservice mesh of related services by computing a change in the number of instances of the service according to the request type.

3. The computer-implemented method of claim 1 , wherein scaling the microservices mesh using the mesh scaler is performed in a single trigger action.

4. The computer-implemented method of claim 1 , wherein the single-service scaler comprises: an automatic service scaler configured to call a monitoring infrastructure to read at least one monitoring parameter at intervals and, for each group of containers running the same service or application, trigger creation / destruction of application / service instances based on the at least one monitoring parameter, the at least one monitoring parameter comprising a load of resources used by the instances, a population-aware scaler configured to calculate the number of instances based on a load of resources used by the instances and the number of instances already running, and A derivative change aware scaler is configured to calculate the number of instances based on the load of resources used by the instances and the speed at which the load changes.

5. The computer-implemented method of claim 1, wherein the grid-type scaler communicates the predicted value to the single-service-type scaler.

6. The computer-implemented method of claim 1, wherein the single-serving scaler: Read monitoring parameters from the monitoring infrastructure of the container executing the instance of the service; a system load of the container in which instances of the service are executed; Collect the load of each service instance; and The average load is calculated by summing the load of all available service instances.

7. The computer-implemented method of claim 1, wherein the single-service scaler triggers creation or destruction of instances of the separate service based on the scaling value.

8. A non-transitory computer-readable medium storing one or more instructions executable by a computer system to perform the following operations, including: selecting a mesh scaler for performing overall mesh-width scaling of a microservices mesh that provides services for containers and microservices-based applications, wherein the applications and services are scaled outward by adding instances to the services and scaled inward by removing instances from the services; scaling the microservices grid using the grid scaler and one or more grid scaling rules to generate predicted values ​​of monitoring parameters; selecting a single-service scaler configured to fine-tune scaling of each individual service; measuring the monitoring parameter using the single serving scaler to generate a measurement value; For a single service associated with the microservice grid, calculating a target value of the monitoring parameter using the predicted value and the measured value; The target value is compared with a defined threshold value to generate a scaling value for determining whether scaling is required for the individual service.

9. The non-transitory computer readable medium of claim 8, wherein the grid-type sealer comprises: a grid scaler configured to monitor the application and scale all related services, and A request-type aware mesh scaler is configured to monitor requests to an application and, based on the request type, scale a microservice mesh of related services by computing a change in the number of instances of the service according to the request type.

10. The non-transitory computer-readable medium of claim 8, wherein scaling the microservices mesh using the mesh scaler is performed in a single trigger action.

11. The non-transitory computer-readable medium of claim 8, wherein the single-service scaler comprises: The automatic service scaler is configured to call the monitoring infrastructure to read at least one monitoring parameter at intervals and, for each group of containers running the same service or application, trigger the creation / destruction of application / service instances based on the at least one monitoring parameter, the at least one monitoring parameter comprising a load of resources used by the instance, a population-aware scaler configured to calculate the number of instances based on a load of resources used by the instances and the number of instances already running, and A derivative change aware scaler is configured to calculate the number of instances based on the load of resources used by the instances and the speed at which the load changes.

12. The non-transitory computer-readable medium of claim 8, wherein the grid-type scaler communicates the predicted value to the single-service-type scaler.

13. The non-transitory computer-readable medium of claim 8, wherein the single-service scaler: Read monitoring parameters from the monitoring infrastructure of the container executing the instance of the service; a system load of the container in which instances of the service are executed; Collect the load of each service instance; and The average load is calculated by summing the load of all available service instances.

14. The non-transitory computer-readable medium of claim 8, wherein the single-service scaler triggers creation or destruction of instances of the separate service based on the scaling value.

15. A computer-implemented system comprising: one or more computers; as well as One or more computer memory devices cooperatively coupled to the one or more computers and having a tangible, non-transitory, machine-readable medium storing one or more instructions that, when executed by the one or more computers, perform one or more operations including: selecting a mesh scaler for performing overall mesh-width scaling of a microservices mesh that provides services for containers and microservices-based applications, wherein the applications and services are scaled outward by adding instances to the services and scaled inward by removing instances from the services; scaling the microservices grid using the grid scaler and one or more grid scaling rules to generate predicted values ​​of monitoring parameters; selecting a single-service scaler configured to fine-tune scaling of each individual service; measuring the monitoring parameter using the single serving scaler to generate a measurement value; For a single service associated with the microservice grid, calculating a target value of the monitoring parameter using the predicted value and the measured value; The target value is compared with a defined threshold value to generate a scaling value for determining whether scaling is required for the individual service.

16. The computer-implemented system of claim 15, wherein the grid-type scaler comprises: a grid scaler configured to monitor the application and scale all related services, and A request-type aware mesh scaler is configured to monitor requests to an application and, based on the request type, scale a microservice mesh of related services by computing a change in the number of instances of the service according to the request type.

17. The computer implemented system of claim 15, wherein scaling of the microservices mesh using the mesh scaler is performed in a single triggered action.

18. The computer-implemented system of claim 15, wherein the single-service scaler comprises: an automatic service scaler configured to call a monitoring infrastructure to read at least one monitoring parameter at intervals and, for each group of containers running the same service or application, trigger creation / destruction of application / service instances based on the at least one monitoring parameter, the at least one monitoring parameter comprising a load of resources used by the instances, a population-aware scaler configured to calculate the number of instances based on a load of resources used by the instances and the number of instances already running, and A derivative change aware scaler is configured to calculate the number of instances based on the load of resources used by the instances and the speed at which the load changes.

19. The computer implemented system of claim 15, wherein the grid scaler communicates the predicted value to the single service scaler.

20. The computer implemented system of claim 15, wherein the single service scaler: Read monitoring parameters from the monitoring infrastructure of the container executing the instance of the service; a system load of the container in which instances of the service are executed; Collect the load of each service instance; Calculate the average load by summing the loads of all available service instances; and Creation or destruction of an instance of the separate service is triggered according to the scaling value.

Images