A micro-service elasticity scaling method and orchestrator

Abstract

The application discloses a micro-service elasticity scaling arrangement method and an arranger, relates to the field of cloud native, and aims to improve response efficiency and resource utilization rate when high-concurrency service requests are received. The application generates a micro-service call chain for each received service request, dynamically allocates resources to each micro-service instance according to the congestion degree of the service request responded by each micro-service instance of each edge server, and determines the route of each micro-service call chain between edge servers with the aim of minimizing the total delay and total load fluctuation of all service requests. The application can minimize system response time while ensuring optimal system load and improving edge server resource utilization.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cloud native technology, and in particular to a microservice elasticity scaling orchestration method and an orchestrator. BACKGROUND

[0002] With the deepening of enterprise digital transformation, microservice architecture gradually replaces traditional monolithic architecture and becomes the mainstream paradigm of current enterprise-level application systems, due to its advantages of independent deployment, independent expansion, independent development, etc. In the microservice architecture, a complete business process often needs the collaborative participation of multiple microservices, and the business goal can be achieved through the calling and interaction between microservices. Although this modular design enhances the maintainability and flexibility of the system, it also brings highly complex service dependency relationships and calling chains. With the continuous expansion of enterprise scale, the increasing diversity of business processes, and the continuous growth of user quantity, the concurrent request volume faced by the system increases exponentially. To ensure the system performance and stability in high concurrency scenarios, users usually deploy a large number of microservice instances in the edge server cluster to improve the throughput capacity and overall elasticity of the service. However, the rapid increase in the number of microservice instances also significantly increases the complexity of the system. Whenever a service request arrives at the system, the orchestrator needs to dynamically combine a complete service calling path (i.e., routing) from the numerous microservice instances deployed on different edge servers in a certain execution order to complete the service request.

[0003] With the increase in the number of microservice instances, the number of instance combination schemes available for responding to requests increases exponentially. Different instance combinations have different overheads in terms of network transmission, computing load, resource occupation, etc., and the system delay generated by each scheme is different. Therefore, how to quickly and efficiently select a group of optimal microservice instance combinations from the vast search space not only enables successful response to service requests, but also minimizes system response time, which has become one of the core challenges faced by microservice architecture in the design and operation stages.

[0004] The Chinese patent document with publication number CN119829255A discloses a service orchestration method based on a multi-tenant microservice architecture, which creates a thread pool and defines a private final pipeline, creates a pipeline builder and specifies each stage in the pipeline, generates a pipeline execution class, initializes the pipeline to process the overall business process steps, and encapsulates different processors, creates a pipeline object, effectively handles concurrent tasks, and performs reasonable processing in the case of timeout, avoiding waste and blocking of system resources, adds each processor to the pipeline and inserts an interceptor in the pipeline, enhancing the scalability and flexibility of the system, merges the output results of each processor into the final result, and performs exception handling and rollback operations, splits the overall business process steps into multiple sub-tasks, forms parallel and serial execution sub-task groups, and determines the execution order, improving the response speed and concurrent processing capability of the system. However, this method also directly leads to the dispersion of service request logic, increases the complexity of the call chain, and once a processor fails, it cannot locate the corresponding service request and fault location, reducing system reliability.

[0005] The Chinese patent document with publication number CN119520624A proposes a service orchestration calling method, device, equipment and storage medium, which includes: obtaining a current target request message to call each target service group; recording target information based on a target flag in the current target request message, saving the target information in a preset cache database; obtaining the target information from the preset cache database through a target service orchestration program to determine the optimal orchestration node order of each target service group corresponding to the target flag; saving the optimal orchestration node order to the preset cache database, so that when a new current target request message is obtained, the new each target service group is determined based on the target flag, the target orchestration node order of the new each target service group is determined from the preset cache database to call each target service group, and jumping to the step of recording target information based on the target flag in the current target request message. This method optimizes the orchestration calling order of microservice instances, but its orchestration node order cache is too dependent on historical data. If the business logic or microservice performance in the system changes, the old orchestration order may no longer be applicable, resulting in an execution path after cache hit that is not the current optimal path, and even performance degradation may be triggered. And the design is biased towards static optimization (determining the optimal path through caching), and dynamic indicators such as load, delay, service availability, network topology changes, etc. when requests arrive in real time are not fully considered, and the current system environment cannot be balanced. SUMMARY

[0006] The invention aims to: for all or part of the above problems, provide a microservice elastic scaling orchestration method and an orchestrator to respond faster to high-concurrency service requests.

[0007] The technical solutions adopted by the present application are as follows:

[0008] A micro-service elasticity scaling arrangement method, comprising:

[0009] receiving service requests;

[0010] generating a corresponding micro-service call chain for each service request to indicate the called micro-service instance and the calling sequence; each micro-service call chain has at least one route between multiple edge servers forming a micro-service cluster; wherein:

[0011] In each edge server, under the total resource limit of the edge server, according to the congestion degree of each micro-service instance in response to the service request, resources are dynamically allocated to each micro-service instance respectively;

[0012] To minimize the total delay and total load fluctuation in response to all service requests, the route of each micro-service call chain between edge servers is determined.

[0013] In addition, the present application also provides a micro-service elasticity scaling arrangement device, comprising a processor and a storage medium, the storage medium stores computer instructions, the processor runs the computer instructions, and executes the micro-service elasticity scaling arrangement method described above.

[0014] As described above, due to the adoption of the above technical solutions, the present application has the following beneficial effects:

[0015] The present application simultaneously considers the performance indicators of the micro-service system in response time and system load, and calculates the micro-service instance route in both aspects, adjusts the dynamic scaling of the micro-service resources, and optimizes the total time delay and total load fluctuation in two dimensions by the scheduler, so as to minimize the response time while ensuring the stability of the system load, effectively reduce the system congestion, and improve the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0016] The present application will be described by way of example and with reference to the accompanying drawings, in which:

[0017] Figure 1 is a flowchart of the micro-service elasticity scaling arrangement method in one embodiment.

[0018] Figure 2 is a schematic diagram of the micro-service arrangement scene.

[0019] Figure 3 is a data flow diagram of the micro-service arrangement.

[0020] Figure 4 is a comparison diagram of the system response time of the present application and the baseline algorithm under different service request quantities.

[0021] Figure 5 is a plot of system response time of the present application versus baseline algorithm at different edge server numbers.

[0022] Figure 6 is a plot of system response time of the present application versus baseline algorithm at different microservice call chain lengths.

[0023] Figure 7 is a plot of system response time of the present application versus baseline algorithm at different microservice numbers.

[0024] Figure 8 is a plot of system load of the present application versus baseline algorithm at different service request volumes.

[0025] Figure 9 is a plot of system load of the present application versus baseline algorithm at different edge server numbers.

[0026] Figure 10 is a plot of system load of the present application versus baseline algorithm at different microservice call chain lengths.

[0027] Figure 11 is a plot of system load of the present application versus baseline algorithm at different microservice numbers. DETAILED DESCRIPTION

[0028] All of the features disclosed in this specification, and / or all of the steps of any method or process specified in this specification, can be combined in any combination, unless specific excluded either in passable paragraphs, above, or specific examples, above.

[0029] Any of the features disclosed in this specification, unless explicitly stated otherwise, or any of the steps of any method or process specified in this specification, unless explicitly stated otherwise, can be replaced by alternatives equivalents or similarly effective particulars. That is, each feature disclosed is only one of many alternatives for equivalent or similar features.

[0030] In a microservice system, whenever a service request is received, the system dynamically generates a call chain to fulfill the response to the request. A call chain is essentially a collection of microservice instances that are executed in a certain order, and these instances work together to handle the complete business process. Each microservice instance usually takes on a specific business logic and has independent processing capabilities. Microservices communicate and transfer data through standard API interfaces. In actual response, if two microservice instances are distributed on different physical servers or virtual nodes, the interaction between them will generate additional network communication latency; on the contrary, if they are deployed on the same edge server, the communication latency can be considered as approximately zero. Therefore, different microservice call chains will result in different response delays.

[0031] Embodiments of the present application are directed to the problem of microservice congestion and serious system latency that can easily occur under high-concurrency service requests in a microservice cluster architecture. A microservice elasticity scaling orchestration method and orchestrator are proposed to reduce the overall response time of the system under high-concurrency service requests and improve the resource utilization of edge servers.

[0032] Achieving the business goal of a service request usually requires calling multiple microservices in sequence to complete the response. The microservice call chain represents which microservices need to be called and the calling order of each microservice. Since different service requests may need to call the same microservice, in order to meet the concurrency requirements of service requests, the same microservice instance is often deployed in multiple different edge servers in the microservice system to achieve the same function, so as to meet the needs of multiple service requests calling at the same time and improve the concurrency processing capability of the microservice system. Therefore, the network environment of the microservice system is composed of edge servers and user devices, and is represented in the form of a two-tuple as . Wherein H represents the overall network environment. E represents a set of edge servers, specifically, , wherein represents the kth edge server, represents the number of edge servers in the microservice cluster. Each edge server has different resources, such as CPU and memory. U represents a set of user devices, which are responsible for initiating service requests. Each edge server and each user device contains corresponding computing resources (CPU, memory, etc.) and attributes such as geographic location information.

[0033] The action of microservice orchestration in response to service requests is completed by an orchestrator. The orchestrator can be represented as a three-tuple . M represents a set of microservice instances, represented as , wherein represents the ith microservice instance, each microservice instance is encapsulated in the form of a container image, and contains the CPU resources and memory resources allocated when deployed on an edge server, each microservice instance has an independent running environment and functional module. S represents a set of microservice call chains of all service requests, and the microservice call chain is denoted as MSN, , which is a directed chain used to describe the microservice link structure relied on by various service requests in the system, wherein f represents a directed chain, represents a microservice instance, This represents the dependencies between microservices, including the amount of data transferred between them. The orchestrator must ensure that it orchestrates the microservices in MSN exactly in the correct order to successfully respond to service requests. R represents the set of service requests currently pending, where each service request can be abstractly represented as a triple. >, among which This represents the microservice call chain of the request. This represents the arrival rate of microservices, obtained based on historical access data statistics. This indicates the duration of the current service request. When a service request arrives, the orchestrator begins its work. In this application, it activates microservice instances in the edge servers based on the microservice call chain topology and the load, location, and resource availability of the edge servers, and routes the service request to the corresponding microservice instance.

[0034] like Figure 1 As shown, in one feasible implementation, this application proposes a microservice elastic scaling orchestration method including:

[0035] S1, Receive service requests.

[0036] A microservice cluster architecture contains a large number of users who may simultaneously initiate service requests. The microservice system receives these concurrent service requests and calls the microservices for each request to achieve the business goal. For example... Figure 2 As shown, user equipment sends service requests through interaction with edge servers, and the resulting requests are aggregated to the orchestrator in the form of a data stream (called a request stream).

[0037] S2. Generate a corresponding microservice call chain for each service request to indicate the microservice instance being called and the order of the calls.

[0038] After receiving the request stream, the orchestrator obtains the geographical location information of the user devices and edge servers, as well as the distribution of microservice instances running on each edge server. Considering the geographical distance difference between user devices and edge servers, additional latency is inevitably introduced during communication. Figure 2 Taking a service request as an example, this service request needs to call the microservices in sequence. To complete the response. Since microservices can be distributed among different edge server nodes, data transmission involves cross-node communication. Moreover, the same microservice can deploy microservice instances on multiple different edge servers, so each microservice call chain will have one or more routes between multiple edge servers forming a microservice cluster. Under the premise of consistent network conditions, edge servers with closer physical distance usually have lower communication delay. The orchestrator prioritizes scheduling service requests to the server node with the smallest delay by calculating the network delay between the user device and each optional edge server to optimize overall service performance.

[0039] However, in user-intensive areas, as the number of service requests continues to increase, when a large number of service requests are concentrated in a nearby edge server, it can cause the edge server node to be overloaded, resulting in congestion, and even failure to affect the processing of large-scale user service requests, reducing user experience. To address this situation, the orchestrator is designed to dynamically adjust the orchestration strategy based on the real-time load of each edge server in the microservice cluster, diverting some service requests to other edge servers with lower load, thereby achieving overall load balancing of the microservice system and improving the stability and response efficiency of microservice jobs.

[0040] S21、As an optional implementation, in the process of orchestrating microservice instances according to the microservice call chain, on each edge server, under the total resource limit of the edge server, the resources are dynamically allocated to each microservice instance according to the congestion degree of the microservice instance responding to service requests.

[0041] Specifically, for each edge server, the higher the congestion degree of the microservice instance deployed on it, the more resources it is dynamically allocated. For example, resources are dynamically allocated to the microservice instance according to the proportion of the congestion degree of the microservice instance in the total sum of the congestion degrees of all microservice instances in the edge server; or the congestion degree of the microservice instance is divided into levels, and each time a level is reached, a corresponding amount of resources is allocated / the resources are scaled by a corresponding level proportion.

[0042] As an optional implementation, the congestion degree includes a delay indicator and an activation indicator for responding to service requests, and the delay indicator and the activation indicator are weighted and summed to obtain a congestion degree indicator.

[0043] The delay indicator indicates the proportion of the average delay of the microservice instance responding to service requests to the system average delay, and is represented as The activation indicator indicates the activation proportion of the microservice instance responding to service requests, and is represented as . Let represent the congestion degree of the microservice instance, then

[0044] Formula (1): ;

[0045] In the formula, respectively represent the weighted weights of the delay index and the activation index.

[0046] The congestion degree of each microservice instance in the edge server when responding to a service request can be calculated by Formula (1). According to the congestion degree, resources are respectively allocated to each microservice instance, and the microservice instance with a larger congestion degree is allocated more resources.

[0047] For example, the resources of each microservice instance can be dynamically allocated according to the congestion degree of each microservice instance in responding to a service request by the following method:

[0048] The system average congestion degree is calculated as , which is the average value of the congestion degrees of all microservice instances in the microservice system.

[0049] If the congestion degree of a microservice instance reaches the system average congestion degree , the resource of the microservice instance is increased according to the set resource scaling rate (assuming ), that is, increased by times; otherwise, if the congestion degree of the microservice instance does not reach the system average congestion degree , the resource of the microservice instance is reduced according to the resource scaling rate , that is, reduced by times. The static resource of each edge server is fixed, so when dynamically adjusting the resource configuration of each microservice instance, it needs to be performed under the limitation of the static resource of the edge server, that is, the total amount of resources of all microservice instances does not exceed the static resource of the edge server. Through real-time load condition dynamic adjustment of the resource allocation strategy of each microservice instance, the elastic scaling of resources is realized, and the system response capability is improved.

[0050] S22, as an optional embodiment, in the process of arranging microservice instances according to the microservice call chain, the routing of each microservice call chain between edge servers is determined to minimize the total delay and total load fluctuation in responding to all service requests.

[0051] The time delay of the microservice system in responding to a service request includes two aspects, one is the time delay of the execution request task link, and the other is the time delay generated by data transmission. Specifically, the total delay is characterized by the execution time delay and the communication time delay of all microservice instances in responding to each service request.

[0052] The execution time delay : in the embodiment of the application, the execution time delay is calculated by the parameter Represents a microservice instance On the edge server Deployment status, This indicates that there is no corresponding microservice instance. Deployed on edge servers superior, (n is a natural number) indicates that there are n microservice instances. Deployed on edge servers Since the static resources of each edge server are fixed, when dynamically scaling microservice instances, the resources used cannot exceed the total resources available to the edge servers, that is, the resource constraints of equations (2) and (3) must be met.

[0053] Equation (2): ;

[0054] Make (3): .

[0055] In the formula, These represent edge servers. Medium CPU resources and medium memory resources.

[0056] In one alternative implementation, the orchestrator employs A queuing model is used to manage service requests. Specifically, this refers to the execution latency of microservice instances. It consists of two parts, including the processing latency of microservice instances. and the queuing delay of the service requests being processed. Handling delays The processing capacity is primarily affected by the performance of the edge server and the resources consumed by the microservice instance. To describe the processing capacity of the microservice instance, this embodiment introduces parameters. To represent microservice instances On the edge server The processing speed.

[0057] Equation (4): ;

[0058] Equation (5): ;

[0059] Equation (6): ;

[0060] Equation (7): ;

[0061] Equation (8): .

[0062] In the formula, This indicates a microservice in request flow F. arrival rate, i.e. the probability of occurrence; denotes the microservice The capability of handling service requests on the edge server , which can be calculated by the processing rate of the microservice instance ; denotes the microservice The probability of being idle in the microservice cluster; and denotes the length of the request stream F in the orchestrator, i.e. the number of service requests to be processed.

[0063] communication latency : there is a physical distance between the user equipment and the edge server, so the communication latency of the user's service request sent to the orchestrator includes the propagation delay , the propagation delay and the physical distance between the two (user equipment and edge server), the network propagation speed between the user equipment and the edge server . In the microservice cluster architecture, a series of microservices need to be executed in a specific order to respond to a service request. Each time a microservice processes a service request, some intermediate data is generated, which needs to be passed to the next microservice instance by the orchestrator, and the data transfer between microservice instances produces a routing delay , which is also part of the communication latency . When two microservice instances and are deployed on the same edge server , the routing delay between microservices can be considered as zero, because the network bandwidth within the same server is almost unconstrained. However, when two microservice instances are deployed on different edge servers and , data transmission is limited by network bandwidth , which will produce a certain routing delay . The routing delay is related to the amount of intermediate data and the network bandwidth between the two edge servers.

[0064] Equation (9): .

[0065] service request is responded to by a microservice call chain within the microservice system, so the end-to-end response time of a service request is from the arrival of the request to the traversal of the microservice call chain Starting with the first microservice instance, until the entire process is completed... This includes all time spent on each microservice instance, encompassing both execution and communication latency. Since a single microservice instance can be shared by multiple service requests, it utilizes... To perform service mapping, when a microservice instance... On the edge server The value is 1 when the function is active and 0 otherwise. This represents the response time of a microservice system. (i.e., total time) can be calculated using equation (10), where This represents the total number of user devices. n Refers to the first n One user device.

[0066] Equation (10): .

[0067] As the number of service requests increases, if the orchestrator routes all service requests to a single edge server, it may overload that edge server, affecting its overall performance, leading to prolonged response times or even edge server crashes. Secondly, since microservice instances typically require time to process requests, when too many service requests are distributed to the same edge server, it may result in an excessively long request queue, potentially increasing the waiting time for each service request. Therefore, when orchestrating microservice instances, this embodiment also considers the load status of each edge server in the microservice cluster, which is reflected in the microservice system as total load fluctuation. Thus, the objective function for orchestrating microservice instances is to minimize the total latency and total load fluctuation in responding to all service requests.

[0068] In one alternative implementation of this application, total load fluctuation is characterized by the variance of resource utilization of all microservice instances in response to each service request.

[0069] Taking CPU and memory, the most important resources, as an example, we define... These represent the variances of CPU resource utilization and memory resource utilization, respectively. (Resource utilization variance) Through the and The weighted sum is obtained. Define resource weights. Then the variance of resource utilization rate The calculation method is as follows:

[0070] Equation (11): .

[0071] As an optional implementation method, CPU resource utilization variance and the variance of memory resource utilization are calculated based on the proportion of CPU and memory used by the microservice instances deployed and activated in each edge server, respectively.

[0072] Definition and respectively represent the proportion of CPU and memory actually used by the activated microservice instances on the edge server to the total resources of the microservice instances deployed on the edge server .

[0073] Equation (12): ;

[0074] Equation (13): .

[0075] By calculating the CPU and memory used by each microservice instance and , the average CPU resource proportion and the memory resource proportion of the microservice instances on the edge server can be calculated, respectively. Then we have: Equation (14):

[0076] ;

[0077] Equation (15): .

[0078] The final constructed objective function is represented as:

[0079] Equation (16): ;

[0080] Equation (17): ;

[0081] Equation (18): s.t. (2), (3).

[0082] That is, the objective function is subject to the constraints of Equation (2) and Equation (3).

[0083] Based on the construction of the objective function, the Markov decision method is used to iteratively solve the objective function to obtain the routing of each microservice call chain among the edge servers. That is, to determine the edge server where each microservice instance in the microservice call chain is located.

[0084] The Markov decision method defines the microservice orchestration as a five-tuple , where represents the state space, i.e., the deployment and resource utilization of the microservice cluster, reflecting the load of the microservice system; represents the action space, i.e., which microservice instances are specifically orchestrated for the microservice call chain;​ This represents the state transition probability, which is the probability of selecting a microservice instance from the microservice cluster. This represents the instant reward function, indicating the amount of reward obtained after calling the microservice instance; This is the discount factor, representing the proportion by which the state space reached after calling a microservice instance contributes to the cumulative reward. The orchestrator continuously interacts with the network environment H, constantly adjusting its settings based on the current state. Select a choreography action from the state space. Whenever the orchestrator performs an action Afterwards, it will receive an instant reward. That is, in the state Next action The reward obtained later This reward reflects the impact of the current action on system load and performance. Based on the action... and state The choice of environment will be based on the state transition probability. Move to the next state. Since a successful response to a service request depends on the collaboration of all microservice instances in the entire microservice call chain, a successful response typically brings a significant reward; therefore, a discount factor is introduced. The reward for a successful response is distributed to each step of the orchestration process, thereby incentivizing the orchestrator to select the optimal service request scheduling strategy.

[0085] As an alternative implementation, a critic-actor network is used to implement the Markov decision-making method. For example... Figure 3 The diagram illustrates the network architecture used in microservice elastic scaling orchestration. In this embodiment, the network architecture includes an environment, a state, and agents. The environment is the microservice system; the state is the deployment and usage of the microservice system at a given moment; and the agent is the entity that performs microservice orchestration. The critic-actor network is deployed within the agent. The actor network infers the edge server where the next microservice instance implementing the objective function resides in the microservice call chain based on the input state space characteristics. The critic network evaluates the cumulative reward obtained from orchestrating microservice instances based on the input state space characteristics. When a user device interacts with the edge server and generates a request, a request flow is generated in the orchestrator. Requests are processed according to… The queuing model orchestrates on edge servers. During each orchestration session, the environment records the current state of the edge servers and inputs this state into the agent. Internally, the agent inputs the state space features into the critic network and the actor network, ultimately generating an orchestration action, entering the next state, and obtaining a reward. By training the critic network and actor network with the objective function in mind, the system can obtain the current state space features each time a microservice instance is identified for microservice call chain orchestration. The actor network then determines the microservice instance to be orchestrated that will yield the maximum cumulative reward.

[0086] Specifically, the state space includes the resource space of each edge server in the microservice cluster and the space of microservice instances running on the edge servers. The space of microservice instances running on the edge servers is represented as follows: This represents the set of microservice instances already deployed in the microservice cluster; the resource space of each edge server includes CPU resource space. and memory resource space , representing the CPU and memory usage of the edge server, respectively. The final state space is represented as .

[0087] The response to the state is reflected in the action space. For the microservice call chain that responds to service requests, the actions of the microservice instances are scheduled. It is the probability of activating each microservice instance in the microservice cluster to respond, denoted as... ,in Indicates the first An edge server can respond to microservices Microservice instance set Activate the first The probability that a microservice instance will respond.

[0088] In one optional implementation, the agent consists of six modules: an actor network, a target actor network, a critic network, a target critic network, an experience recycling pool, and a GRU module. After state vectorization, the agent inputs the state vectors. The GRU module preprocesses the state vectors and then concatenates them to form state space features. Since the number of service requests is affected by time, the GRU module effectively remembers better states and forgets worse states, avoiding gradient explosion during the learning process. The target actor network is the target network with fixed parameters after training, and the target critic network is the critic network with fixed parameters after training. The experience recycling pool caches the state space features obtained after orchestrated actions are applied to the environment, reducing the frequency of agent-environment interaction, improving training efficiency, and providing excellent training samples for the critic and actor networks to enhance the diversity of network training and avoid getting trapped in local optima.

[0089] The cumulative reward obtained from orchestrating actions within the operating environment includes the reward gained from scheduling orchestrated microservice instances within the input state space. It typically also includes the discounted value (multiplied by a discount factor) of the state value (optimization objective, i.e., total latency and total load) obtained after scheduling the orchestrated microservice instances. The reward is the gain in value compared to the state before scheduling (i.e., the difference before and after). The goal of microservice orchestration is to find the optimal microservice instance to activate in the current state, minimizing system response latency and load. The reward for each orchestrated microservice instance scheduled according to the microservice call chain is negatively correlated with the total latency gain after scheduling the microservice instance and positively correlated with the total load. Total latency gain is the change in total latency before and after scheduling.

[0090] Because a successful response to a request requires the entire call chain to complete sequentially, a successful request can only be proven when the last microservice in the call chain is orchestrated successfully. Therefore, the reward is divided into two parts, referred to as the first reward and the second reward. First Reward Consider whether there are responsive microservice instances in the microservice cluster. If there are microservice instances with responsive microservice call chains, a small reward is given. It is a non-zero minimum value, such as 0.01; if there is no respondable microservice instance, the response will definitely fail, and the reward will be set to 0.

[0091] Second reward After receiving the first reward (Non-zero reward) will be considered on top of that. Second reward is to find the lowest response latency and the optimal load scheduling scheme in the set of responsive microservice instances. If the total delay after scheduling the microservice instance is below the total delay of the request optimal scheduling scheme, the second reward is configured according to the difference between the two and the total load after scheduling the microservice instance; if the total delay after scheduling the microservice instance is below the total delay before scheduling, the second reward is configured according to the difference between the two and the total load after scheduling the microservice instance; otherwise, the second reward is configured as 0.

[0092] Specifically, there are:

[0093] Formula (19): ;

[0094] Formula (20): .

[0095] In the formula, represents the minimum response time of the first i-1 microservice instances already scheduled when scheduling the current i-th microservice instance, which is calculated by minimizing formula (10), represents the total delay of the system in state when scheduling the i-th microservice, which can be calculated by formula (10). represents the system load in state when scheduling the i-th microservice, which is characterized by the resource utilization variance in this state, and are adjustable factors. . As an optional implementation, the training method of the critic network and the actor network in the agent includes:

[0096] (1) Experience sampling and storage:

[0097] During interaction with the microservice system, the agent selects an action in state

[0098] , the environment returns a reward and moves to a new state . The interaction data is stored in the experience pool in the form of a four-tuple ). Subscripts t and t+1 represent the current time step and the next time step. When training, a batch of data is randomly sampled from the memory pool to break the time correlation and improve learning efficiency. (2) Update of the critic network:

[0099] The model parameters of the critic network are denoted as , and the goal is to approximate the action value function

[0100] , where s represents the state, and a represents the action. .a action, which represents the selected action, which represents the selected action in state s a the obtained target Q value (i.e. the reward).

[0101] In the training process, first the target actor network generates an action for the next state and computes the target Q value through the target critic network, where represents the model parameters of the target actor network.

[0102] Equation (20): ;

[0103] where the discount factor is used to weigh long-term rewards and immediate rewards, is the model parameter of the target critic network.

[0104] The output of the current critic network is and the mean square error between the current critic network and the target critic network is the loss function of the critic network :

[0105] Equation (21): ;

[0106] where is the number of samples sampled in one update.

[0107] The gradient is calculated by backpropagation, and the model parameters of the critic network are updated using the Adam optimizer:

[0108] Equation (22): ;

[0109] where is the model parameter of the critic network, including weights and biases, etc., such as all the parameters of the layers in the Q network; represents the learning rate of the critic network, which controls the "step size" of each model parameter update; represents the gradient of the loss function with respect to the parameter .

[0110] (3) Update of the actor network:

[0111] The model parameters of the actor network are denoted as , and the goal is to maximize the Q value output by the critic network, i.e. to find the action that can bring the maximum long-term return.

[0112] In the training process, the actor network directly outputs the action in the current state and input it into the critic network to get the corresponding Q value. The optimization goal of the actor network is to maximize the Q value, so its loss function is:

[0113] Equation (23): ;

[0114] The gradient is obtained by back propagation, and the network parameters of the actor network are updated using the Adam optimizer:

[0115] Equation (24): ;

[0116] represents the learning rate of the actor network. The minus sign before the loss function in equation (23) ensures that the optimization process is gradient ascent, that is, the direction of the actor network learning is to improve the Q value of the critic network.

[0117] (4) Soft update of the target network (target critic network and target actor network):

[0118] To avoid unstable training process, the target actor network and the target critic network are introduced in the embodiments of the present application, and the model parameters of the two are updated using a soft update strategy:

[0119] Equation (25): ;

[0120] Equation (26): ;

[0121] wherein, 1 (much less than 1) is the update coefficient. In this way, the change of the parameters can be smoothed to some extent, preventing the training from diverging.

[0122] (5) Training iteration convergence:

[0123] In each iteration, the actor network generates a microservice orchestration scheme (i.e. action) according to the current policy, and the environment feedbacks the reward , the critic network calculates the value estimate and updates the model parameters. Then the actor network adjusts the policy distribution using the advantage function. This process continues until the loss function converges, or the preset performance indicator is reached on the validation set.

[0124] Through the above training method, the critic network and the actor network are trained according to the historical sample data. When the next microservice instance is orchestrated according to the microservice call chain, the state space is input into the agent, and after processing the state space features, they are input into the actor network and the critic network respectively, and the microservice instance with the highest evaluation by the critic network is output by the actor network for orchestration.

[0125] According to the idea of the application, the embodiment of the application further provides a micro-service elasticity scaling orchestrator, which comprises a processor and a storage medium, the storage medium stores computer instructions, and the processor executes the computer instructions to perform the micro-service elasticity scaling orchestration method.

[0126] In the embodiment of the application, the feasibility of the method is verified based on the micro-service running data set disclosed by Alibaba and the server data set of Melbourne cloud. The data set of Alibaba simulates the micro-service chain responding to the request, and the data set of Melbourne cloud provides the geographic location information of each edge server and user. The method considers the influence of heterogeneous edge server resources, more complex link structure, and different request quantities, and compared with the comparative algorithm, the performance is greatly improved.

[0127] In the experiment, different sizes of micro-service systems are simulated by adjusting the number of edge servers (from 4 to 10), the number of users (from 6 to 21), the number of micro-service call chains (from 5 to 10), and the number of micro-services in each call chain (from 4 to 12). The arrival rate of micro-services is set to be between 3 and 5. The CPU resource size of each edge server ranges from 15 (x 1000) to 30 (x 10000) m, and the memory size ranges from 200 MB to 300 MB; the CPU resource consumption of each micro-service instance ranges from 1 (x 1000) to 2 (x 10000) m, and the memory resource consumption ranges from 10 MB to 15 MB. In addition, the dependence relationship data between micro-services is set to be 0 to 3 MB, and the bandwidth between edge servers ranges from 1 to 5 Mbps. The core idea of the algorithm in the application is refined and modified into the comparative algorithm .

[0128] As shown in Figure 4 , as the request quantity rises, the response delay of the micro-service system will increase, and the model proposed in this paper maintains better performance in different request quantity experiments. Especially when the request quantity is large, the application (referred to as Ours) has more obvious improvement compared with the comparative algorithm. Compared with the comparative algorithm , the average system delay performance is improved by 19.91%, 20.23%, 33.52%, and 32.43%. Finding an optimal orchestration scheme needs to pay attention to the resource situation of each edge server, and increasing the number of servers will increase the complexity of the orchestration routing, so under the same request quantity, as the number of edge servers rises, the average delay of the system shows an upward trend. But under different number of edge servers, such as Figure 5As shown, the performance of this application remains excellent, with an average performance improvement of 19.76%, 18.32%, 38.35%, and 37.26% compared to the comparison algorithms. Figure 6 The average latency of the system under different microservice call chains is calculated. Different microservice call chains map different requests. The longer the call chain, the more microservices need to be orchestrated, so the average latency of the system will increase slightly. Compared with the comparison algorithm, the performance improvement of this application is 16.74%, 16.84%, 38.67%, and 39.59%, respectively, which is better than the comparison algorithm. Figure 7 This refers to the average system response time under different numbers of microservice instances. More microservice instances will reduce the queuing time of services to be orchestrated, which will reduce the average system response time to some extent. However, as the number of microservice instances increases, the search space when orchestrating microservices increases, which will increase system latency. After balancing the two, the average system latency shows a downward trend. Compared with the comparison algorithm, the performance improvement of this application is 14.40%, 14.36%, 25.21%, and 43.60%, respectively.

[0129] The result of equation (11) is used to judge the system load. The lower the load index, the better the system's resource utilization, i.e., the better the effect. Comparison Algorithm Focusing solely on edge server resource utilization without considering microservice instance utilization may result in microservice instances being actually idle, but... The algorithm incorrectly assumes that it is making reasonable use of resources for load balancing. That approach doesn't consider load balancing. This application calculates load based on the actual number of microservice instances used and the resources consumed by the collection of microservice instances, resulting in more efficient use of system resources. For example... Figure 8 As shown, the performance ratio of this application under different request volumes These figures represent increases of 31.13%, 40.29%, 70.24%, and 64.36%, respectively. For example... Figure 9 As shown, under different numbers of edge servers performance ratio These figures represent increases of 51.97%, 73.96%, 90.20%, and 89.44%, respectively. For example... Figure 10 As shown, the system load index increased by 33.81%, 53.54%, 79.74%, and 76.20% respectively under different call chain lengths. Figure 11 As shown, when the system has different microservice instances, the system load index is improved by 62.75%, 55.80%, 86.67%, and 80.81% compared with the comparison algorithm.

[0130] Through the above experimental data, it can be proved that the application can effectively improve the response rate and resource utilization rate of the micro-service cluster, reduce the congestion probability of the micro-service instance, reduce the edge server downtime probability, and improve the overall performance of the system.

[0131] The application is not limited to the foregoing specific embodiments. The application extends to any novel one, or any novel combination, of the characteristics disclosed in this specification, and to any novel method or process steps disclosed, or any novel combination thereof.

Claims

1. A microservice elastic scaling orchestration method, characterized in that, include: Receive service requests; Generate a corresponding microservice call chain for each service request to indicate the microservice instance being called and the order of the calls; Each microservice call chain has at least one route among the multiple edge servers forming the microservice cluster; where: On each edge server, within the total resource limit of the edge server, resources are dynamically allocated to each microservice instance based on the congestion level of each deployed microservice instance in responding to service requests. This includes: calculating the system average congestion level; if the congestion level of a microservice instance reaches the system average congestion level, then the resources of that microservice instance are increased according to the set resource scaling ratio; if the congestion level of a microservice instance does not reach the system average congestion level, then the resources of that microservice instance are decreased according to the resource scaling ratio. To minimize the total latency and total load fluctuation in responding to all service requests, the routing of each microservice call chain among the edge servers is determined. This includes: constructing an objective function to minimize the total latency and total load fluctuation; the total latency is characterized by the execution latency and communication latency of all microservice instances responding to each service request; the total load fluctuation is characterized by the variance of resource utilization of all microservice instances responding to each service request; and under the constraint of total resources on the edge servers, the objective function is iteratively solved using a Markov decision method to obtain the routing of each microservice call chain among the edge servers.

2. The microservice elastic scaling orchestration method as described in claim 1, characterized in that, The congestion level includes latency and activation metrics for responding to service requests. A weighted sum of these latency and activation metrics yields the congestion level index; wherein: The latency metric indicates the ratio of the average latency of a microservice instance responding to a service request to the system average latency. The activation metric indicates the activation rate of microservice instances in responding to service requests.

3. The microservice elastic scaling orchestration method as described in claim 1, characterized in that, The execution latency includes the processing latency of microservice instances and the queuing latency of the processed service requests; the communication latency includes the propagation latency of receiving service requests and the routing latency of data transmission between microservice instances.

4. The microservice elastic scaling orchestration method as described in claim 1, characterized in that, The resource utilization variance is obtained by weighted summation of the CPU resource utilization variance and the memory resource utilization variance; the CPU resource utilization variance and the memory resource utilization variance are calculated based on the CPU and memory usage ratios of the microservice instances deployed and activated in each edge server.

5. The microservice elastic scaling orchestration method as described in claim 1, characterized in that, The objective function is solved iteratively using the Markov decision method, including: With the objective function as the goal, a critic network and an actor network are trained. The actor network infers the edge server where the next microservice instance that implements the objective function is located in the microservice call chain based on the input state space features. The critic network evaluates the cumulative reward obtained by orchestrating the microservice instance based on the input state space features. The state space includes the resource space of each edge server in the microservice cluster and the space of microservice instances running on the edge servers. Each time a microservice instance is determined to be part of a microservice call chain orchestration, the state space characteristics of the current state space are obtained, and the actor network determines the microservice instance to be orchestrated that will yield the greatest cumulative reward.

6. The microservice elastic scaling orchestration method as described in claim 5, characterized in that, The cumulative reward includes the reward obtained after scheduling orchestrated microservice instances in the input state space; the reward obtained for each orchestrated microservice instance scheduled according to the microservice call chain is negatively correlated with the total latency gain obtained after scheduling the microservice instance and positively correlated with the total load.

7. The microservice elastic scaling orchestration method as described in claim 6, characterized in that, The rewards include a first reward and a second reward; If the microservice cluster contains microservice instances that can respond to the microservice call chain, then a fixed non-zero reward is configured for the reward. Otherwise, the reward will be set to 0; Based on configuring the first reward for the reward, if the total latency after scheduling the microservice instance is below the total latency of requesting the best orchestration scheme, then configure the second reward according to the difference between the two and the total load after scheduling the microservice instance. If the total latency after scheduling a microservice instance is lower than the total latency before scheduling, then the second reward is configured based on the difference between the two and the total load after scheduling the microservice instance; otherwise, the second reward is configured to 0.

8. A microservice elastic scaling orchestrator, characterized in that, It includes a processor and a storage medium, the storage medium storing computer instructions, the processor executing the computer instructions to perform the microservice elastic scaling orchestration method as described in any one of claims 1-7.

Images