Resource allocation optimization method and system of micro-service system, storage medium and equipment

Abstract

The invention provides a resource allocation optimization method and system of a micro-service system, a storage medium and equipment. The method comprises the following steps: controlling the rate of a data stream entering the micro-service system at the current moment; the micro-service system comprises a plurality of micro-services; obtaining virtual queue lengths corresponding to a plurality of micro-services at the current moment; allocating resources to the data stream corresponding to the micro service based on the virtual queue length; and updating the queue length of the data stream at the current moment to obtain the queue length of the data stream at the next moment. According to the method, resources are dynamically allocated by utilizing real-time delay and SLA feedback information, so that the method is independent of a complex network dependency relationship, and a large amount of offline analysis and prior knowledge learning do not need to be carried out. According to the invention, the resource allocation is adaptively adjusted and the flow control is carried out according to the current network load, the demand degree of the service for the resources and the maximum bearing capacity of the current network, and the data flow time delay is maintained within the SLA safety range to the maximum extent.

Description

Technical Field

[0001] The present application belongs to the field of cloud service technology, and in particular, relates to a resource allocation optimization method, system, storage medium and device for a microservice system. Background Art

[0002] Microservice systems have been increasingly widely used in the cloud service field due to their advantages such as easy development and deployment, high scalability, good fault tolerance, and low maintenance costs. Compared with the traditional monolithic architecture, the microservice architecture breaks down large projects into functionally independent and easy-to-manage components. These components transmit information through application programming interfaces (APIs) or message queues to collaboratively complete the functions of the entire system. The service-level agreement (SLA) defines the various indicators that data flows must meet, among which whether the end-to-end latency meets the SLA is the most concerned issue for developers. A reasonable resource allocation system can effectively reduce the end-to-end latency of each data flow, which is crucial to the smooth operation of the system.

[0003] With the popularity of microservices, it is particularly important to optimize the resource allocation of these microservice components. In a microservice system, each service has its own resource requirements and performance indicators (such as CPU utilization, network bandwidth, etc.). The size of the microservice latency reflects the number of unprocessed tasks in the queue. Existing resource allocation methods mainly include: (1) increasing the resources of the bottleneck microservice; (2) predicting the dependencies between microservices based on the network topology to allocate resources; (3) using a large amount of historical data for training to adjust resources. For example, researchers reduce the impact of bottleneck microservices by increasing the resources of the service through which the data flow with the longest end-to-end latency passes. However, the microservice on the path of the data flow with the longest latency is not necessarily the bottleneck service. When resources are sufficient, the longest queue only means that these microservices will take a relatively long time to process data, but it does not mean that the processing capacity of the microservice is the lowest. The bottleneck service will break the independence between microservices, greatly affect the efficiency of its downstream services, and then lead to low throughput of the entire system. However, how to identify bottleneck microservices and share resources while maintaining the independence between microservices and the stability of the microservice system is still a problem worth exploring in microservice design. The latter two methods often require time and resources to train a large amount of offline data and are highly dependent on intricate network topology relationships. However, the dynamic nature of network topology relationships greatly increases the difficulty of predicting dependency graphs.

[0004] In addition, the accuracy of service dependencies obtained by existing algorithms through prediction and feature analysis of time and space is difficult to guarantee. The resource allocation strategy obtained through offline analysis also has lags and does not take into account real-time data, which is insufficient to explain the rationality of resource allocation. Summary of the invention

[0005] The purpose of this application is to provide a resource allocation optimization method, system, storage medium and device for a microservice system, which can further improve the rationality of the resource allocation results of the microservice system, reduce the SLA violation rate of the data flow, and control its latency within the acceptable range of the SLA.

[0006] In a first aspect, the present application provides a method for optimizing resource allocation of a microservice system, the method comprising:

[0007] Control the rate at which data flows into a microservice system at the current moment; the microservice system includes a number of microservices;

[0008] Obtain the virtual queue lengths corresponding to the microservices at the current moment;

[0009] allocating resources to the data flow corresponding to the microservice based on the virtual queue length;

[0010] The queue length of the data flow at the current moment is updated to obtain the queue length of the data flow at the next moment.

[0011] In an implementation of the first aspect, controlling a rate at which data flows enter the microservice system at a current moment includes:

[0012] Obtain different control parameters according to the current congestion level of the microservice system;

[0013] The rate at which the data stream enters the microservice system at the current moment is controlled based on the control parameter.

[0014] In an implementation of the first aspect, obtaining virtual queue lengths corresponding to the plurality of microservices at a current moment includes:

[0015] Obtain the indicator weights at the current moment; the indicator weights include a first indicator weight of the queue length of the microservice, a second indicator weight of the SLA average delay violation rate, and a third indicator weight of the SLA 90% tail delay violation rate;

[0016] The virtual queue length corresponding to the current moment is obtained based on the indicator weight, the queue length of the microservice, the SLA average delay violation rate, and the SLA 90% tail delay violation rate.

[0017] In an implementation of the first aspect, the indicator weight is dynamically adjusted using a confidence interval upper bound algorithm; wherein:

[0018] The confidence interval upper bound algorithm obtains the indicator weight at the current moment based on the principle of maximizing average return.

[0019] In an implementation of the first aspect, allocating resources to the data flow corresponding to the microservice based on the virtual queue length includes:

[0020] Allocate resources to the corresponding microservice based on the virtual queue length;

[0021] Resources are allocated to each of the data flows corresponding to each of the microservices.

[0022] In an implementation of the first aspect, the method further includes:

[0023] After resources are allocated to each of the data flows corresponding to each of the microservices, for the remaining resources of the microservice system, based on the queue length of the microservice and the queue length of the data flow corresponding to the microservice, probability sampling is performed on the microservice and the data flow corresponding to the microservice in sequence to allocate the remaining resources.

[0024] In an implementation manner of the first aspect, updating the queue length of the data flow at a current moment includes:

[0025] The queue length of the data flow at the current moment is updated based on the queue length of the data flow at the current moment, the arrival rate and the departure rate of the data flow, and the queue length of the data flow at the next moment is obtained.

[0026] In a second aspect, the present application provides a resource allocation optimization system for a microservice system, characterized in that the system comprises:

[0027] A control module, used to control the rate at which data flows into the microservice system at the current moment; the microservice system includes a number of microservices;

[0028] An acquisition module, used to obtain the virtual queue lengths corresponding to the microservices at the current moment;

[0029] an allocation module, configured to allocate resources to the data flow corresponding to the microservice based on the virtual queue length;

[0030] The updating module is used to update the queue length of the data flow at the current moment to obtain the queue length of the data flow at the next moment.

[0031] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above-mentioned resource allocation optimization method of the microservice system.

[0032] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an electronic device, implements the resource allocation optimization method for the above-mentioned microservice system.

[0033] As described above, the resource allocation optimization method, system, storage medium and electronic device of the microservice system described in the present application have the following beneficial effects: the present application uses real-time delay and SLA feedback information to dynamically allocate resources, which is not only independent of complex network dependencies, but also does not require a large amount of offline analysis and prior knowledge learning. The present application adaptively adjusts resource allocation and performs flow control based on the current network load, the service's demand for resources and the maximum bearing capacity of the current network, and maximizes the maintenance of data flow delay within the SLA safety range. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1A Shown is a schematic diagram of an application scenario of the resource allocation optimization method for the microservice system described in this application.

[0035] Figure 1B Shown are structural diagrams of the client-cloud interaction scenarios in these implementations.

[0036] Figure 2A Shown is a schematic diagram of a stable rate data flow mode described in an embodiment of the present application in an embodiment.

[0037] Figure 2B It is a schematic diagram showing a non-steady rate data flow mode in an embodiment of the present application.

[0038] Figure 3 Shown is a flow chart of a method for optimizing resource allocation of a microservice system according to an embodiment of the present application in one embodiment.

[0039] Figure 4 A schematic diagram showing a microservice model schematic diagram described in an embodiment of the present application in one embodiment.

[0040] Figure 5 Shown is a flow chart of a method for optimizing resource allocation of a microservice system according to an embodiment of the present application in one embodiment.

[0041] Figure 6 Shown is a schematic diagram of a network topology structure of a microservice system described in an embodiment of the present application in one embodiment.

[0042] Figure 7 It is shown that the resource allocation optimization method of the microservice system described in the embodiment of the present application and the bottleneck method and the overall expansion method are Figure 6 Average SLA queue delay violation rate in the steady-rate data flow mode obtained by applying in the network.

[0043] Figure 8 It is shown that the resource allocation optimization method of the microservice system described in the embodiment of the present application and the bottleneck method and the overall expansion method are Figure 6 SLA 90% tail queue delay violation rate in steady rate data flow mode applied in the network.

[0044] Fig. 9 It is shown that the resource allocation optimization method of the microservice system described in the embodiment of the present application and the bottleneck method and the overall expansion method are Figure 6 Average queue delay in steady-rate data flow mode applied in the network.

[0045] Fig.10 It is shown that the resource allocation optimization method of the microservice system described in the embodiment of the present application and the bottleneck method and the overall expansion method are Figure 6 90% tail queue latency in steady-rate data flow mode applied in the network.

[0046] Fig.11 The resource allocation optimization method of the microservice system described in the embodiment of the present application is shown in Figure 6 The queue delay in the steady-rate data flow mode obtained by applying it in the network.

[0047] Fig.12 The resource allocation optimization method of the microservice system described in the embodiment of the present application is shown in Figure 6 90% tail queue latency in steady-rate data flow mode applied in the network.

[0048] Fig.13 It is shown that the resource allocation optimization method of the microservice system described in the embodiment of the present application and the bottleneck method and the overall expansion method are Figure 6 Average SLA queue delay violation rate in the non-steady rate data flow mode obtained by applying in the network.

[0049] Fig.14 It is shown that the resource allocation optimization method of the microservice system described in the embodiment of the present application and the bottleneck method and the overall expansion method are Figure 6 The SLA 90% tail queue delay violation rate in the non-steady rate data flow mode obtained by applying in the network.

[0050] Fig.15It is shown that the resource allocation optimization method of the microservice system described in the embodiment of the present application and the bottleneck method and the overall expansion method are Figure 6 The average queue delay in the non-steady rate data flow mode obtained by applying in the network.

[0051] Fig.16 It is shown that the resource allocation optimization method of the microservice system described in the embodiment of the present application and the bottleneck method and the overall expansion method are Figure 6 90% tail queue delay in the non-steady rate data flow mode obtained by applying it in the network.

[0052] Fig.17 The resource allocation optimization method of the microservice system described in the embodiment of the present application is shown in Figure 6 The queue delay in the non-steady rate data flow mode obtained by applying it in the network.

[0053] Fig.18 The resource allocation optimization method of the microservice system described in the embodiment of the present application is shown in Figure 6 90% tail queue delay in the non-steady rate data flow mode obtained by applying it in the network.

[0054] Fig.19 Shown is a schematic diagram of the structure of a resource allocation optimization system for a microservice system described in an embodiment of the present application in one embodiment.

[0055] Fig. 20 Shown is a schematic structural diagram of an electronic device of the present application in one embodiment. DETAILED DESCRIPTION

[0056] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0057] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present application in a schematic manner, and therefore the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0058] In addition, in this application, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0059] Microservice systems have been increasingly widely used in the cloud service field due to their advantages such as easy development and deployment, high scalability, good fault tolerance, and low maintenance costs. Compared with the traditional monolithic architecture, the microservice architecture breaks down large projects into functionally independent and easy-to-manage components. These components transmit information through application programming interfaces or message queues to collaborate to complete the functions of the entire system. The service level agreement defines the various indicators that the data flow must meet, among which whether the end-to-end latency meets the SLA is the most concerned issue for developers. A reasonable resource allocation system can effectively reduce the end-to-end latency of each data flow, which is crucial to the smooth operation of the system. With the popularization of microservices, how to optimize the resource allocation of these microservice components has become particularly important.

[0060] In order to at least solve the above-mentioned technical problems, an embodiment of the present application provides a resource allocation optimization method for a microservice system. By applying the resource allocation optimization method for a microservice system provided by an embodiment of the present application, resource allocation can be adaptively adjusted and flow control can be performed according to the current network load, the service's demand for resources, and the maximum bearing capacity of the current network, so as to maintain the data flow delay within the SLA safety range to the maximum extent.

[0061] The resource allocation optimization method of the microservice system provided in the embodiment of the present application can be applied to Figure 1A The resource allocation optimization device of the microservice system shown in FIG. Figure 1A As shown, the resource allocation optimization device 1 of the microservice system includes a processor 11 and a database 12. The resource allocation optimization method of the microservice system provided in the embodiment of the present application can be applied to the processor 11.

[0062] in, Figure 1A The processor 11 in the embodiment may be a single processor or a processor cluster or a cloud computing center composed of multiple processors, and the specific details are not limited here. Figure 1A Only one processor 11 and one database 12 are shown, but it should be understood that Figure 1AThe examples are only used to understand the present solution, and the specific numbers of processors 11 and databases 12 should be flexibly determined based on actual conditions.

[0063] In some other embodiments, the resource allocation optimization device 1 of the microservice system may not include the database 12, but only include a processor 11 with a storage function. The resource allocation optimization method of the microservice system provided in the embodiment of the present application can be applied to the processor 11. The processor 11 with a storage function may include a tablet computer, a laptop computer, a PDA, a mobile phone, a personal computer (Personal Computer, referred to as PC), etc., which is not limited here.

[0064] In still other embodiments, the resource allocation optimization method for the microservice system described in the present application can be applied to end-cloud interaction scenarios. Figure 1B The following is a schematic diagram of the structure of the client-cloud interaction scenario in these implementations. Figure 1B As shown, the terminal-cloud interaction system 2 includes a terminal 20 and a cloud server 21. The terminal 20 and the cloud server 21 can communicate with each other, and the communication method is not limited to wired or wireless.

[0065] Among them, the terminal 20 can be mobile or fixed, for example, the terminal 20 can be a wireless terminal or a wired terminal. The wireless terminal can refer to a device with wireless transceiver function, which can be deployed indoors, outdoors and in industrial workshops. The terminal 20 can be a mobile phone, a tablet computer, a laptop computer, etc., which is not limited here. The cloud server 21 can include one or more servers, or one or more processing nodes, or one or more virtual machines running on the server. The cloud server 21 can also be called a server cluster, a management platform, a resource allocation optimization center of a microservice system, etc., which is not limited in the embodiments of the present application.

[0066] In order to clearly describe the technical solutions of the embodiments of the present application, the nouns and terms involved in the embodiments of the present application are first defined:

[0067] End-to-end latency: refers to the time it takes for a data stream to enter the network and leave the network (including queuing time and data processing time).

[0068] SLA average latency: the average end-to-end latency specified in the service level agreement.

[0069] SLA 90% tail latency: The value specified in the service level agreement, indicating that the latency of 90% of the data flows is lower than this threshold, and the latency exceeding this value is considered high latency.

[0070] Stable rate data flow mode: refers to the data flow rate at different times fluctuating in a small range around the mean, such as Figure 2A .

[0071] Unsteady rate data flow mode: refers to the data flow rate that varies greatly over time, such as Figure 2B This is a typical diurnal variation pattern.

[0072] It should be noted that the queue length is used in the embodiment of the present application to replace the delay specified by the SLA in the real microservice system, which is called queue delay in the embodiment of the present application. For example, the SLA average queue delay is synonymous with the SLA average delay, and the SLA 90% tail queue delay is synonymous with the SLA 90% tail delay. In addition, the virtual queue length, as the basis for resource allocation, is an estimate of the real delay of the data flow in the microservice system.

[0073] The technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application.

[0074] The following embodiments of the present application provide a method for optimizing resource allocation of a microservice system. For example, Figure 1A The processor 11 or Figure 1B It is implemented by the cloud server 21 shown. Figure 3 The flowchart of the resource allocation optimization method of the microservice system described in the embodiment of the present application is shown as follows: Figure 3 As shown, the resource allocation optimization method of the microservice system includes steps S1 to S4.

[0075] S1. Control the rate at which data flows into a microservice system at the current moment; the microservice system includes a plurality of microservices.

[0076] In some embodiments, the microservice system is represented as a directed acyclic graph in It is a set of nodes, each node represents a microservice; is a collection of links, and the two endpoints of each link represent services with mutual dependencies. The total number of nodes (microservices) and links are N and L respectively. is a collection of data streams, is the set of data flows flowing through microservice n. Figure 4 As shown, each service maintains a queue for each type of data flow passing through it, such as Indicates the queue length of data flow f2 in service 1.

[0077] In some embodiments, at each time t, the microservice system generates a set of mean-determined data flow rates λ(t)=[λ f1 (t), λ f2 (t), λf3 (t)…]. The purpose of the embodiment of the present application is to minimize the total queue delay in the network by optimizing the resource allocation of the microservice system. Therefore, the present application will cleverly use the gap between the current queue length and the SLA queue delay according to the congestion level of the microservice system at the current moment, obtain different control parameters, and control the actual rate at which the data flow enters the microservice system at the current moment based on the control parameters, so as to avoid increasing the congestion level of the microservice system and increasing the risk of SLA violation.

[0078] In some embodiments, for each data flow, after it reaches the source node sf of the microservice system, the actual rate of its entry into the microservice system is controlled by maximizing the value of the following utility function (1), and the remaining packets are queued at the inlet microservice:

[0079] max(ave f -e f (t))logx f (t)-εq sf x f (t)(1)

[0080] Among them, ave f is the average SLA queue delay of data flow f, e f (t) is the queue delay of data flow f at the current time t, x f (t) is the rate at which data flow f actually enters the microservice system at the current time t, and ε is a hyperparameter used to control the number of data packets allowed to enter the system; q sf is the queue length of data flow f at the source node.

[0081] It can be seen that ave f -e f (t) represents the difference between the SLA average queue delay and the current data flow queue delay, that is, the current congestion level of the current microservice system. By solving equation (1), the present application can obtain the x of each data flow entering the microservice system. f (t), thereby controlling the rate at which data flows actually enter the microservice system at the current moment.

[0082] In order to strictly guarantee SLA and avoid potential SLA violations, and timely control large queue delays, this application obtains different control parameters according to the current congestion level of the microservice system, sets a transition zone for high-latency data flows, and flexibly controls the actual rate x of data flows entering the microservice system at the current moment under different degrees of queue delay. f (t), as follows:

[0083] When ave f -e f When (t)≤α, x f (t) = 0;

[0084] When ave f -e f When (t)≤β,

[0085] When ave f -e f When (t)>β,

[0086] Among them, α and β are delay thresholds (α<β) defined according to network structure characteristics and SLA rules. θ1 and θ2 ensure that the amount of data entering the network is within the processable range of the microservice system. In some embodiments, the values ​​of α and β are 50 and 100 respectively; the values ​​of θ1 and θ2 are 10 and 100 respectively.

[0087] As described above, the present application customizes different control parameters according to the congestion level of the microservice system at the current moment to control the rate at which data flows enter the microservice system. In this way, when there are many data flows in a waiting state and the current queue length is long (delay is large), the rate at which data flows enter the system is reduced, and vice versa. Such a negative feedback regulation mechanism ensures that the utility of existing resources is maximized, and while avoiding violations of SLA, it also reduces the potential noise impact caused by congestion (such as excessively long queues may increase the response time of the service).

[0088] S2. Obtain the virtual queue lengths corresponding to the microservices at the current moment.

[0089] When allocating resources, the microservice system in the embodiment of the present application needs to obtain the virtual queue length of the microservice, that is, to estimate the actual delay of the microservice, and then allocate resources based on the estimated actual delay to timely reduce the risk of SLA violations. Figure 5 The flowchart of the resource allocation optimization method of the microservice system described in the embodiment of the present application is shown as follows: Figure 5 As shown, obtaining the virtual queue lengths corresponding to the microservices at the current moment includes steps S21 to S22.

[0090] S21. Obtain the indicator weights at the current moment; the indicator weights include a first indicator weight of the queue length of the microservice, a second indicator weight of the SLA average delay violation rate, and a third indicator weight of the SLA 90% tail delay violation rate.

[0091] In some embodiments, the indicator weight is dynamically adjusted using a confidence interval upper bound algorithm, which includes: the confidence interval upper bound algorithm obtains the indicator weight at the current moment based on the average benefit maximization principle. That is, the joint benefit of the exploration and utilization of the indicator weight is obtained, and the optimal indicator weight at the current moment is selected accordingly.

[0092] In some embodiments, the first indicator weight is set to w1, the second indicator weight is set to w2, and the third indicator weight is set to w3, and the confidence interval upper bound algorithm (UCB algorithm) is used to select the weight [w1, w2, w3] with the largest UCB value in a given weight set. The UCB value calculation rule of the i-th group of indicator weights [w1, w2, w3] at the current moment is:

[0093]

[0094] in, represents the average return of the i-th group of weights at the end of the t-1th moment (the previous moment), m i (t - ) represents the number of times the i-th group of weights is selected at the end of the t-1th moment (the previous moment). In fact, since the benefit has a lag, the embodiment of the present application uses the negative value of the new queue length after allocating resources based on the indicator weights of the previous moment as the utilization benefit of the UCB algorithm at the current moment, and calculates the UCB value in combination with the exploration gain of the weights to select the indicator weights at the current moment.

[0095] It should be noted that the initial return of each group of indicator weights is 0. For the average return of the i-th group of weights, please see the following formula (7).

[0096] It should be noted that weights that have not been selected will be given priority.

[0097] S22. Obtain the virtual queue length corresponding to the current moment based on the indicator weight, the queue length of the microservice, the SLA average delay violation rate, and the SLA 90% tail delay violation rate.

[0098] In some embodiments, after calculating and obtaining the indicator weight at the current moment, the present application uses the three indicators of the queue length of the microservice, the average SLA delay violation rate, and the SLA 90% tail delay violation rate, and combines the indicator weight at the current moment to make a weighted estimate of the actual delay of the microservice. The longer the queue length of the microservice, the greater the possibility of violating the SLA; similarly, the higher the SLA violation rate, the longer the queue length of the microservice.

[0099] In some embodiments, obtaining the virtual queue length corresponding to the microservice at the current moment is expressed as:

[0100]

[0101] in, represents the queue length of the microservice n at the current moment (the sum of the queue lengths of all data flows passing through microservice n), c f (t) represents the SLA average queue delay violation rate of data flow f, represents the sum of the SLA average queue delay violation rates of all data flows passing through microservice n, g f (t) represents the SLA 90% tail queue delay violation rate of data flow f, Represents the sum of the SLA 90% tail queue delay violation rates of all data flows flowing through microservice n.

[0102] Based on the microservice queue length, this application also takes the SLA average delay violation rate and the SLA 90% tail delay violation rate as consideration indicators for the actual delay of the microservice, further improving the accuracy of the estimation result (virtual queue length), and effectively narrowing the gap between the estimated delay and the actual delay. The resource allocation results obtained can also reduce existing and potential SLA violation risks more promptly.

[0103] S3. Allocate resources to the data flow corresponding to the microservice based on the virtual queue length.

[0104] In some embodiments, resources are allocated to the corresponding microservices based on the virtual queue length. That is, resources are allocated to each microservice in proportion to the virtual queue length (estimated latency) of each microservice. For microservice n (the total number of microservices is N), the amount of resources allocated to it is:

[0105]

[0106] in, is the virtual queue length corresponding to microservice n at the current moment, and Y is the total number of resources.

[0107] In some embodiments, resources are allocated to each data flow corresponding to each microservice. That is, after allocating resources to each microservice according to formula (4), resources are allocated proportionally to each data flow of each microservice. For a data flow f in microservice n (the set of data flows flowing through microservice n is ), the amount of resources allocated to it is:

[0108]

[0109] in, is the virtual queue length of data flow f in microservice n at the current moment.

[0110] From formula (4) and formula (5), it can be seen that the present application uses the virtual queue length corresponding to each microservice calculated by formula (3) as the basis for resource allocation, and allocates resources for each microservice. Then, according to the virtual queue length of each data flow in each microservice, resources are allocated for each data flow, thereby minimizing the total queue delay in the network.

[0111] Furthermore, since the total number of resources is Y, and equations (4) and (5) allocate resources in proportion according to the virtual queue length, this means that after allocating resources to each data flow corresponding to each microservice, the total number of resources is not necessarily fully allocated. At this time, for the remaining resources of the microservice system, based on the queue length of the microservice and the queue length of the data flow corresponding to the microservice, probability sampling is performed on the microservice and the data flow corresponding to the microservice in turn to allocate the remaining resources one by one.

[0112] In some embodiments, for the remaining resources, based on the queue length of the microservice and the queue length of the data flow corresponding to the microservice, firstly all microservices are respectively The probability of sampling is then used to sample the sampled microservices n. The probability of sampling each data stream is 1, and the number of its resources is increased by 1. The sampling steps are repeated until all remaining resources are allocated.

[0113] S4. Update the queue length of the data flow at the current moment to obtain the queue length of the data flow at the next moment.

[0114] Specifically, after the resources are allocated, the queue length of the data flow at the current moment needs to be updated to obtain the queue length of the data flow at the next moment, thereby effectively reducing the end-to-end delay of each data flow and ensuring the smooth operation of the microservice system.

[0115] In some embodiments, the queue length of the data flow at the current moment is updated based on the queue length of the data flow at the current moment, the arrival rate and the departure rate of the data flow, and the queue length of the data flow at the next moment is obtained, that is:

[0116]

[0117] in, represents the queue length of data flow f in microservice n at the current time t, represents the arrival rate of data flow f in microservice n at the current time t, The departure rate of data flow f in microservice n at the current time t.

[0118] in, That is, the departure rate (the number of packets leaving the data flow queue) of the data flow f in the microservice n at the current time t is the smaller value between the queue length of the data flow and its allocated resources (processing capacity).

[0119] For the source node, λ f (t) is the arrival rate of data stream f at the current time t.

[0120] It should be noted that in step S21, when obtaining the indicator weight at the current moment, the maximum value of UCB is calculated by the average benefit of the i-th group of weights at the end of the previous moment in formula (2), so as to obtain the indicator weight at the current moment. Therefore, when dynamically adjusting the indicator weight at the next moment using the confidence interval upper bound algorithm, the maximum value of UCB should be obtained based on the average benefit of the i-th group of weights at the current moment. That is, after updating the data flow queue length at the current moment, the benefit should be updated in time, that is, the negative value of the new queue length obtained after allocating resources based on this weight is used as the benefit of the UCB algorithm to obtain the indicator weight at the next moment. That is, for the i-th group of indicator weights selected in step S21 at the current moment, the updated benefit is:

[0121]

[0122] After the income is updated, the process proceeds to the next moment and repeats steps S1 to S4.

[0123] Therefore, the resource allocation optimization method of the microservice system provided in the embodiment of the present application uses real-time delay and SLA feedback information to dynamically allocate resources. This method is not only independent of complex network dependencies, but also does not require a large amount of offline analysis and learning of prior knowledge. The present application adaptively adjusts resource allocation and performs flow control based on the current network load, the service's demand for resources, and the maximum tolerance of the current network, to maximize the maintenance of data flow delay within the SLA safety range.

[0124] by Figure 6 Taking the network shown as an example, the embodiment of the present application builds a lightweight simulation system to simulate the operation of the microservice system. Among them, frontend, search, profile, recommendation, reservation, user, rate, and geo are microservice instances in the system. Figure 6 The data flow in contains backtracking, that is, services that have no direct dependencies on each other need to communicate through a common upstream service. Figure 6 The four data flows in are:

[0125] f1, whose path is (frontend, user, frontend, reservation), has an SLA average queue delay of 500 and an SLA 90% tail queue delay of 600.

[0126] f2, whose path is (frontend, search, geo, search, rate, search, frontend, reservation, frontend, profile), has an SLA average queue latency of 500 and an SLA 90% tail queue latency of 560.

[0127] f3, whose path is (frontend, recommendation, frontend, profile), has an SLA average queue delay of 520 and an SLA 90% tail queue delay of 600.

[0128] f4, whose path is (frontend, user), has an SLA average queue delay of 800 and an SLA 90% tail queue delay of 860.

[0129] exist Figure 6 In the network shown, using Figure 2A The steady rate and Figure 2B The non-steady rate data flow modes shown in the figure are simulated respectively. Figures 7 to 10 , Figures 13 to 16 The method provided in the embodiment of the present application is compared with two other microservice system resource allocation methods (the bottleneck method of identifying the service with the longest queue delay as the bottleneck service and the overall expansion method based on Jackson queuing theory). Figure 6 Comparison of the four data streams in .

[0130] in, Figures 7 to 10 is the result in steady rate data streaming mode, Figures 13 to 16 The results are in the non-steady rate data flow mode. Each method was tested 10 times, and each test contained 5000 iterations. The SLA average queue delay violation rate and SLA 90% tail queue delay violation rate are the average results of the 10 tests, and the average queue delay and 90% tail queue delay are the average values ​​of each test group in the same iteration.

[0131] The experimental results of this embodiment show that, whether in the stable rate data flow mode or the unstable rate data flow mode, the resource allocation optimization method of the microservice system proposed in the embodiment of this application performs significantly better than the other two methods in terms of SLA average queue delay violation rate, SLA 90% tail queue delay violation rate, average queue delay, and 90% tail queue delay, that is, this application achieves more reasonable resource allocation and more efficient resource utilization. In the embodiment provided by this application, the resource allocation optimization method of the microservice system proposed in this application makes Figure 6 The four data flows in the method do not violate the SLA average queue delay and the SLA 90% tail queue delay. In addition, compared with the bottleneck method and the overall expansion method, the data flow queue delay of this embodiment can converge to the vicinity of the SLA faster and does not exceed the SLA, while the data flow delay under the bottleneck method and the overall expansion method shows a continuous upward trend. It can be seen that the present application stabilizes the queue delay of heavily congested data flows within the acceptable range of the SLA by controlling the actual rate at which data flows enter the microservice system, thereby avoiding potential SLA violations.

[0132] Figure 11 to Figure 12 In the steady rate data flow mode, Figure 6 The average queue delay and 90% tail queue delay of the four data flows in each iteration. Figure 17 to Figure 18 In the non-steady rate data flow mode, Figure 6 The average queue delay and 90% tail queue delay of the four data flows in each iteration. Figure 6 In the network shown, due to the actual rate at which the data flow enters the system, the queue delay of the data flow converges to the vicinity of the SLA and always maintains a safe range of the delay threshold β with the SLA. In this way, the resource allocation optimization method for the microservice system proposed in this application strictly maintains the saturation state of the microservice system delay within an acceptable range while making full use of existing resources, effectively avoiding the risk of microservice data flow violating the SLA.

[0133] It should be noted that the resource allocation optimization method for the microservice system provided in the embodiment of the present application is applicable to any other network topology diagram. For different network dependencies or data flow rates, the algorithm parameters can be appropriately adjusted to adapt to different environments.

[0134] The protection scope of the resource allocation optimization method for the microservice system described in the embodiment of the present application is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing, or replacing steps in the prior art based on the principles of the present application are included in the protection scope of the present application.

[0135] The embodiment of the present application also provides a resource allocation optimization system for a microservice system, which can implement the resource allocation optimization method for the microservice system described in the present application. However, the implementation device of the resource allocation optimization system for the microservice system described in the present application includes but is not limited to the structure of the resource allocation optimization system for the microservice system listed in the present embodiment. All structural deformations and replacements of the prior art made according to the principles of the present application are included in the protection scope of the present application.

[0136] Fig.19 The structure diagram of the resource allocation optimization system of the microservice system described in the embodiment of the present application is shown as follows: Fig.19 As shown, the resource allocation optimization system 4 of the microservice system includes a control module 41 , an acquisition module 42 , an allocation module 43 and an update module 44 .

[0137] The control module 41 is used to control the rate at which data flows into the microservice system at the current moment; the microservice system includes a plurality of microservices.

[0138] The acquisition module 42 is used to obtain the virtual queue lengths corresponding to the microservices at the current moment.

[0139] The allocation module 43 is used to allocate resources to the data flow corresponding to the microservice based on the virtual queue length.

[0140] The updating module 44 is used to update the queue length of the data flow at the current moment to obtain the queue length of the data flow at the next moment.

[0141] It should be noted that the structures and principles of the control module 41, the acquisition module 42, the allocation module 43 and the update module 44 correspond one-to-one to the steps in the resource allocation optimization method of the above-mentioned microservice system, so they are not described in detail here.

[0142] In the several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules or units, which can be electrical, mechanical or other forms.

[0143] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0144] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0145] The present application embodiment also provides a computer-readable storage medium. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by a program to instruct the processor, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state hard disk (SSD)), etc.

[0146] The embodiment of the present application further provides an electronic device, wherein the electronic device includes a processor and a memory.

[0147] The memory is used to store computer programs.

[0148] The memory includes: ROM, RAM, disk, USB flash drive, memory card or CD and other media that can store program codes.

[0149] The processor is connected to the memory and is used to execute the computer program stored in the memory so that the electronic device executes the resource allocation optimization method of the microservice system.

[0150] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0151] like Fig. 20 As shown, the electronic device of the present application is in the form of a general computing device. The components of the electronic device may include but are not limited to: one or more processors or processing units 51, a memory 52, and a bus 53 connecting different system components (including the memory 52 and the processing unit 51).

[0152] Bus 53 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0153] Electronic devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0154] The memory 52 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 521 and / or cache memory 522. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 523 may be used to read and write non-removable, non-volatile magnetic media ( Fig. 20 not shown, usually called a "hard drive"). Although Fig. 20Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical medium) may be provided. In these cases, each drive may be connected to the bus 53 via one or more data medium interfaces. The memory 52 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.

[0155] A program / utility 524 having a set (at least one) of program modules 5241 may be stored, for example, in the memory 52, such program modules 5241 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 5241 generally perform the functions and / or methods of the embodiments described herein.

[0156] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, displays, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via input / output (I / O) interface 54. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 55. Fig. 20 As shown, the network adapter 55 communicates with other modules of the electronic device via the bus 53. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0157] The present application embodiment may also provide a computer program product, the computer program product including one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer or data center.

[0158] When the computer program product is executed by a computer, the computer executes the method described in the above method embodiment. The computer program product may be a software installation package, and when the above method is required, the computer program product may be downloaded and executed on a computer.

[0159] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0160] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A resource allocation optimization method for a microservice system, characterized in that: The method comprises: Control the rate at which data flows into a microservice system at the current moment; the microservice system includes a number of microservices; Obtain the virtual queue lengths corresponding to the microservices at the current moment; allocating resources to the data flow corresponding to the microservice based on the virtual queue length; The queue length of the data flow at the current moment is updated to obtain the queue length of the data flow at the next moment.

2. The resource allocation optimization method for a microservice system according to claim 1, characterized in that: Controlling the rate at which data flows into the microservice system at the current moment includes: Obtain different control parameters according to the current congestion level of the microservice system; The rate at which the data stream enters the microservice system at the current moment is controlled based on the control parameter.

3. The resource allocation optimization method for a microservice system according to claim 1, characterized in that: Obtaining the virtual queue lengths corresponding to the microservices at the current moment includes: Obtain the indicator weights at the current moment; the indicator weights include a first indicator weight of the queue length of the microservice, a second indicator weight of the SLA average delay violation rate, and a third indicator weight of the SLA 90% tail delay violation rate; The virtual queue length corresponding to the current moment is obtained based on the indicator weight, the queue length of the microservice, the SLA average delay violation rate, and the SLA 90% tail delay violation rate.

4. The resource allocation optimization method for a microservice system according to claim 3, characterized in that: Dynamically adjust the indicator weight using the confidence interval upper bound algorithm; including: The confidence interval upper bound algorithm obtains the indicator weight at the current moment based on the principle of maximizing average return.

5. The resource allocation optimization method for a microservice system according to claim 1, characterized in that: Allocating resources to the data flow corresponding to the microservice based on the virtual queue length includes: Allocate resources to the corresponding microservice based on the virtual queue length; Resources are allocated to each of the data flows corresponding to each of the microservices.

6. The resource allocation optimization method for a microservice system according to claim 5, characterized in that: The method further comprises: After resources are allocated to each of the data flows corresponding to each of the microservices, for the remaining resources of the microservice system, based on the queue length of the microservice and the queue length of the data flow corresponding to the microservice, probability sampling is performed on the microservice and the data flow corresponding to the microservice in sequence to allocate the remaining resources.

7. The resource allocation optimization method for a microservice system according to claim 1, characterized in that: Updating the queue length of the data flow at the current moment includes: The queue length of the data flow at the current moment is updated based on the queue length of the data flow at the current moment, the arrival rate and the departure rate of the data flow, and the queue length of the data flow at the next moment is obtained.

8. A resource allocation optimization system for a microservice system, characterized in that: The system comprises: A control module, used to control the rate at which data flows into the microservice system at the current moment; the microservice system includes a number of microservices; An acquisition module, used to obtain the virtual queue lengths corresponding to the microservices at the current moment; an allocation module, configured to allocate resources to the data flow corresponding to the microservice based on the virtual queue length; The updating module is used to update the queue length of the data flow at the current moment to obtain the queue length of the data flow at the next moment.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory so that the electronic device executes the resource allocation optimization method for the microservice system according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by an electronic device, the resource allocation optimization method for the microservice system according to any one of claims 1 to 7 is implemented.

Images