A resource management system for stateless microservice architecture

Abstract

The present invention provides a resource management system for a stateless microservice architecture, which is applied to a machine cluster based on a Linux operating system and a Kubernetes container orchestration platform. The system includes: a Linux kernel for specific system calls, which is installed in the cluster; a microservice resource manager, which corresponds one-to-one to a microservice and is installed on a control node of the cluster. The microservice resource manager is provided with a plurality of sub-managers, which are used to drive the corresponding sub-managers to generate buoys of the number of Pod copies in the corresponding state based on statistical data; a coordinator, which corresponds one-to-one to a microservice and is installed on a control node of the cluster. The coordinator is used to control the state transfer, creation, and deletion of the Pod of the corresponding microservice based on the buoys generated by the sub-managers; and a Pod resource manager is installed on each computing node of the cluster. The Pod resource manager is used to monitor the state changes of each Pod and perform corresponding Pod resource management operations. The system of the present invention can significantly improve the utilization of CPU and memory resources while ensuring the end-to-end delay service level.

Description

Technical Field

[0001] The present invention relates to the field of computer cloud computing resource management, and more specifically, to a resource management system for a stateless microservice architecture. Background Art

[0002] The agility, flexibility, and high reliability of cloud computing have driven the development of a new era of cloud computing, further accelerating the emergence of cloud computing concepts and technologies. One of its key developments, cloud native, aims to maximize the dynamic, elastic, and scalable nature of cloud computing environments to develop software. This allows for easy deployment in the cloud, providing stable and reliable network services and leveraging the flexibility of cloud computing to easily cope with fluctuations in request volume. Containerization and microservices architecture are key implementations of the cloud native concept. Containers avoid virtualization while effectively isolating a group of applications into a relatively independent space, greatly facilitating development and deployment. Microservices architectures enable cloud native applications to achieve excellent scalability and flexibility. Using the Kubernetes container orchestration platform to deploy network services has become an industry consensus, and resource management for containerized network service instances (called pods in Kubernetes) is becoming increasingly important.

[0003] Due to the Java language's simplicity, cross-platform nature, and high scalability, numerous Java-based programs are currently running on computer systems worldwide, including programs running in containers. However, the characteristics of the Java language require a warm-up period. This means that Java programs, due to the Java Virtual Machine (JVM), require a period of runtime before reaching peak performance. Therefore, if a Java-based program is running in a container, the container itself will also need to run for a while before reaching peak performance.

[0004] For latency-sensitive network services, maximizing the utilization of resources like CPU and memory while maintaining the hard end-to-end latency service level has always been the primary goal of existing resource management systems. Resource management systems typically improve resource utilization through autoscaling, allocating more resources to running network service instances during high loads and reclaiming excess resources during low loads. Currently, common autoscaling solutions include vertical autoscaling, horizontal autoscaling, and a combination of vertical and horizontal autoscaling. However, each of these approaches still has its own shortcomings.

[0005] The first approach is vertical autoscaling, which typically manages resources by increasing or decreasing the CPU or memory quota of a single Pod. For example, the paper "A Practical Two-Tier Approach to SLO-Oriented Microservice Resource Management" (Lee K, Lee K, Park H, et al. Autothrottle: Satisfying Network Performance Requirements for Containers [J]. IEEE Transactions on Cloud Computing, 2022) uses a responsive strategy to manage CPU resources. This approach has the advantage of being able to quickly adjust resources in response to the current load. In particular, the cost of adjusting CPU resources is minimal, allowing for rapid scaling of the CPU. However, adjusting the memory of a Pod requires creating a new Pod replica, resulting in weak memory resource management capabilities. Furthermore, the scale of vertical scaling of a single container is limited by the maximum resources of a single physical machine, so the maximum capacity expansion is also relatively limited.

[0006] The second approach is horizontal autoscaling, which typically scales microservices by increasing or decreasing the number of pod replicas. For example, the paper "Adaptive Horizontal Pod Autoscaling Systems on Alibaba Cloud Container Service for Kubernetes" (Zhou Z, Zhang C, Ma L, et al. AHPA: Adaptive Horizontal Pod Autoscaling Systems on Alibaba Cloud Container Service for Kubernetes[J].arXiv preprint arXiv:2303.03640,2023.) uses a predictive scaling strategy to manage the number of pod replicas. This approach is the easiest and most common autoscaling solution. Microservice scaling can be achieved by simply increasing or decreasing the number of pod replicas and is not limited by the resource size of a single physical machine. In theory, a well-designed microservice can scale indefinitely. However, its drawbacks are that creating pod replicas takes time for initialization, which is costly, and the response time is slow, resulting in a lag in real-time load response. When the load fluctuates, repeated creation and deletion operations within a short period of time waste significant resources.

[0007] The third approach is a combined horizontal and vertical autoscaling solution. For example, the paper "Resource-efficient provisioning with tail latency guarantees for microservices" (Cai B, Wang B, Yang M, et al. AutoMan: Resource-efficient provisioning with tail latency guarantees for microservices [J]. Future Generation Computer Systems, 2023, 143:61-75) uses a reinforcement learning algorithm to adaptively control the horizontal and vertical resource scaling of microservices. This approach combines the advantages of both solutions one and two, enabling rapid adjustment of CPU resources to load fluctuations within a short period of time, and the upper limit of capacity expansion is not limited by the upper limit of a single physical machine's resources. However, this approach also introduces greater operational complexity, and its biggest challenge is the inability to quickly determine the optimal combined horizontal and vertical scaling method. Therefore, previous research typically builds black-box models for this problem and then uses heuristic methods or reinforcement learning to solve it. However, these approaches often face high barriers to deployment and debugging, hindering widespread application. Summary of the Invention

[0008] The present invention aims to overcome at least one defect (shortcoming) of the above-mentioned prior art and provide a resource management system for a stateless microservice architecture, which can improve the utilization of CPU and memory resources of microservices while ensuring service quality, thereby saving resource usage overhead.

[0009] The technical solution adopted by the present invention is a resource management system for a stateless microservice architecture, which is applied to a machine cluster based on the Linux operating system and the Kubernetes container orchestration platform. The system includes:

[0010] A Linux kernel, used for specific system calls, installed in the machine cluster;

[0011] A microservice resource manager is installed on the control node of the machine cluster and is provided with several sub-managers. It is used to drive the corresponding sub-managers to generate buoys of the number of Pod copies in the corresponding state based on statistical data. Each microservice corresponds to a microservice resource manager;

[0012] A coordinator, installed on the control node of the machine cluster, is used to control the state transfer and creation and deletion of the Pod of the corresponding microservice according to the buoys generated by the sub-managers. Each microservice corresponds to a coordinator;

[0013] The Pod resource manager is installed on each computing node in the machine cluster and is used to monitor the status changes of each Pod and perform corresponding Pod resource management operations.

[0014] In the above solution, the sub-managers include responsive sub-managers, short-term predictive sub-managers, and long-term predictive sub-managers;

[0015] The responsive sub-manager is configured to generate an initial value of the first buoy w1 in a responsive manner based on the resource usage of the microservice over a period of time;

[0016] The short-term prediction sub-manager is used to perform short-term prediction using the interval prediction algorithm EnbPI and the single-step prediction algorithm SVR to generate an initial value of the second buoy w2;

[0017] The long-term prediction sub-manager is used to perform long-term prediction using the periodic prediction algorithm Prophet, the interval prediction algorithm EnbPI and the single-step prediction algorithm SVR to generate an initial value of the third buoy w3.

[0018] In the above solution, the running states of the Pod instance include initialization state, preheating state, running state, suspended-level 1 state, and suspended-level 2 state;

[0019] The initialization state is a state in which the Pod is undergoing initialization operations after creation and cannot be used to provide service computing;

[0020] The preheating state is a state in which the Pod is used to provide service computing after initialization is completed, but its execution speed and response time are not optimal;

[0021] The running state is the state in which the Pod executes the corresponding request normally at the highest execution speed and response time;

[0022] The suspended-level 1 state is a state in which part or all of the CPU of the Pod is reclaimed and no new load requests are accepted;

[0023] The suspended-level 2 state is a state in which most of the memory of the Pod is reclaimed based on the suspended-level 1 state;

[0024] The first buoy w1 is used to indicate the number of Pods in the running state of the microservice;

[0025] The second buoy w2 is used to indicate the sum of the number of Pods in the running state and the suspended-level 1 state of the microservice;

[0026] The third buoy w3 is used to indicate the sum of the number of Pods in all states of the microservice.

[0027] In the above scheme,

[0028] The coordinator decides whether to perform the buoy value repair operation based on the size relationship between the initial values ​​of the first buoy w1, the second buoy w2, and the third buoy w3. Specifically:

[0029] Under normal circumstances, the relative size relationship of the three buoy values ​​is w1≤w2≤w3;

[0030] In abnormal situations, if the relative size relationship of the three buoy values ​​does not satisfy w1≤w2≤w3, the coordinator will correct the values ​​of each buoy to satisfy the normal relative size relationship;

[0031] Under the premise that w1≤w2≤w3 is satisfied, the coordinator adjusts the status of the corresponding Pod according to the state transition logic so that the number of replicas of each Pod state is consistent with the number of corresponding buoy values. The logic of the coordinator correcting the value of each buoy is as follows:

[0032] use Indicates the values ​​of the first buoy w1, the second buoy w2 and the third buoy w3 generated last time, Indicates the values ​​of the first buoy w1, the second buoy w2 and the third buoy w3 currently generated. Must be satisfied Quantitative relationship of

[0033] First, The smaller value is reassigned Assign the larger value to After this operation, and Already satisfied Quantitative relationship of

[0034] like Will The larger value is assigned and After correction and Already satisfied The quantitative relationship ends the repair logic;

[0035] like Will The larger value is assigned After correction and Already satisfied The quantitative relationship ends the repair logic;

[0036] After the repair logic, Must be satisfied At this time, the subsequent operations can be continued according to the normal operation logic of the relative size of the buoy values.

[0037] In the above solution, the coordinator adjusts the status of the corresponding Pod according to the state transition logic so that the number of replicas of each Pod state is consistent with the number of corresponding buoy values. Specifically:

[0038] like First create a quantity of Pod; if Then first delete the number The priority of deleting Pods is in the following states: Initialization, Preheating, Suspended-Level 2, Suspended-Level 1, and Running.

[0039] Then prioritize adjusting the number of running Pods to When the number of running pods needs to be increased, the priority for converting pods in various states to the running state is Pending-Level 1, Pending-Level 2, and Preheating. When the number of running pods needs to be reduced, the excess pods are directly converted to Pending-Level 1.

[0040] When the number of running Pods is After the number matches, start adjusting the number of Pods in the pending-level 1 state to When the number of Pods in the Pending-Level 1 state needs to be increased, Pods in various states are converted to the Pending-Level 1 state with priority given to the Pending-Level 2 state and the Preheating state. When the number of Pods in the Pending-Level 1 state needs to be reduced, the excess Pods are converted to the Pending-Level 2 state.

[0041] In the above solution, the Pod Resource Manager controls the resources of a specific Pod according to the following logic:

[0042] To control CPU resources, the Pod Resource Manager counts the CPU usage time of each Pod over the past period of time, as well as the total time when the Pod requested CPU but received no response. This total time is called CPU duration, and the Pod Resource Manager adjusts the corresponding Pod's resources according to the following logic:

[0043] If the Pod status remains in the Running state, the CPU duration limit of the Pod is set to the maximum CPU duration within the time window. However, if the maximum CPU duration limit exceeds the maximum available CPU duration set for the Pod, the maximum available CPU duration set for the Pod is used.

[0044] If the Pod state changes from Running to Suspended-Level 1 or Suspended-Level 2, the CPU duration limit of the Pod is set to (1+) times the average CPU duration in the time window; γ is a hyperparameter;

[0045] If a Pod transitions directly from Pending-Level 1 or Pending-Level 2 to Running, its resources are set to the maximum available CPU resources of all Pods in the same microservice.

[0046] If the number of running Pods decreases within this control loop, the number of running Pods before the decrease is p, and the number of running Pods after the decrease is q, then the CPU time limit of all Pods is set to the current CPU time limit and The larger value of the CPU duration at the previous moment.

[0047] Set the burst CPU usage of all Pods to the maximum available CPU usage of the Pod minus the CPU usage limit of the current Pod;

[0048] and / or,

[0049] In terms of memory resource control, the Pod Resource Manager counts the number of page faults in Pods over a period of time and adaptively allocates available memory to Pods in the Suspended-Level 2 state based on a limit of h page faults. The Pod Resource Manager adjusts the available memory size of Pods using the following policies:

[0050] For a Pod that enters the suspended-level 2 state for the first time, set its available memory to rMB;

[0051] For Pods that are already in the Suspended Level 2 state, if the average number of page faults for the Pod over the past period is greater than h, rMB of memory resources will be added to the Pod.

[0052] For the Pod that leaves the suspended-level 2 state for the first time, its available memory is set to the original maximum memory usage, and the specific system call is called to swap all memory pages remaining on the hard disk into the memory.

[0053] In the above solution, the responsive sub-manager and the short-term predictive sub-manager generate the first buoy w1 and the second buoy w2 in the following manner:

[0054] The current buoy value is determined by the proportional relationship between the previous buoy value and the current statistical data. Its mathematical description is:

[0055]

[0056] In formula (1), w x It can represent any one of w1 and w2. represents the buoy w at time t x The value of represents the buoy w at time t+1 xThe value of M 当前 Indicates the current value of a certain indicator M, M 标准 Indicates the ideal value of indicator M;

[0057] Among them, M 当前 With w x There is a negative correlation, that is, when other variables are the same, the larger the w x Corresponding to smaller M 当前 If the selected indicator does not have this negative correlation, mathematical changes are made to make it satisfy this relationship.

[0058] M 标准 is a preset value, the indicator is an indicator related to the Pod, and the indicator M adopted by the responsive sub-manager is different from that adopted by the short-term predictive sub-manager;

[0059] The responsive sub-manager calculates the indicator changes within the sliding time window, takes the average value, and adds the product of the preset percentile threshold a and the standard deviation as its M 标准 One of the sources is as shown in formula (2):

[0060] (2)

[0061] In formula (2), Represents the average value of indicator M in a time window of size m, Indicates the percentile threshold of indicator M in a time window of size m, represents the standard deviation of the indicator M in the time window of size m, and v1 is the standard deviation of the indicator M generated by the responsive sub-manager. 当前 One of the important sources of

[0062] (3)

[0063] In formula (3), represents the maximum value of the indicator M within the time window of size n, and v2 is the maximum value of the responsive sub-manager M generated by 当前 Another important source of

[0064] M 当前 =max(v1,2); (4)

[0065] The M current of the responsive sub-manager is obtained by formula (2)(3)(4). Substituting the calculated M current into formula (1), the corresponding buoy of the microservice at the next moment can be obtained.

[0066] In the above scheme, the short-term prediction sub-manager uses SVR as the underlying regression algorithm for single-step prediction, and then uses the EnbPI algorithm for interval prediction. The short-term prediction sub-manager continuously collects V+N indicators over a period of time as a new set of training data for SVR. The first V indicators constitute the input value of the training data, and the maximum value of the next N indicators serves as the label value of the training data set. This set of data is then added to the training data. When the number of training data sets meets the minimum requirement for the number of training sets, the SVR prediction set is obtained through the following two methods:

[0067] ① When the number of collected training data sets first meets the training requirements, a total of B new SVRs are trained on the collected training data sets to obtain the initial SVR prediction set;

[0068] ② In the case of an existing SVR prediction group, delete the oldest set of training data and the k SVR predictors trained using the oldest data in the EnbPI algorithm, and then train k new SVR predictors on the new training data, so that the number of predictors in the SVR prediction group remains unchanged.

[0069] According to the execution mode of EnbPI algorithm, the confidence level α is set, and the existing SVR prediction group is used to predict the maximum value of the future N indicators as y through the recent V indicators, and the probability that the true maximum value does not exceed the maximum value y of the predicted value is

[0070] The short-term prediction sub-manager uses the maximum value y of the prediction value of the above EnbPI algorithm and SVR algorithm as its M 当前 , substituted into formula (1) to obtain the value of the second buoy w2.

[0071] In the above solution, the long-term prediction sub-manager first uses the Prophet algorithm to directly predict the number of Pod replicas required at the next moment based on the previous load request cycle changes, which serves as the initial value of the third buoy w3;

[0072] If the value of the third buoy w3 generated by the long-term prediction formula sub-manager is smaller than the value of the second buoy w2 generated by the short-term prediction formula sub-manager, the value of the third buoy w3 is corrected.

[0073] In the above solution, the value of the third buoy w3 is corrected by combining EnbPI and SVR, specifically:

[0074] Using the method combining EnbPI and SVR, the confidence β is set, and the predicted difference between the second buoy w2 and the original third buoy w3 is z. The original third buoy w3 plus the predicted value z is used as the initial value of the new third buoy w3.

[0075] Compared with the prior art, the present invention has the following beneficial effects:

[0076] 1) This invention implements a combined horizontal and vertical container scaling method by configuring three custom components in the corresponding control nodes of the machine cluster: a microservice resource manager, a coordinator, and a Pod resource manager. This significantly enhances the resource management capabilities of the Kubernetes container orchestration platform. The stateless microservice architecture system of this invention significantly improves CPU and memory resource utilization while maintaining end-to-end latency service levels.

[0077] 2) The Pod in this invention has five states, of which the Preheat state, Suspend-Level 1 state, and Suspend-Level 2 state are the states introduced in this invention. The Preheat state preheats the Pod in advance, significantly reducing the end-to-end latency spike caused by the program not being preheated. The Suspend-Level 1 and Suspend-Level 2 states, combined with a hierarchical resource recovery strategy, enable the recycling of multiple resources at different frequencies, improving the efficiency of resource recovery and reallocation.

[0078] 3) The specific system call described in the present invention can swap the memory pages swapped out of the program to the hard disk into the memory in advance at one time. Combined with the predictive swap algorithm, it can effectively reduce the impact of the memory swap mechanism on program performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 This is a schematic diagram of the distribution of system components in a resource management system for a stateless microservice architecture according to the present invention.

[0080] Figure 2 This is a flow chart of the deployment and operation steps of a resource management system for a stateless microservice architecture of the present invention.

[0081] Figure 3 This is a schematic diagram of the Pod state transition of the present invention.

[0082] Figure 4 Schematic diagram of a complete buoy of the present invention.

[0083] Figure 5 This is a schematic diagram of the buoy during stability of the present invention. DETAILED DESCRIPTION

[0084] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present invention. To better illustrate the following embodiments, some components in the accompanying drawings may be omitted, enlarged, or reduced in size, and do not represent actual product dimensions. Those skilled in the art will appreciate that some well-known structures and their descriptions may be omitted from the accompanying drawings.

[0085] Example 1

[0086] This embodiment provides a resource management system for a stateless microservice architecture, which can be applied to a machine cluster in a Linux operating system and Kubernetes container orchestration platform environment; preferably, it is applied to a machine cluster built based on a Linux kernel operating system with kernel version 3.11 or above and a Kubernetes container orchestration platform with version 1.22 or above; the resource management system described in this embodiment controls the status of the Pod and further controls the available resources of each Pod by setting three custom components in the corresponding control node of the machine cluster: a microservice resource manager, a coordinator, and a Pod resource manager; introduces different adjustment frequencies and adjustment mechanisms for different types of resources, and manages the CPU, memory and other resources in the system in a more targeted manner, thereby greatly improving resource utilization and saving resource usage overhead.

[0087] Specifically, if Figure 1 As shown, the system includes:

[0088] A Linux kernel (not shown in the figure), used for specific system calls, which is installed in the machine cluster;

[0089] A microservice resource manager is installed on the control node of the machine cluster and is provided with several sub-managers. It is used to drive the corresponding sub-managers to generate buoys of the number of Pod copies in the corresponding state based on statistical data. Each microservice corresponds to a microservice resource manager;

[0090] A coordinator, installed on the control node of the machine cluster, is used to control the state transfer and creation and deletion of the Pod of the corresponding microservice according to the buoys generated by the sub-managers. Each microservice corresponds to a coordinator;

[0091] The Pod resource manager is installed on each computing node in the machine cluster and is used to monitor the status changes of each Pod and perform corresponding Pod resource management operations.

[0092] Specifically, such as Figure 2 As shown, the deployment and operation of the resource management system includes the following steps:

[0093] S1: Install the Linux kernel with specific system calls in the machine cluster where the system is deployed;

[0094] Specifically, in this embodiment, the name of the specific system call is reload_swappage. Adding a specific system call to the Linux kernel means adding a new system call without changing the original Linux kernel system call, which is used to swap in all memory pages of a specific process that have been swapped out to the hard disk. The swapped-out memory pages refer to memory pages that are temporarily swapped to the disk using memory swapping technology in a computer system that uses virtual addresses. The swap-in operation refers to the operation of swapping the memory pages swapped to the disk back into the memory using memory swapping technology. Therefore, the specific function of the specific system call is to reload all memory pages of a specific process into the memory.

[0095] Preferably, in this step, the PID may be used to obtain the Linux kernel process descriptor first, and then the shmem_unuse function in the Linux kernel code may be used to swap the memory page of the corresponding program into the memory.

[0096] S2: Install a microservice resource manager and coordinator for each microservice on the control node in the machine cluster;

[0097] Specifically, in this embodiment, the control node is a control node in the Kubernetes container orchestration platform in the machine cluster, which is used to deploy the core components of the machine cluster. It is not responsible for executing specific microservice calculations, but is only used to control the operation of the entire Kubernetes container orchestration platform itself.

[0098] The microservice resource manager is an important system component in the present invention. Figure 1 As shown in the "Microservice Resource Manager" component, each microservice resource manager corresponds to a specific microservice deployed in the cluster. This means the number of microservice resource managers is equal to the number of microservices deployed in the cluster. This component controls the number of replicas of pods in each state within the corresponding microservice. The microservice resource manager is configured with several sub-managers.

[0099] The coordinator is an important system component in the present invention, such as Figure 1 As shown in the "Coordinator" section, the coordinator corresponds one-to-one with the specific microservices deployed in the machine cluster, that is, the number of coordinators is equal to the number of microservices deployed in the machine cluster. This component is used to coordinate the buoy values ​​given by several sub-managers in the microservice resource manager. When the buoy values ​​given by the sub-managers do not conflict, it directly controls the number of replicas of the Pod in each state of the corresponding microservice. When the buoy values ​​given by the sub-managers conflict, it first coordinates the initial buoy values, and then controls the number of replicas of the Pod in each state of the corresponding microservice.

[0100] S3: Install the Pod resource manager on each computing node in the machine cluster;

[0101] Specifically, in this embodiment, the computing node is a computing node in the Kubernetes container orchestration platform in the cluster, which is used to deploy instances of various microservices and execute the computing logic of the corresponding microservices.

[0102] The Pod resource manager is an important system component in the present invention. Figure 1 As shown in the "Pod Resource Manager" in the figure, the Pod Resource Manager is deployed in the compute nodes of the machine cluster, and each compute node is deployed with only one Pod Resource Manager. The Pod Resource Manager directly communicates with the operating system of the compute node based on the status of the Pod and adjusts the available resources of the corresponding Pod, which in this embodiment are CPU resources and memory resources.

[0103] S4: Each of the microservice resource managers drives its corresponding sub-manager to generate a buoy indicating the number of Pod replicas in the corresponding state based on the statistical data;

[0104] S5: The coordinator controls the state transfer of the Pod of the corresponding microservice and the creation and deletion of the Pod according to the indicator buoys generated by the three sub-managers;

[0105] S6: The Pod resource manager monitors the status changes of each Pod and performs corresponding Pod resource management operations.

[0106] Specifically, in this embodiment, the state refers to a label used by the system of the present invention to mark the number of requests that a Pod should currently be allocated and the number of available resources.

[0107] like Figure 3 As shown. In this embodiment, Pods have five states: Initializing, Warming, Running, Idle-Level 1, and Idle-Level 2. Specifically, the state label values ​​can be set to Initializing, Warming, Running, L1Suspended, and L2Suspended. The coordinator identifies the state of each Pod by controlling the label value of genesis.io / state. There is a transition relationship between states, and state transitions must follow the transition relationship between states.

[0108] The transfer relationship refers to the rules that the Pod needs to follow when its status changes, such as Figure 3As shown, the two states can transition in the direction of the arrow. An example of a feasible state transition is transitioning from the Running state to the Suspended-Level 1 state. Multiple state transitions can occur within a control cycle. An example of a feasible multiple state transition is transitioning directly from the Running state to the Suspended-Level 2 state. An infeasible example is transitioning from the Running state to the Preheating state. Since there is no arrow pointing from the Running state to the Preheating state, the Running state cannot transition to the Preheating state.

[0109] In this embodiment, the control cycle refers to the period from steps S4 to S6 during which each component completes a round of operations. This cycle begins when the microservice resource manager analyzes statistical data and ends when the pod resource manager adjusts resources for each pod. Each microservice has its own unique control cycle, and its process execution is not affected by the control cycles of other microservices.

[0110] The creation operation refers to the operation of the Kubernetes container orchestration platform to create a new container instance based on the container template.

[0111] The initialization state refers to the state in which Kubernetes is performing initialization operations on a new Pod after it is created. In the initialization state, Kubernetes needs to perform operations such as allocating an IP address, allocating storage volumes, and setting environment variables for the Pod. The Pod in this state cannot execute the business logic of the corresponding microservice, so load requests cannot be forwarded to the Pod in the initialization state.

[0112] The warm-up state refers to a state in which a Pod can execute the business logic of the corresponding microservice after completing initialization, but its execution speed and response time are not optimal. This state is commonly seen in programs written in Java. Not all Pods experience this state; Pods that do not require warm-up can skip this state. In some cases, such as when the Pod contains a JVM, it takes a while to reach its optimal operating state. Therefore, the warm-up state refers to the state of the Pod after completing initialization but before reaching its optimal operating state.

[0113] The running state refers to the state in which the Pod can normally execute the corresponding request at the highest execution speed and response time. In this system, the vast majority of load requests will be distributed to the Pod in the running state, so as to ensure that the response time of most requests is within the normal range.

[0114] The Suspended-Level 1 state is a Pod state unique to this system. Pods in this state should have their CPU resources partially or fully reclaimed, with the specific extent of reclaiming depending on the Pod's background tasks and resource reclaim operations by the Pod Resource Manager. In theory, a Pod in this state will not receive new load requests, but will continue to execute load requests received before entering the Suspended-Level 1 state.

[0115] The Idle-Level 2 state is a Pod state unique to this system. Pods in this state have their CPU and memory resources partially or fully reclaimed. The specific extent of reclaim depends on the Pod's background tasks and resource reclaim operations performed by the Pod Resource Manager. In theory, a Pod in this state will not receive new load requests, but will continue to execute load requests received before entering the Suspended-Level 2 state.

[0116] The deletion refers to the operation of deleting the Pod from the machine cluster by the Kubernetes container orchestration platform.

[0117] The buoy is used to indicate the sum of the number of containers in various states. There are five types of buoys involved in this system, which are respectively recorded as the first buoy w1, the second buoy w2, the third buoy w3, the fourth buoy w a 、The fifth buoy w b , its intuitive physical meaning and quantitative relationship are as follows Figure 4 As shown. When the load is stable, the fourth buoy w a and the fifth buoy w b The number of is 0, that is, a microservice only has Pods in the running state, suspended-level 1 state, and suspended-level 2 state when the load is stable. Therefore, the buoy relationship diagram under the load stability is as follows Figure 5 shown.

[0118] In a specific implementation process, the sub-managers of the microservice resource manager include a responsive sub-manager, a short-term predictive sub-manager, and a long-term predictive sub-manager;

[0119] The responsive sub-manager is used to generate the initial value of the first buoy w1 in a responsive manner based on the resource usage of the microservice over a period of time; the first buoy w1 is the number of Pods in the running state of the microservice.

[0120] The short-term prediction sub-manager is used to perform short-term prediction using the interval prediction algorithm EnbPI and the single-step prediction algorithm SVR to generate the initial value of the second buoy w2; the second buoy w2 is the sum of the number of Pods in the running state and the suspended-level 1 state of the microservice.

[0121] The long-term prediction sub-manager is used to perform long-term prediction using the periodic prediction algorithm Prophet, the interval prediction algorithm EnbPI, and the single-step prediction algorithm SVR to generate an initial value of the third buoy w3. The third buoy w3 is the sum of the number of pods in all states of the microservice.

[0122] In a preferred embodiment, the responsive sub-manager generates the first buoy w1 at a frequency of 5 seconds, the short-term predictive sub-manager generates the second buoy w2 at a frequency of 1 minute, and the long-term predictive sub-manager generates the third buoy w3 at a frequency of 1 day. The control cycle takes the shortest buoy generation time interval, which is 5 seconds.

[0123] Fourth buoy w a Indicates the sum of the number of Pods in a single microservice in the system of the present invention that are undergoing Pod initialization operations or have completed Pod initialization operations but are still waiting for preheating.

[0124] When the newly generated w3 of the long-term predictive sub-manager is greater than the number of existing Pod copies, the coordinator calculates the number of Pods that need to be created, drives the Kubernetes container orchestration platform to create new Pods, and then preheats the newly created containers in a certain proportion.

[0125] Fifth buoy w b Indicates the number of Pods that are being preheated in a single microservice in the system of the present invention, to avoid deploying too many microservices in the preheating state, which would cause the end-to-end delay of the microservice to increase too much and fail to meet the service level. b The value of w a The value of is converted.

[0126] In this embodiment, the statistical data refers to various indicators that can reflect the current load pressure of the microservice. Specifically, in this embodiment, the average CPU utilization of the microservice, the average memory usage of the microservice, and the average number of requests received per minute by the microservice are collected. The statistical data can be directly observed through the microservice resource manager and stored using Prometheus and TimescaleDB tools.

[0127] The responsive sub-manager and the short-term predictive sub-manager use the same logic to generate buoys w1 and w2. Both use the proportional relationship between the buoy value at the previous moment and the statistical data at the current moment to determine the buoy value at the current moment. The mathematical description is:

[0128]

[0129] In formula (1), w x It can represent any one of w1 and w2. represents the buoy w at time t x The value of represents the buoy w at time t+1 x The value of M 当前 Indicates the current value of a certain indicator M, M 标准 Indicates the ideal value of indicator M.

[0130] Among them, M 当前 Should be with w x There is a negative correlation, that is, when other variables are the same, the larger the w x Should correspond to a smaller M 当前 If the selected indicator does not have this negative correlation, it should be made to satisfy this relationship through mathematical changes.

[0131] M 标准 is the ideal value of indicator M, which is a preset value, such as the ideal Pod average CPU utilization, the average number of requests per minute for a single Pod, etc. For different microservices, M 标准 The value of is usually different, so M 标准 It needs to be measured and set manually. It usually does not need to be adjusted after setting. However, if it is necessary to change the M 标准 The value can also be readjusted by the component.

[0132] The responsive sub-manager and the short-term prediction sub-manager both generate corresponding buoy values ​​through formula (1). The difference lies in the different indicators M adopted by the two sub-managers and the generation of M 当前 The process and methods are different.

[0133] In this embodiment, the responsive sub-manager calculates the indicator changes within the sliding time window, takes the average value and adds the product of the percentile threshold a and the standard deviation as its M 标准 One of the sources is shown in formula (2):

[0134]

[0135] The percentile threshold a can be set to 95%; in formula (2), Represents the average value of the indicator sequence M in the time window of size m, represents the 95% position of the indicator sequence M in the time window of size m, represents the standard deviation of the indicator sequence M within the time window of size m, and v1 is the standard deviation of the indicator sequence M generated by the responsive sub-manager. 当前 Specifically, in this embodiment, the indicator is collected every 5 seconds, and the time window size m is 60.

[0136]

[0137] In this embodiment, the formula (3) represents the maximum value of the indicator sequence M in the time window of size n, and v2 is the maximum value of the indicator sequence M generated by the responsive sub-manager. 当前 In this embodiment, the indicator is collected every 5 seconds, and the time window size n is 120.

[0138] M 当前 =max(v1,2); (4)

[0139] The responsive sub-management M 当前 From formula (2)(3)(4), we can get the calculated M 当前 Substituting into formula (1), we can get the corresponding buoy of the microservice at the next moment

[0140] The short-term prediction sub-manager uses a method that combines the interval prediction algorithm EnbPI (Ensemble Batch Prediction Intervals) and the single-step prediction algorithm SVR (Support Vector Regression) to generate the buoy w2. EnbPI is an advanced interval prediction algorithm, which is characterized by not making any assumptions about the data distribution of the time series. It is suitable for non-stationary time series and can be used in conjunction with various underlying regression algorithms. The underlying regression algorithm of the system of the present invention is SVR. The SVR algorithm is a classic regression algorithm based on support vector machines. It maps data to a high-dimensional data space through nonlinear mapping, so that the independent variables and dependent variables in the high-dimensional data feature space have good linear regression characteristics, and are returned to the original space after fitting in the feature space.

[0141] In this embodiment, the short-term prediction sub-manager uses SVR as the underlying regression algorithm for single-step prediction, and then uses the EnbPI algorithm for interval prediction. More intuitively, the short-term prediction sub-manager continuously collects V+N observation indicators over a period of time as a new set of training data for SVR, such as V+N=300+12. Among them, the first 300 observation indicators constitute the input value of the training data, and the maximum value of the last 12 observation indicators is used as the label value of the group of training data, and the group of data is added to the training data. When the number of groups of training data meets the minimum requirement of the number of training groups, the SVR prediction group is obtained in two ways:

[0142] ① When the number of collected training data sets meets the training requirements for the first time, a total of B new SVRs are trained on the collected training data sets, such as training 20 new SVRs, to obtain the initial SVR prediction group.

[0143] ② In the case of an existing SVR prediction group, delete the oldest set of training data and the k SVR predictors trained using the oldest data in the EnbPI algorithm, and then train k new SVR predictors on the new training data, so that the number of predictors in the SVR prediction group remains unchanged at 20.

[0144] In this embodiment, according to the execution mode of the EnbPI algorithm, the confidence level α is set to 95%, and the existing SVR prediction group is used to predict the maximum value of the next 12 observation indicators as y through the most recent 300 observation indicators, and the probability that the true maximum value does not exceed the maximum value y of the predicted value is

[0145] The short-term prediction sub-manager uses the maximum value y of the prediction value of the above EnbPI algorithm and SVR algorithm as its M 当前 , substituted into formula (1) to obtain the value of buoy w2.

[0146] The long-term prediction sub-manager uses the Prophet algorithm to predict specific indicators within a future period based on information from a longer period. The Prophet algorithm is an open-source time series prediction algorithm developed by Facebook and is widely used worldwide for its simplicity, efficiency, and minimal parameter adjustments.

[0147] The period is related to the actual request load, and a common period is one day or one week. The request volume will show periodic changes over time.

[0148] The long-term prediction sub-manager first uses the Prophet algorithm to directly predict the number of Pod replicas required at the next moment based on past load request cycle changes. This is used as the initial value of buoy w3. If the value of buoy w3 generated by the long-term prediction sub-manager is lower than the value of buoy w2 generated by the short-term prediction sub-manager, the value of buoy w3 is corrected using the aforementioned method combining EnbPI and SVR.

[0149] In more detail, this system uses a method combining EnbPI and SVR, sets the confidence β, and predicts that the difference between the buoy w2 and the original third buoy w3 is z. The original third buoy w3 plus the predicted value z is used as the initial value of the new third buoy w3.

[0150] The goal of the long-term predictive sub-manager is to generate a third buoy w3 that is as small as possible without the need for passive triggering of Pod creation operations, that is, the third buoy w3 should be higher than the actual value of the required number of Pods, but not too overestimated. The reason for allowing the third buoy w3 to be higher than the actual value of the required number of Pods is that the system of the present invention has a unique resource-saving mechanism. Even if excess Pods are created, they can be converted into suspended-level 1 and idle-level 2 states, thereby saving over-allocated resources. However, if the third buoy w3 is lower than the actual value of the required number of Pods, it will cause the creation operation of the Pod, and the overhead of the Pod creation operation is significantly higher than the overhead of converting the suspended-level 1 state and the suspended-level 2 state into the running state. Therefore, it is reasonable in this system to tend to set the value of the third buoy w3 higher than the actual value required.

[0151] Because the operation overhead introduced by various operations is inconsistent, the frequencies at which the three resource sub-managers update the corresponding buoys are also inconsistent.

[0152] Updating the first buoy w1 within a reasonable range will result in the allocation and recycling of CPU resources, and the overhead introduced by the change of CPU resources is relatively small. Therefore, the responsive sub-manager updates the first buoy w1 most frequently.

[0153] Updating the second buoy w2 within a reasonable range will result in the allocation and recycling of memory resources. In the system of the present invention, the overhead introduced by memory resource reallocation is caused by swapping in and out memory pages, which is greater than the allocation and recycling of CPU resources. Therefore, the frequency of updating the second buoy w2 by the short-term and predictive sub-managers is lower than the update frequency of the first buoy w1.

[0154] Updating the third buoy, w3, results in the creation and deletion of Pods. Pod creation incurs initialization and warmup overhead, while Pod deletion triggers Pod cleanup and resource recycling, both of which introduce significant operational overhead. To minimize the overhead associated with Pod creation and deletion, the long-term predictive sub-manager updates the third buoy, w3, at the lowest possible frequency.

[0155] In the specific implementation process, the coordinator decides whether to perform the buoy value repair operation according to the size relationship between the initial values ​​of the first buoy w1, the second buoy w2 and the third buoy w3, specifically:

[0156] Under normal circumstances, the relative size relationship of the three buoy values ​​is w1≤w2≤w3;

[0157] In abnormal situations, if the relative size relationship of the three buoy values ​​does not satisfy w1≤w2≤w3, the coordinator will correct the values ​​of each buoy to satisfy the normal relative size relationship;

[0158] Under the premise that w1≤w2≤w3 is satisfied, the coordinator adjusts the status of the corresponding Pod according to the state transition logic so that the number of replicas of each Pod state is consistent with the number of corresponding buoy values. The logic of the coordinator correcting the value of each buoy is as follows:

[0159] use Indicates the values ​​of the first buoy w1, the second buoy w2 and the third buoy w3 generated last time, Indicates the values ​​of the first buoy w1, the second buoy w2 and the third buoy w3 currently generated. Must be satisfied Quantitative relationship of

[0160] First, The smaller value is reassigned Assign the larger value to After this operation, and Already satisfied Quantitative relationship of

[0161] like Will The larger value is assigned and After correction and Already satisfied The quantitative relationship ends the repair logic; if Will The larger value is assigned After correction and Already satisfied The quantitative relationship ends the repair logic;

[0162] After the repair logic, Must be satisfied At this time, the subsequent operations can be continued according to the normal operation logic of the relative size of the buoy values.

[0163] After generating the three new buoy values, the coordinator needs to adjust the Pod status so that the number of replicas of each Pod status is consistent with the number of corresponding buoy values. The specific adjustment logic is as follows:

[0164] like First create a quantity of Pod; if Then first delete the number

[0165] The priority of deleting Pods is in the following states: Initialization, Preheating, Suspended-Level 2, Suspended-Level 1, and Running.

[0166] Then prioritize adjusting the number of running Pods to When the number of running Pods needs to be increased, the priority for converting Pods in various states to running is Pending-Level 1, Pending-Level 2, and Preheating. When the number of running Pods needs to be reduced, the excess Pods are directly converted to Pending-Level 1.

[0167] When the number of running Pods is After the number matches, start adjusting the number of Pods in the pending-level 1 state to When the number of Pods in the Pending-Level 1 state needs to be increased, Pods in various states are prioritized for transitioning to the Pending-Level 1 state, with priority given to Pending-Level 2 and Warming-Up. When the number of Pods in the Pending-Level 1 state needs to be reduced, the remaining Pods are transitioned to the Pending-Level 2 state.

[0168] Furthermore, the Pod resource management operation refers to the operation by which the Pod resource manager modifies the CPU availability, memory availability, and other attributes of the Pod based on the Pod status change on the local machine. The detailed state change operation is expected to achieve the following objectives:

[0169] The initialization state changes to the preheating state, and the Pod resource manager does not operate;

[0170] When the preheating state changes to the running state, the Pod resource manager does not perform any operations;

[0171] The running state changes to the suspended-level 1 state, and the Pod Resource Manager reduces the CPU available to the Pod;

[0172] The Suspended-Level 1 state changes to the Suspended-Level 2 state, and the Pod Resource Manager reduces the amount of memory available to the Pod;

[0173] The Pending-Level 2 state transitions to the Pending-Level 1 state, and the Pod Resource Manager increases the memory available to the Pod to its originally limited value.

[0174] The Suspended-Level 1 state transitions to the Running state, and the Pod Resource Manager increases the CPU available to the Pod to its originally limited value.

[0175] Since the state of the Pod is allowed to change continuously within a control loop, a Pod may directly change from the Running state to the Suspended-Level 2 state. In this case, resource control operations such as reducing CPU resources and memory resources need to be performed on the Pod at the same time.

[0176] In this embodiment, the Pod Resource Manager controls CPU resources by counting the CPU usage time of each Pod over the past period of time and the total time when the Pod requested the CPU but received no response. The system of the present invention refers to this total time as the CPU duration. Specifically, the resources of the corresponding Pod are adjusted according to the following logic:

[0177] If the Pod status remains in the Running state, the CPU duration limit of the Pod is set to the maximum CPU duration within the time window. However, if the maximum CPU duration limit exceeds the maximum available CPU duration set for the Pod, the maximum available CPU duration set for the Pod is used.

[0178] If a Pod transitions from Running to Suspended - Level 1 or Suspended - Level 2, the Pod's CPU duration limit is set to (1+) times the average CPU duration in the time window. γ is a hyperparameter that can be adjusted as needed and defaults to 0.5. A larger value improves service quality but consumes more resources.

[0179] If a Pod transitions directly from Pending-Level 1 or Pending-Level 2 to Running, its resources are set to the maximum available CPU resources of all Pods in the same microservice.

[0180] If the number of running Pods decreases within this control loop, the number of running Pods before the decrease is p, and the number of running Pods after the decrease is q, then the CPU time limit of all Pods is set to the current CPU time limit and The larger value of the CPU duration at the previous moment.

[0181] In addition, the burst CPU usage of all Pods is set to the maximum available CPU usage of the Pod minus the CPU usage limit of the current Pod.

[0182] In this embodiment, the CPU duration limit of the Pod is implemented by adjusting the cpu.cfs_quota_us field in the Linux cgroup, and the burst CPU duration limit of the Pod is implemented by adjusting the cpu.cfs_burst_us field in the Linux cgroup.

[0183] In this embodiment, the Pod Resource Manager counts the number of page faults in a Pod over a period of time and uses h page faults as a limit (preferably, h = 1024) to adaptively allocate available memory to Pods in the Suspended-Level 2 state. Specifically, the Pod Resource Manager adjusts the available memory size of a Pod using the following strategy:

[0184] For a Pod that enters the Suspended-Level 2 state for the first time, set its available memory to rMB, preferably r=10;

[0185] For Pods that are already in the Suspended Level 2 state, if the average number of page faults for the Pod over the past period is greater than h, rMB of memory resources will be added to the Pod.

[0186] For the Pod that leaves the suspended-level 2 state for the first time, its available memory is set to the original maximum memory usage, and the specific system call is called to swap all memory pages remaining on the hard disk into the memory.

[0187] In this embodiment, the available memory of the Pod is achieved by adjusting the memory.high field in the Linux cgroup.

[0188] During implementation, steps S4 through S6 are executed in a loop. Each control cycle involves generating new buoy values, adjusting Pod status, and adjusting individual Pod resources. The three buoys (w1, w2, and w3) have different update frequencies. If a corresponding buoy value is not updated during a control cycle, the previous value is used as the value for the current control cycle.

[0189] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A resource management system for stateless microservice architecture, applied to machine clusters based on the Linux operating system and the Kubernetes container orchestration platform, characterized by: The system comprises: A Linux kernel, used for specific system calls, installed in the machine cluster; A microservice resource manager is installed on the control node of the machine cluster and is provided with several sub-managers. It is used to drive the corresponding sub-managers to generate buoys of the number of Pod copies in the corresponding state based on statistical data. Each microservice corresponds to a microservice resource manager; A coordinator, installed on the control node of the machine cluster, is used to control the state transfer and creation and deletion of the Pod of the corresponding microservice according to the buoys generated by the sub-managers. Each microservice corresponds to a coordinator; The Pod resource manager is installed on each computing node in the machine cluster to monitor the status changes of each Pod and perform corresponding Pod resource management operations; The sub-managers include a responsive sub-manager, a short-term prediction sub-manager, and a long-term prediction sub-manager; The responsive sub-manager is used to generate a first buoy in a responsive manner based on the resource usage of the microservice over a period of time. The initial value of The short-term prediction sub-manager is used to use the interval prediction algorithm EnbPI and the single-step prediction algorithm SVR to perform short-term prediction and generate a second buoy The initial value of The long-term prediction sub-manager is used to use the periodic prediction algorithm Prophet, the interval prediction algorithm EnbPI and the single-step prediction algorithm SVR to perform long-term prediction and generate the third buoy The initial value of The running state of the Pod instance includes initialization state, preheating state, running state, suspended-level 1 state, and suspended-level 2 state; The initialization state is a state in which the Pod is undergoing initialization operations after creation and cannot be used to provide service computing; The preheating state is a state in which the Pod is used to provide service computing after initialization is completed, but its execution speed and response time are not optimal; The running state is the state in which the Pod executes the corresponding request normally at the highest execution speed and response time; The suspended-level 1 state is a state in which part or all of the CPU of the Pod is reclaimed and no new load requests are accepted; The suspended-level 2 state is a state in which most of the memory of the Pod is reclaimed based on the suspended-level 1 state; Wherein, the first buoy Used to indicate the number of Pods in the running state of the microservice; The second buoy The sum of the number of Pods in the running state and suspended-level 1 state for the microservice; The third buoy Indicates the sum of all Pod states of the microservice.

2. The resource management system for stateless microservice architecture according to claim 1, characterized in that: The coordinator is based on the first buoy , second buoy and the third buoy The size of the initial value determines whether to perform the float value repair operation, specifically: Under normal circumstances, the relative size relationship of the three buoy values ​​is ; In abnormal cases, the relative size relationship of the three buoy values ​​does not meet ,The coordinator corrects the values ​​of each buoy so that they satisfy the normal relative size relationship; In satisfaction Under the premise of , the coordinator adjusts the status of the corresponding Pod according to the state transition logic so that the number of replicas of each Pod state is consistent with the number of corresponding buoy values; The logic for the coordinator to correct the values ​​of each buoy is as follows: use , , Indicates the first buoy generated last time , second buoy and the third buoy The value of , , Indicates the first buoy currently generated , second buoy and the third buoy The value of the last generated , , Must be satisfied Quantitative relationship of First, 、 The smaller value is reassigned , assign the larger value to After this operation, and Already satisfied Quantitative relationship of like ,Will 、 The larger value is assigned and , after correction 、 and Already satisfied The quantitative relationship ends the repair logic; if ,Will 、 The larger value is assigned , after correction 、 and Already satisfied The quantitative relationship ends the repair logic; After the repair logic, , , Must be satisfied At this time, the subsequent operations can be continued according to the normal operation logic of the relative size of the buoy values.

3. The resource management system for stateless microservice architecture according to claim 2, characterized in that: The coordinator adjusts the state of the corresponding Pod according to the state transfer logic so that the number of replicas of each Pod state is consistent with the number of corresponding buoy values. Specifically: like , then first create a quantity of Pod; if , then first delete the number The priority of deleting Pods is in the following states: Initialization, Preheating, Suspended-Level 2, Suspended-Level 1, and Running. Then prioritize adjusting the number of running Pods to When the number of running Pods needs to be increased, the priority for converting Pods in various states to the running state is Pending-Level 1, Pending-Level 2, and Preheating. When the number of running Pods needs to be reduced, the excess Pods are directly converted to Pending-Level 1. When the number of running Pods is After the number matches, start adjusting the number of Pods in the pending-level 1 state to When the number of Pods in the Suspended-Level 1 state needs to be increased, the priority of converting Pods in various states to the Suspended-Level 1 state is Suspended-Level 2 state and Preheating state; when the number of Pods in the Suspended-Level 1 state needs to be reduced, the excess Pod states are converted to the Suspended-Level 2 state.

4. The resource management system for stateless microservice architecture according to any one of claims 1 to 3, characterized in that: The Pod Resource Manager controls the resources of a specific Pod according to the following logic: To control CPU resources, the Pod Resource Manager counts the CPU usage time of each Pod over the past period of time, as well as the total time when the Pod requested CPU but received no response. This total time is called CPU duration, and the Pod Resource Manager adjusts the corresponding Pod's resources according to the following logic: If the Pod status remains in the Running state, the CPU duration limit of the Pod is set to the maximum CPU duration within the time window. However, if the maximum CPU duration limit exceeds the maximum available CPU duration set for the Pod, the maximum available CPU duration set for the Pod is used. If the Pod state changes from running to suspended-level 1 or suspended-level 2, the CPU time limit of the Pod is set to times the average CPU time in the time window; is a hyperparameter; If a Pod transitions directly from Pending-Level 1 or Pending-Level 2 to Running, its resources are set to the maximum available CPU resources of all Pods in the same microservice. If the number of running Pods decreases within this control loop, the number of running Pods before the decrease is recorded as , after reducing the number of running Pods to , then the CPU duration limit of all Pods is set to the current CPU duration limit and The larger value of the CPU duration at the previous moment. Set the burst CPU usage of all Pods to the maximum available CPU usage of the Pod minus the CPU usage limit of the current Pod; and / or, In terms of memory resource control, the Pod Resource Manager counts the number of page faults of the Pod in the past period of time and uses The Pod Resource Manager adaptively allocates available memory to Pods in the Suspended-Level 2 state based on the number of page faults. The Pod Resource Manager adjusts the available memory size of the Pod using the following strategies: For a Pod that enters the suspended-level 2 state for the first time, set its available memory to MB; For a Pod that has been in the Suspended-Level 2 state, if the average number of page faults of the Pod in the past period is greater than , then increase MB of memory resources; For the Pod that leaves the suspended-level 2 state for the first time, its available memory is set to the original maximum memory usage, and the specific system call is called to swap all memory pages remaining on the hard disk into the memory.

5. The resource management system for stateless microservice architecture according to any one of claims 1 to 3, characterized in that: The responsive sub-manager and the short-term predictive sub-manager generate a first buoy , second buoy The way is as follows: The current buoy value is determined by the proportional relationship between the previous buoy value and the current statistical data. Its mathematical description is: In formula (1), Can be expressed 、 Any one of Indicates the buoy at time t The value of express Time Buoy The value of Indicates the current indicator The value of Indicator Ideal value; in, and negatively correlated, meaning that when other variables are the same, the greater For smaller If the selected indicator does not have this negative correlation, mathematical changes are made to make it satisfy this relationship. is a preset value, the indicator is an indicator related to the Pod, and the indicator taken by the responsive sub-manager and the short-term predictive sub-manager different; The responsive sub-manager calculates the indicator changes within the sliding time window, takes the average value, and adds the product of the preset percentile threshold a and the standard deviation as its average value. One of the sources is as shown in formula (2): In formula (2), Indicates size Indicators within the time window The average value of Indicates size Indicators within the time window The percentile threshold of Indicates size Indicators within the time window The standard deviation of The responsive sub-manager generates One of the important sources of In formula (3), Indicates size Indicators within the time window The maximum value of The responsive sub-manager generates Another important source of The responsive sub-manager According to formula (2) (3) (4), the calculated Substituting into formula (1), we can get the corresponding buoy of the microservice at the next moment .

6. The resource management system for stateless microservice architecture according to claim 5, characterized in that: The short-term prediction sub-manager uses SVR as the underlying regression algorithm for single-step prediction, and then uses the EnbPI algorithm for interval prediction; the short-term prediction sub-manager continuously collects data from the past period of time. indicators as a new set of training data for SVR; The indicators constitute the input value of the training data. The maximum value of the indicators is used as the label value of the training data group, and the data group is added to the training data. When the number of training data groups meets the minimum requirement of the number of training groups, the SVR prediction group is obtained by the following two methods: According to the execution mode of EnbPI algorithm, set the confidence level , using the existing SVR prediction set, by the recent Indicators predict the future The maximum value within the index is , and the true maximum value does not exceed the maximum value of the predicted value The probability of ; The short-term prediction sub-manager uses the maximum value of the prediction value of the above-mentioned EnbPI algorithm and SVR algorithm As its , substituted into formula (1) to obtain the second buoy value.

7. The resource management system for stateless microservice architecture according to claim 6, characterized in that: The long-term prediction sub-manager first uses the Prophet algorithm to directly predict the number of Pod copies required at the next moment based on the previous load request cycle changes as the third buoy The initial value of If the third buoy generated by the long-term prediction sub-manager The value is less than the second buoy generated by the short-term prediction sub-manager The value of the third buoy Correct the value.

8. The resource management system for stateless microservice architecture according to claim 7, characterized in that: The third buoy was treated by combining EnbPI with SVR. The value of is modified, specifically: Use the method of combining EnbPI and SVR to set the confidence level , predicted the second buoy With the original third buoy The difference is , the original third buoy Add the predicted value As the new third buoy The initial value of .

Images