A Kubernetes container automatic scaling method and system based on dual agents

Abstract

The present invention provides a dual-agent based Kubernetes container automatic scaling method and system in the field of cloud computing resource management technology, the method comprising: step S1, based on a dual-agent module, a state space hierarchical module, and an attention experience playback module, creating a scaling model and setting a reward function of the scaling model; step S2, acquiring a large amount of historical workload data of Kubernetes containers to build a data set; step S3, dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, verifying the trained scaling model through the validation set, and testing the verified scaling model through the test set; step S4, scheduling the cloud computing resources of the Kubernetes container through the tested scaling model. The advantages of the present invention are: greatly improving the resource utilization, efficiency, and reliability of cloud computing resource scheduling of Kubernetes containers.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing resource management, and in particular to a dual-agent-based Kubernetes container automatic scaling method and system. Background Art

[0002] With the rapid development of cloud-native technologies, containerization has become the mainstream paradigm for modern application deployment, and Kubernetes, as the dominant container orchestration platform, plays an increasingly important role in enterprise IT architecture. However, how to achieve efficient and automatic expansion of cloud computing resources for Kubernetes containers to meet dynamic workload requirements while balancing resource utilization and service quality has become a key issue that needs to be urgently addressed in the field of cloud-native technologies. Traditionally, automatic expansion of Kubernetes containers has mainly faced the following challenges:

[0003] 1. Insufficient workload prediction accuracy. Currently, the HPA (Horizontal Pod Autoscaler) function of the Kubernetes container relies on simple threshold rules to make expansion decisions, which cannot effectively predict and respond to complex workload patterns, such as periodic fluctuations and burst traffic, causing the supply of cloud computing resources to lag behind actual demand. In a production environment, this simple reactive expansion strategy may lead to up to 35% over-configuration of cloud computing resources.

[0004] 2. It is difficult to coordinate and optimize multi-dimensional resources. The application of Kubernetes containers usually involves cloud computing resources in multiple dimensions, such as CPU, memory, and network. There are complex dependencies between cloud computing resources in different dimensions. Traditional single-dimensional expansion methods are difficult to capture the coupling effect between cloud computing resources, resulting in unbalanced configuration of cloud computing resources and affecting overall performance. Ignoring the dependencies between cloud computing resources may lead to SLA breaches of about 25%.

[0005] 3. The real-time and stability of expansion decisions are in conflict. The operating environment of Kubernetes containers has high requirements for the response speed of expansion decisions, especially when facing sudden workloads; however, an overly aggressive expansion strategy may cause system shocks, while an overly conservative expansion strategy may miss the opportunity for expansion.

[0006] 4. System complexity and scalability. As the cluster size of Kubernetes containers continues to grow, the state space grows exponentially, and traditional methods are difficult to cope with such a high-dimensional decision space; at the same time, heterogeneous computing resources, multi-tenant environments, network topology and other factors further increase the complexity of the automatic expansion problem.

[0007] In response to the above challenges, some scholars have proposed a workload prediction method based on LSTM, which captures the time dependency of workload sequences through deep learning; some scholars have designed a container scheduling framework based on reinforcement learning to achieve dynamic optimization of cloud computing resource allocation; in industrial practice, Google's Autopilot system uses machine learning methods to optimize the resource efficiency of Kubernetes container clusters and has achieved certain results. However, existing research still has the following limitations: 1. It only focuses on the optimization of a single aspect and lacks overall consideration of workload prediction and resource adjustment, resulting in unsatisfactory utilization of cloud computing resources; 2. When dealing with high-dimensional state spaces and complex decision sequences, the learning efficiency and convergence still need to be improved, resulting in poor scheduling efficiency of cloud computing resources; 3. It ignores the effective use of empirical knowledge, making it difficult to give full play to the value of historical data, affecting the accuracy of workload prediction, and thus affecting the reliability of cloud computing resource scheduling.

[0008] Therefore, how to provide a dual-agent-based Kubernetes container automatic scaling method and system to improve the resource utilization, efficiency and reliability of cloud computing resource scheduling of Kubernetes containers has become a technical problem that needs to be solved urgently. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide a dual-agent-based Kubernetes container automatic scaling method and system to improve the resource utilization, efficiency and reliability of cloud computing resource scheduling of Kubernetes containers.

[0010] In a first aspect, the present invention provides a dual-agent-based Kubernetes container automatic scaling method, comprising the following steps:

[0011] Step S1, based on a dual agent module including a workload prediction agent unit and a resource adjustment agent unit, a state space hierarchical module, and an attention experience playback module, a scaling model for cloud computing resource scheduling for Kubernetes containers is created, and a reward function of the scaling model is set;

[0012] The workload prediction agent unit is used to output workload prediction results to the state space hierarchical module based on the input workload data;

[0013] The state space hierarchical module is used to output a system state vector including a workload state vector, a resource utilization state vector and a performance state vector to the resource adjustment agent unit according to the input workload prediction result and workload data;

[0014] The resource adjustment agent unit is used to output a cloud computing resource scheduling strategy according to an input system state vector;

[0015] The attention experience playback module is used to generate scheduling experience according to the system state vector, resource scheduling strategy and reward value of reward function, calculate the attention weight of each scheduling experience, and train and optimize the scaling model by screening the scheduling experience based on the attention weight;

[0016] Step S2: Obtain a large amount of historical workload data of Kubernetes containers, and construct a data set after preprocessing each of the historical workload data;

[0017] Step S3: dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, validating the trained scaling model through the validation set, and testing the validated scaling model through the test set;

[0018] Step S4: Schedule the cloud computing resources of the Kubernetes container using the scaling model that has passed the test.

[0019] Furthermore, in step S1, the workload prediction agent unit is constructed based on the LSTM network, and the formula is:

[0020] ;

[0021] ;

[0022] in, Represents the workload prediction result, that is, the workload vector predicted at time t+1 , Represents the CPU usage vector predicted at time t+1, Represents the memory usage vector predicted at time t+1, Represents the predicted request quantity vector at time t+1; The network function representing the workload prediction agent unit; h t represents the hidden state of LSTM at time t, which is used to capture the historical information of the workload vector; c t Represents the cell state of LSTM at time t, which is used to maintain the long-term memory of the workload vector; The attention weight vector representing the workload vector; , , Both represent learnable weight matrices; represents the workload vector input at time t , represents the CPU usage vector at time t, represents the memory usage vector at time t, represents the request quantity vector at time t; b represents the bias vector.

[0023] Furthermore, in step S1, the formula of the state space hierarchical module is:

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] Where S represents the system state vector; represents the workload state vector; represents the resource utilization state vector; represents the performance state vector; represents the workload vector input at time t; represents the workload vector predicted at time t+1; represents the load variance, which is calculated based on the workload prediction results and workload data; Indicates the load change rate, which is calculated based on the workload prediction results and workload data; Represents the CPU usage vector at time t; Represents the memory usage vector at time t; represents the request quantity vector at time t; Indicates the SLA achievement status, with a value range of [0,1); Indicates the queue length of requests; Indicates the average response delay of the request; represents the SLA default rate;

[0029] The resource adjustment agent unit is constructed based on the DQN network, and the formula is: ;

[0030] in, represents the system state vector at time t; represents the extended action at time t; the action space of the extended action is defined as , Indicates horizontal expansion action, used to adjust the number of Pods of Kubernetes containers. Indicates a vertical scaling action, which is used to adjust the resource limits of a Pod; express and Corresponding cloud computing resource scheduling strategy; A network function representing a resource adjustment agent unit; Represents the reward value of the reward function at time t; represents the discount factor; represents the extended action at time t+1; Represents the system state vector at time t+1; maxQ() represents the DQN network.

[0031] Furthermore, in step S1, the formula of the scheduling experience generated by the attention experience playback module is: ;in, Indicates scheduling experience; represents the system state vector at time t; represents the extended action at time t; Represents the reward value of the reward function at time t; Represents the system state vector at time t+1; Indicates the system status; Indicates an extended action; Represents the reward value; Indicates the next system state;

[0032] The calculation formula of the attention weight is: ;

[0033] in, represents the attention weight at time t; e i represents the embedding vector of the i-th scheduling experience; e j represents the embedding vector of the jth scheduling experience; T represents transpose; represents the empirical projection matrix; An embedding vector representing the current system state;

[0034] The training and optimization of the scaling model based on the attention weight screening scheduling experience is specifically as follows:

[0035] Calculate the sampling probability based on the attention weights: ;

[0036] in, represents the sampling probability of the ith scheduling experience; Indicates the temperature parameter, the value is 2; represents the i-th attention weight; represents the jth attention weight;

[0037] Training and optimizing the scaling model based on the sampling probability screening scheduling experience;

[0038] The formula of the reward function is: ;

[0039] in, represents the system state vector at time t; represents the extended action at time t; express and The corresponding reward value; Both represent weight factors, and ; express and Corresponding resource utilization rewards; express and Corresponding cost rewards; express and Corresponding SLA rewards.

[0040] Furthermore, in step S1, the workload data includes at least CPU usage, memory usage vector, number of requests, SLA achievement status, queue length of requests, average response delay of requests, and SLA breach rate;

[0041] The step S2 is specifically as follows:

[0042] Obtain a large amount of historical workload data of Kubernetes containers, and construct a data set after performing preprocessing on each of the historical workload data, including at least null value processing, data standardization processing, business key value processing, data merging, and missing value processing;

[0043] The step S3 is specifically as follows:

[0044] Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, and continuously optimizing the hyperparameters of the scaling model including at least a learning rate, a training batch, a random dropout rate, and a weight decay rate until a preset convergence condition is met;

[0045] Verifying the trained scaling model through the verification set to determine whether the scheduling accuracy of the cloud computing resource scheduling strategy is greater than a preset accuracy threshold, if not, expanding the training set to continue training; if yes, the verification is passed;

[0046] The verified scaling model is tested using the test set to determine whether the confidence of the cloud computing resource scheduling strategy is greater than a preset confidence threshold. If not, the training set is expanded to continue training; if yes, the test passes.

[0047] In a second aspect, the present invention provides a dual-agent-based Kubernetes container automatic scaling system, comprising the following modules:

[0048] A scaling model creation module, used to create a scaling model for cloud computing resource scheduling for Kubernetes containers based on a dual agent module including a workload prediction agent unit and a resource adjustment agent unit, a state space hierarchical module, and an attention experience playback module, and set a reward function for the scaling model;

[0049] The workload prediction agent unit is used to output workload prediction results to the state space hierarchical module based on the input workload data;

[0050] The state space hierarchical module is used to output a system state vector including a workload state vector, a resource utilization state vector and a performance state vector to the resource adjustment agent unit according to the input workload prediction result and workload data;

[0051] The resource adjustment agent unit is used to output a cloud computing resource scheduling strategy according to an input system state vector;

[0052] The attention experience playback module is used to generate scheduling experience according to the system state vector, resource scheduling strategy and reward value of reward function, calculate the attention weight of each scheduling experience, and train and optimize the scaling model by screening the scheduling experience based on the attention weight;

[0053] A data set construction module is used to obtain a large amount of historical workload data of Kubernetes containers, and to construct a data set after preprocessing each of the historical workload data;

[0054] a scaling model training module, used for dividing the data set into a training set, a validation set and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, validating the trained scaling model through the validation set, and testing the validated scaling model through the test set;

[0055] The container automatic scaling module is used to schedule the cloud computing resources of the Kubernetes container through the scaling model that has passed the test.

[0056] Furthermore, in the scaling model creation module, the workload prediction agent unit is constructed based on the LSTM network, and the formula is:

[0057] ; ;

[0058] in, Represents the workload prediction result, that is, the workload vector predicted at time t+1 , Represents the CPU usage vector predicted at time t+1, Represents the memory usage vector predicted at time t+1, Represents the predicted request quantity vector at time t+1; The network function representing the workload prediction agent unit; h t represents the hidden state of LSTM at time t, which is used to capture the historical information of the workload vector; c t Represents the cell state of LSTM at time t, which is used to maintain the long-term memory of the workload vector; The attention weight vector representing the workload vector; , , Both represent learnable weight matrices; represents the workload vector input at time t , represents the CPU usage vector at time t, represents the memory usage vector at time t, represents the request quantity vector at time t; b represents the bias vector.

[0059] Furthermore, in the scaling model creation module, the formula of the state space layering module is:

[0060] ;

[0061] ;

[0062] ;

[0063] ;

[0064] Where S represents the system state vector; represents the workload state vector; represents the resource utilization state vector; represents the performance state vector; represents the workload vector input at time t; represents the workload vector predicted at time t+1; represents the load variance, which is calculated based on the workload prediction results and workload data; Indicates the load change rate, which is calculated based on the workload prediction results and workload data; Represents the CPU usage vector at time t; Represents the memory usage vector at time t; represents the request quantity vector at time t; Indicates the SLA achievement status, with a value range of [0,1); Indicates the queue length of requests; Indicates the average response delay of the request; represents the SLA default rate;

[0065] The resource adjustment agent unit is constructed based on the DQN network, and the formula is: ;

[0066] in, represents the system state vector at time t; represents the extended action at time t; the action space of the extended action is defined as , Indicates horizontal expansion action, used to adjust the number of Pods of Kubernetes containers. Indicates a vertical scaling action, which is used to adjust the resource limits of a Pod; express and Corresponding cloud computing resource scheduling strategy; A network function representing a resource adjustment agent unit; Represents the reward value of the reward function at time t; represents the discount factor; represents the extended action at time t+1; Represents the system state vector at time t+1; maxQ() represents the DQN network.

[0067] Furthermore, in the scaling model creation module, the formula for the scheduling experience generated by the attention experience playback module is: ;

[0068] in, Indicates scheduling experience; represents the system state vector at time t; represents the extended action at time t; Represents the reward value of the reward function at time t; Represents the system state vector at time t+1; Indicates the system status; Indicates an extended action; Represents the reward value; Indicates the next system state;

[0069] The calculation formula of the attention weight is: ;

[0070] in, represents the attention weight at time t; e i represents the embedding vector of the i-th scheduling experience; e j represents the embedding vector of the jth scheduling experience; T represents transpose; represents the empirical projection matrix; An embedding vector representing the current system state;

[0071] The training and optimization of the scaling model based on the attention weight screening scheduling experience is specifically as follows:

[0072] Calculate the sampling probability based on the attention weights: ;

[0073] in, represents the sampling probability of the ith scheduling experience; Indicates the temperature parameter, the value is 2; represents the i-th attention weight; represents the jth attention weight;

[0074] Training and optimizing the scaling model based on the sampling probability screening scheduling experience;

[0075] The formula of the reward function is: ;

[0076] in, represents the system state vector at time t; represents the extended action at time t; express and The corresponding reward value; Both represent weight factors, and ; express and Corresponding resource utilization rewards; express and Corresponding cost rewards; express and Corresponding SLA rewards.

[0077] Furthermore, in the scaling model creation module, the workload data includes at least CPU usage, memory usage vector, number of requests, SLA achievement status, queue length of requests, average response delay of requests, and SLA breach rate;

[0078] The data set construction module is specifically used for:

[0079] Obtain a large amount of historical workload data of Kubernetes containers, and construct a data set after performing preprocessing on each of the historical workload data, including at least null value processing, data standardization processing, business key value processing, data merging, and missing value processing;

[0080] The telescopic model training module is specifically used for:

[0081] Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, and continuously optimizing the hyperparameters of the scaling model including at least a learning rate, a training batch, a random dropout rate, and a weight decay rate until a preset convergence condition is met;

[0082] Verifying the trained scaling model through the verification set to determine whether the scheduling accuracy of the cloud computing resource scheduling strategy is greater than a preset accuracy threshold, if not, expanding the training set to continue training; if yes, the verification is passed;

[0083] The verified scaling model is tested using the test set to determine whether the confidence of the cloud computing resource scheduling strategy is greater than a preset confidence threshold. If not, the training set is expanded to continue training; if yes, the test passes.

[0084] The advantages of the present invention are:

[0085] Through the dual-agent module, the state-space hierarchical module and the attention experience replay module, a scaling model for cloud computing resource scheduling of Kubernetes containers is created, and the reward function of the scaling model is set. The dual-agent module includes a workload prediction agent unit and a resource adjustment agent unit. Then, a large amount of historical workload data of Kubernetes containers is obtained for preprocessing to construct a data set, and the data set is divided into a training set, a validation set and a test set. The scaling model is trained with the training set, and the scaling model is continuously optimized with the reward function during the training process. The trained scaling model is verified with the validation set, and the verified scaling model is tested with the test set. Finally, the cloud computing resources of the Kubernetes container are scheduled with the scaling model that has passed the test. That is, cloud computing resources are scheduled with the scaling model trained with a large amount of historical workload data, and the workload prediction agent unit of the scaling model is used to output the workload prediction result to the state-space hierarchical module based on the input workload data; the state-space hierarchical module is used to output the workload prediction result to the state-space hierarchical module based on the input workload prediction result and the workload Load data, outputs the system state vector including workload state vector, resource utilization state vector and performance state vector to the resource adjustment agent unit; the resource adjustment agent unit is used to output the cloud computing resource scheduling strategy according to the input system state vector; the attention experience replay module is used to generate scheduling experience according to the system state vector, resource scheduling strategy and the reward value of the reward function, calculate the attention weight of each scheduling experience, and train and optimize the scaling model based on the attention weight screening scheduling experience; that is, in the cloud computing resource scheduling process of the Kubernetes container, the workload prediction results and the cloud computing resource scheduling strategy are comprehensively considered through the dual agent module, the multi-dimensional characteristics of the Kubernetes container are comprehensively characterized through the state space hierarchical module to improve the rationality of cloud computing resource scheduling, and the value of historical data is fully utilized through the attention experience replay module, the accuracy of workload prediction is effectively improved, and the learning efficiency of the scaling model and the optimization of the cloud computing resource scheduling strategy are improved, which ultimately greatly improves the resource utilization, efficiency and reliability of the cloud computing resource scheduling of the Kubernetes container. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0087] Figure 1 It is a flow chart of a dual-agent-based Kubernetes container automatic scaling method of the present invention.

[0088] Figure 2 It is a structural schematic diagram of a dual-agent-based Kubernetes container automatic scaling system of the present invention. DETAILED DESCRIPTION

[0089] The technical solution in the embodiment of the present application has the following general idea: scheduling cloud computing resources through a scaling model trained with a large amount of historical workload data; the workload prediction agent unit of the scaling model is used to output the workload prediction result to the state space hierarchical module based on the input workload data; the state space hierarchical module is used to output the system state vector including the workload state vector, the resource utilization state vector and the performance state vector to the resource adjustment agent unit based on the input workload prediction result and workload data; the resource adjustment agent unit is used to output the cloud computing resource scheduling strategy based on the input system state vector; the attention experience replay module is used to generate the scheduling strategy based on the system state vector, the resource scheduling strategy and the reward value of the reward function. The attention weight of each scheduling experience is calculated, and the scheduling experience is screened based on the attention weight to train and optimize the scaling model; that is, in the cloud computing resource scheduling process of Kubernetes containers, the workload prediction results and cloud computing resource scheduling strategies are comprehensively considered through the dual agent module, and the multi-dimensional characteristics of Kubernetes containers are comprehensively characterized through the state space hierarchical module to improve the rationality of cloud computing resource scheduling. The value of historical data is fully utilized through the attention experience playback module, which effectively improves the accuracy of workload prediction, improves the learning efficiency of the scaling model and optimizes the cloud computing resource scheduling strategy, thereby improving the resource utilization, efficiency and reliability of cloud computing resource scheduling of Kubernetes containers.

[0090] Please refer to Figure 1 to Figure 2 As shown, a preferred embodiment of a dual-agent-based Kubernetes container automatic scaling method of the present invention includes the following steps:

[0091] Step S1, based on a dual agent module including a workload prediction agent unit (WPA) and a resource adjustment agent unit (RAA), a state space hierarchical module, and an attention experience playback module, a scaling model for cloud computing resource scheduling for Kubernetes containers is created, and a reward function of the scaling model is set; the workload prediction agent unit and the resource adjustment agent unit achieve collaborative work through synchronization parameters and shared experience learning;

[0092] The workload prediction agent unit is used to output workload prediction results to the state space hierarchical module based on the input workload data, that is, it is responsible for capturing the timing dependency of the workload and making accurate predictions;

[0093] The state space hierarchical module is used to output a system state vector including a workload state vector, a resource utilization state vector and a performance state vector to the resource adjustment agent unit based on the input workload prediction results and workload data, that is, it is responsible for comprehensively characterizing the multi-dimensional features of the Kubernetes container;

[0094] The resource adjustment agent unit is used to output a cloud computing resource scheduling strategy according to the input system state vector, that is, to optimize resource allocation decisions based on the prediction results;

[0095] The attention experience playback module is used to generate scheduling experience according to the system state vector, resource scheduling strategy and reward value of the reward function, calculate the attention weight of each scheduling experience, screen the scheduling experience based on the attention weight to train and optimize the scaling model, realize context-based intelligent experience selection to optimize the strategy, and improve learning efficiency through priority sampling and fixed capacity buffer management;

[0096] Step S2: Obtain a large amount of historical workload data of Kubernetes containers, and construct a data set after preprocessing each of the historical workload data;

[0097] Step S3: dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, validating the trained scaling model through the validation set, and testing the validated scaling model through the test set;

[0098] Step S4: Schedule the cloud computing resources of the Kubernetes container using the scaling model that has passed the test.

[0099] The present invention decouples workload prediction from cloud computing resource adjustment through a dual-agent module, and improves their respective performance through LSTM and DQN; comprehensively optimizes multi-dimensional resource characteristics through a state space hierarchical module and a reward function; significantly improves learning efficiency and system adaptability through an attention experience replay module; and maximizes cloud computing resource utilization while meeting service quality requirements through a hierarchical design of a state space hierarchical module and collaborative learning of dual-agent modules.

[0100] In step S1, the workload prediction agent unit is constructed based on the LSTM network, and the formula is: ; ;

[0101] in, Represents the workload prediction result, that is, the workload vector predicted at time t+1 , Represents the CPU usage vector predicted at time t+1, Represents the memory usage vector predicted at time t+1, Represents the predicted request quantity vector at time t+1; The network function representing the workload prediction agent unit; h t represents the hidden state of LSTM at time t, which is used to capture the historical information of the workload vector; c t Represents the cell state of LSTM at time t, which is used to maintain the long-term memory of the workload vector; The attention weight vector representing the workload vector; , , Both represent learnable weight matrices; represents the workload vector input at time t , represents the CPU usage vector at time t, represents the memory usage vector at time t, represents the request quantity vector at time t; b represents the bias vector. The Softmax() function represents the normalization function.

[0102] The workload prediction agent unit adopts a multi-head attention structure, configured with 4 attention heads, and the hidden dimension of each attention head is 64; the attention head obtains the attention score by calculating the similarity between the query vector and the key vector, and then normalizes it with the value vector and weighted sums it to generate the context vector. This mechanism can adaptively capture the long-term and short-term dependencies in the time series. The LSTM network adopts a two-layer structure, and the expression ability of the model is improved by increasing the network depth; in terms of network configuration, the input dimension is determined according to the number of features, the hidden layer dimension is set to 128, and a dropout mechanism of 0.2 is introduced to prevent overfitting. The network output is mapped to the target dimension through a fully connected layer to predict resource usage at future moments.

[0103] The workload prediction agent unit captures long-term dependencies in workload data by introducing an attention mechanism, thereby significantly improving prediction accuracy.

[0104] In step S1, the formula of the state space hierarchical module is:

[0105] ;

[0106] ;

[0107] ;

[0108] ;

[0109] Where S represents the system state vector; represents the workload state vector; represents the resource utilization state vector; represents the performance state vector; represents the workload vector input at time t; represents the workload vector predicted at time t+1; It represents the load variance, which is calculated based on the workload prediction results and workload data and is used to measure volatility; Indicates the load change rate, which is calculated based on the workload prediction results and workload data; Represents the CPU usage vector at time t; Represents the memory usage vector at time t; represents the request quantity vector at time t; Indicates the SLA achievement status, with a value range of [0,1); Indicates the queue length of requests; Indicates the average response delay of requests; Indicates the SLA breach rate;

[0110] In the cloud environment of Kubernetes containers, in order to achieve accurate dynamic adjustment of cloud computing resources, the present invention designs a three-layer state space structure to comprehensively capture the system status from three dimensions: workload status, resource utilization status, and performance status. This hierarchical structure can evaluate the expansion needs of cloud computing resources from different levels and make more accurate decisions.

[0111] The first layer is the workload status, which focuses on load characteristics and changing trends. For example, the system collects current workload data every 15 seconds, including real-time indicators such as CPU utilization (such as 75%), memory utilization (such as 60%), and number of requests per second (such as 1,000). At the same time, based on the workload prediction results, it records the predicted load for the next 5 minutes. For example, it is expected that the CPU utilization will reach 85% and the number of requests will increase to 1,200 per second. In addition, the load fluctuation and change speed are also calculated. For example, the CPU utilization fluctuation range is ±10%, and the CPU utilization increases by 2% every minute. These indicators together constitute the system's comprehensive understanding of the workload.

[0112] The second layer is the resource utilization status, which focuses on the usage of cloud computing resources and monitors the resource usage details of each Pod, such as Pod1's CPU usage is 80% and its memory usage is 2GB; it records the resource load level of the entire Kubernetes container cluster, such as the average CPU usage of the Kubernetes container cluster is 78% and its memory usage is 65%; it also records the usage of network resources, such as the current bandwidth usage of 500Mbps and the number of active connections of 1,000; in order to ensure service quality, it also monitors the achievement of the service level agreement (SLA), such as 95% of requests can be responded to within 100ms and the service availability reaches 99.9%.

[0113] The third layer is the performance status, which focuses on the system operation effect and records the request queue status of each service. For example, service A currently has 100 requests waiting, and 10 requests are added every minute. It monitors the response time of different types of requests, such as the average response time of read operations is 50ms and that of write operations is 80ms. It also counts SLA breaches, such as 5% of the request response time exceeds the predetermined threshold. These indicators directly reflect the service quality of the system and are an important basis for resource adjustment decisions.

[0114] These three layers of state information are interrelated and influence each other. For example, when workload forecasts show that future loads will increase significantly, and current resource utilization is already high (such as CPU utilization exceeding 80%), and performance indicators show that the request queue begins to grow (such as an increase of 10 requests per minute), the expansion operation is started in advance. Specifically, the number of Pods may be increased first (such as from 3 to 4), or the resource limits of existing Pods may be increased (such as increasing the number of CPU cores from 2 to 3). Through this layered state space design, changes in the cloud environment can be fully and timely perceived, possible resource pressures can be predicted, and reasonable expansion decisions can be made in advance.

[0115] The resource adjustment agent unit is constructed based on the DQN network, and the formula is: ;

[0116] in, represents the system state vector at time t; represents the extended action at time t; the action space of the extended action is defined as , Indicates horizontal expansion action, used to adjust the number of Pods of Kubernetes containers. Represents a vertical expansion action, which is used to adjust the resource limit of the Pod. The action space is used to comprehensively capture multi-dimensional resource characteristics. , -1 means reducing one Pod, +2 means adding two Pods; , C represents the CPU configuration set, M represents the memory configuration set, such as (2,4096) means setting 2 CPU cores and 4 GB memory; express and Corresponding cloud computing resource scheduling strategy; A network function representing a resource adjustment agent unit; Represents the reward value of the reward function at time t; represents the discount factor, the value is 0.99; represents the extended action at time t+1; Represents the system state vector at time t+1; maxQ() represents the DQN network.

[0117] In step S1, the formula for the scheduling experience generated by the attention experience playback module is: ;

[0118] in, Indicates scheduling experience; represents the system state vector at time t; represents the extended action at time t; Represents the reward value of the reward function at time t; represents the system state vector at time t; Indicates the system status; Indicates an extended action; Represents the reward value; Indicates the next system state; in specific implementation, the scheduling experience is stored in the experience playback buffer;

[0119] The calculation formula of the attention weight is: ;

[0120] in, represents the attention weight at time t; e i represents the embedding vector of the i-th scheduling experience; e j represents the embedding vector of the jth scheduling experience; T represents transpose; represents the empirical projection matrix; An embedding vector representing the current system state;

[0121] The training and optimization of the scaling model based on the attention weight screening scheduling experience is specifically as follows:

[0122] Calculate the sampling probability based on the attention weights: ;

[0123] in, represents the sampling probability of the ith scheduling experience; Indicates the temperature parameter, the value is 2; represents the i-th attention weight; represents the jth attention weight;

[0124] Training and optimizing the scaling model based on the sampling probability screening scheduling experience;

[0125] The formula of the reward function is: ;

[0126] in, represents the system state vector at time t; represents the extended action at time t; express and The corresponding reward value; Both represent weight factors, and , used to balance the three optimization goals of resource utilization, cost, and SLA; express and Corresponding resource utilization rewards; express and Corresponding cost rewards; express and The corresponding SLA reward. util (), R cost (), R sla () all adopt a segmented design, that is, differentiated punishments are imposed on violations of different degrees.

[0127] ;

[0128] in, represents the target utilization; Indicates the current resource utilization; Indicates the maximum utilization; and Both represent penalty coefficients, which are used to adjust the intensity of rewards or penalties within different ranges; ;

[0129] in, represents the current cost; represents the basic cost; represents the maximum cost; represents the rate of change of cost; and Both represent weight parameters, which are used to control the impact of current cost and cost change respectively;

[0130] ;

[0131] in, represents the SLA default rate; Indicates the SLA default rate threshold; and Both represent penalty coefficients, which are used to control the intensity of rewards or penalties within different default rate ranges.

[0132] In step S1, the workload data at least includes CPU usage, memory usage vector, number of requests, SLA achievement status, request queue length, average request response delay, and SLA breach rate; in the cloud environment of Kubernetes containers, the workload data is expressed as a series of time series data. In specific implementation, the workload data needs to be standardized to eliminate the dimensionality effect and improve the generalization ability of the scaling model;

[0133] The step S2 is specifically as follows:

[0134] Obtain a large amount of historical workload data of Kubernetes containers, and construct a data set after performing preprocessing on each of the historical workload data, including at least null value processing, data standardization processing, business key value processing, data merging, and missing value processing;

[0135] The step S3 is specifically as follows:

[0136] Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, and continuously optimizing the hyperparameters of the scaling model including at least a learning rate, a training batch, a random dropout rate, and a weight decay rate until a preset convergence condition is met;

[0137] Verifying the trained scaling model through the verification set to determine whether the scheduling accuracy of the cloud computing resource scheduling strategy is greater than a preset accuracy threshold, if not, expanding the training set to continue training; if yes, the verification is passed;

[0138] The verified scaling model is tested using the test set to determine whether the confidence of the cloud computing resource scheduling strategy is greater than a preset confidence threshold. If not, the training set is expanded to continue training; if yes, the test passes.

[0139] A preferred embodiment of a dual-agent-based Kubernetes container automatic scaling system of the present invention includes the following modules:

[0140] A scaling model creation module is used to create a scaling model for cloud computing resource scheduling for Kubernetes containers based on a dual agent module including a workload prediction agent unit (WPA) and a resource adjustment agent unit (RAA), a state space hierarchical module, and an attention experience playback module, and set a reward function for the scaling model; the workload prediction agent unit and the resource adjustment agent unit achieve collaborative work through synchronization parameters and shared experience learning;

[0141] The workload prediction agent unit is used to output workload prediction results to the state space hierarchical module based on the input workload data, that is, it is responsible for capturing the timing dependency of the workload and making accurate predictions;

[0142] The state space hierarchical module is used to output a system state vector including a workload state vector, a resource utilization state vector and a performance state vector to the resource adjustment agent unit based on the input workload prediction results and workload data, that is, it is responsible for comprehensively characterizing the multi-dimensional features of the Kubernetes container;

[0143] The resource adjustment agent unit is used to output a cloud computing resource scheduling strategy according to the input system state vector, that is, to optimize resource allocation decisions based on the prediction results;

[0144] The attention experience playback module is used to generate scheduling experience according to the system state vector, resource scheduling strategy and reward value of the reward function, calculate the attention weight of each scheduling experience, screen the scheduling experience based on the attention weight to train and optimize the scaling model, realize context-based intelligent experience selection to optimize the strategy, and improve learning efficiency through priority sampling and fixed capacity buffer management;

[0145] A data set construction module is used to obtain a large amount of historical workload data of Kubernetes containers, and to construct a data set after preprocessing each of the historical workload data;

[0146] a scaling model training module, used for dividing the data set into a training set, a validation set and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, validating the trained scaling model through the validation set, and testing the validated scaling model through the test set;

[0147] The container automatic scaling module is used to schedule the cloud computing resources of the Kubernetes container through the scaling model that has passed the test.

[0148] The present invention decouples workload prediction from cloud computing resource adjustment through a dual-agent module, and improves their respective performance through LSTM and DQN; comprehensively optimizes multi-dimensional resource characteristics through a state space hierarchical module and a reward function; significantly improves learning efficiency and system adaptability through an attention experience replay module; and maximizes cloud computing resource utilization while meeting service quality requirements through a hierarchical design of a state space hierarchical module and collaborative learning of dual-agent modules.

[0149] In the scaling model creation module, the workload prediction agent unit is constructed based on the LSTM network, and the formula is:

[0150] ;

[0151] ;

[0152] in, Represents the workload prediction result, that is, the workload vector predicted at time t+1 , Represents the CPU usage vector predicted at time t+1, Represents the memory usage vector predicted at time t+1, Represents the predicted request quantity vector at time t+1; The network function representing the workload prediction agent unit; h t represents the hidden state of LSTM at time t, which is used to capture the historical information of the workload vector; c t Represents the cell state of LSTM at time t, which is used to maintain the long-term memory of the workload vector; The attention weight vector representing the workload vector; , , Both represent learnable weight matrices; represents the workload vector input at time t , represents the CPU usage vector at time t, represents the memory usage vector at time t, represents the request quantity vector at time t; b represents the bias vector. The Softmax() function represents the normalization function.

[0153] The workload prediction agent unit adopts a multi-head attention structure, configured with 4 attention heads, and the hidden dimension of each attention head is 64; the attention head obtains the attention score by calculating the similarity between the query vector and the key vector, and then normalizes it with the value vector and weighted sums it to generate the context vector. This mechanism can adaptively capture the long-term and short-term dependencies in the time series. The LSTM network adopts a two-layer structure, and the expression ability of the model is improved by increasing the network depth; in terms of network configuration, the input dimension is determined according to the number of features, the hidden layer dimension is set to 128, and a dropout mechanism of 0.2 is introduced to prevent overfitting. The network output is mapped to the target dimension through a fully connected layer to predict resource usage at future moments.

[0154] The workload prediction agent unit captures long-term dependencies in workload data by introducing an attention mechanism, thereby significantly improving prediction accuracy.

[0155] In the scaling model creation module, the formula of the state space layering module is:

[0156] ;

[0157] ;

[0158] ;

[0159] ;

[0160] Where S represents the system state vector; represents the workload state vector; represents the resource utilization state vector; represents the performance state vector; represents the workload vector input at time t; represents the workload vector predicted at time t+1; It represents the load variance, which is calculated based on the workload prediction results and workload data and is used to measure volatility; Indicates the load change rate, which is calculated based on the workload prediction results and workload data; Represents the CPU usage vector at time t; Represents the memory usage vector at time t; represents the request quantity vector at time t; Indicates the SLA achievement status, with a value range of [0,1); Indicates the queue length of requests; Indicates the average response delay of the request; represents the SLA default rate;

[0161] In the cloud environment of Kubernetes containers, in order to achieve accurate dynamic adjustment of cloud computing resources, the present invention designs a three-layer state space structure to comprehensively capture the system status from three dimensions: workload status, resource utilization status, and performance status. This hierarchical structure can evaluate the expansion needs of cloud computing resources from different levels and make more accurate decisions.

[0162] The first layer is the workload status, which focuses on load characteristics and changing trends. For example, the system collects current workload data every 15 seconds, including real-time indicators such as CPU utilization (such as 75%), memory utilization (such as 60%), and number of requests per second (such as 1,000). At the same time, based on the workload prediction results, it records the predicted load for the next 5 minutes. For example, it is expected that the CPU utilization will reach 85% and the number of requests will increase to 1,200 per second. In addition, the load fluctuation and change speed are also calculated. For example, the CPU utilization fluctuation range is ±10%, and the CPU utilization increases by 2% every minute. These indicators together constitute the system's comprehensive understanding of the workload.

[0163] The second layer is the resource utilization status, which focuses on the usage of cloud computing resources and monitors the resource usage details of each Pod, such as Pod1's CPU usage is 80% and its memory usage is 2GB; it records the resource load level of the entire Kubernetes container cluster, such as the average CPU usage of the Kubernetes container cluster is 78% and its memory usage is 65%; it also records the usage of network resources, such as the current bandwidth usage of 500Mbps and the number of active connections of 1,000; in order to ensure service quality, it also monitors the achievement of the service level agreement (SLA), such as 95% of requests can be responded to within 100ms and the service availability reaches 99.9%.

[0164] The third layer is the performance status, which focuses on the system operation effect and records the request queue status of each service. For example, service A currently has 100 requests waiting, and 10 requests are added every minute. It monitors the response time of different types of requests, such as the average response time of read operations is 50ms and that of write operations is 80ms. It also counts SLA breaches, such as 5% of the request response time exceeds the predetermined threshold. These indicators directly reflect the service quality of the system and are an important basis for resource adjustment decisions.

[0165] These three layers of state information are interrelated and influence each other. For example, when workload forecasts show that future loads will increase significantly, and current resource utilization is already high (such as CPU utilization exceeding 80%), and performance indicators show that the request queue begins to grow (such as an increase of 10 requests per minute), the expansion operation is started in advance. Specifically, the number of Pods may be increased first (such as from 3 to 4), or the resource limits of existing Pods may be increased (such as increasing the number of CPU cores from 2 to 3). Through this layered state space design, changes in the cloud environment can be fully and timely perceived, possible resource pressures can be predicted, and reasonable expansion decisions can be made in advance.

[0166] The resource adjustment agent unit is constructed based on the DQN network, and the formula is: ;

[0167] in, represents the system state vector at time t; represents the extended action at time t; the action space of the extended action is defined as Indicates horizontal expansion action, used to adjust the number of Pods of Kubernetes containers. Represents a vertical expansion action, which is used to adjust the resource limit of the Pod. The action space is used to comprehensively capture multi-dimensional resource characteristics. , -1 means reducing one Pod, +2 means adding two Pods; , C represents the CPU configuration set, M represents the memory configuration set, such as (2,4096) means setting 2 CPU cores and 4 GB memory; express and Corresponding cloud computing resource scheduling strategy; A network function representing a resource adjustment agent unit; Represents the reward value of the reward function at time t; represents the discount factor, the value is 0.99; represents the extended action at time t+1; Represents the system state vector at time t+1; maxQ() represents the DQN network.

[0168] In the scaling model creation module, the formula for the scheduling experience generated by the attention experience playback module is: ;

[0169] in, Indicates scheduling experience; represents the system state vector at time t; represents the extended action at time t; Represents the reward value of the reward function at time t; Represents the system state vector at time t+1; Indicates the system status; Indicates an extended action; Represents the reward value; Indicates the next system state; in specific implementation, the scheduling experience is stored in the experience playback buffer;

[0170] The calculation formula of the attention weight is: ;

[0171] in, represents the attention weight at time t; e i represents the embedding vector of the i-th scheduling experience; e j represents the embedding vector of the jth scheduling experience; T represents transpose; represents the empirical projection matrix; An embedding vector representing the current system state;

[0172] The training and optimization of the scaling model based on the attention weight screening scheduling experience is specifically as follows:

[0173] Calculate the sampling probability based on the attention weights: ;

[0174] in, represents the sampling probability of the ith scheduling experience; Indicates the temperature parameter, the value is 2; represents the i-th attention weight; represents the jth attention weight;

[0175] Training and optimizing the scaling model based on the sampling probability screening scheduling experience;

[0176] The formula of the reward function is: ;

[0177] in, represents the system state vector at time t; represents the extended action at time t; express and The corresponding reward value; Both represent weight factors, and , used to balance the three optimization goals of resource utilization, cost, and SLA; express and Corresponding resource utilization rewards; express and Corresponding cost rewards; express and The corresponding SLA reward. util (), R cost (), R sla () all adopt a segmented design, that is, differentiated punishments are imposed on violations of different degrees.

[0178] ;

[0179] in, represents the target utilization; Indicates the current resource utilization; Indicates the maximum utilization; and Both represent penalty coefficients, which are used to adjust the intensity of rewards or penalties within different ranges; ;

[0180] in, represents the current cost; represents the basic cost; represents the maximum cost; represents the rate of change of cost; and Both represent weight parameters, which are used to control the impact of current cost and cost change respectively; ;

[0181] in, represents the SLA default rate; Indicates the SLA default rate threshold; and Both represent penalty coefficients, which are used to control the intensity of rewards or penalties within different default rate ranges.

[0182] In the scaling model creation module, the workload data includes at least CPU usage, memory usage vector, number of requests, SLA achievement status, request queue length, average request response delay, and SLA breach rate; in the cloud environment of Kubernetes containers, the workload data is expressed as a series of time series data. In specific implementation, the workload data needs to be standardized to eliminate the dimensionality effect and improve the generalization ability of the scaling model;

[0183] The data set construction module is specifically used for:

[0184] Obtain a large amount of historical workload data of Kubernetes containers, and construct a data set after performing preprocessing on each of the historical workload data, including at least null value processing, data standardization processing, business key value processing, data merging, and missing value processing;

[0185] The telescopic model training module is specifically used for:

[0186] Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, and continuously optimizing the hyperparameters of the scaling model including at least a learning rate, a training batch, a random dropout rate, and a weight decay rate until a preset convergence condition is met;

[0187] Verifying the trained scaling model through the verification set to determine whether the scheduling accuracy of the cloud computing resource scheduling strategy is greater than a preset accuracy threshold, if not, expanding the training set to continue training; if yes, the verification is passed;

[0188] The verified scaling model is tested using the test set to determine whether the confidence of the cloud computing resource scheduling strategy is greater than a preset confidence threshold. If not, the training set is expanded to continue training; if yes, the test passes.

[0189] In order to facilitate understanding of the present invention, further examples are given:

[0190] In a typical e-commerce platform's order processing system, a cloud computing resource scheduling strategy at a certain time on a certain day is used as an example to illustrate.

[0191] The workload prediction agent analyzes historical workload data: The current status of the Kubernetes container is: CPU usage 65%, memory usage 50%, and the workload prediction agent outputs the workload prediction results: ;

[0192] in, Contains information about workload changes over the past 24 hours, especially the trend of a 15% increase in CPU usage in the last hour.

[0193] Attention weight vector Calculated by the following formula: ;

[0194] in, represents the workload vector input at time t .

[0195] Through this calculation, the workload prediction agent unit focuses on the characteristics of rapid load increase in the last hour, and predicts that the system status in 15 minutes will become: CPU utilization rate 85%, request volume 1200 times / second.

[0196] Based on the workload prediction results of the workload prediction agent unit, a system status map is constructed. Enter the resource adjustment agent unit.

[0197] The resource adjustment agent unit evaluates the value of different expansion actions based on the system state vector: ;

[0198] The workload prediction results of the workload prediction agent unit are included, so that the resource adjustment agent unit can prepare for the predicted load growth in advance.

[0199] During the evaluation process, the expected reward value of each extended action is calculated through the reward function: ;

[0200] Considering the load growth predicted by the workload prediction agent, the action of adding a Pod obtains the highest expected reward value. Execute the cloud computing resource scheduling strategy, store this scheduling experience in the experience playback buffer, and calculate the attention weight of the scheduling experience: ;

[0201] Among them, e i It contains key information about the current scheduling experience: the prediction accuracy of the workload prediction agent (prediction error < 5%), the actual effect of the cloud computing resource scheduling strategy (CPU utilization dropped to 70%, response time maintained at 120 ms), etc. The attention weight of the scheduling experience determines the probability of the scheduling experience being selected in future training: ;

[0202] After 15 minutes, calculate the actual reward value:

[0203] R util =0.85, because the CPU usage is maintained in the ideal range of 70%-75%;

[0204] R cost =-0.2, the cost of adding a Pod is 2 yuan / hour;

[0205] R sla =0.9, the response time of 95% requests is less than 150ms;

[0206] The final comprehensive reward value is 0.57, which verifies the effectiveness of this cloud computing resource scheduling strategy.

[0207] This complete decision cycle demonstrates the connection between various modules: the workload prediction results of the workload prediction agent unit are passed to the resource adjustment agent unit through the state space hierarchical module, affecting the selection of expansion actions; the results after the execution of the cloud computing resource scheduling strategy are evaluated through the reward function to evaluate the quality of the decision, and the attention experience replay module is used to guide future decisions. This interlocking design ensures that the system can accurately predict workload changes, adjust cloud computing resource configuration in a timely manner, and continuously learn and optimize from experience.

[0208] In summary, the advantages of the present invention are:

[0209] Through the dual-agent module, the state-space hierarchical module and the attention experience replay module, a scaling model for cloud computing resource scheduling of Kubernetes containers is created, and the reward function of the scaling model is set. The dual-agent module includes a workload prediction agent unit and a resource adjustment agent unit. Then, a large amount of historical workload data of Kubernetes containers is obtained for preprocessing to construct a data set, and the data set is divided into a training set, a validation set and a test set. The scaling model is trained with the training set, and the scaling model is continuously optimized with the reward function during the training process. The trained scaling model is verified with the validation set, and the verified scaling model is tested with the test set. Finally, the cloud computing resources of the Kubernetes container are scheduled with the scaling model that has passed the test. That is, cloud computing resources are scheduled with the scaling model trained with a large amount of historical workload data, and the workload prediction agent unit of the scaling model is used to output the workload prediction result to the state-space hierarchical module based on the input workload data; the state-space hierarchical module is used to output the workload prediction result to the state-space hierarchical module based on the input workload prediction result and the workload Load data, outputs the system state vector including workload state vector, resource utilization state vector and performance state vector to the resource adjustment agent unit; the resource adjustment agent unit is used to output the cloud computing resource scheduling strategy according to the input system state vector; the attention experience replay module is used to generate scheduling experience according to the system state vector, resource scheduling strategy and the reward value of the reward function, calculate the attention weight of each scheduling experience, and train and optimize the scaling model based on the attention weight screening scheduling experience; that is, in the cloud computing resource scheduling process of the Kubernetes container, the workload prediction results and the cloud computing resource scheduling strategy are comprehensively considered through the dual agent module, the multi-dimensional characteristics of the Kubernetes container are comprehensively characterized through the state space hierarchical module to improve the rationality of cloud computing resource scheduling, and the value of historical data is fully utilized through the attention experience replay module, the accuracy of workload prediction is effectively improved, and the learning efficiency of the scaling model and the optimization of the cloud computing resource scheduling strategy are improved, which ultimately greatly improves the resource utilization, efficiency and reliability of the cloud computing resource scheduling of the Kubernetes container.

[0210] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A dual-agent based Kubernetes container automatic scaling method, characterized by: The steps include: Step S1, based on a dual agent module including a workload prediction agent unit and a resource adjustment agent unit, a state space hierarchical module, and an attention experience playback module, a scaling model for cloud computing resource scheduling for Kubernetes containers is created, and a reward function of the scaling model is set; The workload prediction agent unit is used to output workload prediction results to the state space hierarchical module based on the input workload data; The state space hierarchical module is used to output a system state vector including a workload state vector, a resource utilization state vector and a performance state vector to the resource adjustment agent unit according to the input workload prediction result and workload data; The resource adjustment agent unit is used to output a cloud computing resource scheduling strategy according to an input system state vector; The attention experience playback module is used to generate scheduling experience according to the system state vector, resource scheduling strategy and reward value of reward function, calculate the attention weight of each scheduling experience, and train and optimize the scaling model by screening the scheduling experience based on the attention weight; Step S2: Obtain a large amount of historical workload data of Kubernetes containers, and construct a data set after preprocessing each of the historical workload data; Step S3: dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, validating the trained scaling model through the validation set, and testing the validated scaling model through the test set; Step S4: Scheduling the cloud computing resources of the Kubernetes container by using the scaling model that has passed the test; In step S1, the workload prediction agent unit is constructed based on the LSTM network, and the formula is: ; ; in, Represents the workload prediction result, that is, the workload vector predicted at time t+1 , Represents the CPU usage vector predicted at time t+1, Represents the memory usage vector predicted at time t+1, Represents the predicted request quantity vector at time t+1; The network function representing the workload prediction agent unit; h t represents the hidden state of LSTM at time t, which is used to capture the historical information of the workload vector; c t Represents the cell state of LSTM at time t, which is used to maintain the long-term memory of the workload vector; The attention weight vector representing the workload vector; , , Both represent learnable weight matrices; represents the workload vector input at time t , represents the CPU usage vector at time t, represents the memory usage vector at time t, represents the request quantity vector at time t; b represents the bias vector; In step S1, the formula of the state space hierarchical module is: ; ; ; ; in, represents the system state vector at time t; represents the workload state vector; represents the resource utilization state vector; represents the performance state vector; represents the workload vector input at time t; represents the workload vector predicted at time t+1; represents the load variance, which is calculated based on the workload prediction results and workload data; Indicates the load change rate, which is calculated based on the workload prediction results and workload data; Represents the CPU usage vector at time t; Represents the memory usage vector at time t; represents the request quantity vector at time t; Indicates the SLA achievement status, with a value range of [0,1); Indicates the queue length of requests; Indicates the average response delay of the request; represents the SLA default rate; The resource adjustment agent unit is constructed based on the DQN network, and the formula is: ; in, represents the system state vector at time t; represents the extended action at time t; the action space of the extended action is defined as , Indicates horizontal expansion action, used to adjust the number of Pods of Kubernetes containers. Indicates a vertical scaling action, which is used to adjust the resource limits of a Pod; express and Corresponding cloud computing resource scheduling strategy; A network function representing a resource adjustment agent unit; represents the reward value of the reward function at time t; γ represents the discount factor; represents the extended action at time t+1; Represents the system state vector at time t+1.

2. A dual-agent based Kubernetes container automatic scaling method as described in claim 1, characterized in that: In step S1, the formula for the scheduling experience generated by the attention experience playback module is: ; in, Indicates scheduling experience; represents the system state vector at time t; represents the extended action at time t; Represents the reward value of the reward function at time t; Represents the system state vector at time t+1; Indicates the system status; Indicates an extended action; Represents the reward value; Indicates the next system state; The calculation formula of the attention weight is: ; in, represents the attention weight at time t; e i represents the embedding vector of the i-th scheduling experience; e j represents the embedding vector of the j-th scheduling experience; T represents transposition; represents the empirical projection matrix; An embedding vector representing the current system state; The training and optimization of the scaling model based on the attention weight screening scheduling experience is specifically as follows: Calculate the sampling probability based on the attention weights: ; in, represents the sampling probability of the ith scheduling experience; Indicates the temperature parameter, the value is 2; represents the i-th attention weight; represents the jth attention weight; Training and optimizing the scaling model based on the sampling probability screening scheduling experience; The formula of the reward function is: ; in, represents the system state vector at time t; represents the extended action at time t; express and The corresponding reward value; , , are weight factors, and ; express and Corresponding resource utilization rewards; express and Corresponding cost rewards; express and Corresponding SLA rewards.

3. A dual-agent based Kubernetes container automatic scaling method as described in claim 1, characterized in that: In step S1, the workload data includes at least CPU usage, memory usage vector, number of requests, SLA achievement status, queue length of requests, average response delay of requests, and SLA breach rate; The step S2 is specifically as follows: Obtain a large amount of historical workload data of Kubernetes containers, and construct a data set after performing preprocessing on each of the historical workload data, including at least null value processing, data standardization processing, business key value processing, data merging, and missing value processing; The step S3 is specifically as follows: Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, and continuously optimizing the hyperparameters of the scaling model including at least a learning rate, a training batch, a random dropout rate, and a weight decay rate until a preset convergence condition is met; Verifying the trained scaling model through the verification set to determine whether the scheduling accuracy of the cloud computing resource scheduling strategy is greater than a preset accuracy threshold, if not, expanding the training set to continue training; if yes, the verification is passed; The verified scaling model is tested using the test set to determine whether the confidence of the cloud computing resource scheduling strategy is greater than a preset confidence threshold. If not, the training set is expanded to continue training; if yes, the test passes.

4. A dual-agent based Kubernetes container automatic scaling system, characterized by: Includes the following modules: A scaling model creation module, used to create a scaling model for cloud computing resource scheduling for Kubernetes containers based on a dual agent module including a workload prediction agent unit and a resource adjustment agent unit, a state space hierarchical module, and an attention experience playback module, and set a reward function for the scaling model; The workload prediction agent unit is used to output workload prediction results to the state space hierarchical module based on the input workload data; The state space hierarchical module is used to output a system state vector including a workload state vector, a resource utilization state vector and a performance state vector to the resource adjustment agent unit according to the input workload prediction result and workload data; The resource adjustment agent unit is used to output a cloud computing resource scheduling strategy according to an input system state vector; The attention experience playback module is used to generate scheduling experience according to the system state vector, resource scheduling strategy and reward value of reward function, calculate the attention weight of each scheduling experience, and train and optimize the scaling model by screening the scheduling experience based on the attention weight; A data set construction module is used to obtain a large amount of historical workload data of Kubernetes containers, and to construct a data set after preprocessing each of the historical workload data; a scaling model training module, used for dividing the data set into a training set, a validation set and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, validating the trained scaling model through the validation set, and testing the validated scaling model through the test set; The container automatic scaling module is used to schedule the cloud computing resources of the Kubernetes container through the scaling model that has passed the test; In the scaling model creation module, the workload prediction agent unit is constructed based on the LSTM network, and the formula is: ; ; in, Represents the workload prediction result, that is, the workload vector predicted at time t+1 , Represents the CPU usage vector predicted at time t+1, Represents the memory usage vector predicted at time t+1, Represents the predicted request quantity vector at time t+1; The network function representing the workload prediction agent unit; h t represents the hidden state of LSTM at time t, which is used to capture the historical information of the workload vector; c t Represents the cell state of LSTM at time t, which is used to maintain the long-term memory of the workload vector; The attention weight vector representing the workload vector; , , Both represent learnable weight matrices; represents the workload vector input at time t , represents the CPU usage vector at time t, represents the memory usage vector at time t, represents the request quantity vector at time t; b represents the bias vector; In the telescoping model creation module, the formula of the state space layering module is: ; ; ; ; in, represents the system state vector at time t; represents the workload state vector; represents the resource utilization state vector; represents the performance state vector; represents the workload vector input at time t; represents the workload vector predicted at time t+1; represents the load variance, which is calculated based on the workload prediction results and workload data; Indicates the load change rate, which is calculated based on the workload prediction results and workload data; Represents the CPU usage vector at time t; Represents the memory usage vector at time t; represents the request quantity vector at time t; Indicates the SLA achievement status, with a value range of [0,1); Indicates the queue length of requests; Indicates the average response delay of the request; represents the SLA default rate; The resource adjustment agent unit is constructed based on the DQN network, and the formula is: ; in, represents the system state vector at time t; represents the extended action at time t; the action space of the extended action is defined as , Indicates horizontal expansion action, used to adjust the number of Pods of Kubernetes containers. Indicates a vertical scaling action, which is used to adjust the resource limits of a Pod; express and Corresponding cloud computing resource scheduling strategy; A network function representing a resource adjustment agent unit; represents the reward value of the reward function at time t; γ represents the discount factor; represents the extended action at time t+1; Represents the system state vector at time t+1.

5. A dual-agent based Kubernetes container automatic scaling system as described in claim 4, characterized in that: In the scaling model creation module, the formula for the scheduling experience generated by the attention experience playback module is: ; in, Indicates scheduling experience; represents the system state vector at time t; represents the extended action at time t; Represents the reward value of the reward function at time t; Represents the system state vector at time t+1; Indicates the system status; Indicates an extended action; Indicates the reward value; Indicates the next system state; The calculation formula of the attention weight is: ; in, represents the attention weight at time t; e i represents the embedding vector of the i-th scheduling experience; e j represents the embedding vector of the j-th scheduling experience; T represents transposition; represents the empirical projection matrix; An embedding vector representing the current system state; The training and optimization of the scaling model based on the attention weight screening scheduling experience is specifically as follows: Calculate the sampling probability based on the attention weights: ; in, represents the sampling probability of the ith scheduling experience; Indicates the temperature parameter, the value is 2; represents the i-th attention weight; represents the jth attention weight; Training and optimizing the scaling model based on the sampling probability screening scheduling experience; The formula of the reward function is: ; in, represents the system state vector at time t; represents the extended action at time t; express and The corresponding reward value; , , Both represent weight factors, and ; express and Corresponding resource utilization rewards; express and Corresponding cost rewards; express and Corresponding SLA rewards.

6. A dual-agent-based Kubernetes container automatic scaling system as described in claim 4, characterized in that: In the scaling model creation module, the workload data includes at least CPU usage, memory usage vector, number of requests, SLA achievement status, queue length of requests, average response delay of requests, and SLA breach rate; The data set construction module is specifically used for: Obtain a large amount of historical workload data of Kubernetes containers, and construct a data set after performing preprocessing on each of the historical workload data, including at least null value processing, data standardization processing, business key value processing, data merging, and missing value processing; The telescopic model training module is specifically used for: Dividing the data set into a training set, a validation set, and a test set based on a preset ratio, training the scaling model through the training set, continuously optimizing the scaling model through the reward function during the training process, and continuously optimizing the hyperparameters of the scaling model including at least a learning rate, a training batch, a random dropout rate, and a weight decay rate until a preset convergence condition is met; Verifying the trained scaling model through the verification set to determine whether the scheduling accuracy of the cloud computing resource scheduling strategy is greater than a preset accuracy threshold, if not, expanding the training set to continue training; if yes, the verification is passed; The verified scaling model is tested using the test set to determine whether the confidence of the cloud computing resource scheduling strategy is greater than a preset confidence threshold. If not, the training set is expanded to continue training; if yes, the test passes.

Images