Resource scheduling method and resource scheduling system

Abstract

The invention provides a resource scheduling method and a resource scheduling system, and relates to the technical field of computers. The method comprises the following steps: acquiring current observation data of a cluster; wherein the current observation data comprises a cluster resource utilization rate, SLO state data, Node observation data and Pod observation data; predicting the resource condition of the cluster based on the Node observation data and the Pod observation data by using a graph neural network to obtain a resource prediction score; generating a current state vector of the cluster according to the resource prediction score, the cluster resource utilization rate and the SLO state data; and generating a resource scheduling strategy based on the current state vector through the reinforcement learning model, and scheduling the resources of the cluster according to the resource scheduling strategy. According to the method, prediction is performed by using the graph neural network based on the acquired cluster observation data, the resource scheduling strategy is determined by combining reinforcement learning and the prediction result, and the cluster resources are scheduled according to the resource scheduling strategy, so that the resource scheduling effect is optimized.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a resource scheduling method and a resource scheduling system. Background Technology

[0002] With the rapid popularization of cloud computing and microservices, information systems are gradually evolving from monolithic architectures to highly elastic, loosely coupled integrated platforms. To achieve resource scheduling within these integrated platforms, existing technologies employ a combination of rule-based and metric-based horizontal / vertical Pod Autoscalers (HPA / VPA) and the default Kubernetes scheduler. The core idea is to perform passive, reactive scheduling and scaling based on pre-defined rules and current / historical resource metrics. However, this approach suffers from poor resource scheduling performance. Summary of the Invention

[0003] The purpose of this application is to provide a resource scheduling method and a resource scheduling system, which uses graph neural networks to make predictions based on collected cluster observation data, and combines reinforcement learning and prediction results to determine resource scheduling strategies, and schedules cluster resources according to the resource scheduling strategies, thereby optimizing resource scheduling effects.

[0004] In a first aspect, embodiments of this application provide a resource scheduling method, the method comprising: acquiring current observation data of a cluster; wherein the current observation data includes cluster resource utilization, SLO status data, Node observation data, and Pod observation data; using a graph neural network, based on the Node observation data and Pod observation data, predicting the resource status of the cluster to obtain a resource prediction score; generating a current state vector of the cluster based on the resource prediction score, cluster resource utilization, and SLO status data; generating a resource scheduling strategy based on the current state vector using a reinforcement learning model, and scheduling the cluster resources according to the resource scheduling strategy.

[0005] In this embodiment, current observation data of the cluster is acquired, and a graph neural network is used to predict the cluster's resource status based on Node and Pod observation data. This allows for the identification of potential resource bottlenecks in the cluster over a future period, enabling proactive intervention and predictive resource scheduling. Furthermore, based on the prediction results, a reinforcement learning model is used to generate a resource scheduling strategy based on the current state vector. This allows for the continuous and automatic identification of optimal parameter configurations that ensure application performance while maximizing cluster resource utilization, based on the cluster's real-time load and historical patterns. This achieves refined resource allocation and optimizes resource scheduling.

[0006] In some embodiments, a graph neural network is used to predict the resource status of the cluster based on Node observation data and Pod observation data to obtain a resource prediction score. This includes: generating a network topology graph based on Node and Pod observation data; wherein the network topology graph includes Node nodes and Pod nodes; Node nodes include Node feature vectors, and Pod nodes include Pod feature vectors; inputting the network topology graph into the graph neural network, using the message passing mechanism of the graph neural network to update the Node feature vectors based on the feature vectors of the Node nodes' neighboring nodes, and updating the Pod feature vectors based on the feature vectors of the Pod nodes' neighboring nodes, to obtain the Node global feature vectors and Pod global feature vectors of the Pod nodes; and using the prediction mechanism of the graph neural network, predicting the cluster resource status based on the Node global feature vectors and Pod global feature vectors to obtain a resource prediction score.

[0007] In this embodiment, a network topology graph is generated based on Node and Pod observation data. This graph represents the hierarchical relationships between Nodes and Pods, as well as the calling relationships between different Pods. A message passing mechanism is used to capture the dependencies and communication patterns between Nodes and Pods, and between different Pods, generating global feature vectors for Nodes and Pods. Based on these global feature vectors, the cluster's resource status is predicted, ensuring that the obtained resource prediction score reflects a global understanding of the cluster, thus improving prediction accuracy. This prediction result directly influences the determination of resource scheduling strategies, thereby optimizing resource scheduling performance.

[0008] In some embodiments, the resource prediction score includes a Node resource pressure score and a Pod resource risk score. The prediction mechanism of a graph neural network is used to predict the cluster's resource status based on the Node global feature vector and the Pod global feature vector to obtain the resource prediction score. This includes: calculating the Node resource pressure score of a Node based on the Node global feature vector and the Pod global feature vectors of all Pod nodes adjacent to the Node; and calculating the Pod resource risk score of a Pod based on the Pod global feature vector and the global feature vectors of the Node nodes adjacent to the Pod and other Pod nodes.

[0009] In this embodiment, the resource prediction score includes a Node resource pressure score and a Pod resource risk score. The Node resource pressure score is calculated using the global feature vectors of the Node and its neighboring nodes, and the Pod resource risk score is calculated using the global feature vectors of the Pod and its neighboring nodes. By considering the global feature vectors of neighboring nodes, the accuracy of the prediction results is improved, thereby optimizing the resource scheduling effect.

[0010] In some embodiments, generating the current state vector of the cluster based on resource prediction scores, cluster resource utilization, and SLO status data includes: concatenating the Node resource pressure score of each Node node, the Pod resource risk score of each Pod node, the cluster resource utilization, and the SLO status data to generate the current state vector of the cluster.

[0011] In this embodiment, the current state vector is generated by concatenating the Node resource pressure score of each Node node, the Pod resource risk score of each Pod node, the cluster resource utilization rate, and the SLO status data. This current state vector is a digital representation of the overall health status of the cluster in the current and short-term future. It will serve as the "eyes" of the reinforcement learning agent, improve the accuracy of decision-making, and thus optimize the resource scheduling effect.

[0012] In some embodiments, a resource scheduling policy is generated based on the current state vector using a reinforcement learning model, and the cluster resources are scheduled according to the resource scheduling policy. This includes: determining a target action based on the current state vector using a policy network of the reinforcement learning model; wherein the target action is used to generate the resource scheduling policy; and scheduling the cluster resources according to the resource scheduling policy.

[0013] In this embodiment, a reinforcement learning model is used to determine the target action based on the current state vector to generate a resource scheduling strategy. Since the current state vector is a digital representation of the cluster's global health status in the current and short-term future, and the reinforcement learning model can dynamically adjust cluster resource parameters based on the target action, decision-making accuracy is improved, thereby optimizing resource scheduling performance.

[0014] In some embodiments, the target action includes adjusting the Pod identifier and the CPU resource request change value; scheduling the cluster resources according to the resource scheduling policy, including: determining the target Pod to be adjusted based on the Pod identifier; adjusting the CPU resource request value of the target Pod to be adjusted according to the CPU resource request change value, so as to schedule the cluster resources.

[0015] In this embodiment, by dynamically adjusting the CPU resource request value of the Pod, the refined supply and dynamic automation of cluster resources are realized, the cluster resource utilization rate is improved, and the cluster scheduling effect is optimized.

[0016] In some embodiments, the target action includes a migration Pod identifier and a target Node identifier; scheduling cluster resources according to a resource scheduling policy includes: determining the target Pod to be migrated based on the migration Pod identifier, determining the target Node based on the target Node identifier; and migrating the target Pod to be migrated to the target Node in order to schedule cluster resources.

[0017] In this embodiment of the application, by migrating the Pod to be migrated to the target Node, the cluster resources are dynamically adjusted, the cluster resource utilization is improved, and the cluster scheduling effect is optimized.

[0018] In some embodiments, after generating a resource scheduling policy based on the current state vector using a reinforcement learning model and scheduling the cluster's resources according to the resource scheduling policy, the method further includes: obtaining the latest observation data after the cluster executes the resource scheduling policy; determining the latest state vector based on the latest observation data and calculating the reward score; updating the policy network parameters of the reinforcement learning model according to the current state vector, the target action, the latest state vector, and the reward score to obtain the updated reinforcement learning model.

[0019] In this embodiment, the parameters of the reinforcement learning model are updated using the state data of the cluster, thereby forming a closed-loop learning system from data to decision to feedback. This enables the reinforcement learning model to continuously adapt to load changes, improves the performance of the reinforcement learning model, and thus improves the accuracy of decision-making.

[0020] In some embodiments, the formula for calculating the reward score is as follows: ;in, , , These are the weighting coefficients. Indicates the reward score. This represents the average CPU utilization of the cluster after implementing the resource scheduling policy. This indicates the number of SLO violations after the resource scheduling policy is executed. This indicates the overhead of executing the resource scheduling strategy.

[0021] In this embodiment of the application, SLO status data is considered when calculating the reward score. SLO status data reflects business stability, that is, whether the user experience meets the standard. This makes the ultimate goal of the reinforcement learning model no longer to optimize abstract cluster resource indicators, but to ensure that every scheduling decision (adjusting resources, migrating Pods) is directly aimed at ensuring the end user experience, thus realizing business value-driven resource scheduling.

[0022] Secondly, embodiments of this application provide a resource scheduling system, which includes a data acquisition module, a prediction module, and a decision engine module. The data acquisition module is used to collect observation data of the cluster. The current observation data includes cluster resource utilization, SLO status data, Node observation data, and Pod observation data. The prediction module is used to use a graph neural network to predict the resource status of the cluster based on the Node observation data and Pod observation data, and obtain a resource prediction score. The prediction module generates a current state vector of the cluster based on the resource prediction score, cluster resource utilization, and SLO status data. The decision engine module generates a resource scheduling strategy based on the current state vector using a reinforcement learning model, and schedules the cluster resources according to the resource scheduling strategy.

[0023] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions, when executed by the processor, are capable of performing the method steps of any embodiment of the first aspect.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium, including: computer program instructions stored on the computer-readable storage medium, wherein the computer program instructions are executed by a processor to perform the method steps of any embodiment of the first aspect.

[0025] Fifthly, embodiments of this application provide a computer program product, including: computer program instructions, which are executed by a processor to perform the method steps of any embodiment of the first aspect.

[0026] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a resource scheduling method provided in an embodiment of this application; Figure 2 This application provides a schematic diagram of the structure of a resource scheduling system according to an embodiment of the present application; Figure 3 This is a schematic diagram of the electronic device structure provided in an embodiment of this application. Detailed Implementation

[0029] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0030] It should be noted that all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0031] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0032] With the rapid popularization of cloud computing and microservices, information systems are gradually evolving from monolithic architectures to highly elastic, loosely coupled integrated platforms. To achieve resource scheduling within these integrated platforms, existing technologies employ a combination of rule-based and metric-based horizontal / vertical Pod Autoscaling (HPA / VPA) and the default Kubernetes scheduler. The specific solution is as follows: a) Scheduling line (static decision-making process) Execution entity: Default scheduler (kube-scheduler) Core task: Select the most suitable Node for the newly created Pod.

[0033] Workflow: (1) Trigger: When a new Pod needs to be created, the scheduler is triggered to select a Node.

[0034] (2) Filter: Based on the static resource requirements declared by the Pod (such as requests and limits), filter out all Node nodes that do not meet the conditions (such as insufficient resources or mismatched node selectors).

[0035] (3) Scoring: The filtered Node nodes are scored in multiple dimensions, taking into account factors such as resource availability, affinity / anti-affinity rules, and data locality.

[0036] Bind: Select the node with the highest score and bind the Pod to that node.

[0037] b) Expansion line (dynamic adjustment process) Execution entities: HPA (Horizontal Pod Autoscaler) and VPA (Vertical Pod Autoscaler) Core task: Dynamically adjust application resource allocation based on real-time load.

[0038] Workflow: (1) Monitoring: The monitoring system (such as Prometheus) continuously collects resource usage metrics (such as CPU / memory utilization) of Pods and Nodes.

[0039] (2) Horizontal scaling (HPA): Automatically adjusts the number of Pod replicas according to preset targets (such as an average CPU utilization of 50%) to match load changes.

[0040] (3) Vertical scaling (VPA): Analyze the historical resource usage patterns of Pods and automatically suggest or adjust the requests and limits values ​​of individual Pods to achieve fine-grained resource allocation.

[0041] As discussed above, existing solutions combining rule-based and metric-based horizontal / vertical Pod Autoscaling (HPA / VPA) with the Kubernetes default scheduler only make decisions based on the instantaneous state of the cluster at the time of Pod creation, failing to recognize subsequent resource changes. Furthermore, they rely entirely on fixed resource requests and limits preset by developers, and cannot dynamically adjust according to actual runtime needs.

[0042] Furthermore, the entire process is passive and delayed. It only triggers action after monitoring metrics (such as CPU utilization) have exceeded thresholds, making it impossible to predict and intervene before bottlenecks occur. Moreover, the entire process—from metric exceeding threshold -> detection -> decision -> execution -> new Pod ready—involves significant delays, making it difficult to handle rapid load fluctuations or sensitive business operations.

[0043] Furthermore, HPA / VPA typically makes decisions only for a single application, lacking awareness of the overall resource status of the cluster and the dependencies between microservices, which can easily lead to conflicting or suboptimal decisions.

[0044] Therefore, the core idea behind the combination of rule-based and metric-based horizontal / vertical Pod Autoscalers (HPA / VPA) with the Kubernetes default scheduler is to perform passive, reactive scheduling and scaling based on pre-defined rules and current / historical resource metrics. However, this approach suffers from poor resource scheduling performance.

[0045] To address the aforementioned issues, this application provides a resource scheduling method that uses graph neural networks to make predictions based on collected cluster observation data, combines reinforcement learning with the prediction results to determine a resource scheduling strategy, and schedules cluster resources according to the resource scheduling strategy, thereby optimizing the resource scheduling effect.

[0046] Figure 1 This is a flowchart illustrating a resource scheduling method provided in an embodiment of this application. It is understood that the resource scheduling method provided in this application can be applied to terminal devices (also known as electronic devices) and servers; specifically, terminal devices can be smartphones, tablets, computers, personal digital assistants (PDAs), etc.; servers can be application servers or web servers. To facilitate understanding of the technical solutions provided in this application, the following uses a server as the execution subject to describe the application scenarios of the resource scheduling method provided in this application.

[0047] like Figure 1 As shown, the method includes: Step S101: Obtain the current observation data of the cluster; wherein, the current observation data includes cluster resource utilization, SLO status data, Node observation data and Pod observation data.

[0048] A cluster consists of multiple Node nodes, and each Node node runs multiple Pods. A Node node is the unit of work in the cluster; it is the physical or virtual host on which a Pod runs. A Pod is the smallest and simplest deployable and manageable unit of computing.

[0049] The current observation data of the cluster refers to a multi-dimensional data set that can comprehensively reflect the cluster's basic configuration and its operating workload status, covering various aspects such as infrastructure configuration, runtime performance, and service topology relationships.

[0050] Cluster resource utilization refers to the measurement data on the degree and efficiency of the use of infrastructure resources such as computing, storage, and network in a cluster, reflecting the rationality of resource allocation and the overall load status of the cluster.

[0051] Cluster resource utilization includes data such as CPU utilization, memory utilization, storage utilization, and network resource utilization.

[0052] SLO status data refers to metrics used to measure and record whether a service meets predefined service level objectives. It is usually calculated based on application performance indicators and reflects the quality and compliance of the service.

[0053] SLO status data includes delayed SLO-related data (request processing delay distribution, delayed SLO threshold, number of delayed SLO violations, delay compliance rate), availability SLO-related data, throughput SLO-related data, overall SLO status, and details of SLO violation events.

[0054] Node observation data refers to specific observation data for cluster worker nodes, covering node hardware configuration, resource capacity, running status, load, and aggregated information of Pods running on the node.

[0055] Node observation data includes Node hardware configuration data, Node resource data, Node runtime status data, Node performance and load data (real-time CPU utilization, real-time memory utilization, disk utilization, system load, network traffic, etc.), and Node aggregated Pod data (a list of all Pods running on the current node, total resource requests and actual usage aggregated for Pods on the node), etc.

[0056] Pod observation data refers to specific observation data for Pods, the smallest scheduling unit in a cluster. It covers the Pod's resource configuration, running status, performance indicators, and relationship information with other entities.

[0057] Pod observation data includes Pod configuration metadata, Pod resource specification data, Pod runtime status data, Pod performance metrics data (real-time CPU utilization, real-time memory usage, network traffic, etc.), and relationship data (belonging Node information, inter-service call relationships, etc.).

[0058] In the above implementation process, the current observation data of the cluster is collected through a preset data acquisition process to obtain relevant data.

[0059] In one implementation, the Kubernetes API Server's List-Watch mechanism is used to continuously collect cluster metadata in real time, including configuration information for all Nodes and Pods (such as CPU Requests / Limits, tags, etc.). Simultaneously, real-time time-series metrics, such as CPU utilization, memory utilization, and network throughput for each Pod, are collected from monitoring systems like Prometheus. Furthermore, data such as the call topology between Pods and network latency are collected from the control plane of the service mesh (such as Istio).

[0060] Step S102: Using a graph neural network, based on Node observation data and Pod observation data, predict the resource status of the cluster and obtain a resource prediction score.

[0061] In the above implementation process, the graph neural network is a pre-trained network model, such as using the GraphSAGE or GAT architecture, and is pre-trained using historical Node observation data and historical Pod observation data to obtain a graph neural network suitable for resource prediction scenarios.

[0062] The collected Node and Pod observation data are input into a graph neural network for data processing to obtain resource prediction scores related to the Node and Pod observation data.

[0063] Step S103: Generate the current state vector of the cluster based on the resource prediction score, cluster resource utilization rate and SLO status data.

[0064] In the above implementation process, the resource prediction score, cluster resource utilization rate and SLO status data are concatenated to generate the current state vector.

[0065] Since the resource prediction score describes future resource bottlenecks, while the cluster resource utilization rate and SLO status data describe the current cluster status, the current state vector is used to digitally represent the global health status of the cluster in the current and short-term future.

[0066] As can be seen from the above, cluster resource utilization includes various types of utilization data, such as CPU utilization, memory utilization, and storage utilization. SLO status data also includes various types of SLO-related data, such as latency SLO-related data (request processing latency distribution, latency SLO threshold, latency SLO violation count, latency compliance rate), availability SLO-related data, and details of SLO violation events.

[0067] Therefore, when generating the current state vector, one or more of them can be selected as one of the parameters and concatenated to generate the current state vector.

[0068] The specific parameters to be selected can be set according to the actual situation, and this application does not make specific restrictions on this.

[0069] Step S104: Using a reinforcement learning model, a resource scheduling strategy is generated based on the current state vector, and the cluster's resources are scheduled according to the resource scheduling strategy.

[0070] Reinforcement learning is a core paradigm in machine learning. Its goal is to enable an agent to learn to perform a series of actions through interaction with its environment in order to obtain the maximum cumulative reward.

[0071] In the above implementation process, a decision engine is formed through a reinforcement learning model. In this decision engine, the reinforcement learning model acts as an agent, used to generate resource scheduling strategies based on the current state vector. The reinforcement learning model can be the PPO algorithm.

[0072] In the resource scheduling scenario, the state space of the reinforcement learning model is defined as: the current state vector S_t; the action space of the reinforcement learning model is defined as: adjustment action, transfer action, and no action.

[0073] Specifically, the adjustment action could be: selecting a Pod and increasing / decreasing its CPURequest value by one unit (e.g., 0.1 cores). The migration action could be: selecting a Pod and a target Node and performing a migration operation.

[0074] Based on the current state S_t, the agent selects an action A_t through its policy network π(A_t|S_t).

[0075] For the computer to understand this abstract action, the abstract action A_t needs to be translated into a concrete, executable resource scheduling policy to schedule cluster resources according to the policy. For example, A_t can be translated into a concrete, executable Kubernetes API call.

[0076] In the above implementation process, the generated current state vector is input into the reinforcement learning model to generate a resource scheduling policy, and the cluster resources are scheduled according to the resource scheduling policy.

[0077] In this embodiment, current observation data of the cluster is acquired, and a graph neural network is used to predict the cluster's resource status based on Node and Pod observation data. This allows for the identification of potential resource bottlenecks in the cluster over a future period, enabling proactive intervention and predictive resource scheduling. Furthermore, based on the prediction results, a reinforcement learning model is used to generate a resource scheduling strategy based on the current state vector. This allows for the continuous and automatic identification of optimal parameter configurations that ensure application performance while maximizing cluster resource utilization, based on the cluster's real-time load and historical patterns. This achieves refined resource allocation and optimizes resource scheduling.

[0078] In some embodiments, a graph neural network is used to predict the resource status of the cluster based on Node observation data and Pod observation data to obtain a resource prediction score. This includes: generating a network topology graph based on Node and Pod observation data; wherein the network topology graph includes Node nodes and Pod nodes; Node nodes include Node feature vectors, and Pod nodes include Pod feature vectors; inputting the network topology graph into the graph neural network, using the message passing mechanism of the graph neural network to update the Node feature vectors based on the feature vectors of the Node nodes' neighboring nodes, and updating the Pod feature vectors based on the feature vectors of the Pod nodes' neighboring nodes, to obtain the Node global feature vectors and Pod global feature vectors of the Pod nodes; and using the prediction mechanism of the graph neural network, predicting the cluster resource status based on the Node global feature vectors and Pod global feature vectors to obtain a resource prediction score.

[0079] In the above implementation process, the specific construction method for generating the network topology graph based on Node observation data and Pod observation data is as follows: Each physical Node and each Pod is abstracted as a node in the network topology graph. Two types of edges are defined: (1) Membership edge: pointing from a Pod node to its own Node node, representing a scheduling relationship; (2) Call edge: pointing from one Pod node to another Pod node, representing a network call dependency relationship between microservices. Among them, the weight of the membership edge can be set to 1, representing a strong association; the weight of the call edge can be determined according to the standardized value of network latency or request rate.

[0080] In addition, corresponding feature vectors are generated for each Node and each Pod in the network topology graph. The Node feature vector is represented as [total CPU cores, available CPU cores, total memory, available memory, disk I / O rate, network bandwidth]. The Pod feature vector is represented as [CPU Request value, CPU Limit value, current CPU utilization, memory Request value, memory Limit value, current memory utilization].

[0081] In summary, the structure of the network topology graph can be represented as G=(V,E,X,W), where G represents the network topology graph, V represents the set of nodes (V={Nod node set ∪ Pod node set}), E represents the set of edges (E={Member edge ∪ Call edge}), X represents the set of node feature vectors, and W represents the set of edge weights. The neighboring nodes of each node in the graph can be determined based on edge relationships.

[0082] To facilitate the processing of network topology graphs by graph neural networks, the network topology graphs are converted into adjacency matrices and feature matrices for representation.

[0083] After the transformed data is input into the graph neural network, the message passing mechanism of the graph neural network is used to update the Node feature vector based on the feature vectors of the Node's neighboring nodes, and to update the Pod feature vector based on the feature vectors of the Pod's neighboring nodes, so as to obtain the Node global feature vector and the Pod global feature vector.

[0084] In this process, the message passing mechanism of graph neural networks is usually a multi-layer message passing mechanism. In each layer, each node aggregates the feature information of its neighboring nodes connected by edges; after multiple iterations, each node obtains a feature representation containing global information about its topological position.

[0085] Specifically, for the k-th layer (k≥1), neighbor aggregation and node update operations are performed on each node v in the graph. Neighbor aggregation operation: a_v^(k)=AGGREGATE_k({h_u^(k-1)|u∈N(v)}); where a_v^(k) represents the feature representation of all neighbor nodes after aggregating their feature vectors at layer (k-1), N(v) represents the set of all neighbor nodes of node v, h_u^(k-1) is the feature representation of neighbor node u at layer (k-1), and AGGREGATE_k represents the aggregation function, which can be any one of mean aggregation, maximum aggregation, summation, or attention-weighted summation. Node update operation: h_v^(k)=UPDATE_k(h_v^(k-1),a_v^(k)); where h_v^(k-1) is the feature representation of node v at layer (k-1), and h_v^(k) represents the updated feature representation of node v at layer k.

[0086] For example, suppose there is a 2-layer GNN model. For the central node A, in the first layer: A collects information from its direct neighbors (B, C, D) and updates it to obtain the Layer-1 embedding h_A^(1). In the second layer: Since B, C, and D are also aggregated and updated in the first layer as central nodes, the Layer-1 embeddings h_B^(1), h_C^(1), and h_D^(1) of B, C, and D are used when calculating h_A^(2). Among them, h_B^(1), h_C^(1), and h_D^(1) already contain the information of their respective neighbors.

[0087] Therefore, h_A^(2) actually indirectly and generally integrates the information of A's multi-level neighbors (such as B's neighbor E), and the information of E is transmitted to A through B as an intermediary.

[0088] Finally, the prediction mechanism of graph neural networks is used to predict the resource status of the cluster based on the global feature vectors of nodes and pods, and a resource prediction score is obtained.

[0089] In this embodiment, a network topology graph is generated based on Node and Pod observation data. This graph represents the hierarchical relationships between Nodes and Pods, as well as the calling relationships between different Pods. A message passing mechanism is used to capture the dependencies and communication patterns between Nodes and Pods, and between different Pods, generating global feature vectors for Nodes and Pods. Based on these global feature vectors, the cluster's resource status is predicted, ensuring that the obtained resource prediction score reflects a global understanding of the cluster, thus improving prediction accuracy. This prediction result directly influences the determination of resource scheduling strategies, thereby optimizing resource scheduling performance.

[0090] In some embodiments, the resource prediction score includes a Node resource pressure score and a Pod resource risk score. The prediction mechanism of a graph neural network is used to predict the cluster's resource status based on the Node global feature vector and the Pod global feature vector to obtain the resource prediction score. This includes: calculating the Node resource pressure score of a Node based on the Node global feature vector and the Pod global feature vectors of all Pod nodes adjacent to the Node; and calculating the Pod resource risk score of a Pod based on the Pod global feature vector and the global feature vectors of the Node nodes adjacent to the Pod and other Pod nodes.

[0091] In the above implementation process, considering that the network topology graph includes Node nodes and Pod nodes, when making predictions, it is possible to predict the resource status of only one type of node, or to predict the resource status of both types of node.

[0092] The Node resource stress score is used to measure the current and future resource load stress of a single Node.

[0093] The Pod resource risk score measures the probability that a Pod will face operational risks due to resource contention or Node pressure.

[0094] Specifically, it predicts and outputs a future resource pressure score for each Node in the network topology graph, and predicts and outputs a resource contention risk score for each Pod.

[0095] For example, a Node may have PodA (database), PodB (web server), and PodD (monitoring agent) running on it.

[0096] Regarding the resource stress score of the Node, the model considers that the current resource utilization of the Node is not high (CPU 44%, memory 44%), but the running Pods have high priority (database + monitoring), and the Web service may have the risk of sudden traffic surges. Therefore, the resource stress score of the Node is 0.65 (medium to high).

[0097] For the resource risk score of PodA (database), the direct risk is the health of PodA itself; the risk at the Node level: Node pressure score 0.65 → medium risk; the risk of competition from neighboring Pods: if PodB experiences a surge in traffic, it may compete for CPU, but PodB has a lower priority than A, and Kubernetes will prioritize PodA; the potential risk: if the Node needs maintenance, migrating a high-priority PodA is difficult; the overall calculation result: PodA risk score = 0.40 (low to medium), and its main risks come from: Node pressure + potential competition from neighboring Pods.

[0098] In this embodiment, the resource prediction score includes a Node resource pressure score and a Pod resource risk score. The Node resource pressure score is calculated using the global feature vectors of the Node and its neighboring nodes, and the Pod resource risk score is calculated using the global feature vectors of the Pod and its neighboring nodes. By considering the global feature vectors of neighboring nodes, the accuracy of the prediction results is improved, thereby optimizing the resource scheduling effect.

[0099] In some embodiments, generating the current state vector of the cluster based on resource prediction scores, cluster resource utilization, and SLO status data includes: concatenating the Node resource pressure score of each Node node, the Pod resource risk score of each Pod node, the cluster resource utilization, and the SLO status data to generate the current state vector of the cluster.

[0100] In the above implementation process, the Node resource pressure score of each Node represents the resource load pressure of each Node; the Pod resource risk score of each Pod represents the probability of the operational risk of each Pod. Therefore, by concatenating the Node resource pressure score of each Node, the Pod resource risk score of each Pod, the cluster resource utilization and SLO status data, a current state vector of the cluster is generated, which represents the current and short-term global health status of the cluster.

[0101] It should be noted that cluster resource utilization and SLO status data can each be selected from one or more specific data points.

[0102] In this embodiment, the current state vector is used as the "eyes" of the reinforcement learning agent to improve decision-making accuracy, thereby optimizing resource scheduling.

[0103] In some embodiments, a resource scheduling policy is generated based on the current state vector using a reinforcement learning model, and the cluster resources are scheduled according to the resource scheduling policy. This includes: determining a target action based on the current state vector using a policy network of the reinforcement learning model; wherein the target action is used to generate the resource scheduling policy; and scheduling the cluster resources according to the resource scheduling policy.

[0104] As can be seen from the above embodiments, the action space of the reinforcement learning model includes three types. In the actual execution process, the agent selects a target action A_t according to the current state S_t through its policy network π(A_t|S_t), and converts the selected target action into a resource scheduling policy for resource scheduling.

[0105] In one implementation, the target action includes adjusting the Pod identifier and the CPU resource request change value; scheduling the cluster resources according to the resource scheduling policy, including: determining the target Pod to be adjusted based on the Pod identifier; and adjusting the CPU resource request value of the target Pod to be adjusted according to the CPU resource request change value, so as to schedule the cluster resources.

[0106] Specifically, if the target action is a tweak, the Kubernetes API is called, and the PATCH corresponds to the Pod specification (spec.containers[].resources.requests.cpu).

[0107] In this process, by dynamically adjusting the CPU resource request value of the Pod, the fine-grained supply and dynamic automation of cluster resources are realized, the utilization rate of cluster resources is improved, and the cluster scheduling effect is optimized.

[0108] In one implementation, the target action includes a migration Pod identifier and a target Node identifier; scheduling cluster resources according to a resource scheduling policy includes: determining the target Pod to be migrated based on the migration Pod identifier, determining the target Node based on the target Node identifier; and migrating the target Pod to be migrated to the target Node to schedule cluster resources.

[0109] Specifically, if the target action is a migration action, the target Pod to be migrated is first evicted from its original Node, and then the Kubernetes default scheduler reschedules the target Pod to be migrated to the target Node according to the new resource constraints.

[0110] In this process, by migrating the Pods to be migrated to the target Node, the cluster resources are dynamically adjusted, the cluster resource utilization is improved, and the cluster scheduling effect is optimized.

[0111] In this embodiment, a reinforcement learning model is used to determine the target action based on the current state vector to generate a resource scheduling strategy. Since the current state vector is a digital representation of the cluster's global health status in the current and short-term future, and the reinforcement learning model can dynamically adjust cluster resource parameters based on the target action, decision-making accuracy is improved, thereby optimizing resource scheduling performance.

[0112] In some embodiments, after generating a resource scheduling policy based on the current state vector using a reinforcement learning model and scheduling the cluster's resources according to the resource scheduling policy, the method further includes: obtaining the latest observation data after the cluster executes the resource scheduling policy; determining the latest state vector based on the latest observation data and calculating the reward score; updating the policy network parameters of the reinforcement learning model according to the current state vector, the target action, the latest state vector, and the reward score to obtain the updated reinforcement learning model.

[0113] In the above implementation process, based on the properties of the reinforcement learning model, after scheduling the cluster's resources according to the resource scheduling strategy, the latest observation data of the cluster is collected, and the new state S_{t+1} is determined.

[0114] The reward calculation module in the reinforcement learning model calculates the actual reward score R_t obtained after executing the target action A_t based on the latest SLO state data and cluster resource utilization data.

[0115] In one implementation, the formula for calculating the reward score is as follows: ;in, , , These are the weighting coefficients. Indicates the reward score. This represents the average CPU utilization of the cluster after implementing the resource scheduling policy. This indicates the number of SLO violations after the resource scheduling policy is executed. This indicates the overhead of executing the resource scheduling strategy.

[0116] Store this experience (S_t, A_t, R_t, S_{t+1}) into the experience replay buffer of the reinforcement learning agent.

[0117] The reinforcement learning agent periodically samples data from the buffer to update its policy network parameters, thereby learning better scheduling strategies. This forms a closed-loop learning system from data to decision to feedback, enabling the reinforcement learning model to continuously adapt to load changes and obtain better reinforcement learning models.

[0118] In this embodiment of the application, SLO status data is considered when calculating the reward score. SLO status data reflects business stability, that is, whether the user experience meets the standard. This makes the ultimate goal of the reinforcement learning model no longer to optimize abstract cluster resource indicators, but to ensure that every scheduling decision (adjusting resources, migrating Pods) is directly aimed at ensuring the end user experience, thus realizing business value-driven resource scheduling.

[0119] Furthermore, by updating the parameters of the reinforcement learning model using the cluster's state data, a closed-loop learning system from data to decision to feedback is formed. This enables the reinforcement learning model to continuously adapt to load changes, improving its performance and thus increasing decision accuracy.

[0120] Figure 2 This application provides a schematic diagram of the structure of a resource scheduling system, as shown in the embodiment. Figure 2 As shown, the system 20 includes a data acquisition module 201, a prediction module 202, and a decision engine module 203. The data acquisition module 201 is used to collect observation data of the cluster. The current observation data includes cluster resource utilization, SLO status data, Node observation data, and Pod observation data. The prediction module 202 is used to use a graph neural network to predict the resource status of the cluster based on the Node observation data and Pod observation data, and obtain a resource prediction score. It also generates the current state vector of the cluster based on the resource prediction score, cluster resource utilization, and SLO status data. The decision engine module 203 is used to generate a resource scheduling strategy based on the current state vector through a reinforcement learning model, and schedule the cluster resources according to the resource scheduling strategy.

[0121] In summary, the beneficial effects of this application are as follows: The reinforcement learning component dynamically adjusts the Requests / Limits parameters of Pods. The reinforcement learning agent uses "adjusting resource parameters" as a core action, continuously and automatically fine-tuning it based on real-time application load fluctuations. This achieves refined and dynamic automation of resource provisioning, significantly improving cluster resource utilization.

[0122] A graph neural network is introduced to analyze the global topology of the cluster (between Pods and Nodes, and between Pods), learning Pod call relationships and network performance as edge features. Furthermore, when the reinforcement learning agent decides to "migrate a Pod," its policy network is influenced by the topology information output by the graph neural network. The reinforcement learning agent tends to schedule closely communicating Pod groups to the same rack, availability zone, or node with the lowest network latency. This provides global topology awareness, enabling it to make globally optimal scheduling decisions and reduce application latency.

[0123] By introducing graph neural networks to predict stress and risk scores, and using this prediction as a critical state input, the reinforcement learning agent makes proactive decisions (such as migrating some non-critical Pods from a given Node) to mitigate potential resource contention. This transforms the system from a passive response to proactive prediction and intervention, effectively preventing performance bottlenecks and enhancing system stability.

[0124] Figure 3 This is a schematic diagram of the electronic device structure provided in the embodiments of this application, such as... Figure 3 As shown, the electronic device includes a processor 301, a memory 302, and a bus 303; wherein the processor 301 and the memory 302 communicate with each other through the bus 303. The processor 301 is used to call program instructions in the memory 302 to execute the methods provided in the above-described method embodiments.

[0125] Processor 301 can be an integrated circuit chip with signal processing capabilities. The aforementioned processor 301 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0126] The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0127] This application provides a computer program product, including: computer program instructions, which are executed by a processor to perform the methods provided in the above-described method embodiments.

[0128] This application provides a computer-readable storage medium, including: computer program instructions stored on the computer-readable storage medium, which, when executed by a processor, perform the methods provided in the above-described method embodiments.

[0129] Computer-readable storage media can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0130] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0131] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0133] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A resource scheduling method, characterized in that, The method includes: Obtain the current observation data of the cluster; wherein, the current observation data includes cluster resource utilization, SLO status data, Node observation data, and Pod observation data; Using a graph neural network, based on the observation data of the Node and the observation data of the Pod, the resource status of the cluster is predicted to obtain a resource prediction score; The current state vector of the cluster is generated based on the resource prediction score, the cluster resource utilization rate, and the SLO status data. A resource scheduling strategy is generated based on the current state vector using a reinforcement learning model, and the resources of the cluster are scheduled according to the resource scheduling strategy.

2. The method according to claim 1, characterized in that, The method of using a graph neural network to predict the resource status of the cluster based on the node observation data and the pod observation data, and obtaining a resource prediction score, includes: Based on the Node observation data and the Pod observation data, a network topology graph is generated; wherein, the network topology graph includes Node nodes and Pod nodes; the Node nodes include Node feature vectors, and the Pod nodes include Pod feature vectors; The network topology graph is input into the graph neural network. Using the message passing mechanism of the graph neural network, the feature vector of the Node is updated based on the feature vector of the neighboring nodes of the Node, and the feature vector of the Pod is updated based on the feature vector of the neighboring nodes of the Pod, so as to obtain the global feature vector of the Node and the global feature vector of the Pod. The resource status of the cluster is predicted based on the global feature vectors of the Nodes and the global feature vectors of the Pods using the prediction mechanism of the graph neural network, and a resource prediction score is obtained.

3. The method according to claim 2, characterized in that, in, The resource prediction score includes Node resource pressure score and Pod resource risk score; The method of using the graph neural network prediction mechanism to predict the resource status of the cluster based on the global feature vectors of the nodes and the global feature vectors of the pods, and obtaining a resource prediction score, includes: Based on the Node global feature vector of the Node and the Pod global feature vectors of all Pod nodes adjacent to the Node, calculate the Node resource pressure score of the Node. The Pod resource risk score of the Pod node is calculated based on the Pod global feature vector of the Pod node and the global feature vectors of the adjacent Node nodes and other Pod nodes.

4. The method according to claim 3, characterized in that, The step of generating the current state vector of the cluster based on the resource prediction score, the cluster resource utilization rate, and the SLO status data includes: The Node resource pressure score of each Node, the Pod resource risk score of each Pod, the cluster resource utilization, and the SLO status data are concatenated to generate the current state vector of the cluster.

5. The method according to any one of claims 1-4, characterized in that, The step of generating a resource scheduling strategy based on the current state vector using a reinforcement learning model, and scheduling the cluster's resources according to the resource scheduling strategy, includes: The target action is determined based on the current state vector through the policy network of the reinforcement learning model; wherein the target action is used to generate the resource scheduling policy. The resources of the cluster are scheduled according to the resource scheduling policy.

6. The method according to claim 5, characterized in that, in, The target action includes adjusting the Pod identifier and CPU resource request change values; the scheduling of resources in the cluster according to the resource scheduling policy includes: The target Pod to be adjusted is determined based on the adjusted Pod identifier; Based on the change in CPU resource request value, the CPU resource request value of the target Pod is adjusted to schedule the resources of the cluster.

7. The method according to claim 5, characterized in that, in, The target action includes migrating Pod identifiers and target Node identifiers; the scheduling of cluster resources according to the resource scheduling policy includes: The target Pod to be migrated is determined based on the migration Pod identifier, and the target Node is determined based on the target Node identifier; The target Pod to be migrated is migrated to the target Node in order to schedule the resources of the cluster.

8. The method according to claim 5, characterized in that, After generating a resource scheduling policy based on the current state vector using a reinforcement learning model, and scheduling the cluster's resources according to the resource scheduling policy, the method further includes: Obtain the latest observation data after the cluster executes the resource scheduling policy; The latest state vector is determined based on the latest observation data, and the reward score is calculated. The policy network parameters of the reinforcement learning model are updated based on the current state vector, the target action, the latest state vector, and the reward score to obtain the updated reinforcement learning model.

9. The method according to claim 8, characterized in that, The formula for calculating the reward score is as follows: ;in, , , These are the weighting coefficients. Indicates the reward score. This represents the average CPU utilization of the cluster after implementing the resource scheduling strategy. This indicates the number of SLO violations after the resource scheduling policy is executed. This indicates the overhead of executing the resource scheduling strategy.

10. A resource scheduling system, characterized in that, The system includes a data acquisition module, a prediction module, and a decision engine module; The data acquisition module is used to collect observation data of the cluster; wherein, the current observation data includes cluster resource utilization, SLO status data, Node observation data, and Pod observation data; The prediction module is used to use a graph neural network to predict the resource status of the cluster based on the node observation data and the pod observation data, and obtain a resource prediction score; and generate the current state vector of the cluster based on the resource prediction score, the cluster resource utilization rate and the SLO status data. The decision engine module is used to generate a resource scheduling strategy based on the current state vector using a reinforcement learning model, and to schedule the resources of the cluster according to the resource scheduling strategy.

Images