Instance-specification-aware resource scheduling method, apparatus, device, and medium

Abstract

The application is suitable for the technical field of cloud computing resource management and scheduling, and provides a resource scheduling method and device based on instance specification awareness, equipment and medium, the method comprises: collecting multi-dimensional resource information and task load information of a plurality of computing nodes in a cloud computing platform in real time, obtaining a target scheduling action that meets the instance specification requirement at the current time through a pre-trained multi-dimensional resource specification awareness scheduling model according to the multi-dimensional resource information and the task load information, wherein the multi-dimensional resource specification awareness scheduling model is trained through a population-based reinforcement learning training mechanism, task resource allocation is performed according to the target scheduling action, the task execution result is monitored, the multi-dimensional resource specification awareness scheduling model is adaptively updated based on the feedback information obtained through monitoring, thereby improving the cloud environment resource utilization efficiency, the task matching rate and the dynamic environment response speed.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of cloud computing resource management and scheduling, and particularly relates to a resource scheduling method and device based on instance specification perception, equipment and medium. BACKGROUND

[0002] With the rapid development of cloud computing, big data and artificial intelligence technologies, cloud data centers have become the core infrastructure supporting various Internet services and enterprise applications. In a typical cloud computing environment, service providers generally use standardized "instance specification templates" to encapsulate and deliver computing instances. These templates fixedly combine multiple dimensions of resources such as central processing units (CPUs), memories, graphics processing units (GPUs), and network bandwidths, aiming to simplify resource management and improve instance delivery efficiency. However, existing resource scheduling methods mostly rely on heuristic rules or static matching strategies, lacking systematic modeling of the overall characteristics and dynamic changes of resource specifications, often leading to over-occupancy of some resource dimensions while others are idle, further exacerbating the problem of resource fragmentation and inefficient utilization.

[0003] Some existing technologies propose solutions to optimize resource scheduling, for example, patent document CN202510732796 "Resource scheduling method and system based on multi-dimensional load prediction" proposes to realize prediction and dynamic scheduling optimization of future load through historical data modeling and multi-dimensional time series prediction. Although this solution has certain advantages in prediction accuracy and scheduling adaptability, it mainly focuses on time series prediction of load fluctuations and task priority evaluation, lacks targeted structural modeling and global scheduling consideration of "instance specification templates" as a resource composite, and is difficult to avoid resource dimension imbalance and subsequent task mismatching problems from the specification level; for another example, patent document CN202511264457 "Cost optimization method for resource scheduling management of cloud data center" proposes to realize balanced scheduling among computing, storage, bandwidth and energy consumption through sliding time window prediction and multi-objective game model, which can to some extent balance resource utilization and cost. However, this solution mainly targets the game between multi-dimensional resources and cost optimization, does not consider the combination constraints of standardized specification templates, and lacks dynamic perception mechanism for future specification request distribution and key resource scarcity, so in the scenario of continuous arrival and diversified task requests, it may still lead to early depletion of key specification resources, thereby reducing the overall scheduling success rate.

[0004] It is evident that existing technologies generally suffer from one or more of the following shortcomings: First, they lack a holistic modeling mechanism for instance specification templates, making it difficult to avoid uneven consumption among multi-dimensional resources; second, they do not fully consider the dynamic distribution of future specification requests, and early scheduling can easily lead to the premature occupation of critical specification resources; third, they lack a systematic characterization of resource balance and specification distribution characteristics, making it difficult to maintain efficient scheduling and high resource utilization in diverse, continuously arriving task environments. These problems collectively lead to increased resource fragmentation in cloud data centers, reduced task matching rates, and decreased overall utilization. Therefore, a new multi-dimensional resource scheduling technology is urgently needed to overcome the limitations of existing methods and improve the resource utilization efficiency and service quality of cloud data centers. Summary of the Invention

[0005] The purpose of this invention is to provide a resource scheduling method, apparatus, device, and medium based on instance specification awareness, aiming to solve the problems of severe resource fragmentation, poor specification distribution adaptability, and slow response in dynamic environments caused by existing technologies.

[0006] In a first aspect, the present invention provides a resource scheduling method based on instance specification awareness, the method comprising the following steps:

[0007] Real-time collection of multi-dimensional resource information and task load information from multiple computing nodes in the cloud computing platform;

[0008] Based on the multidimensional resource information and the task load information, the target scheduling action that meets the instance specification requirements at the current moment is obtained through a pre-trained multidimensional resource specification-aware scheduling model. The multidimensional resource specification-aware scheduling model is trained through a population-based reinforcement learning training mechanism.

[0009] The task resource allocation is performed according to the target scheduling action, and the task execution results are monitored.

[0010] The multidimensional resource specification-aware scheduling model is adaptively updated based on the feedback information obtained from monitoring.

[0011] In some embodiments, before the step of outputting the target scheduling action that meets the instance specification requirements at the current moment through a pre-trained multi-dimensional resource specification-aware scheduling model, the method further includes:

[0012] A population consisting of multiple agents is randomly initialized, wherein each agent has independent network parameters and hyperparameters;

[0013] Based on the collected multidimensional resource status data and specification request information, a reinforcement learning environment for training the population is constructed.

[0014] In the reinforcement learning environment, each agent in the population is trained in parallel reinforcement learning according to a pre-constructed target reward function, so as to optimize the scheduling strategy through the interaction between the agent and the environment.

[0015] When the preset convergence condition is met, the agent with the highest average cumulative reward is selected as the multidimensional resource specification-aware scheduling model.

[0016] In some embodiments, the step of performing parallel reinforcement learning training on each agent in the population includes:

[0017] Based on the reward signal calculated by the target reward function, the comprehensive performance of each agent in the population is periodically evaluated to obtain the comprehensive performance evaluation value of each agent.

[0018] Based on the comprehensive performance evaluation values, elite individuals in the population are identified;

[0019] The population is updated based on the elite individuals.

[0020] In some embodiments, the step of periodically evaluating the comprehensive performance of each agent in the population includes:

[0021] Using a comprehensive performance evaluation function Periodically evaluate the comprehensive performance of each agent in the population to obtain a comprehensive performance evaluation value for each agent, wherein... These represent the adjustable weighting coefficients. Indicates the first The average cumulative reward of an agent in recent rounds of interaction Indicates the first The overall performance evaluation value of an agent. Indicates the first Multidimensional resource balance indicators under an agent scheduling strategy Indicates the first Future specification matching capability under individual agent scheduling strategies.

[0022] In some embodiments, the target reward function is constructed jointly based on a multidimensional resource balance index and future specification matching capability, wherein the multidimensional resource balance index is used to measure the balance of resource utilization, and the future specification matching capability is used to measure the adaptability of the scheduling strategy to future loads.

[0023] In some embodiments, the calculation formula for the multidimensional resource balance index is as follows: ,in, This represents the number of resource dimensions in the reinforcement learning environment. Indexes representing resource dimensions. This represents the index of the computation node in the reinforcement learning environment. Represents a computing node In the Dimension's remaining resources Represents a computing node In the Total capacity of dimensions Represents a computing node At any moment A multidimensional resource balance index.

[0024] In some embodiments, the formula for calculating the future specification matching capability is as follows: ,in, This represents a collection of specification templates. Indicates the index of the specification template. Specification template Normalized popularity Specification template In the Dimensional resource requirements, Represents a computing node At any moment The ability to match future specifications.

[0025] Secondly, the present invention provides an instance-specification-aware resource scheduling apparatus, the apparatus comprising:

[0026] The information acquisition unit is used to collect multi-dimensional resource information and task load information of multiple computing nodes in the cloud computing platform in real time.

[0027] The scheduling acquisition unit is used to acquire the target scheduling action that meets the instance specification requirements at the current moment through a pre-trained multi-dimensional resource specification awareness scheduling model based on the multi-dimensional resource information and the task load information. The multi-dimensional resource specification awareness scheduling model is trained through a population-based reinforcement learning training mechanism.

[0028] The scheduling execution unit is used to perform task resource allocation according to the target scheduling action and monitor the task execution results;

[0029] The model update unit is used to adaptively update the multi-dimensional resource specification-aware scheduling model based on the feedback information obtained from monitoring.

[0030] Thirdly, the present invention also provides a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0031] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0032] This invention collects multi-dimensional resource information and task load information from multiple computing nodes in a cloud computing platform in real time. Based on the multi-dimensional resource information and task load information, a pre-trained multi-dimensional resource specification-aware scheduling model is used to obtain the target scheduling action that meets the instance specification requirements at the current moment. The multi-dimensional resource specification-aware scheduling model is trained through a population-based reinforcement learning training mechanism. The model performs task resource allocation according to the target scheduling action and monitors the task execution results. Based on the feedback information obtained from the monitoring, the multi-dimensional resource specification-aware scheduling model is adaptively updated, thereby improving the resource utilization efficiency, task matching rate, and dynamic environment response speed of the cloud environment. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the instance specification-aware resource scheduling method provided in Embodiment 1 of the present invention.

[0034] Figure 2 This is a flowchart illustrating the instance specification-aware resource scheduling method provided in Embodiment 2 of the present invention.

[0035] Figure 3 This is a schematic diagram of the resource scheduling device based on instance specification awareness provided in Embodiment 3 of the present invention;

[0036] Figure 4 This is a schematic diagram of the structure of the computing device provided in Embodiment 4 of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0038] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described feature, integral, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. Furthermore, the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. The terms "first," "second," and similar words do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Words such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Above," "below," "left," "right," etc., are used only to indicate relative positional relationships, which may change accordingly when the absolute position of the described object changes. The term "a plurality" refers to two or more, and other quantifiers are similar.

[0039] To keep the following description of the embodiments of the present invention clear and concise, detailed descriptions of some known functions and known components are omitted in this specification.

[0040] The specific implementation of the present invention will be described in detail below with reference to specific embodiments:

[0041] Example 1:

[0042] Figure 1 The implementation flow of the instance specification-aware resource scheduling method provided in Embodiment 1 of the present invention is illustrated. For ease of explanation, only the parts related to the embodiments of the present invention are shown, and are described in detail below:

[0043] In step S101, multi-dimensional resource information and task load information of multiple computing nodes in the cloud computing platform are collected in real time.

[0044] This invention applies to computing devices, such as personal computers and servers. In this invention, a computing node refers to a physical server or virtualized instance (such as a virtual machine or container) that provides computing resources in a cloud computing environment. Each node contains multiple resource dimensions (such as CPU, memory, GPU, bandwidth, etc.), and its resource status (remaining capacity, total capacity) is monitored in real time as the basis for resource scheduling decisions to meet the resource requirements of the instance specification template. Here, multi-dimensional resource information and task load information of various computing nodes in the cloud computing platform are collected in real time to construct a complete environment state vector, reflecting the cluster resource distribution, task requirements, and dynamic load changes. The multi-dimensional resource information includes, but is not limited to, indicators such as CPU utilization, GPU memory and computing power utilization, memory usage, network bandwidth, and I / O throughput. The task load information includes the resource requirements of tasks for the instance specification template, used to characterize the specific consumption demands of the scheduled tasks on multi-dimensional resources.

[0045] In step S102, based on multidimensional resource information and task load information, the target scheduling action that meets the instance specification requirements at the current moment is obtained through a pre-trained multidimensional resource specification-aware scheduling model. The multidimensional resource specification-aware scheduling model is trained through a population-based reinforcement learning training mechanism.

[0046] In this embodiment of the invention, an environmental state vector composed of multidimensional resource information and task load information is input into a pre-trained multidimensional resource specification-aware scheduling model. The multidimensional resource specification-aware scheduling model extracts features and makes decision inferences from the input environmental state vector through a deep neural network structure, and outputs the optimal target scheduling action that meets the instance specification requirements at the current moment. The target scheduling action includes, but is not limited to, task placement location, resource specification selection, task migration strategy, and elastic scaling scheme, so as to achieve fine-grained allocation of heterogeneous resources. The multidimensional resource specification-aware scheduling model is an agent model trained based on a population-based reinforcement learning training mechanism (PBT).

[0047] In one feasible embodiment, the training process of the multidimensional resource specification-aware scheduling model is detailed in the following embodiment two, and will not be repeated here.

[0048] In step S103, task resource allocation is performed according to the target scheduling action, and the task execution results are monitored.

[0049] In this embodiment of the invention, based on the target scheduling action output by the multi-dimensional resource specification awareness scheduling model, the underlying resource scheduling control module is invoked to execute the corresponding task resource allocation. During the execution of the scheduling action, the task execution result is monitored. The underlying resource scheduling control module includes, but is not limited to, the Kubernetes scheduler, the OpenStackNova component, or a self-developed scheduling engine. Without affecting business continuity, it completes instance migration, resource reallocation, and scaling adjustment, thereby achieving dynamic scheduling and optimal resource configuration within the cloud platform.

[0050] In step S104, the multidimensional resource specification awareness scheduling model is adaptively updated based on the feedback information obtained from monitoring.

[0051] In this embodiment of the invention, after the scheduling action is completed, key performance indicators, including resource utilization, task completion time, resource fragmentation rate, system throughput and energy efficiency ratio, are continuously monitored during the task execution process. Feedback information is generated and returned to the multi-dimensional resource specification-aware scheduling model for online fine-tuning and incremental retraining of the model. When environmental changes (including resource specification expansion or workload characteristic changes) are detected, the model adaptive update mechanism is automatically triggered to maintain the long-term adaptability and stability of the agent.

[0052] In one feasible embodiment, the trained multi-dimensional resource specification-aware scheduling model is deployed as a pluggable scheduling strategy plugin and modularly integrated with the existing cloud resource management platform. This allows for the automatic selection of the corresponding scheduling model based on different business scenarios (such as AI training tasks, inference tasks, and big data processing tasks), enabling multi-model collaboration and dynamic strategy switching. This achieves adaptive resource scheduling optimization in multi-tenant, heterogeneous resource environments.

[0053] In this embodiment of the invention, multidimensional resource information and task load information of multiple computing nodes in a cloud computing platform are collected in real time. Based on the multidimensional resource information and task load information, a pre-trained multidimensional resource specification-aware scheduling model is used to obtain the target scheduling action that meets the instance specification requirements at the current moment. The multidimensional resource specification-aware scheduling model is trained through a population-based reinforcement learning training mechanism. Task resource allocation is performed according to the target scheduling action, and the task execution results are monitored. The multidimensional resource specification-aware scheduling model is adaptively updated based on the feedback information obtained from the monitoring. Thus, by deploying the PBT-optimized multidimensional resource specification-aware scheduling model, intelligent and adaptive scheduling of cloud platform resources is realized, which significantly improves resource utilization and system throughput, reduces resource fragmentation rate and task migration overhead, and has good dynamic environment adaptability and long-term performance maintenance characteristics. It can be flexibly migrated and reused between different cloud environments and heterogeneous resource pools, providing an efficient and scalable solution for the autonomous scheduling and intelligent operation and maintenance of cloud computing platforms.

[0054] Example 2:

[0055] Figure 2 The implementation flow of the instance specification-aware resource scheduling method provided in Embodiment 2 of the present invention is illustrated. For ease of explanation, only the parts related to the embodiments of the present invention are shown, and are described in detail below:

[0056] The multidimensional resource specification-aware scheduling model is trained through the following steps:

[0057] In step S201, a population consisting of multiple agents is randomly initialized, wherein each agent has independent network parameters and hyperparameters.

[0058] In this embodiment of the invention, a population consisting of multiple agents is initialized. Each agent has independent and randomly initialized network parameters and hyperparameters. The agent is the core decision-making module of the resource scheduling system. Based on a deep reinforcement learning algorithm, it learns the optimal scheduling strategy by interacting with the environment. It receives the environmental status of the cloud platform (such as node resources and task load) and outputs specific scheduling actions (such as task placement location and resource specification selection).

[0059] In one feasible embodiment, each agent comprises a policy network (Actor network) and a value network (Critic network). Here, the parameters of the agent's policy network and value network are randomly initialized, and parameter perturbation terms are introduced. To ensure that each agent has a distinct policy search space and prevent them from becoming homogenized in the early training phase, a policy network is used to generate the probability distribution of scheduled actions, and a value network is used to evaluate the expected reward of state-action pairs. Represents random variables Follows a mean of 0 and a variance of It follows a normal distribution (Gaussian distribution).

[0060] In another feasible embodiment, the policy network employs a multi-layer perceptron (MLP) structure, which uses the current environment state vector as the basis for its implementation. As input, output the probability distribution of scheduling actions for each computing node. That is, in the first The state corresponding to the time Below, strategy π is determined by parameters. Generate Actions The probability distribution is determined, and the network continuously adjusts its parameters through backpropagation to achieve adaptive optimization of the policy. The value network also adopts a multi-layer fully connected neural network structure and shares some feature extraction layers with the policy network to improve training efficiency and stability. The input to the Critic network is the system state vector. The output is the expected reward of the state-action pair. ,in, and These represent the parameter sets of the policy network and the value network, respectively.

[0061] In another feasible embodiment, a set of randomly sampled hyperparameter vectors is assigned to each agent. These hyperparameter vectors include at least the learning rate, discount factor, GAE smoothing coefficient, and entropy coefficient. The hyperparameter vectors are represented as follows: Where i represents the index of the agent in the population. This represents the hyperparameter vector of the i-th agent. This represents the learning rate of the i-th agent. This represents the discount factor for the i-th agent. Represents the GAE smoothing coefficient of the i-th agent. Let represent the entropy coefficient of the i-th agent, thereby forming a diverse population through randomized hyperparameters.

[0062] In step S202, a reinforcement learning environment for population training is constructed based on the collected multidimensional resource status data and specification request information.

[0063] In this embodiment of the invention, a reinforcement learning environment for population training is constructed based on multi-dimensional resource status data and specification request information collected historically or in real time. The specification request information includes specification templates, which can be derived from cloud service configuration files, container specification lists, or task submission interfaces to achieve standardized descriptions of different specification requests.

[0064] In one feasible embodiment, the reinforcement learning environment is constructed through the following steps:

[0065] (S202.1) Construct a state space based on the collected multidimensional resource status data and specification request information;

[0066] In this embodiment of the invention, the specification templates in the specification request information are parsed to extract their specific requirements across multiple resource dimensions, including CPU, memory, GPU, and bandwidth. The multi-dimensional requirement vector corresponding to each specification template is represented as follows: , Indicates the index of the specification template. Indicates the total number of resource dimensions Indicates the first Each specification template in the resource dimension The system calculates the demand for resources and constructs a node's remaining resource matrix from multi-dimensional resource status data. This matrix includes remaining capacity for CPU, memory, GPU, and other dimensions. The node's remaining resource matrix is ​​represented as follows: ,in, Represents the total number of computing nodes. The index representing the compute node, followed by the multidimensional demand vector of the current scheduling specification. and the remaining resource matrix of all nodes. Combining them forms a state space. This serves as the state input for the reinforcement learning environment, enabling agent training to more closely reflect the actual resource distribution.

[0067] In a feasible embodiment, the multidimensional demand vector is normalized to map the resource demand of each dimension to the interval [0,1], thereby ensuring the comparability of different dimensions and facilitating unified processing by the intelligent agent.

[0068] (S202.2) Construct the action space, which consists of all selectable computation nodes;

[0069] In this embodiment of the invention, an action space is constructed. The action space consists of all selectable computation nodes, represented as ,in, Indicates the selection of the first The actions of each computing node as the target scheduling location.

[0070] In one feasible embodiment, when resource constraints or computing node unavailability exist, an action mask is generated in the action space. ,pass Valid actions are selected by allowing the agent to choose actions only on the set of valid computing nodes, thereby preventing the agent from choosing illegal nodes, effectively reducing the policy search space, avoiding illegal decisions, and improving the training convergence speed.

[0071] (S202.3) Set state transition rules;

[0072] In this embodiment of the invention, when the intelligent agent performs an action (i.e., select a node) After scheduling, if the action is effective (node ​​resources meet task requirements), the remaining resource matrix of that node is updated immediately. And generate a new system state. If an action is invalid (due to insufficient node resources or the action being masked), a "skip" state transition is triggered, maintaining the current resource state to prevent invalid actions from affecting training stability. This represents the remaining resource matrix of the node after the update.

[0073] In step S203, in the reinforcement learning environment, each agent in the population is trained in parallel reinforcement learning according to the pre-constructed target reward function, so as to optimize the scheduling strategy through the interaction between the agent and the environment.

[0074] In this embodiment of the invention, each agent in the population trains asynchronously in parallel within a reinforcement learning environment. Through continuous interaction with the environment, it learns performance feedback under different resource allocation strategies to continuously optimize its own policy network and value network. Specifically, firstly, in each round of interaction, each agent outputs a scheduling action from its own Actor network based on the current environmental state input, which is used to indicate the scheduling decision of the target node. To enhance exploratory behavior, the agent introduces random noise or probabilistic strategies in action selection to balance the "exploration-exploitation" relationship. Next, the environment executes virtual task scheduling according to the scheduling action output by the agent, updates the node resource occupancy, and calculates the immediate reward signal based on the target reward function. Subsequently, the environment returns the new state vector and the corresponding immediate reward value to the agent, realizing a state transition and reward feedback process. Finally, the agent stores the "state-action-reward-next state" quadruple generated during the interaction into the experience replay pool. The algorithm uses a buffer and a gradient update rule based on the Actor-Critic algorithm to optimize the parameters of the policy network and the value network respectively. The Critic network is used to estimate the action value function to reduce variance, while the Actor network optimizes the long-term cumulative reward expectation through the policy gradient method.

[0075] In one feasible embodiment, the objective reward function is jointly constructed based on multidimensional resource balance indicators and future specification matching capability to reflect the joint optimization objective of short-term equilibrium and long-term adaptability. Specifically, the objective reward function is expressed as follows: ,in, MRBI stands for Multidimensional Resource Balance Index, used to measure the balance of resource utilization. MRBI ranges from 0 to 1. When resource utilization is perfectly balanced across all dimensions, MRBI is close to 1. MFFP stands for Future Specification Fit, used to measure the adaptability of scheduling strategies to future loads. This represents the number of valid nodes that satisfy the resource constraints. This represents the sum of MRBI and MFFP under the current action. The objective reward function represents the maximum sum of MRBI and MFFP among all currently valid nodes. By jointly modeling MRBI and MFFP indices in a dynamic normalized form, we can balance short-term resource balance with long-term specification matching. We can also filter illegal actions through "effective node constraints", so that the agent's learning process has multi-objective self-balancing ability, avoids getting trapped in local optima (such as single-dimensional resource optimization but overall imbalance), and achieves stable convergence in dynamic environments.

[0076] In a feasible embodiment, the formula for calculating the multidimensional resource balance index is as follows: This metric, for the first time, measures the balanced utilization of multi-dimensional resources (such as CPU, memory, GPU, etc.) across computing nodes at the specification template level. It overcomes the limitations of traditional scheduling, which relies solely on single-dimensional utilization or average load. By introducing MRBI as a reinforcement learning reward signal, it proactively avoids single-dimensional resource overload and multi-dimensional imbalance during training, significantly reducing resource fragmentation and achieving higher overall cluster utilization. This represents the number of resource dimensions in a reinforcement learning environment. Indexes representing resource dimensions. This represents the index of a computation node in a reinforcement learning environment. Represents a computing node In the Dimension's remaining resources Represents a computing node In the Total capacity of dimensions Represents a computing node At any moment A multidimensional resource balance index.

[0077] In one feasible embodiment, the formula for calculating future specification matching capability is as follows: This metric incorporates the future specification request distribution characteristics into the scheduling optimization objective. By statistically analyzing the popularity of different specification types and the differences in multidimensional demands, it achieves predictive awareness and adaptive modeling of future workload distribution. Thus, by introducing MFFP (Multi-Functional Flow Planning), scheduling decisions are no longer limited to current task allocation but can reserve critical specification resources, preventing high-value specifications from being exhausted prematurely, thereby improving task matching rate and long-term scheduling success rate. This represents a collection of specification templates. Indicates the index of the specification template. Specification template Normalized popularity Specification template In the Dimensional resource requirements, Represents a computing node At any moment The ability to match future specifications.

[0078] In a feasible embodiment, parallel reinforcement learning training of each agent in the population is achieved through the following steps:

[0079] (S203.1) Based on the reward signal calculated by the objective reward function, periodically evaluate the comprehensive performance of each agent in the population to obtain the comprehensive performance evaluation value of each agent;

[0080] In this embodiment of the invention, after completing several training episodes, the comprehensive performance of all agents in the PBT population is periodically evaluated based on the reward signal calculated by the target reward function, so as to obtain the comprehensive performance evaluation value of each agent, to determine its performance in the resource scheduling task, and to provide a decision basis for subsequent evolution and hyperparameter update.

[0081] In one feasible embodiment, a comprehensive performance evaluation function is constructed based on a multi-dimensional resource optimization objective. A comprehensive performance evaluation function is used to periodically evaluate the comprehensive performance of each agent in the population, resulting in a comprehensive performance evaluation value for each agent. The multidimensional resource optimization objective includes MRBI, MFFP, and a reward signal calculated based on the objective reward function. These represent adjustable weighting coefficients, used to balance immediate gains and long-term optimization goals. Indicates the first The average cumulative reward of an agent in recent rounds of interaction Indicates the first The overall performance evaluation value of an agent. Indicates the first Multidimensional resource balance indicators under an agent scheduling strategy Indicates the first Future specification matching capability under individual agent scheduling strategies.

[0082] In a feasible embodiment, the task execution data of each agent in the current evaluation period are summarized, and its utilization distribution, resource fragmentation rate, task waiting time and other indicators in dimensions such as CPU, memory, and GPU are calculated. The final comprehensive performance evaluation value is generated by normalization and weighted averaging, thereby ensuring that the indicators under different dimensions are comparable.

[0083] (S203.2) Based on the comprehensive performance evaluation value, identify the elite individuals in the population;

[0084] In this embodiment of the invention, agents are sorted in descending order based on their comprehensive performance evaluation values. The network parameters, hyperparameter configurations, and historical performance trajectories of each agent are recorded. Agents with superior performance are marked as "elite individuals," and their parameters are preferentially retained or copied to lower-performing agents in subsequent evolutionary stages to achieve experience transfer and performance inheritance. Specifically, based on the descending sorting results, a predetermined proportion of agents are selected as elite individuals. The network parameters, hyperparameters, and performance indicators of these elite individuals are retained, and their strategies are considered the globally optimal reference for the current stage. This elite retention mechanism prevents high-performance strategies from being destroyed by random mutations during evolution, thereby ensuring the stable convergence of the algorithm.

[0085] In one specific embodiment, the top 20% of agents are selected as elite individuals based on the descending sorting results.

[0086] (S203.3) Update the population based on elite individuals.

[0087] In this embodiment of the invention, population renewal is achieved through the following steps:

[0088] (S203.3.1) The policy network parameters, value network parameters and hyperparameter combinations of elite individuals are copied to agents with lower overall performance evaluation values ​​in the population;

[0089] In this embodiment of the invention, for agents with lower performance in the descending sort results (i.e., non-elite individuals in the population), an elite individual is randomly selected from the set of elite individuals, and the policy network parameters (including the weights of Actor and Critic) and corresponding hyperparameters (such as learning rate, discount factor, exploration noise amplitude, etc.) of the elite individual are copied. This copying process enables inferior individuals to directly inherit the excellent policy structure and training experience, realizes the knowledge transfer of excellent policies, thereby accelerating the convergence speed and improving the overall population quality.

[0090] (S203.3.2) Perform mutation operation on the agent that has completed the copying operation to generate a new candidate agent.

[0091] In this embodiment of the invention, a perturbation formula is adopted. By applying random perturbations to the hyperparameters of the replicated non-elite individuals, new combinations of hyperparameters are generated, forming new candidate agents. This avoids the population getting trapped in local optima and maintains exploration diversity. Indicates the first The combination of hyperparameters after perturbation of each agent The set of hyperparameters representing elite individuals. The random perturbation term, which follows a normal distribution, can be dynamically adjusted to balance global exploration and local exploitation.

[0092] The above steps (S203.3.1) to (S203.3.2) are based on the selected elite individuals and adopt an adaptive evolutionary mechanism of "elite parameter replication - hyperparameter mutation" to dynamically update the agent population, so as to achieve continuous improvement of overall performance and diversified exploration of strategies.

[0093] The steps (S203.1) to (S203.3) above introduce the PBT mechanism during the reinforcement learning training phase, enabling parallel search and knowledge transfer among agents, completing online hyperparameter optimization and policy self-evolution of agents. Compared with the traditional single agent training method, this mechanism can simultaneously take into account exploration and stability in multidimensional resource scheduling problems, effectively preventing premature convergence to local optima, and overcoming the problems of fixed hyperparameters, insufficient exploration, and local optima convergence in traditional single agent reinforcement learning. This significantly improves the adaptability of agents to complex dynamic cloud environments and long-term decision-making performance.

[0094] In step S204, when the preset convergence condition is met, the agent with the highest average cumulative reward is selected as the multidimensional resource specification-aware scheduling model.

[0095] In this embodiment of the invention, after multiple rounds of parallel training and evolutionary updates, it is determined whether the overall agent population has reached a performance convergence or stable state, thereby deciding whether to terminate training and output the final model. By continuously monitoring the performance change trend and reward function convergence characteristics of the agents during training, it is ensured that the model maintains continuous optimization in long-term learning and eventually obtains a stable optimal policy. After the PBT training process converges, all agents are finally ranked according to the average cumulative reward (reflecting the overall benefit level of the scheduling policy). The agent with the highest score is the globally optimal individual. This globally optimal individual has adaptive multi-dimensional resource scheduling capabilities and can achieve efficient and robust scheduling decisions in actual cloud computing environments. Here, this globally optimal individual is used as a multi-dimensional resource specification-aware scheduling model, and its policy network and value network parameters are frozen. The frozen model will no longer participate in training but will be used as the final output model for deployment and online inference stages. Specifically, the multi-dimensional resource specification-aware scheduling model will be saved as a standardized model file format (such as ONNX or PyTorch weight files), recording the corresponding environment configuration, input / output interface definitions, and version information. Simultaneously with model export, the system generates a corresponding inference interface description file, ensuring that the model can be directly loaded and called in different deployment environments (such as Kubernetes schedulers or cloud management platforms), achieving plug-and-play deployment. The exported model is deployed to the resource scheduling module of the cloud computing platform, serving as the core decision engine for real-time task allocation and resource optimization. During actual operation, the model performs decision-making inference based on real-time input specification requests and node status information, outputting the optimal node allocation result to achieve dynamic balanced scheduling of multi-dimensional resources. Simultaneously, the system can periodically collect operational data feedback to support periodic retraining and adaptive updates of the model, realizing a closed-loop mechanism of "training—deployment—feedback—re-optimization."

[0096] In a feasible embodiment, to determine whether the training has converged, the global average reward growth rate and the Population Diversity Index (PDI) are calculated after each generation of evolution. The training is considered to have converged when any of the following conditions are met: "the global average reward growth rate is lower than a set threshold and remains unchanged for several generations", "the population diversity index is lower than a preset lower limit and the performance improvement is stable", or "the maximum number of iterations or the training time limit is reached". If the convergence conditions are not met, a new round of training and evolution is continued. By introducing a systematic convergence monitoring and multi-index judgment mechanism, the model can be ensured to achieve stable convergence during the evolution process, avoiding overfitting and invalid iterations. At the same time, the training cost and time overhead are significantly reduced while ensuring optimal model performance.

[0097] In one feasible embodiment, the global average reward growth rate The calculation formula is: ,in, Indicates the current generation The average reward, For the front Average reward of the generation Indicates generational interval, when When the percentage remains consistently below a set threshold (e.g., 0.5%), it indicates that the training is stabilizing.

[0098] In one feasible implementation, the population diversity index The calculation formula is: ,in, Indicates the number of individual intelligent agents. Indicates the first The policy network parameter vector of each agent Used to measure the differences in policies among different agents, when When the value is large, it indicates that there are significant differences in strategies among the agents, and the population is in a high exploration state. When the percentage continues to drop below the preset lower limit (e.g., 5%), it indicates that the individuals in the population are becoming homogeneous and the training has gradually converged.

[0099] In this embodiment of the invention, a population consisting of multiple agents is randomly initialized, where each agent has independent network parameters and hyperparameters. Based on the collected multidimensional resource state data and specification request information, a reinforcement learning environment for population training is constructed. In the reinforcement learning environment, each agent in the population is trained in parallel according to a pre-constructed target reward function to optimize the scheduling strategy through interaction between the agent and the environment. When a preset convergence condition is met, the agent with the highest average cumulative reward is selected as the multidimensional resource specification-aware scheduling model. Thus, by training multiple agents with different hyperparameters in parallel and searching and updating hyperparameters in parallel among multiple agents, adaptive evolution and performance improvement of the agent model are achieved, improving the training convergence speed and enhancing the stability and generalization ability of the final strategy.

[0100] Example 3:

[0101] Figure 3 The structure of the instance specification-aware resource scheduling device provided in Embodiment 3 of the present invention is shown. For ease of explanation, only the parts related to the embodiments of the present invention are shown, including:

[0102] Information acquisition unit 31 is used to collect multi-dimensional resource information and task load information of multiple computing nodes in the cloud computing platform in real time;

[0103] The scheduling acquisition unit 32 is used to acquire the target scheduling action that meets the instance specification requirements at the current moment based on multidimensional resource information and task load information through a pre-trained multidimensional resource specification awareness scheduling model. The multidimensional resource specification awareness scheduling model is trained through a population-based reinforcement learning training mechanism.

[0104] The scheduling execution unit 33 is used to perform task resource allocation according to the target scheduling action and monitor the task execution results;

[0105] The model update unit 34 is used to adaptively update the multi-dimensional resource specification-aware scheduling model based on the feedback information obtained from monitoring.

[0106] Preferably, the instance specification-aware resource scheduling device of this embodiment further includes:

[0107] The population initialization unit is used to randomly initialize a population consisting of multiple agents, where each agent has independent network parameters and hyperparameters.

[0108] The environment construction unit is used to construct a reinforcement learning environment for population training based on the collected multidimensional resource status data and specification request information.

[0109] The population training unit is used to perform parallel reinforcement learning training on each agent in the population in a reinforcement learning environment according to a pre-built target reward function, so as to optimize the scheduling strategy through the interaction between the agent and the environment.

[0110] The agent selection unit is used to select the agent with the highest average cumulative reward as the multidimensional resource specification-aware scheduling model when the preset convergence conditions are met.

[0111] In this embodiment of the invention, for the sake of convenience and brevity, only the division of the above-described functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to achieve all or part of the functions described above. Each unit and module of the device can be implemented by corresponding hardware or software units. Each unit and module can be an independent hardware or software unit, or it can be integrated into a single hardware or software unit, which is not intended to limit the invention. In addition, the specific names of each functional unit and module are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the device can be referred to the corresponding description in the foregoing method embodiments, and will not be repeated here.

[0112] Example 4:

[0113] Figure 4The structure of the computing device provided in Embodiment 4 of the present invention is shown. For ease of explanation, only the parts related to the embodiments of the present invention are shown.

[0114] The computing device 4 of this embodiment includes a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40. When the processor 40 executes the computer program 42, it implements the steps described in the above embodiment of the instance-specification-aware resource scheduling method, for example... Figure 1 The steps S101 to S104 are shown. Alternatively, when the processor 40 executes the computer program 42, it implements the functions of each unit in the above-described device embodiments, for example... Figure 3 The functions of units 31 to 34 are shown.

[0115] In this embodiment of the invention, multidimensional resource information and task load information of multiple computing nodes in the cloud computing platform are collected in real time. Based on the multidimensional resource information and task load information, the target scheduling action that meets the instance specification requirements at the current moment is obtained through a pre-trained multidimensional resource specification awareness scheduling model. The multidimensional resource specification awareness scheduling model is trained through a population-based reinforcement learning training mechanism. The task resource allocation is performed according to the target scheduling action, and the task execution results are monitored. The multidimensional resource specification awareness scheduling model is adaptively updated based on the feedback information obtained from the monitoring, thereby improving the resource utilization efficiency, task matching rate and dynamic environment response speed of the cloud environment.

[0116] The computing device in this embodiment of the invention can be a personal computer. The steps implemented by the processor 40 in the computing device 4 when executing the computer program 42 to implement the instance specification-aware resource scheduling method can be referred to the description of the foregoing method embodiments, and will not be repeated here.

[0117] Example 5:

[0118] In this embodiment of the invention, a computer-readable storage medium is provided, which stores a computer program. When executed by a processor, the computer program implements the steps described in the embodiment of the instance-specification-aware resource scheduling method. For example... Figure 1 The steps S101 to S104 are shown. Alternatively, when the computer program is executed by the processor, it implements the functions of each unit in the above-described device embodiments, for example... Figure 3 The functions of units 31 to 34 are shown.

[0119] In this embodiment of the invention, multidimensional resource information and task load information of multiple computing nodes in the cloud computing platform are collected in real time. Based on the multidimensional resource information and task load information, the target scheduling action that meets the instance specification requirements at the current moment is obtained through a pre-trained multidimensional resource specification awareness scheduling model. The multidimensional resource specification awareness scheduling model is trained through a population-based reinforcement learning training mechanism. The task resource allocation is performed according to the target scheduling action, and the task execution results are monitored. The multidimensional resource specification awareness scheduling model is adaptively updated based on the feedback information obtained from the monitoring, thereby improving the resource utilization efficiency, task matching rate and dynamic environment response speed of the cloud environment.

[0120] The computer-readable storage medium of this invention can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EEPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0121] The above embodiments are merely illustrative of the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the scope of disclosure involved in the above embodiments is not limited to technical solutions formed by specific combinations of the above technical features, but should also cover other technical solutions formed by arbitrary combinations of the above technical features or their equivalent features without departing from the above-disclosed concept. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0122] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

Claims

1. A resource scheduling method based on instance specification awareness, characterized in that, The method includes the following steps: Real-time collection of multi-dimensional resource information and task load information from multiple computing nodes in the cloud computing platform; Based on the multidimensional resource information and the task load information, the target scheduling action that meets the instance specification requirements at the current moment is obtained through a pre-trained multidimensional resource specification-aware scheduling model. The multidimensional resource specification-aware scheduling model is trained through a population-based reinforcement learning training mechanism. The task resource allocation is performed according to the target scheduling action, and the task execution results are monitored. The multi-dimensional resource specification-aware scheduling model is adaptively updated based on the feedback information obtained from monitoring. Before the step of outputting the target scheduling action that meets the instance specification requirements at the current moment through the pre-trained multi-dimensional resource specification-aware scheduling model, the method further includes: A population consisting of multiple agents is randomly initialized, wherein each agent has independent network parameters and hyperparameters; Based on the collected multidimensional resource status data and specification request information, a reinforcement learning environment for training the population is constructed. In the reinforcement learning environment, each agent in the population undergoes parallel reinforcement learning training based on a pre-constructed target reward function. This allows for scheduling strategy optimization through agent-environment interaction. The target reward function is jointly constructed based on a multi-dimensional resource balance index and future specification matching capability. The multi-dimensional resource balance index measures the balance of resource utilization, while the future specification matching capability measures the adaptability of the scheduling strategy to future loads. The calculation formula for the multi-dimensional resource balance index is as follows: ,in, This represents the number of resource dimensions in the reinforcement learning environment. Indexes representing resource dimensions. This represents the index of the computation node in the reinforcement learning environment. Represents a computing node In the Dimension's remaining resources Represents a computing node In the Total capacity of dimensions Represents a computing node At any moment Multidimensional resource balance indicators; When the preset convergence condition is met, the agent with the highest average cumulative reward is selected as the multidimensional resource specification-aware scheduling model.

2. The method as described in claim 1, characterized in that, The steps for parallel reinforcement learning training of each agent in the population include: Based on the reward signal calculated by the target reward function, the comprehensive performance of each agent in the population is periodically evaluated to obtain the comprehensive performance evaluation value of each agent. Based on the comprehensive performance evaluation values, elite individuals in the population are identified; The population is updated based on the elite individuals.

3. The method as described in claim 2, characterized in that, The steps for periodically evaluating the comprehensive performance of each agent in the population include: Using a comprehensive performance evaluation function Periodically evaluate the comprehensive performance of each agent in the population to obtain a comprehensive performance evaluation value for each agent, wherein... These represent the adjustable weighting coefficients. Indicates the first The average cumulative reward of an agent in recent rounds of interaction Indicates the first The overall performance evaluation value of an agent. Indicates the first Multidimensional resource balance indicators under an agent scheduling strategy Indicates the first Future specification matching capability under individual agent scheduling strategies.

4. The method as described in claim 1, characterized in that, The formula for calculating the future specification matching capability is as follows: ,in, This represents a collection of specification templates. Indicates the index of the specification template. Specification template Normalized popularity Specification template In the Dimensional resource requirements, Represents a computing node At any moment The ability to match future specifications.

5. A resource scheduling device based on instance specification awareness, characterized in that, The device includes: The information acquisition unit is used to collect multi-dimensional resource information and task load information of multiple computing nodes in the cloud computing platform in real time. The scheduling acquisition unit is used to acquire the target scheduling action that meets the instance specification requirements at the current moment through a pre-trained multi-dimensional resource specification awareness scheduling model based on the multi-dimensional resource information and the task load information. The multi-dimensional resource specification awareness scheduling model is trained through a population-based reinforcement learning training mechanism. The scheduling execution unit is used to perform task resource allocation according to the target scheduling action and monitor the task execution results; The model update unit is used to adaptively update the multi-dimensional resource specification-aware scheduling model based on the feedback information obtained from monitoring. The device further includes: A population initialization unit is used to randomly initialize a population consisting of multiple agents, wherein each agent has independent network parameters and hyperparameters. The learning environment construction unit is used to construct a reinforcement learning environment for the population training based on the collected multidimensional resource status data and specification request information. An agent training unit is used to perform parallel reinforcement learning training on each agent in the population according to a pre-constructed target reward function in the reinforcement learning environment. This aims to optimize the scheduling strategy through agent-environment interaction. The target reward function is jointly constructed based on a multi-dimensional resource balance index and future specification matching capability. The multi-dimensional resource balance index measures the balance of resource utilization, and the future specification matching capability measures the adaptability of the scheduling strategy to future loads. The calculation formula for the multi-dimensional resource balance index is as follows: ,in, This represents the number of resource dimensions in the reinforcement learning environment. Indexes representing resource dimensions. This represents the index of the computation node in the reinforcement learning environment. Represents a computing node In the Dimension's remaining resources Represents a computing node In the Total capacity of dimensions Represents a computing node At any moment Multidimensional resource balance indicators; The agent selection unit is used to select the agent with the highest average cumulative reward as the multidimensional resource specification-aware scheduling model when the preset convergence conditions are met.

6. The apparatus as claimed in claim 5, characterized in that, The agent training unit, when performing parallel reinforcement learning training on each agent in the population, includes: Based on the reward signal calculated by the target reward function, the comprehensive performance of each agent in the population is periodically evaluated to obtain the comprehensive performance evaluation value of each agent. Based on the comprehensive performance evaluation values, elite individuals in the population are identified; The population is updated based on the elite individuals.

7. The apparatus as claimed in claim 6, characterized in that, The agent training unit, when performing periodic comprehensive performance evaluations of each agent in the population, includes: Using a comprehensive performance evaluation function Periodically evaluate the comprehensive performance of each agent in the population to obtain a comprehensive performance evaluation value for each agent, wherein... These represent the adjustable weighting coefficients. Indicates the first The average cumulative reward of an agent in recent rounds of interaction Indicates the first The overall performance evaluation value of an agent. Indicates the first Multidimensional resource balance indicators under an agent scheduling strategy Indicates the first Future specification matching capability under individual agent scheduling strategies.

8. The apparatus as claimed in claim 5, characterized in that, The formula for calculating the future specification matching capability is as follows: ,in, This represents a collection of specification templates. Indicates the index of the specification template. Specification template Normalized popularity Specification template In the Dimensional resource requirements, Represents a computing node At any moment The ability to match future specifications.

9. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images