Resource scheduling method and device, storage medium and computer program product

Abstract

The invention provides a resource scheduling method and device, a storage medium and a computer program product, and the method comprises the steps: obtaining training data which at least comprises task list information, heterogeneous resource information of various task processing nodes, and clock information; determining a resource scheduling strategy corresponding to one or more tasks based on task list information included in the training data, heterogeneous resource information of various task processing nodes, clock information and a first model; based on a preset reward function and a resource scheduling strategy, the first model is trained, the trained first model is obtained, resource scheduling is conducted on one or more target tasks in the to-be-scheduled task list through the trained first model, and the preset reward function comprises a resource usage amount function and a task processing time function.

Description

Technical Field

[0001] This application relates to the field of resource scheduling technology, and in particular to a resource scheduling method, device, storage medium and computer program product. Background Technology

[0002] With the deepening development of artificial intelligence applications, the complexity and diversity of computing tasks are increasing, and the demand for computing resources is also showing a high degree of heterogeneity. This heterogeneity is not only reflected in the diversity of hardware architecture, but also involves multiple dimensions such as computing performance, power consumption, and cost, posing a huge challenge to task scheduling, especially in high-load scenarios. Most related technologies perform resource scheduling for a single task or for a specific resource. However, when scheduling resources for multiple concurrent tasks, it is difficult to guarantee the minimization of task completion time and the maximization of heterogeneous resource utilization. Summary of the Invention

[0003] In view of this, embodiments of this application provide a resource scheduling method, device, storage medium, and computer program product that can ensure the minimization of task completion time and the maximization of heterogeneous resource utilization when scheduling resources for multiple concurrent tasks.

[0004] The technical solution of this application is implemented as follows: Firstly, this application provides a resource scheduling method, the method comprising: Acquire training data, which includes at least: task list information, heterogeneous resource information of various task processing nodes, and clock information; Based on the training data, including task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model, determine the resource scheduling strategy for one or more tasks. Based on a preset reward function and resource scheduling strategy, the first model is trained to obtain the trained first model. The trained first model is then used to schedule resources for one or more target tasks in the task list to be scheduled. The preset reward function includes a resource usage function and a task processing time function.

[0005] Secondly, this application provides a resource scheduling device, the resource scheduling device comprising: The acquisition unit is used to acquire training data, wherein the training data includes at least: task list information, heterogeneous resource information of various task processing nodes, and clock information; The determining unit is used to determine the resource scheduling strategy for one or more tasks based on the task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model included in the training data. The training unit is used to train the first model based on a preset reward function and a resource scheduling strategy to obtain the trained first model. The trained first model is then used to schedule resources for one or more target tasks in the task list to be scheduled. The preset reward function includes a resource usage function and a task processing time function.

[0006] Thirdly, this application provides a resource scheduling device, which includes a processor and a memory; the processor implements the above-described resource scheduling method when executing a running program stored in the memory.

[0007] Fourthly, this application provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the above-described resource scheduling method.

[0008] Fifthly, this application provides a computer program product, including a computer program that implements the above-described resource scheduling method when executed by a processor.

[0009] This application provides a resource scheduling method, device, storage medium, and computer program product. The method includes: acquiring training data, wherein the training data includes at least: task list information, heterogeneous resource information of various task processing nodes, and clock information; determining a resource scheduling strategy corresponding to one or more tasks based on the task list information, heterogeneous resource information of various task processing nodes, clock information, and a first model included in the training data; training the first model based on a preset reward function and the resource scheduling strategy to obtain a trained first model, and using the trained first model to perform resource scheduling on one or more target tasks in the task list to be scheduled, wherein the preset reward function includes a resource usage function and a task processing time function. Using the above implementation scheme, when training the first model, the resource scheduling strategy for one or more tasks is first determined by the first model for the tasks in the task list, combined with the heterogeneous resource information and clock information input to the first model. Since the preset reward function includes a resource usage function and a task processing time function, when training the first model using the preset reward function and the resource scheduling strategy, the resource scheduling strategy for one or more tasks determined by the first model is optimized from the aspects of resource usage and task processing time. Therefore, by using this method to train the first model, the resource scheduling result for one or more target tasks is determined by the trained first model. This resource scheduling result can ensure that the task completion time is minimized and the utilization rate of heterogeneous resources is maximized when multiple tasks are concurrent. Attached Figure Description

[0010] Figure 1 A flowchart illustrating a resource scheduling method provided in an embodiment of this application; Figure 2 A schematic diagram illustrating the process of extracting feature vectors using a deep ensemble model (DS) provided in this application embodiment; Figure 3 This application provides a schematic diagram of a process for determining a reward value. Figure 4 A schematic diagram of the overall process of a cloud-edge-device resource scheduling method based on the DS-Proximal Policy Optimization (DS-PPO) model provided for embodiments of this application; Figure 5 This is a schematic diagram of the composition structure of a resource scheduling device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the composition structure of a resource scheduling device provided in an embodiment of this application. Detailed Implementation

[0011] To gain a more detailed understanding of the features and technical content of the embodiments of this application, the technical solution of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments. The accompanying drawings are for reference only and are not intended to limit the embodiments of this application.

[0012] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0013] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. It should also be noted that the terms "first / second / third" used in the embodiments of this application are merely for distinguishing similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein.

[0014] With the rapid development of Internet of Things (IoT), big data, and Artificial Intelligence (AI) technologies, computing frameworks are undergoing unprecedented changes. Against this backdrop, the cloud-edge-device (cloud computing) computing framework, as a new computing model, integrates the global processing capabilities of cloud computing, the real-time response advantages of edge computing, and the widespread distribution of terminal devices, forming a highly efficient and collaborative computing ecosystem. However, with the deepening development of AI applications, the complexity and diversity of computing tasks are increasing, and the demand for computing resources is exhibiting a high degree of heterogeneity. This heterogeneity is not only reflected in the diversity of hardware architectures but also involves multiple dimensions such as computing performance, power consumption, and cost, posing significant challenges to task scheduling, especially in high-load scenarios (such as sudden AI task disruptions). Therefore, designing a scheduling algorithm that can perform real-time online decision-making in a vast decision space while ensuring optimal time efficiency and resource allocation is crucial.

[0015] There are currently some innovative solutions to the problem of optimizing cloud-edge-device resource scheduling: Method 1: A multi-user fine-grained task offloading and scheduling method under cloud-edge-device collaboration is proposed. This method jointly considers latency, energy consumption and server rental costs. First, application tasks are divided and subtask priorities are designed. Then, a multi-user subtask scheduling scheme is proposed. An improved simulated annealing particle swarm algorithm is designed to solve for the minimum total system cost, thereby achieving the optimal offloading decision.

[0016] Method 2: This method proposes an optimization scheme for resource scheduling that adapts to the characteristics of two typical cloud-edge collaboration scenarios. It analyzes the scenario breakdown, scheduling objectives, and solution methods in turn.

[0017] Method 3: A multi-queue task scheduling method based on HGIC time difference is proposed to realize heterogeneous task management in hybrid cloud scenarios.

[0018] Method 4: A cloud-edge-device resource scheduling optimization method based on a two-layer graph neural network is proposed. The main steps include constructing a cloud-edge-device resource scheduler based on a two-layer graph neural network. The first layer graph neural network is used to implement cloud-edge-device resource allocation, and the second layer graph neural network is used for industrial terminal layer scheduling. Heterogeneous graphs are used to represent the state in the actual manufacturing environment, capturing the complex relationships between workpiece processes and available equipment. A directed acyclic graph is constructed as the input to the second layer graph neural network. A framework model for phased scheduling and offloading is constructed, where the phased scheduling and offloading framework model uses an optimization strategy to optimize task scheduling. The optimization strategy adopts a three-stage adaptive hybrid heterogeneous resource allocation and task scheduling method with a dynamic multi-task mode. The first stage performs task pre-sorting, the second stage performs resource pre-allocation, and the third stage performs dynamic task scheduling to coordinate the richness of cloud computing resources and the low transmission latency of local and edge resources, thereby optimizing the scheduling results.

[0019] Based on the solutions mentioned above, the following problems still exist: The multi-user fine-grained task offloading scheduling method proposed in Method 1, under cloud-edge-device collaboration, provides a new perspective and approach for improving the utilization of dense network resources and solving the problem of multi-user fine-grained task offloading. Through cloud-edge-device collaborative computing, it effectively balances latency, energy consumption, and server rental costs. However, this research mainly focuses on the offloading problem of a single application on a user device and does not fully consider the scenario of multi-application collaborative offloading, which limits the generalization of the method. Furthermore, although Method 1 considers the dependencies between tasks, the model does not fully cover all complex application scenarios, such as dynamic changes between tasks and real-time adjustments to resource competition.

[0020] Although Method 2 proposes resource scheduling optimization schemes for different scenarios, its scheme still has the following shortcomings. First, the scheme fails to make full use of the computing resources of the cloud, edge and industrial terminals, resulting in the overall resource utilization rate not reaching the optimal level. Second, the scheme fails to effectively solve the scheduling problems that occur during dynamic data updates, making it difficult to adapt to the needs of real-time data updates. Moreover, the applicability of the scheme in actual production environments has not been verified.

[0021] The multi-queue scheduling method proposed in Method 3 effectively solves the problem of heterogeneous task scheduling in hybrid clouds. By considering various time-varying factors, it achieves a balance between cost-effectiveness and performance. However, this method still has shortcomings in dealing with dynamic changes in hybrid cloud environments. For example, its ability to adjust resource allocation and execution order under high load scenarios has not been fully verified. Furthermore, the performance of cloud heuristic optimization methods depends on parameter settings. In practical applications, how to quickly and effectively optimize parameters according to specific scenarios and needs is a challenge.

[0022] The cloud-edge-device resource scheduling method proposed in Method 4, based on a two-layer graph neural network, can achieve rational utilization of resources in the cloud, edge, and industrial terminals in the context of intelligent manufacturing. However, it still has some limitations in practical applications. The two-layer graph neural network requires a lot of computing resources and time for training and inference, which greatly reduces the feasibility and efficiency of actual deployment. The two-layer graph neural network model does not respond quickly enough in dynamic and uncertain environments. When faced with sudden tasks or unconventional situations, its real-time adjustment capability is insufficient. Furthermore, it is difficult to directly apply to other application scenarios or environments, resulting in poor generalization.

[0023] To address the aforementioned issues, this application proposes a novel cloud-edge-device collaborative real-time resource scheduling strategy based on reinforcement learning optimization to efficiently resolve the heterogeneity problem of tasks and computing resources. Simultaneously, it incorporates optimization of PPO generalization capabilities based on the DeepSets neural network model; considering sudden task occurrences, an intelligent queue system is designed to prioritize tasks, comprehensively considering heterogeneous computing capabilities and network impact, ultimately ensuring optimal task allocation and maximizing overall computing resource utilization.

[0024] The detailed technical solutions of this application embodiment are as follows: This application provides a resource scheduling method, such as... Figure 1 As shown, the method may include: S101. Obtain training data.

[0025] The training data includes at least: task list information, heterogeneous resource information of various task processing nodes, and clock information.

[0026] In this embodiment, an improved reinforcement learning PPO model is primarily used to schedule heterogeneous computing resources for each task. A reinforcement learning agent (also known as an intelligent agent) is employed as the core of the task scheduler to achieve efficient task allocation and resource management. The intelligent agent allocates tasks based on the current environmental state (such as the computing power of heterogeneous resources between nodes, resource usage, task requirements, and device communication costs). The reinforcement learning process aims to maximize cumulative rewards, and through continuous interaction with the environment, the intelligent agent continuously optimizes its decision-making strategy.

[0027] In this embodiment, training data is used to input the agent for action decision-making. In this embodiment, the training data is the observation environment of the task scheduler. The observation environment of the task scheduler consists of a task queue (i.e., task list information), clock information, and heterogeneous computing resources coordinated by cloud, edge, and device. The observation environment of the task scheduler is the current environmental state of the agent input into the PPO model.

[0028] In this embodiment of the application, the task queue is used to provide task information, which includes the computing resources required for different computing methods and the estimated computing time.

[0029] In the embodiments of this application, clock information is used to provide the current system time and record the arrival time, queuing time, allocated execution time, and final completion time of each task.

[0030] In this embodiment, the heterogeneous computing resources of cloud-edge-device collaboration are used to provide comprehensive environmental information, including basic information about each cluster, such as available computing resources, network environment, and the number of completed tasks. The collaborative computing clusters include cloud clusters, edge clusters, and end device clusters.

[0031] In this embodiment of the application, the environmental information composed of heterogeneous computing resource information from cloud, edge, and terminal can be denoted as: , The set of all nodes contained in all clusters within the system can be specifically represented as Where k represents the total number of nodes, and each node's feature vector N is a tuple containing various resource attributes of that node. It can be defined as the following formula (1): (1) in, Represents a node The number of available cores in heterogeneous processors / accelerators (such as central processing units (CPUs), graphics processing units (GPUs), neural processing units (NPUs), etc.). Represents a node The amount of available memory on heterogeneous processors / accelerators (such as CPUs, GPUs, NPUs, etc.). Represents a node Available storage space Represents a node Network communication capabilities.

[0032] In this embodiment of the application, clock information This represents the system's built-in time step, recording the elapsed time since the system began scheduling the first task. When there are no tasks to be scheduled in the system, or all computing resources are idle, The value is reset to zero until the next task arrives and the timer restarts.

[0033] In this embodiment of the application, the task information provided by the task list information can be denoted as: , This represents the waiting time for the computational tasks currently being scheduled by the system, the computational tasks currently waiting to be scheduled, and the waiting time for the computational tasks waiting to be scheduled. Specifically, it can be expressed as: Where x represents the number of tasks. , This is the waiting time for the task. This can be expressed as the following formula (2): (2) in, Indicates task In the The estimated number of cores required to run on this type of computing unit. Indicates task In the The estimated memory required to run on this type of computing unit. Indicates task In the Estimated storage space required under various execution strategies. Indicates task In the The estimated duration of operation under a given execution strategy.

[0034] S102. Based on the task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model included in the training data, determine the resource scheduling strategy corresponding to one or more tasks.

[0035] In this embodiment of the application, the first model is the PPO model.

[0036] In this embodiment, the acquired training data is used as the current environmental state of the agent input into the PPO model. After acquiring the training data, before inputting it into the agent of the PPO model, in order to improve the generalization ability of the PPO model, a pre-trained second model can be used to extract features from the task list information, heterogeneous resource information of various processing nodes, and clock information included in the training data. The extracted feature information corresponding to the task list information, heterogeneous resource information, and clock information is aggregated to obtain the feature vector corresponding to the training data, so as to determine the resource scheduling strategy using the first model and the feature vector.

[0037] In this embodiment, the second model employs the DeepSets neural network model, which is a deep learning model specifically designed for processing set data. The architecture of the DeepSets neural network model is as follows: Figure 2 As shown, the core idea is to use a feature extraction function. To achieve data feature extraction, feature aggregation functions are used. Feature aggregation is achieved, making the network insensitive to the order of the input set.

[0038] It should be noted that DeepSets possesses permutation invariance, flexibility, efficiency, and strong generalization ability, while PPO, as an improved policy gradient algorithm, introduces a probability ratio shearing function to limit the policy update step size, thereby improving training stability and efficiency. By combining DeepSets with the PPO algorithm, the powerful modeling capabilities of DeepSets for ensemble data can be leveraged to handle complex task sets in heterogeneous cloud-edge-device collaborative scenarios, further enhancing the performance of the PPO algorithm in resource allocation and task scheduling, while also improving its generalization ability.

[0039] It should be noted that the process of pre-training the DeepSets neural network model can be found in relevant technical documents, and will not be elaborated here.

[0040] In the embodiments of this application, the following are adopted: Figure 2 The model architecture shown uses task list information, heterogeneous resource information, clock information, etc., included in the training data as a set. , will set Feature extraction function from a pre-trained DeepSets neural network model Feature extraction is performed, followed by summing and aggregating the extracted features, and then applying the feature aggregation function. Feature aggregation is performed to transform a set into a final feature vector representation.

[0041] In this embodiment, after processing the training data using the DeepSets neural network model to obtain feature vectors, the obtained feature vectors are input into the PPO model. The PPO model then outputs the action space for the current environmental state. The action space is the set of all possible actions that the agent can take after observing the current environmental state.

[0042] In the embodiments of this application, the action space mainly includes two types of actions: the first is task allocation action, and the second is waiting action. The first can be understood as the agent, after observing the current environmental state, allocating resource scheduling strategies to one or more tasks; the second can be understood as causing one or more tasks in the task list to enter a waiting queue (or cache queue) to wait for more suitable scheduling resources.

[0043] Based on this, in the embodiments of this application, after the feature vector is input into the intelligent agent (scheduler), one approach is for the intelligent agent to directly output, based on the heterogeneous resource information in the feature vector and combined with clock information, the task allocation and execution on the specified cluster and nodes according to the current environmental state.<Cluster,Node,T> The expression indicates that Cluster represents the cluster that executes the task, Node represents the node that executes the task, and T represents the method that executes the task.

[0044] In this embodiment of the application, for one or more tasks in the task list information, if the resource scheduling strategy corresponding to one or more tasks is not determined based on the task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model included in the training data, one or more tasks are added to the waiting task list, so that the resource scheduling strategy corresponding to one or more tasks in the waiting task list can be determined again when the preset time is reached.

[0045] In this embodiment, if the agent (scheduler) fails to determine a suitable resource scheduling strategy for one or more tasks after inputting the feature vector, such as lacking heterogeneous resource information to meet the needs of one or more tasks, then a second waiting action is adopted, whereby the agent (scheduler) allows the current task to wait for a period of time. Then, scheduling resources are allocated to the tasks in the waiting queue to satisfy specific scheduling strategies or to wait for better resource conditions.<Cluster,Node,T,P> express.

[0046] In summary, in this embodiment of the application, by inputting training data into the DS-PPO model, processing it through the Deepsets neural network model to obtain feature vectors, and then processing the feature vectors through the PPO model, the output includes two types: resource scheduling strategy or adding the task to the waiting queue for rescheduling of resources.

[0047] S103. Based on the preset reward function and resource scheduling strategy, the first model is trained to obtain the trained first model, which is then used to schedule resources for one or more target tasks in the task list to be scheduled.

[0048] The preset reward functions include a resource usage function and a task processing time function.

[0049] In this embodiment of the application, resource usage may include resource utilization rate.

[0050] In this embodiment of the application, the task processing time may include task waiting time and task execution time.

[0051] In this embodiment, the preset reward function includes a resource utilization function, a task waiting time function, and a task execution time function.

[0052] In this embodiment, the resource scheduling strategy corresponding to the task in the task list output by the agent of the PPO model is used, and a reward mechanism is adopted to provide the agent with direct feedback on the impact of the output action space on the environment. The feedback is usually reflected in the form of reward value.

[0053] In this embodiment, the agent executes the output action. After the action is executed, a preset reward function is used to evaluate the reward value of the resource scheduling strategy output by the agent. With the help of the feedback mechanism of the reward mechanism, the agent can identify the quality of the action and adjust the agent's behavior strategy to achieve the goal of optimizing the PPO model parameters.

[0054] In this embodiment, the reward value obtained by the preset reward function from evaluating the resource scheduling strategy mainly has two types: The first type is -1, indicating that the agent (scheduler)'s current action has not caused any substantial change to the environment, that is, the current task is arranged to wait by the agent (scheduler) instead of being executed immediately. The second type is a positive value greater than 0. The magnitude of this type of reward value is determined by the quality of the decision made by the agent (scheduler) on the optimization objective, and further affects the subsequent reward value and the adjustment of the resource scheduling strategy.

[0055] In this embodiment, the model parameters of the PPO model are updated and adjusted based on the reward value obtained from the preset reward function. After multiple training cycles, the trained PPO model can be obtained.

[0056] In this embodiment of the application, the first model is trained based on a preset reward function and a resource scheduling strategy to obtain the trained first model. This can be achieved in the following way: Based on the preset reward function, the reward value corresponding to the resource scheduling strategy is determined, wherein the reward value includes a first reward value corresponding to the heterogeneous resource usage and a second reward value corresponding to the processing time of each task in the task list information; Based on the first reward value and the second reward value, the model parameters of the first model are updated to obtain the trained first model.

[0057] In this embodiment of the application, the flowchart for obtaining reward values ​​from input training data during reinforcement learning is as follows: Figure 3 As shown, for a given task, the reward for each action is formalized as a preset reward function. The expression is shown in the following formula (3): (3) in, Defined as follows (4): (4) Wherein, C is defined by the following formula (5): (5) W is defined by the following formula (6): (6) Defined as follows (7): (7) In the above formula, Let W represent the contribution of resource utilization rate to the reward, W represent the contribution of task waiting time to the reward, and C represent the contribution of task completion time to the reward. This represents the total contribution to the reward, i.e., the reward value received.

[0058] This indicates the situation where the task is waiting. This indicates the scheduling situation.

[0059] This represents the total runtime of all tasks completed up to the current time. Indicates task The estimated runtime; similarly, This represents the total waiting time for all completed tasks. Indicates task The estimated waiting time.

[0060] Indicates task The amount of resources used when executing on node j. Represents a node The total amount of resources available.

[0061] variable This indicates the number of tasks that have been completed. , and For the weight parameters, satisfying Furthermore, within the range [0,1], it is used to balance the completion time, waiting time, and resource utilization in the reward function.

[0062] In the embodiments of this application, reference is made to Figure 3 As shown, the training data is processed by DS-PPO based on the aforementioned implementation method, and the action space (resource scheduling strategy or entering the waiting queue) for the current environment state is output, denoted as <cluster, node, method (T), waiting time (P)>. The reward value for the agent's output action can be determined by combining the action space with the preset reward function defined above (i.e., formula (3)). That is, a positive reward is obtained if heterogeneous computing resources are successfully scheduled, and a negative reward (i.e. -1) is obtained if heterogeneous computing resources are not successfully scheduled.

[0063] In this embodiment of the application, the intelligent agent (scheduler) executes the action and determines the reward value of the output action from three perspectives: resource utilization, task waiting time, and task execution time, using the preset reward function defined above. That is, the contribution value of resource utilization to the reward and the contribution value of task waiting time and task execution time to the reward are calculated using the formulas (5) to (7) above. After obtaining the contribution value of resource utilization to the reward and the contribution value of task waiting time and task execution time to the reward, the model parameters of the PPO model are updated based on the contribution value of resource utilization to the reward and the contribution value of task waiting time and task execution time to the reward, so as to obtain the first model after training.

[0064] In this embodiment of the application, after calculating the contribution value of resource utilization rate to the reward and the contribution value of task waiting time and task execution time to the reward using the above formulas (5) to (7), the model parameters of the PPO model are updated based on the contribution value of resource utilization rate to the reward and the contribution value of task waiting time and task execution time to the reward. Specifically, this can be achieved in the following way: Based on the first reward value (i.e., the contribution value of resource utilization rate to the reward) and the second reward value (i.e., the contribution value of task waiting time and task execution time to the reward), the target reward value is determined; based on the target reward value and the preset loss function, the model parameters of the first model are updated to obtain the trained first model.

[0065] It should be noted that after determining the target reward value, the next environmental state data can be collected, and this process can be repeated to train the model until the trained PPO model is obtained.

[0066] In this embodiment, the contribution of resource utilization to the reward and the contribution of task waiting time and task execution time to the reward are substituted into the above formula (4) for summation to obtain the reward value corresponding to the resource scheduling strategy executed in the current environment. .

[0067] It should be noted that if the output action is to add the task to the waiting queue, the reward value is -1.

[0068] In this embodiment of the application, under the framework of reinforcement learning, the system calculates the total reward value over a period of time. This reward value is accumulated from each action and its corresponding reward value, and future reward values ​​are also taken into account in the total reward.

[0069] It should be noted that the reward value generated by each action is only used as a piece of data for memory during the training process in the RL model.

[0070] In this application embodiment, in the foregoing embodiment, when the scheduler receives the current environment state ,in It is a collection that contains multiple elements and features (such as the heterogeneous computing resource status of each node in the cluster, task information, etc.). The following formula (8) is used to represent it: (8) in, It includes specific computing resources and task information, clock information, etc. for each node, where i takes values ​​from 1 to n.

[0071] Continue execution, for each input element Application of feature extraction function This yields the feature representation of each element, and the feature extraction function. The process is implemented using a neural network, as shown in the following formula (9): (9) Summing and aggregating all element features, then applying the aggregation function. The final feature vector representation is shown in the following formula (10): (10) in, It is the result of applying the feature extraction function to each input element. This represents summing over all features. This represents an aggregate function.

[0072] In this embodiment of the application, after obtaining the feature vector, the feature vector is input into the PPO model. The PPO model's agent provides a resource scheduling strategy for scheduling tasks. At the same time, the preset reward function also outputs a reward value for the action output. Changes in the reward value will affect the advantage function, thereby causing changes in the loss function in the PPO model.

[0073] It should be noted that the PPO model includes a policy network. and value network Inputting feature vectors into the PPO model essentially means inputting the feature vectors into the PPO model. As a policy network and value network The inputs are used to obtain the probability distribution of the output actions of the policy network. The current state value estimated by the value network , It is expressed as the following formula (11). This can be expressed as the following formula (12): (11) (12) in, Represents the learnable parameters of the policy network. Represents the strategy function.

[0074] In this embodiment of the application, the intelligent agent generates and collects states during the interaction with the environment. ,action Instant rewards and the next state This data is then stored in an experience pool for subsequent training. Using the collected training data, a value network is then established. Calculate the advantage function , can be obtained As shown in the following formula (13): (13) in, For instant rewards, As a discount factor, Let be the eigenvector at time t. for The eigenvector at time step 1.

[0075] In this embodiment of the application, through the advantage function The pruning loss function of the PPO model is obtained. As shown in the following formula (14): (14) in, To be the minimum constant that limits the policy update magnitude, The probability ratio of the new and old strategies can be defined by the following formula (15): (15) in, Characterization of new strategies Representing the old strategy.

[0076] It should be noted that the clip operation will... Limited to [1- ,1+ ]between.

[0077] In this embodiment of the application, in order to make the predicted value of the value function close to the actual return (i.e., advantage estimation), the mean squared error form is used to represent the regression loss function of the value function. The formula is shown in the following formula (16): (16) in, This represents the expectation at time step t across all sampled trajectories.

[0078] In this embodiment, to prevent the strategy from converging prematurely to a suboptimal solution and to ensure the diversity of exploration, an entropy loss of the strategy is added. The greater the policy entropy, the more random the distribution of action choices, and the greater the entropy loss of the policy. The formula is shown in the following formula (17): (17) In this embodiment of the application, the final comprehensive loss function is shown in the following formula (18): (18) in, and The hyperparameters are used to adjust the weights of various items, in order to balance the weights of different loss terms.

[0079] In this embodiment of the application, the network parameters are evaluated using the gradient descent method. Update to minimize the loss function It can be expressed as the following formula (19): (19) in, For learning rate, loss function Regarding parameters The gradient.

[0080] In the embodiments of this application, by minimizing the total loss function, the PPO algorithm can optimize the policy function and the value function, thereby enabling more stable and efficient policy learning and scheduling optimization in dynamic environments.

[0081] In this embodiment of the application, the model parameters of the PPO model are continuously adjusted using the calculated target reward value and the loss function until the loss function is minimized. The PPO model trained at this point is then used as the final model for resource scheduling.

[0082] It should be noted that the implementation process of the above formulas (8) to (9) can also be understood as the specific process of optimizing the scheduling strategy based on the DS-PPO model.

[0083] In this embodiment of the application, after obtaining the trained PPO model, the resource scheduling of the task to be scheduled can be carried out using the trained PPO model in the following way: obtain heterogeneous resource information and clock information of one or more target tasks, task processing nodes of each type in the task to be scheduled list; extract feature information corresponding to one or more target tasks, heterogeneous resource information and clock information respectively, and determine feature vectors based on feature information; input the feature vectors into the trained first model, and output the target resource scheduling result for one or more target tasks through the first model.

[0084] In this embodiment of the application, after the PPO model is trained in the above manner, in a specific application, one or more target tasks to be scheduled are collected. The one or more target tasks can be obtained from the task queue list, as well as heterogeneous computing resources and clock information of all processing nodes obtained from all clusters.

[0085] In this embodiment, the DeepSets neural network model is used to extract features from the collected information and further determine the feature vector.

[0086] It should be noted that the process of using the DeepSets neural network model to determine the feature vector of the input environment state can be referred to the aforementioned embodiments, and will not be repeated here.

[0087] In this embodiment, the obtained feature vector is input into the trained PPO model, which then outputs the target resource scheduling result for one or more target tasks in the task list. Since the PPO model training process considers the impact of resource utilization and task processing time on the reward value, this target resource scheduling result is the optimal resource scheduling result, i.e., it uses the least amount of time and achieves the highest utilization of heterogeneous computing resources.

[0088] In the above embodiments of this application, the tasks in the task list information involved in the above embodiments can be sorted based on priority and then scheduled according to priority order when performing heterogeneous computing resource scheduling. Therefore, based on the task list information, heterogeneous resource information of various task processing nodes, clock information and the first model included in the training data, the resource scheduling strategy corresponding to one or more tasks can be determined. Specifically, it can also be implemented in the following way: obtaining the scheduling priority information of one or more tasks in the task queue information; determining the scheduling priority order corresponding to one or more tasks according to the scheduling priority information; and determining the resource scheduling strategy corresponding to one or more tasks according to the scheduling priority order based on the task list information, heterogeneous resource information of various task processing nodes, clock information and the first model included in the training data.

[0089] In this embodiment of the application, in order to ensure that high-priority tasks are scheduled first, the scheduling priority information can be determined according to the urgency of each task in the task queue information. The tasks in the task queue information can be sorted in descending order of scheduling priority information to determine the priority order of all tasks in the task queue.

[0090] In this embodiment of the application, when scheduling computational resources for tasks in the task queue, the PPO model can be adopted according to the task priority order. For one or more tasks in the task list information, the resource scheduling strategy corresponding to one or more tasks is determined based on heterogeneous resource information and the clock information.

[0091] It should be noted that, based on the task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model included in the training data, the specific implementation of determining the resource scheduling strategy corresponding to one or more tasks can be referred to the foregoing embodiments, and will not be repeated here.

[0092] It should be noted that after the first model is trained, in the process of practical application, the tasks to be scheduled can be prioritized. For example, when a sudden task arrives, the sudden task can be prioritized for scheduling, and then the priority order can be used to determine whether to prioritize the scheduling of heterogeneous resources for the sudden task.

[0093] It is understood that the resource scheduling method provided in this application embodiment, when training the first model, firstly determines the resource scheduling strategy for one or more tasks based on the tasks in the task list and the heterogeneous resource information and clock information input to the first model. Since the preset reward function includes a resource usage function and a task processing time function, when training the first model using the preset reward function and the resource scheduling strategy, the resource scheduling strategy for one or more tasks determined by the first model is optimized from the aspects of resource usage and task processing time. Therefore, by using this method to train the first model, the resource scheduling result for one or more target tasks is determined using the trained first model. This resource scheduling result can ensure that the task completion time is minimized and the utilization rate of heterogeneous resources is maximized when multiple tasks are concurrent.

[0094] Based on the above embodiments, this application also provides an overall flowchart of a cloud-edge-device resource scheduling method based on the DS-PPO model, as shown below. Figure 4 As shown, the specific implementation process is as follows: State Observation: After observing the current environment state, which includes task information (Task info), clock information, and environment information (Env info) in the task list, the scheduler first extracts key features of the current environment state through the DeepSets neural network model to obtain a feature vector. Then, the feature vector is used as the input of the reinforcement learning (RL) agent (i.e., the scheduler).

[0095] The RL agent uses this data to determine whether a task should wait—that is, whether to allocate the task immediately or wait for a period of time. When resource conditions meet the task requirements, task allocation is performed immediately. The scheduler directly outputs the cluster, node number, and execution method (e.g., whether the task needs to be executed by a model or a regular process) based on the current environment state. This can be represented as...<Cluster,Node,T> Otherwise, the scheduler places the task in the task cache queue (also known as the task waiting queue) to wait and restarts the scheduling process. The scheduler executes the task scheduling strategy based on the output action, assigning the task to the appropriate node. After the action is executed, the reward generator simultaneously evaluates the reward value generated by the action at the current time based on the agent's action and feeds the reward value back to the scheduler. The scheduler further optimizes and adjusts the model parameters based on the reward value after the action is executed. Finally, the trained PPO model, under conditions of high task concurrency, maximizes the utilization of heterogeneous resources and minimizes task waiting time and task completion time during task allocation.

[0096] Based on the above embodiments, the resource scheduling in this application mainly manifests in the following aspects: The core of this cloud-edge-device collaborative real-time scheduling strategy, optimized using reinforcement learning, lies in its novel approach. This strategy integrates reinforcement learning and cloud-edge-device collaboration to efficiently address the heterogeneity of tasks and computing resources. By using a reinforcement learning agent as the core scheduler, the agent can dynamically and in real-time allocate tasks based on the current environmental state (including the computing power and resource usage of heterogeneous resources among nodes), thereby optimizing resource utilization and task execution efficiency.

[0097] Optimization of PPO Generalization Ability Based on DeepSets Neural Network: In order to further improve the performance of scheduling policies in handling complex tasks and heterogeneous resources, this application adopts DeepSets neural network to optimize the generalization ability of proximal policy optimization (PPO). The DeepSets model extracts key features of the observation environment of task queues, system clocks and heterogeneous computing resources, and uses the obtained feature vectors as input to RL agent. This process not only helps the agent to more accurately judge the best time and method of task allocation, but also enhances its adaptability in different scenarios, so that it can still maintain efficient scheduling performance when facing diverse tasks and heterogeneous resources.

[0098] The intelligent queue system prioritizes tasks: In the event of a sudden task shortage, the intelligent queue system can prioritize tasks and automatically adjust the execution order of tasks based on factors such as urgency and resource requirements, ensuring that critical tasks are processed first.

[0099] Optimal task allocation and maximization of overall resource utilization: In this embodiment, the optimal allocation of tasks and maximization of overall resource utilization are achieved by designing the objective function of the reward mechanism. After observing the current environmental state, the scheduler extracts features through the DeepSets model, and the RL agent determines the task allocation strategy based on these features. When the resource conditions meet the task requirements, the scheduler will immediately allocate the task and allocate the reward. If the conditions are not met, the task will be placed in the task queue for subsequent processing, and the reward will be set to -1. In this way, the system can minimize the task completion time and waiting time, and improve the utilization of computing resources.

[0100] Compared with related technologies, the technical solution adopted in this application has the following technical advantages: 1. Employing reinforcement learning optimization strategies as the core of the scheduler, the agent dynamically and in real-time allocates tasks based on the current environmental state (such as the computing power of heterogeneous resources among nodes, resource usage, task requirements, and device communication costs). Through continuous trial and error and learning in a cloud-edge-device collaborative environment, the agent can automatically adapt to the heterogeneous characteristics of different tasks and computing resources, and learn the optimal scheduling strategy. Compared with existing technologies, this approach can more efficiently solve the problem of heterogeneity in tasks and computing resources, achieving optimal task allocation.

[0101] 2. The Proximal Policy Optimization (PPO) model is optimized using the DeepSets neural network. The DeepSets model has the characteristics of permutation invariance, flexibility, efficiency and strong generalization ability. After extracting key features from the input data through the DeepSets neural network model, the data is then fed into the RL agent, which enables the scheduling policy to show good generalization performance in different application scenarios.

[0102] 3. By adopting an intelligent queue system for task priority sorting, it can make real-time dynamic adjustments based on the urgency of tasks and resource availability. This ensures a rapid response and optimal scheduling decision under high load or sudden task conditions, improving real-time scheduling and dynamic adjustment capabilities.

[0103] 4. By designing the objective function of the reward mechanism, the optimal allocation of tasks and the maximization of overall resource utilization can be achieved.

[0104] Based on this, the technical solution of this application significantly improves the allocation efficiency of heterogeneous computing resources by optimizing the scheduling strategy through reinforcement learning, combined with DeepSets neural networks and intelligent queue systems. It is applicable to high-demand scenarios such as cloud computing, edge computing, industrial IoT, autonomous driving, smart cities, and 5G communications. This implementation can bring significant commercial value to multi-cluster and mobile cloud computing environments. Through efficient resource scheduling and management, it meets the computing needs of enterprises and data centers in complex, dynamic, and heterogeneous environments, effectively promoting the improvement of mobile cloud computing service quality and the reduction of operating costs.

[0105] Based on the above embodiments, another embodiment of this application provides a resource scheduling device 1, such as... Figure 5 As shown, the resource scheduling device 1 includes: The acquisition unit 10 is used to acquire training data, wherein the training data includes at least: task list information, heterogeneous resource information of various task processing nodes, and clock information.

[0106] The determining unit 11 is used to determine the resource scheduling strategy corresponding to the one or more tasks based on the task list information, heterogeneous resource information of various task processing nodes, clock information and the first model included in the training data.

[0107] Training unit 12 is used to train the first model based on a preset reward function and the resource scheduling strategy to obtain a trained first model, so as to use the trained first model to perform resource scheduling on one or more target tasks in the task list to be scheduled, wherein the preset reward function includes a resource usage function and a task processing time function.

[0108] In one embodiment, the determining unit 11 is further configured to determine the reward value corresponding to the resource scheduling strategy based on the preset reward function, wherein the reward value includes a first reward value corresponding to the heterogeneous resource usage and a second reward value corresponding to the processing time of each task in the task list information.

[0109] Training unit 12 is also used to update the model parameters of the first model based on the first reward value and the second reward value, so as to obtain the trained first model.

[0110] In one embodiment, the determining unit 11 is further configured to determine a target reward value based on the first reward value and the second reward value.

[0111] Training unit 12 is also used to update the model parameters of the first model based on the target reward value and the preset loss function, so as to obtain the trained first model.

[0112] In one embodiment, the resource scheduling device 1 may further include: an extraction unit and an aggregation unit.

[0113] The extraction unit is used to extract features from the training data, including task list information, heterogeneous resource information of various processing nodes, and clock information, using a pre-trained second model.

[0114] The aggregation unit is used to aggregate the feature information corresponding to the extracted task list information, heterogeneous resource information and clock information respectively to obtain the feature vector corresponding to the training data, so as to determine the resource scheduling strategy using the first model and the feature vector.

[0115] In one embodiment, the resource scheduling device 1 may further include an adding unit.

[0116] The addition unit is used to add one or more tasks in the task list information to a waiting task list when the resource scheduling strategy corresponding to the one or more tasks has not been determined based on the task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model, so that the resource scheduling strategy corresponding to the one or more tasks in the waiting task list can be determined again when a preset time is reached.

[0117] In one embodiment, the acquisition unit 10 is further configured to acquire scheduling priority information of one or more tasks in the task queue information.

[0118] The determining unit 11 is further configured to determine the scheduling priority order corresponding to the one or more tasks according to the scheduling priority information.

[0119] The determining unit 11 is further configured to determine the resource scheduling strategy corresponding to the one or more tasks according to the scheduling priority order, based on the task list information, heterogeneous resource information of various task processing nodes, clock information and the first model included in the training data.

[0120] In one embodiment, the acquisition unit 10 is further configured to acquire heterogeneous resource information and clock information of one or more target tasks, task processing nodes of various types, in the task list to be scheduled.

[0121] The extraction unit is further configured to extract feature information corresponding to the one or more target tasks, the heterogeneous resource information and the clock information respectively, and determine feature vectors based on the feature information.

[0122] The determining unit 11 is further configured to input the feature vector into the trained first model and output the target resource scheduling result for the one or more target tasks through the first model.

[0123] This application provides a resource scheduling device that acquires training data, wherein the training data includes at least: task list information, heterogeneous resource information of various task processing nodes, and clock information; based on the task list information, heterogeneous resource information of various task processing nodes, and clock information included in the training data, and a first model, a resource scheduling strategy corresponding to one or more tasks is determined; based on a preset reward function and the resource scheduling strategy, the first model is trained to obtain a trained first model, and the trained first model is used to perform resource scheduling on one or more target tasks in the task list to be scheduled, wherein the preset reward function includes a resource usage function and a task processing time function. Therefore, the resource scheduling device proposed in this application provides a resource scheduling device for the first task. When training a model, the first model first determines the resource scheduling strategy for one or more tasks in the task list, taking into account the heterogeneous resource information and clock information input to the first model. Since the preset reward function includes a resource usage function and a task processing time function, when training the first model using the preset reward function and the resource scheduling strategy, the resource scheduling strategy for one or more tasks determined by the first model is optimized from the aspects of resource usage and task processing time. Therefore, this method is used to train the first model, and the resource scheduling result of one or more target tasks is determined by the trained first model. This resource scheduling result can ensure that the task completion time is minimized and the utilization of heterogeneous resources is maximized when multiple tasks are running concurrently.

[0124] Figure 6 This is a schematic diagram of the composition structure of a resource scheduling device 2 provided in an embodiment of this application. In practical applications, based on the same disclosed concept of the above embodiments, such as... Figure 6 As shown, the resource scheduling device 2 in this embodiment includes a processor 20, a memory 21, and a communication bus 22.

[0125] In specific embodiments, the acquisition unit 10, determination unit 11, training unit 12, extraction unit, aggregation unit, and addition unit described above can be implemented by the processor 20 located on the resource scheduling device 2. The processor 20 can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), CPU, controller, microcontroller, and microprocessor. It is understood that for different devices, the electronic device used to implement the above processor functions can also be other types, and this application embodiment does not impose specific limitations.

[0126] In this embodiment, the communication bus 22 is used to realize the connection communication between the processor 20 and the memory 21; when the processor 20 executes the running program stored in the memory 21, it implements the following resource scheduling method: Acquire training data, which includes at least: task list information, heterogeneous resource information of various task processing nodes, and clock information; based on the task list information, heterogeneous resource information of various task processing nodes, and clock information included in the training data, and a first model, determine a resource scheduling strategy corresponding to one or more tasks; train the first model based on a preset reward function and resource scheduling strategy to obtain a trained first model, and use the trained first model to perform resource scheduling on one or more target tasks in the task list to be scheduled, wherein the preset reward function includes a resource usage function and a task processing time function.

[0127] In one embodiment, the processor 20 is further configured to determine, based on the preset reward function, the reward value corresponding to the resource scheduling strategy, wherein the reward value includes a first reward value corresponding to the heterogeneous resource usage and a second reward value corresponding to the processing time of each task in the task list information; and update the model parameters of the first model based on the first reward value and the second reward value to obtain the trained first model.

[0128] In one embodiment, the processor 20 is further configured to determine a target reward value based on the first reward value and the second reward value; and update the model parameters of the first model based on the target reward value and a preset loss function to obtain the trained first model.

[0129] In one embodiment, the processor 20 is further configured to use a pre-trained second model to extract features from the task list information, heterogeneous resource information of various processing nodes, and clock information included in the training data; aggregate the extracted feature information corresponding to the task list information, the heterogeneous resource information, and the clock information to obtain a feature vector corresponding to the training data, so as to determine the resource scheduling strategy using the first model and the feature vector.

[0130] In one embodiment, the processor 20 is further configured to, for one or more tasks in the task list information, add the one or more tasks to a waiting task list if the resource scheduling strategy corresponding to the one or more tasks is not determined based on the task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model included in the training data, so as to determine the resource scheduling strategy corresponding to the one or more tasks in the waiting task list again when a preset time is reached.

[0131] In one embodiment, the processor 20 is further configured to obtain scheduling priority information of one or more tasks in the task queue information; determine the scheduling priority order corresponding to the one or more tasks according to the scheduling priority information; and determine the resource scheduling strategy corresponding to the one or more tasks according to the scheduling priority order, based on the task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model included in the training data.

[0132] In one embodiment, the processor 20 is further configured to acquire heterogeneous resource information and clock information of one or more target tasks, task processing nodes of various types, in a list of tasks to be scheduled; extract feature information corresponding to the one or more target tasks, the heterogeneous resource information, and the clock information respectively, and determine feature vectors based on the feature information; input the feature vectors into the trained first model, and output target resource scheduling results for the one or more target tasks through the first model.

[0133] Based on the above embodiments, this application provides a storage medium storing a computer program thereon. The computer-readable storage medium stores one or more programs, which can be executed by one or more processors and applied in a resource scheduling device. The computer program implements the resource scheduling method described above.

[0134] Based on the above embodiments, this application provides a computer program product, including a computer program that can be executed by one or more processors and applied in a resource scheduling device. The computer program implements the resource scheduling method described above.

[0135] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause an image display device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the embodiments of this application.

[0137] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the protection scope of this application.

Claims

1. A resource scheduling method, characterized in that, The method includes: Acquire training data, wherein the training data includes at least: task list information, heterogeneous resource information of various task processing nodes, and clock information; Based on the training data, including task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model, a resource scheduling strategy corresponding to one or more tasks is determined. Based on the preset reward function and the resource scheduling strategy, the first model is trained to obtain the trained first model, which is then used to schedule resources for one or more target tasks in the task list to be scheduled. The preset reward function includes a resource usage function and a task processing time function.

2. The method according to claim 1, characterized in that, The step of training the first model based on the preset reward function and the resource scheduling strategy to obtain the trained first model includes: Based on the preset reward function, the reward value corresponding to the resource scheduling strategy is determined, wherein the reward value includes a first reward value corresponding to the heterogeneous resource usage and a second reward value corresponding to the processing time of each task in the task list information. Based on the first reward value and the second reward value, the model parameters of the first model are updated to obtain the trained first model.

3. The method according to claim 2, characterized in that, The step of updating the model parameters of the first model based on the first reward value and the second reward value to obtain the trained first model includes: Based on the first reward value and the second reward value, determine the target reward value; Based on the target reward value and the preset loss function, the model parameters of the first model are updated to obtain the trained first model.

4. The method according to claim 1, characterized in that, After acquiring the training data, the method further includes: The pre-trained second model is used to extract features from the training data, including the task list information, heterogeneous resource information of various processing nodes, and clock information. The extracted task list information, heterogeneous resource information, and clock information are aggregated to obtain the feature vector corresponding to the training data, so as to determine the resource scheduling strategy using the first model and the feature vector.

5. The method according to claim 1, characterized in that, After acquiring the training data, the method further includes: For one or more tasks in the task list information, if the resource scheduling strategy corresponding to the one or more tasks cannot be determined based on the task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model included in the training data, the one or more tasks are added to the waiting task list, so that the resource scheduling strategy corresponding to the one or more tasks in the waiting task list can be determined again when a preset time is reached.

6. The method according to claim 1, characterized in that, Based on the training data, including task list information, heterogeneous resource information of various task processing nodes, clock information, and the first model, the resource scheduling strategy for one or more tasks is determined, including: Obtain the scheduling priority information of one or more tasks from the task queue information; The scheduling priority order of the one or more tasks is determined according to the scheduling priority information. Based on the scheduling priority order, the resource scheduling strategy corresponding to the one or more tasks is determined according to the task list information, heterogeneous resource information of various task processing nodes, clock information and the first model included in the training data.

7. The method according to claim 1, characterized in that, After training the first model based on the preset reward function and the resource scheduling strategy to obtain the trained first model, the method further includes: Obtain heterogeneous resource information and clock information of one or more target tasks, task processing nodes of various types, from the list of tasks to be scheduled; Extract feature information corresponding to the one or more target tasks, the heterogeneous resource information, and the clock information respectively, and determine feature vectors based on the feature information; The feature vector is input into the trained first model, and the first model outputs the target resource scheduling result for the one or more target tasks.

8. A resource scheduling device, characterized in that, The resource scheduling device includes a processor and a memory; when the processor executes the running program stored in the memory, it implements the method as described in any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images