Task scheduling method, device and electronic equipment

Abstract

The embodiment of the present invention discloses a task scheduling method, device and electronic device, the method comprising: receiving a task processing request for a target task sent by a terminal device; in response to the task processing request, determining a computing node for processing the task, and constructing a first graph structure data according to the link bandwidth between the computing nodes; splitting the target task into a plurality of subtasks with dependencies, and constructing directed graph structure data according to the dependencies between the subtasks; determining a resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing nodes, and a state space containing information required to determine the resource allocation strategy; and sending the subtasks to corresponding computing nodes for processing according to the resource allocation strategy.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a task scheduling method, device and electronic equipment. Background Art

[0002] With the rapid development of communication services and computer technology, edge computing nodes have been widely used due to their wide distribution and low latency, allowing users to access and use near-field computing resources more conveniently.

[0003] Scheduling of edge computing nodes is to assign pending tasks to edge computing nodes for processing. Tasks can usually be assigned randomly.

[0004] However, due to the differences in latency, load, etc. of different edge computing nodes, random allocation will result in unbalanced node loads, leading to poor node resource utilization, affecting global resource utilization and scheduling continuity. Therefore, a technical solution is needed to improve resource utilization and scheduling continuity during task scheduling. Summary of the invention

[0005] The purpose of the embodiment of the present invention is to provide a task scheduling method to solve the problems of poor data utilization and scheduling continuity in the task scheduling process in the prior art.

[0006] To solve the above technical problems, the embodiments of the present invention are implemented as follows: In a first aspect, an embodiment of the present invention provides a task scheduling method, the method comprising: Receiving a task processing request for a target task sent by a terminal device; In response to the task processing request, determining a computing node for processing the task, and constructing first graph structure data according to link bandwidths between the computing nodes; Splitting the target task into a plurality of subtasks with dependency relationships, and constructing directed graph structure data according to the dependency relationships between the subtasks; Determine the resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing node, and a state space containing information required to determine the resource allocation strategy, wherein the state space contains feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated during the resource allocation strategy generation process; According to the resource allocation strategy, the subtasks are respectively sent to corresponding computing nodes for processing.

[0007] In a second aspect, an embodiment of the present invention provides a task scheduling device, the device comprising: A request receiving module, used for receiving a task processing request for a target task sent by a terminal device; A first construction module, configured to determine, in response to the task processing request, a computing node for processing the task, and construct first graph structure data according to a link bandwidth between the computing nodes; The second construction module is used to split the target task into a plurality of subtasks with dependency relationships, and construct directed graph structure data according to the dependency relationships between the subtasks; a strategy generation module, configured to determine the resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing node, and a state space containing information required to determine the resource allocation strategy, wherein the state space contains feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated during the resource allocation strategy generation process; The task allocation module is used to send the subtasks to corresponding computing nodes for processing according to the resource allocation strategy.

[0008] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the task scheduling method provided in the above embodiment.

[0009] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the task scheduling method provided in the above embodiment are implemented.

[0010] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the task scheduling method provided in the above embodiment.

[0011] It can be seen from the technical solution provided by the above embodiments of the present invention that the embodiments of the present invention receive a task processing request for a target task sent by a terminal device, respond to the task processing request, determine a computing node for processing the task, and construct a first graph structure data according to the link bandwidth between the computing nodes, split the target task into multiple subtasks with dependencies, and construct a directed graph structure data according to the dependencies between the subtasks, determine the resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing nodes, and a state space containing information required to determine the resource allocation strategy, the state space contains feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated in the process of generating the resource allocation strategy, and according to the resource allocation strategy, the subtasks are sent to corresponding computing nodes for processing. In this way, since the first graph structure data is constructed based on the link bandwidth between computing nodes, and the directed graph structure data is constructed based on the dependency relationship between subtasks, therefore, according to the state space constructed by the characteristic information of the first graph structure data, the characteristic information of the directed graph structure data, and the characteristic information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, the resource allocation strategy for the target task can be accurately determined through the pre-trained strategy determination model, so as to allocate the subtasks to the corresponding computing nodes according to the resource allocation strategy, thereby improving resource utilization and ensuring the continuity of task scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0013] Figure 1 A flowchart of a task scheduling method of the present invention; Figure 2 A schematic diagram of a subtask with dependency relationship according to the present invention; Figure 3 A schematic diagram of a task processing process of the present invention; Figure 4 A flowchart of another task scheduling method of the present invention; Figure 5 A flowchart of another task scheduling method of the present invention; Figure 6 A schematic diagram of a resource allocation strategy of the present invention; Figure 7It is a structural schematic diagram of a task scheduling device of the present invention; Figure 8 The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0014] The embodiments of the present invention provide a task scheduling method, device and electronic device.

[0015] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0016] Embodiment 1 like Figure 1 As shown, an embodiment of the present invention provides a task scheduling method, the execution subject of the method may be a server, the server may be an independent server, or a server cluster composed of multiple servers. The method may specifically include the following steps: In S102, a task processing request for a target task sent by a terminal device is received.

[0017] Among them, the terminal device can be any device that can trigger the execution of the target task. For example, the terminal device can be a mobile terminal device such as a mobile phone, a tablet computer, or a terminal device such as a computer. The target task can be any task that can provide services to users. For example, the target task can be a video viewing task, a communication task, a resource transfer task, etc.

[0018] In implementation, with the rapid development of communication services and computer technology, edge computing nodes have been widely used due to their wide distribution and low latency, allowing users to access and use near-field computing resources more conveniently. Scheduling for edge computing nodes is to assign tasks to be processed to edge computing nodes for processing, and tasks can usually be assigned in a random manner. However, due to the differences in latency, load, etc. of different edge computing nodes, there will be an imbalance in node load through random assignment, resulting in poor node resource utilization, affecting global resource utilization and scheduling continuity. Therefore, a technical solution is needed to improve resource utilization and scheduling continuity during task scheduling. To this end, the embodiments of this specification provide a technical solution that can solve the above problems. For details, please refer to the following content.

[0019] Taking the target task as a video watching task as an example, the user can trigger the execution of a video watching task for a certain video in the terminal device. When the terminal device detects the trigger execution instruction, it can send a task processing request for the target task to the server.

[0020] Taking the server as a management node located at a base station as an example, the base station and the management node located at the base station can constitute an intelligent agent, and the intelligent agent can receive a task processing request for a target task sent by a terminal device.

[0021] In S104, in response to the task processing request, a computing node for processing the task is determined, and first graph structure data is constructed according to the link bandwidth between the computing nodes.

[0022] Among them, the computing nodes can be multiple edge computing nodes, and the intelligent agent can offload tasks to the edge computing nodes for processing to improve task processing efficiency.

[0023] In implementation, the intelligent agent can obtain an edge computing node that can carry services in a mobile edge computing (MEC) network in response to a task processing request, and determine the obtained edge computing node as a computing node for processing the task.

[0024] The intelligent agent can obtain the link bandwidth between every two computing nodes, and determine the edge weight in the first graph structure data according to the obtained link bandwidth.

[0025] In addition, the agent can also obtain characteristic information of the computing node to determine the attribute information of the node in the first graph structure data according to the characteristic information of the computing node. The characteristic information of the computing node may include the task volume of the computing node currently processing tasks, the maximum task volume that the computing node can carry, the computing capacity of the computing node, and other characteristic information.

[0026] In S106, the target task is split into a plurality of subtasks with dependency relationships, and directed graph structure data is constructed according to the dependency relationships between the subtasks.

[0027] Among them, the dependency relationship can be a data dependency relationship between subtasks. For example, if the data required by subtask 2 during processing is the data obtained after the processing of subtask 1, then there is a dependency relationship between subtask 1 and subtask 2, and the dependency relationship between subtask 1 and subtask 2 is a directed dependency relationship, that is, subtask 2 depends on subtask 1.

[0028] In implementation, the agent can split the target task into multiple subtasks with dependencies according to the task processing logic of the target task. For example, taking the target task as a video viewing task, the agent can split the video viewing task into subtask 1, subtask 2, and subtask 3, where subtask 1 can be an account login subtask, subtask 2 can be a video data acquisition subtask, and subtask 3 can be a video data sending subtask. The following dependencies exist between these three subtasks: subtask 2 can only be processed after subtask 1 is processed, and subtask 3 can only be processed after subtask 2 is processed.

[0029] Since the dependencies between subtasks are directed dependencies, the agent can construct directed graph structure data based on the dependencies between subtasks. For example, taking subtask 1, subtask 2, and subtask 3 as an example, according to the directed dependencies between these three subtasks, the following can be constructed: Figure 2 The directed graph structure data shown, wherein node 1 is the node corresponding to subtask 1, node 2 is the node corresponding to subtask 2, and node 3 is the node corresponding to subtask 3.

[0030] In addition, the above description is based on a one-to-one dependency relationship. In addition, the dependency relationship between subtasks can also include many-to-many, many-to-one, one-to-many and other different dependency relationships.

[0031] In S108, according to the pre-trained strategy determination model, a resource allocation strategy for the target task is determined based on a preset reward function, an action space constructed by computing nodes, and a state space containing information required to determine the resource allocation strategy.

[0032] Among them, the strategy determination model can be an algorithm for determining the resource allocation strategy based on the deep reinforcement learning (DRL) algorithm. The state space can include the feature information of the first graph structure data, the feature information of the directed graph structure data, and the feature information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process. It can also include the feature information of the subtasks and computing nodes corresponding to the intermediate resource allocation strategy. The preset reward function can be used to improve the node resource utilization and reduce the task completion time.

[0033] In implementation, the agent can determine the resource allocation strategy for the target task based on the four elements of the DRL algorithm and the pre-trained strategy determination model. The four elements of the DRL algorithm are a preset reward function, an action space constructed by computing nodes, a state space containing the information required to determine the resource allocation strategy, and a transition probability.

[0034] In addition, the intelligent agent can use the model constructed by the fully connected layer to extract features of the intermediate resource allocation strategies generated in the process of resource allocation strategy generation, as well as the subtasks and computing nodes corresponding to the intermediate resource allocation strategies, to obtain feature information of the intermediate resource allocation strategies, as well as feature information of the subtasks and computing nodes corresponding to the intermediate resource allocation strategies.

[0035] Among them, the intermediate resource allocation strategy, as well as the subtasks and computing nodes corresponding to the intermediate resource allocation strategy can be non-graph structure data ,in, is the number of the subtask that currently needs to be scheduled (i.e., the subtask corresponding to the intermediate resource allocation strategy). is the current scheduling result (i.e., the intermediate resource allocation strategy), is the currently available computing node (i.e., the computing node corresponding to the intermediate resource allocation strategy). The output result obtained after processing by the L-layer fully connected layer (i.e., the feature information of the intermediate resource allocation strategy, and the feature information of the subtasks and computing nodes corresponding to the intermediate resource allocation strategy) can be , where when L=0, .

[0036] In S110, according to the resource allocation strategy, the subtasks are respectively sent to corresponding computing nodes for processing.

[0037] In implementation, Figure 3 As shown, the intelligent agent constructed by the base station and the management node can receive task processing requests sent by multiple terminal devices (such as terminal device 1~terminal device n), and the intelligent agent can be used to manage m subordinate computing nodes.

[0038] After determining the resource allocation strategy, the agent can send the subtask to the corresponding computing node for processing according to the resource allocation strategy. In this way, through the autonomous decision-making of the agent and the collaborative work between multiple agents, task collaborative scheduling and resource collaborative scheduling on a large-scale network can be achieved, meeting the computing resource and bandwidth requirements for massive edge data processing, and also meeting the real-time and privacy requirements of the application.

[0039] An embodiment of the present invention provides a task scheduling method, which receives a task processing request for a target task sent by a terminal device, determines a computing node for processing the task in response to the task processing request, and constructs a first graph structure data according to the link bandwidth between the computing nodes, splits the target task into multiple subtasks with dependencies, and constructs directed graph structure data according to the dependencies between the subtasks, determines a resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing nodes, and a state space containing information required to determine the resource allocation strategy, the state space contains feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, and sends the subtasks to corresponding computing nodes for processing according to the resource allocation strategy. In this way, since the first graph structure data is constructed based on the link bandwidth between computing nodes, and the directed graph structure data is constructed based on the dependency relationship between subtasks, therefore, according to the state space constructed by the characteristic information of the first graph structure data, the characteristic information of the directed graph structure data, and the characteristic information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, the resource allocation strategy for the target task can be accurately determined through the pre-trained strategy determination model, so as to allocate the subtasks to the corresponding computing nodes according to the resource allocation strategy, thereby improving resource utilization and ensuring the continuity of task scheduling.

[0040] Embodiment 2 like Figure 4 and Figure 5 As shown, an embodiment of the present invention provides a task scheduling method, the execution subject of the method may be a terminal device, wherein the terminal device may be a device such as a personal computer, or a mobile terminal device such as a mobile phone or a tablet computer, and the terminal device may be a terminal device that allows a user to use a communication service based on a communication number. The method may specifically include the following steps: In S102, a task processing request for a target task sent by a terminal device is received.

[0041] In S104, in response to the task processing request, a computing node for processing the task is determined, and first graph structure data is constructed according to the link bandwidth between the computing nodes.

[0042] In S106, the target task is split into a plurality of subtasks with dependency relationships, and directed graph structure data is constructed according to the dependency relationships between the subtasks.

[0043] In S202, a first-order adjacency matrix, a second-order in-degree adjacency matrix, and a second-order out-degree adjacency matrix corresponding to the directed graph structure data are determined according to the edges between the nodes in the directed graph structure data.

[0044] The directed graph structure data may include nodes constructed based on subtasks and edges between nodes constructed based on dependencies between subtasks, and the feature information of the directed graph structure data may include a second embedding vector corresponding to the directed graph structure data.

[0045] In implementation, this can be done by To represent directed graph structure data, is a node in the directed graph structure data, is the edge between nodes in the directed graph structure data, where , is the i-th node in the directed graph structure data, .

[0046] The first-order adjacency matrix can be constructed based on the edges between nodes in the directed graph structure data. , second-order in-degree adjacency matrix And the second-order out-degree adjacency matrix , where the first-order adjacency matrix ,, , , is the edge between the i-th node and the j-th node in the directed graph structure data. If there is an edge between the i-th node and the j-th node, is 1, otherwise, is 0, Represents the element at (m,n) in the matrix.

[0047] In S204, linear transformation is performed on the first-order adjacency matrix, the second-order in-degree adjacency matrix, and the second-order out-degree adjacency matrix to obtain a processed first-order adjacency matrix, a processed second-order in-degree adjacency matrix, and a processed second-order out-degree adjacency matrix.

[0048] In implementation, the first-order adjacency matrix can be linearly transformed by the following formula to obtain the processed first-order adjacency matrix: , in, is the processed first-order adjacency matrix, is a linear transformation operation based on parameters {W, b}, , is the preset linear transformation coefficient, , for The degree matrix of , is the initial feature matrix, .

[0049] The second-order in-degree adjacency matrix can be linearly transformed by the following formula to obtain the processed second-order in-degree adjacency matrix: , in, is the processed second-order in-degree adjacency matrix, , , for The degree matrix of .

[0050] The second-order out-degree adjacency matrix can be linearly transformed by the following formula to obtain the processed second-order out-degree adjacency matrix: , in, is the processed second-order out-degree adjacency matrix, , , for The degree matrix of .

[0051] In S206, according to the pre-trained first extraction model, based on the processed first-order adjacency matrix, the processed second-order in-degree adjacency matrix, and the processed second-order out-degree adjacency matrix, a second embedding vector corresponding to the directed graph structure data is determined.

[0052] The first extraction model may be a model constructed based on a directed graph convolutional network (DGCN) algorithm.

[0053] In implementation, the intelligent agent can construct a first extraction model based on the DGCN algorithm, and train the first extraction model according to historical data to obtain a trained first extraction model for feature extraction processing of the adjacency matrix.

[0054] The second embedding vector output by the first extraction model , that is, the convolutional layer output of the first extraction model can be Flattening is performed to obtain a second embedding vector, which is the output of the first extraction model, where , and are learnable parameters.

[0055] In S208, a first adjacency matrix for characterizing the link bandwidth between the computing nodes and a second adjacency matrix for characterizing the delay between the computing nodes are determined according to the edges between the nodes in the first graph structure data.

[0056] Among them, the first graph structure data may include nodes constructed based on computing nodes, edges between nodes determined based on connection relationships between computing nodes, and weights of edges between nodes determined based on link bandwidth between computing nodes, and the feature information of the first graph structure data may include a first embedding vector corresponding to the first graph structure data.

[0057] In implementation, the first graph structure data may be a topological graph for characterizing an undirected potential collaborative relationship between computing nodes. The attribute characteristics of the nodes in the first graph structure data may be determined based on characteristics such as the CPU resources and storage resources of the computing nodes. The edge weights of the first graph structure data may be determined based on the link bandwidth and latency between the computing nodes.

[0058] In S210, according to the pre-trained second extraction model, based on the first adjacency matrix and the second adjacency matrix, a first embedding vector corresponding to the first graph structure data is determined.

[0059] Among them, the second extraction model can be a model built based on a graph convolutional network algorithm (Graph Convolutional Network, GCN).

[0060] In implementation, the agent can use link bandwidth and delay as adjacency matrix elements to construct the first adjacency matrix and the second adjacency matrix ,in, , is the link bandwidth between the i-th node and the j-th node in the first graph structure data, , is the delay between the i-th node and the j-th node in the first graph structure data. If there is no edge between the i-th node and the j-th node in the first graph structure data, then, , =0.

[0061] The intelligent agent can use the second lifting model to aggregate the neighborhood information of the first adjacency matrix and the second adjacency matrix respectively, and then splice the output results to obtain a first embedding vector.

[0062] Each convolutional layer of the second extraction model obtains the summary value of the neighbor features of each node by multiplying the adjacency matrix and the feature matrix. Then, this summary value is multiplied by the parameter matrix and nonlinear changes are made to the city to obtain the matrix of aggregated neighbor vertex features. , where the propagation rule of each convolutional layer in the second extraction model is: ,in, is the output of the lth convolutional layer For and After splicing, Including based on What you get , and based on What you get ,and , is the initial feature matrix of the first graph structure data, , Used to represent the CPU resources of the mth node, is the storage resource of the mth node. Normalize the adjacency matrix (i.e. ) is to maintain the original distribution of the feature matrix during information transmission, and to prevent some vertices with higher and lower degrees from having large differences in feature distribution. Finally, after L layers of convolution processing, and After concatenation, flattening is performed to obtain the first embedding vector .

[0063] In S212, historical tasks, historical computing nodes, target resource allocation strategies corresponding to the historical tasks, and historical state spaces corresponding to the historical tasks are obtained.

[0064] Among them, the historical state space may include an embedding vector corresponding to graph structure data constructed by the link bandwidth between historical computing nodes, an embedding vector corresponding to directed graph structure data constructed by the dependency relationship between multiple historical subtasks obtained by splitting historical tasks, and a fourth embedding vector, which is determined according to the historical intermediate resource allocation strategy generated in the process of generating the target resource allocation strategy, and the historical subtasks and historical computing nodes corresponding to the historical intermediate resource allocation strategy.

[0065] In S214, according to the first network structure in the strategy determination model, a first resource allocation strategy for the historical task is determined based on the historical state space, the preset reward function, and the action space constructed by the historical computing nodes.

[0066] Among them, the preset reward function may include an immediate reward and a delayed reward. The immediate reward can be determined based on the resource overhead of the historical computing nodes and the link bandwidth overhead between the historical computing nodes. The delayed reward can be determined based on the task execution time of the historical tasks and the resource utilization of the historical computing nodes.

[0067] In S216, according to the second network structure in the strategy determination model, based on the historical state space, the preset reward function and the action space constructed by the historical computing nodes, a second resource allocation strategy for the historical task is determined.

[0068] In S218, a loss value is determined according to the target resource allocation strategy, the first resource allocation strategy, and the second resource allocation strategy.

[0069] In S220, it is determined based on the loss value whether the strategy determination model converges. If it is determined that the strategy determination model has not converged, the network parameters of the first network structure are updated according to the first update algorithm, and the network parameters of the second network structure are updated according to the second update algorithm to obtain an updated strategy determination model.

[0070] The first updating algorithm is different from the second updating algorithm.

[0071] In S222, based on the historical state space, the preset reward function, and the action space constructed by the historical computing nodes, the updated policy determination model continues to be trained until the policy determination model converges to obtain a trained policy determination model.

[0072] In implementation, when the agent receives a task processing request for a historical task, it can split the historical task into multiple historical subtasks with dependencies. Then, the agent can add the historical subtasks to the scheduling queue through a topological sorting algorithm.

[0073] The intelligent agent can construct a historical state space according to the embedded vector corresponding to the graph structure data constructed by the link bandwidth between historical computing nodes, the embedded vector corresponding to the directed graph structure data constructed by the dependency relationship between multiple historical subtasks obtained by splitting the historical task, and the fourth embedding vector after obtaining the directed acyclic graph (DAG) information, network resources and link information of the historical tasks. The fourth embedding vector is determined according to the historical intermediate resource allocation strategy generated in the process of generating the target resource allocation strategy, and the historical subtasks and historical computing nodes corresponding to the historical intermediate resource allocation strategy.

[0074] In the process of training the strategy determination model, the agent can determine the state at time t based on the historical state space. , select the action at time t through the action space constructed by the historical calculation nodes The agent assigns the corresponding historical calculation node to the historical subtask according to the action value and calculates the reward at time t (including immediate rewards and / or delayed rewards), and get the state at the next moment (t+1) , and store [ , , ] to the memory pool.

[0075] When the current step is greater than the exploration step, the agent can randomly select a small batch of data from the memory pool to train the policy determination model. The purpose of training the policy determination model is to obtain the minimized loss value, which can be determined by the following formula: , in, is the loss value, The weight is The policy network (i.e., the weight value is the output of the first resource allocation strategy or the second resource allocation strategy, is the target Q value, which can be determined by the following formula : , in, is the learning rate, is the discount rate, Parameters for the target network (i.e., target resource allocation strategy).

[0076] The target network and the policy network have the same network structure, and the target network periodically copies the weights of the policy network to provide more stable training results.

[0077] In addition, the policy determination model can also adopt the Actor-Critic architecture. For example, the intelligent agent actor network can be written as , whose parameters are , you can use Indicates that the parameter is Actor network Similarly, you can use Indicates that the parameter is Critic Network , the input state and action of the actor network (i.e. the first network structure) and the critic network (i.e. the second network structure), and the output is the Q value.

[0078] According to the centralized training distributed execution architecture and SAC algorithm, we can get the minimum target that agent i needs to minimize when updating the actor network: .

[0079] The experience replay pool D is used to store the agent's current state x, behavior a, reward r, and state x' at the next moment. The four-tuple As a storage in D, where , , , .

[0080] Each agent can use a separate critic network , the input is the states of all agents sampled from the experience replay pool D and actions In order to increase the stability of agent training, the current agent's actor strategy can be used to generate actions Output Select actions for agent u Q value.

[0081] Actor network for decentralized execution Using only its own observation actions The probability distribution of the action is obtained, and the agent obtains the specific action to be performed by sampling the obtained probability distribution In order to ensure that it is still differentiable after adoption, we can set , for the action probability distribution Perform sampling operation. The expectation in can be approximated by adopting from the experience replay pool D, and the actor network of the agent u can be updated by the SGD algorithm (i.e., the first update algorithm), and the critic network parameters This can be done by minimizing the Bellman error of agent u (i.e. the second update algorithm) To update: , in, .

[0082] in, The target critic network for agent u The parameters are randomly selected in the experience revisit pool D to obtain a fixed batch of sample pairs The agent u obtains the state corresponding to the next moment by observing Enter the target actor network Then, the next action of the agent can be obtained by sampling the obtained action probability distribution .

[0083] In order to ensure stable update of the algorithm, the SAC agent can use two sets of networks. Because of the Actor-Critic architecture, the agent can have four deep neural networks, namely the evaluation actor network (i.e. the first network structure), the evaluation critic network (i.e. the second network structure), the target actor network and the target critic network. During the training process, only the evaluation network (i.e. the evaluation actor network and the evaluation critic network) can be trained. The target actor network and the target critic network can be used to store historical resource allocation measurements to stabilize the learning effect of the deep learning network through the target actor network and the target critic network. After the parameters of the evaluation actor network and the evaluation critic network have been updated several times, the parameters of the target actor network can be adjusted. and the parameters of the target critic network To update: , .

[0084] in, To control the hyperparameters of the target actor network and the target critic network update, To train the latest parameters of the actor network, To train the latest parameters of the critic network.

[0085] The algorithm process of the above model training process can be shown as follows:

[0086] In the reinforcement learning algorithm, the agent can obtain the state of the environment, and then the agent selects an action according to a predefined strategy. After performing the operation, the system will obtain a new state by interacting with the environment in the next time period. Then, the agent will repeat the above operation of selecting a new action based on the expected reward size. For subtasks with dependencies, the agent needs to handle the dependencies between subtasks. Since GNN can effectively process graph data, the information between vertices in the directed graph structure data can be extracted according to the model constructed by the directed graph convolutional network (DGCN) algorithm (i.e., the first extraction model), and the information of the computational node relationship graph can be extracted according to the model constructed by the GCN algorithm (i.e., the second extraction model).

[0087] Since deep learning has limitations in processing special structured data, and convolutional neural networks (CNNs) perform well in processing grid data (such as images), the relationship between nodes and edges in graph structured data can be effectively processed directly based on graph neural networks (GNNs). In addition, combining GNN's processing capabilities for graph structured data and reinforcement learning's advantages in decision-making, the policy determination model can perform effective policy learning and decision-making while understanding complex network structures. Therefore, the graph reinforcement learning multi-agent collaborative management mechanism can achieve task collaborative scheduling and resource collaborative scheduling on a large-scale network through the autonomous decision-making of agents and the collaboration between multiple agents.

[0088] When making scheduling decisions, the system state can be composed of graph structure information such as tasks and computing nodes, as well as non-graph structure information such as the number of subtasks currently to be scheduled and scheduling results. Graph neural networks can convert high-dimensional non-Euclidean information of graph structures into low-dimensional feature vectors, regard graph representation problems as machine learning tasks themselves, and use data-driven methods to learn embeddings that encode graph structures. Therefore, by embedding tasks (i.e., the second embedding vector) and computing nodes (i.e., the first embedding necklace) in the task graph (i.e., directed graph structure data) and the network graph (i.e., the first graph structure data), and using a fully connected network to extract non-graph structure information features (i.e., the third embedding vector), the redundancy of high-dimensional input information can be reduced and the information required for scheduling decisions can be extracted.

[0089] Combining deep reinforcement learning (DRL) and graph neural network (GNN), distributed collaboration of computing power on the edge can be achieved, and the allocation of computing power resources on the terminal can be completed reasonably, which improves the collaborative scheduling of tasks and resources on a large scale network. It meets the requirements of computing resources and bandwidth for massive edge data processing, and also meets the real-time and privacy requirements of applications.

[0090] In S224, the first network structure of the model is determined according to the pre-trained strategy, and the first prediction allocation strategy for the target task and the weight value corresponding to the first prediction allocation strategy are determined based on the state space, the preset reward function and the action space constructed by the computing nodes.

[0091] Among them, Figure 6 As shown, the first prediction allocation strategy may include multiple resource allocation links, and each resource allocation link may correspond to a weight value.

[0092] In S226, a second network structure of the model is determined according to a pre-trained strategy, and a second prediction allocation strategy for the target task and a weight value corresponding to the second prediction allocation strategy are determined based on the state space, a preset reward function, and an action space constructed by computing nodes.

[0093] In S228, a resource allocation strategy for the target task is determined according to the weight value corresponding to the first prediction allocation strategy and the weight value corresponding to the second prediction allocation strategy.

[0094] Among them, the reward function may include an immediate reward and a delayed reward. The immediate reward is determined according to the resource overhead of the computing node and the link bandwidth overhead between the computing nodes, and the delayed reward is determined according to the task execution time of the target task and the resource utilization of the computing node.

[0095] In implementation, the intelligent agent may determine the resource allocation strategy for the target task according to the weight values ​​corresponding to the resource allocation links included in the first prediction allocation strategy and the second prediction allocation strategy.

[0096] In S110, according to the resource allocation strategy, the subtasks are respectively sent to corresponding computing nodes for processing.

[0097] An embodiment of the present invention provides a task scheduling method, which receives a task processing request for a target task sent by a terminal device, determines a computing node for processing the task in response to the task processing request, and constructs a first graph structure data according to the link bandwidth between the computing nodes, splits the target task into multiple subtasks with dependencies, and constructs directed graph structure data according to the dependencies between the subtasks, determines a resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing nodes, and a state space containing information required to determine the resource allocation strategy, the state space contains feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, and sends the subtasks to corresponding computing nodes for processing according to the resource allocation strategy. In this way, since the first graph structure data is constructed based on the link bandwidth between computing nodes, and the directed graph structure data is constructed based on the dependency relationship between subtasks, therefore, according to the state space constructed by the characteristic information of the first graph structure data, the characteristic information of the directed graph structure data, and the characteristic information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, the resource allocation strategy for the target task can be accurately determined through the pre-trained strategy determination model, so as to allocate the subtasks to the corresponding computing nodes according to the resource allocation strategy, thereby improving resource utilization and ensuring the continuity of task scheduling.

[0098] Embodiment 3 The above is a task scheduling method provided by an embodiment of the present invention. Based on the same idea, an embodiment of the present invention also provides a task scheduling device, such as Figure 7 shown.

[0099] The task scheduling device includes: a request receiving module 701, a first building module 702, a second building module 703, a strategy generating module 704 and a task allocating module 705, wherein: The request receiving module 701 is used to receive a task processing request for a target task sent by a terminal device; A first construction module 702, configured to determine, in response to the task processing request, a computing node for processing the task, and construct first graph structure data according to the link bandwidth between the computing nodes; The second construction module 703 is used to split the target task into multiple subtasks with dependency relationships, and construct directed graph structure data according to the dependency relationships between the subtasks; A strategy generation module 704 is used to determine the resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing node, and a state space containing information required to determine the resource allocation strategy, wherein the state space contains feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated during the resource allocation strategy generation process; The task allocation module 705 is used to send the subtasks to corresponding computing nodes for processing according to the resource allocation strategy.

[0100] In an embodiment of the present invention, the directed graph structure data includes nodes constructed based on the subtasks, and edges between the nodes constructed based on the dependency relationships between the subtasks, and the feature information of the directed graph structure data includes a second embedding vector corresponding to the directed graph structure data. The device also includes: A first determination module is used to determine a first-order adjacency matrix, a second-order in-degree adjacency matrix, and a second-order out-degree adjacency matrix corresponding to the directed graph structure data according to the edges between the nodes in the directed graph structure data; A matrix transformation module, used for performing linear transformation processing on the first-order adjacency matrix, the second-order in-degree adjacency matrix and the second-order out-degree adjacency matrix respectively, to obtain a processed first-order adjacency matrix, a processed second-order in-degree adjacency matrix and a processed second-order out-degree adjacency matrix; A first extraction module is used to determine a second embedding vector corresponding to the directed graph structure data based on a pre-trained first extraction model, the processed first-order adjacency matrix, the processed second-order in-degree adjacency matrix, and the processed second-order out-degree adjacency matrix, wherein the first extraction model is a model constructed based on a directed graph convolutional network algorithm.

[0101] In an embodiment of the present invention, the first graph structure data includes nodes constructed based on the computing nodes, edges between the nodes determined based on the connection relationship between the computing nodes, and weights of the edges between the nodes determined based on the link bandwidth between the computing nodes, and the feature information of the first graph structure data includes a first embedding vector corresponding to the first graph structure data, The device also includes: A second determination module, configured to determine, based on the edges between the nodes in the first graph structure data, a first adjacency matrix for characterizing the link bandwidth between the computing nodes and a second adjacency matrix for characterizing the delay between the computing nodes; The second extraction module is used to determine the first embedding vector corresponding to the first graph structure data based on the first adjacency matrix and the second adjacency matrix according to a pre-trained second extraction model, and the second extraction model is a model constructed based on a graph convolutional network algorithm.

[0102] In an embodiment of the present invention, the reward function includes an immediate reward and a delayed reward. The immediate reward is determined based on the resource overhead of the computing node and the link bandwidth overhead between the computing nodes, and the delayed reward is determined based on the task execution time of the target task and the resource utilization of the computing node.

[0103] In an embodiment of the present invention, the device comprises: A data acquisition module, used to acquire historical tasks, historical computing nodes, target resource allocation strategies corresponding to the historical tasks, and historical state spaces corresponding to the historical tasks; A first prediction module, configured to determine a first network structure in the strategy-determined model, and to determine a first resource allocation strategy for the historical task based on the historical state space, the preset reward function, and the action space constructed by the historical computing nodes; A second prediction module is used to determine a second network structure in the strategy determination model, and to determine a second resource allocation strategy for the historical task based on the historical state space, the preset reward function, and the action space constructed by the historical computing nodes; A loss determination module, configured to determine a loss value according to the target resource allocation strategy, the first resource allocation strategy, and the second resource allocation strategy; a model updating module, configured to determine whether the policy determination model converges according to the loss value, and if it is determined that the policy determination model has not converged, update the network parameters of the first network structure according to a first updating algorithm, and update the network parameters of the second network structure according to a second updating algorithm, to obtain an updated policy determination model, wherein the first updating algorithm is different from the second updating algorithm; A model training module is used to continue training the updated policy determination model based on the historical state space, the preset reward function and the action space constructed by the historical calculation node until the policy determination model converges to obtain a trained policy determination model.

[0104] In the embodiment of the present invention, the policy generation module 704 is used to: Determine a first network structure of the model according to the pre-trained strategy, and determine a first prediction allocation strategy for the target task and a weight value corresponding to the first prediction allocation strategy based on the state space, the preset reward function, and the action space constructed by the computing node; Determine a second network structure of the model according to the pre-trained strategy, and determine a second prediction allocation strategy for the target task and a weight value corresponding to the second prediction allocation strategy based on the state space, the preset reward function, and the action space constructed by the computing node; The resource allocation strategy for the target task is determined according to the weight value corresponding to the first prediction allocation strategy and the weight value corresponding to the second prediction allocation strategy.

[0105] An embodiment of the present invention provides a task scheduling device, which receives a task processing request for a target task sent by a terminal device, determines a computing node for processing the task in response to the task processing request, and constructs a first graph structure data according to the link bandwidth between the computing nodes, splits the target task into multiple subtasks with dependencies, and constructs directed graph structure data according to the dependencies between the subtasks, determines a resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing nodes, and a state space containing information required to determine the resource allocation strategy, the state space containing feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, and sends the subtasks to corresponding computing nodes for processing according to the resource allocation strategy. In this way, since the first graph structure data is constructed based on the link bandwidth between computing nodes, and the directed graph structure data is constructed based on the dependency relationship between subtasks, therefore, according to the state space constructed by the characteristic information of the first graph structure data, the characteristic information of the directed graph structure data, and the characteristic information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, the resource allocation strategy for the target task can be accurately determined through the pre-trained strategy determination model, so as to allocate the subtasks to the corresponding computing nodes according to the resource allocation strategy, thereby improving resource utilization and ensuring the continuity of task scheduling.

[0106] Embodiment 4 Figure 8 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention is provided. The electronic device 800 includes but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, a processor 810, and a power supply 811. Those skilled in the art will appreciate that Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In the embodiments of the present invention, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted terminal, a wearable device, and a pedometer.

[0107] Among them, the processor 810 is used to receive a task processing request for a target task sent by a terminal device; in response to the task processing request, determine a computing node for processing the task, and construct a first graph structure data according to the link bandwidth between the computing nodes; split the target task into multiple subtasks with dependencies, and construct directed graph structure data according to the dependencies between the subtasks; determine the resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing node, and a state space containing information required to determine the resource allocation strategy, the state space contains feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated during the resource allocation strategy generation process; according to the resource allocation strategy, send the subtasks to corresponding computing nodes for processing.

[0108] In addition, the directed graph structure data includes nodes constructed based on the subtasks, and edges between nodes constructed based on the dependency relationships between the subtasks. The feature information of the directed graph structure data includes a second embedding vector corresponding to the directed graph structure data. The processor 810 is also used to determine the first-order adjacency matrix, the second-order in-degree adjacency matrix and the second-order out-degree adjacency matrix corresponding to the directed graph structure data according to the edges between the nodes in the directed graph structure data; perform linear transformation processing on the first-order adjacency matrix, the second-order in-degree adjacency matrix and the second-order out-degree adjacency matrix respectively to obtain a processed first-order adjacency matrix, a processed second-order in-degree adjacency matrix and a processed second-order out-degree adjacency matrix; determine the second embedding vector corresponding to the directed graph structure data based on the processed first-order adjacency matrix, the processed second-order in-degree adjacency matrix and the processed second-order out-degree adjacency matrix according to a pre-trained first extraction model, and the first extraction model is a model constructed based on a directed graph convolutional network algorithm.

[0109] In addition, the first graph structure data includes nodes constructed based on the computing nodes, edges between the nodes determined based on the connection relationship between the computing nodes, and weights of the edges between the nodes determined based on the link bandwidth between the computing nodes. The feature information of the first graph structure data includes a first embedding vector corresponding to the first graph structure data. The processor 810 is also used to determine a first adjacency matrix for characterizing the link bandwidth between the computing nodes and a second adjacency matrix for characterizing the time delay between the computing nodes based on the edges between the nodes in the first graph structure data; according to a pre-trained second extraction model, based on the first adjacency matrix and the second adjacency matrix, determine the first embedding vector corresponding to the first graph structure data, and the second extraction model is a model constructed based on a graph convolutional network algorithm.

[0110] In addition, the reward function includes an immediate reward and a delayed reward, the immediate reward is determined according to the resource overhead of the computing node and the link bandwidth overhead between the computing nodes, and the delayed reward is determined according to the task execution time of the target task and the resource utilization of the computing node.

[0111] In addition, the processor 810 is further used to obtain historical tasks, historical computing nodes, target resource allocation strategies corresponding to the historical tasks, and historical state spaces corresponding to the historical tasks; determine a first resource allocation strategy for the historical tasks according to a first network structure in the strategy-determined model based on the historical state space, the preset reward function, and the action space constructed by the historical computing nodes; determine a second resource allocation strategy for the historical tasks according to a second network structure in the strategy-determined model based on the historical state space, the preset reward function, and the action space constructed by the historical computing nodes; determine a second resource allocation strategy for the historical tasks according to the target resource allocation strategy, the first resource allocation strategy, and the action space constructed by the historical computing nodes. The strategy and the second resource allocation strategy are used to determine the loss value; according to the loss value, it is determined whether the strategy determination model converges, and when it is determined that the strategy determination model has not converged, the network parameters of the first network structure are updated according to the first update algorithm, and the network parameters of the second network structure are updated according to the second update algorithm to obtain an updated strategy determination model, wherein the first update algorithm is different from the second update algorithm; based on the historical state space, the preset reward function and the action space constructed by the historical calculation nodes, the updated strategy determination model is continued to be trained until the strategy determination model converges to obtain a trained strategy determination model.

[0112] In addition, the processor 810 is also used to determine the first network structure of the model according to the pre-trained strategy, determine the first prediction allocation strategy for the target task and the weight value corresponding to the first prediction allocation strategy based on the state space, the preset reward function and the action space constructed by the computing node; determine the second network structure of the model according to the pre-trained strategy, determine the second prediction allocation strategy for the target task and the weight value corresponding to the second prediction allocation strategy based on the state space, the preset reward function and the action space constructed by the computing node; determine the resource allocation strategy for the target task according to the weight value corresponding to the first prediction allocation strategy and the weight value corresponding to the second prediction allocation strategy.

[0113] An embodiment of the present invention provides an electronic device, which receives a task processing request for a target task sent by a terminal device, determines a computing node for processing the task in response to the task processing request, and constructs a first graph structure data according to the link bandwidth between the computing nodes, splits the target task into multiple subtasks with dependencies, and constructs directed graph structure data according to the dependencies between the subtasks, determines a resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing nodes, and a state space containing information required to determine the resource allocation strategy, the state space containing feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, and sends the subtasks to corresponding computing nodes for processing according to the resource allocation strategy. In this way, since the first graph structure data is constructed based on the link bandwidth between computing nodes, and the directed graph structure data is constructed based on the dependency relationship between subtasks, therefore, according to the state space constructed by the characteristic information of the first graph structure data, the characteristic information of the directed graph structure data, and the characteristic information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, the resource allocation strategy for the target task can be accurately determined through the pre-trained strategy determination model, so as to allocate the subtasks to the corresponding computing nodes according to the resource allocation strategy, thereby improving resource utilization and ensuring the continuity of task scheduling.

[0114] It should be understood that in the embodiment of the present invention, the radio frequency unit 801 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving downlink data from the base station, it is sent to the processor 810 for processing; in addition, uplink data is sent to the base station. Generally, the radio frequency unit 801 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 801 can also communicate with the network and other electronic devices through a wireless communication system.

[0115] The electronic device provides users with wireless broadband Internet access through the network module 802, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0116] The audio output unit 803 can convert the audio data received by the RF unit 801 or the network module 802 or stored in the memory 809 into an audio signal and output it as sound. Moreover, the audio output unit 803 can also provide audio output related to a specific function performed by the electronic device 800 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 803 includes a speaker, a buzzer, a receiver, etc.

[0117] The input unit 804 is used to receive audio or video signals. The input unit 804 may include a graphics processor (GPU) 8041 and a microphone 8042. The graphics processor 8041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 806. The image frame processed by the graphics processor 8041 can be stored in the memory 809 (or other storage medium) or sent via the radio frequency unit 801 or the network module 802. The microphone 8042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 801 in the case of a telephone call mode.

[0118] The electronic device 800 also includes at least one sensor 805, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 8061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 8061 and / or the backlight when the electronic device 800 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 805 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0119] The display unit 806 is used to display information input by the user or information provided to the user. The display unit 806 may include a display panel 8061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0120] The user input unit 807 can be used to receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 807 includes a touch panel 8071 and other input devices 8072. The touch panel 8071, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch panel 8071). The touch panel 8071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact point coordinates, and then sends it to the processor 810, receives the command sent by the processor 810 and executes it. In addition, the touch panel 8071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 8071, the user input unit 807 may also include other input devices 8072. Specifically, other input devices 8072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0121] Furthermore, the touch panel 8071 may be overlaid on the display panel 8061. When the touch panel 8071 detects a touch operation on or near it, it transmits the information to the processor 810 to determine the type of the touch event. Then, the processor 810 provides a corresponding visual output on the display panel 8061 according to the type of the touch event. Figure 8 In the figure, the touch panel 8071 and the display panel 8061 are used as two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 8071 and the display panel 8061 can be integrated to realize the input and output functions of the electronic device, which is not limited here.

[0122] The interface unit 808 is an interface for connecting an external device to the electronic device 800. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 808 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic device 800 or may be used to transmit data between the electronic device 800 and an external device.

[0123] The memory 809 can be used to store software programs and various data. The memory 809 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 809 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0124] The processor 810 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 809 and calling data stored in the memory 809, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 810 may include one or more processing units; preferably, the processor 810 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 810.

[0125] The electronic device 800 may also include a power source 811 (such as a battery) for supplying power to various components. Preferably, the power source 811 may be logically connected to the processor 810 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0126] Preferably, an embodiment of the present invention further provides an electronic device, comprising a processor 810, a memory 809, and a computer program stored in the memory 809 and executable on the processor 810. When the computer program is executed by the processor 810, each process of the above-mentioned task scheduling method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0127] Embodiment 5 An embodiment of the present invention also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the various processes of the above-mentioned task scheduling method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0128] Embodiment 6 The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned task scheduling method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0129] An embodiment of the present invention provides a computer-readable storage medium, which receives a task processing request for a target task sent by a terminal device, determines a computing node for processing the task in response to the task processing request, and constructs a first graph structure data according to the link bandwidth between the computing nodes, splits the target task into multiple subtasks with dependencies, and constructs directed graph structure data according to the dependencies between the subtasks, determines a resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing nodes, and a state space containing information required to determine the resource allocation strategy, the state space containing feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, and sends the subtasks to corresponding computing nodes for processing according to the resource allocation strategy. In this way, since the first graph structure data is constructed based on the link bandwidth between computing nodes, and the directed graph structure data is constructed based on the dependency relationship between subtasks, therefore, according to the state space constructed by the characteristic information of the first graph structure data, the characteristic information of the directed graph structure data, and the characteristic information of the intermediate resource allocation strategy generated in the resource allocation strategy generation process, the resource allocation strategy for the target task can be accurately determined through the pre-trained strategy determination model, so as to allocate the subtasks to the corresponding computing nodes according to the resource allocation strategy, thereby improving resource utilization and ensuring the continuity of task scheduling.

[0130] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0135] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0136] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0138] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A task scheduling method, characterized in that: The method comprises: Receiving a task processing request for a target task sent by a terminal device; In response to the task processing request, determining a computing node for processing the task, and constructing first graph structure data according to link bandwidths between the computing nodes; Splitting the target task into a plurality of subtasks with dependency relationships, and constructing directed graph structure data according to the dependency relationships between the subtasks; Determine the resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing node, and a state space containing information required to determine the resource allocation strategy, wherein the state space contains feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated during the resource allocation strategy generation process; According to the resource allocation strategy, the subtasks are respectively sent to corresponding computing nodes for processing.

2. The method according to claim 1, characterized in that The directed graph structure data includes nodes constructed based on the subtasks and edges between the nodes constructed based on dependency relationships between the subtasks, and the feature information of the directed graph structure data includes a second embedding vector corresponding to the directed graph structure data. Before determining the resource allocation strategy for the target task according to the pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing node, and a state space containing information required to determine the resource allocation strategy, the method further includes: Determine, according to the edges between the nodes in the directed graph structure data, a first-order adjacency matrix, a second-order in-degree adjacency matrix, and a second-order out-degree adjacency matrix corresponding to the directed graph structure data; Performing linear transformation processing on the first-order adjacency matrix, the second-order in-degree adjacency matrix, and the second-order out-degree adjacency matrix, respectively, to obtain a processed first-order adjacency matrix, a processed second-order in-degree adjacency matrix, and a processed second-order out-degree adjacency matrix; According to a pre-trained first extraction model, a second embedding vector corresponding to the directed graph structure data is determined based on the processed first-order adjacency matrix, the processed second-order in-degree adjacency matrix, and the processed second-order out-degree adjacency matrix, wherein the first extraction model is a model constructed based on a directed graph convolutional network algorithm.

3. The method according to claim 2, characterized in that The first graph structure data includes nodes constructed based on the computing nodes, edges between the nodes determined based on connection relationships between the computing nodes, and weights of the edges between the nodes determined based on link bandwidths between the computing nodes, and the feature information of the first graph structure data includes a first embedding vector corresponding to the first graph structure data, Before determining the resource allocation strategy for the target task according to the pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing node, and a state space containing information required to determine the resource allocation strategy, the method further includes: Determine, according to the edges between the nodes in the first graph structure data, a first adjacency matrix for characterizing the link bandwidth between the computing nodes and a second adjacency matrix for characterizing the delay between the computing nodes; According to a pre-trained second extraction model, based on the first adjacency matrix and the second adjacency matrix, a first embedding vector corresponding to the first graph structure data is determined, and the second extraction model is a model constructed based on a graph convolutional network algorithm.

4. The method according to claim 3, characterized in that The reward function includes an immediate reward and a delayed reward, wherein the immediate reward is determined according to the resource overhead of the computing node and the link bandwidth overhead between the computing nodes, and the delayed reward is determined according to the task execution time of the target task and the resource utilization rate of the computing node.

5. The method according to claim 3, characterized in that: Before determining the resource allocation strategy for the target task according to the pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing node, and a state space containing information required to determine the resource allocation strategy, the method further includes: Acquire historical tasks, historical computing nodes, target resource allocation strategies corresponding to the historical tasks, and historical state spaces corresponding to the historical tasks; Determine a first resource allocation strategy for the historical task based on the historical state space, the preset reward function, and the action space constructed by the historical computing nodes according to the first network structure in the strategy determination model; Determine, according to the second network structure in the strategy determination model, a second resource allocation strategy for the historical task based on the historical state space, the preset reward function, and the action space constructed by the historical computing nodes; Determining a loss value according to the target resource allocation strategy, the first resource allocation strategy, and the second resource allocation strategy; Determining whether the policy determination model converges according to the loss value, and when it is determined that the policy determination model does not converge, updating the network parameters of the first network structure according to a first updating algorithm, and updating the network parameters of the second network structure according to a second updating algorithm to obtain an updated policy determination model, wherein the first updating algorithm is different from the second updating algorithm; Based on the historical state space, the preset reward function and the action space constructed by the historical calculation node, the updated policy determination model continues to be trained until the policy determination model converges to obtain a trained policy determination model.

6. The method according to claim 5, characterized in that The determining model according to the pre-trained strategy determines the resource allocation strategy for the target task based on a preset reward function, an action space constructed by the computing node, and a state space containing information required to determine the resource allocation strategy, including: Determine a first network structure of the model according to the pre-trained strategy, and determine a first prediction allocation strategy for the target task and a weight value corresponding to the first prediction allocation strategy based on the state space, the preset reward function, and the action space constructed by the computing node; Determine a second network structure of the model according to the pre-trained strategy, and determine a second prediction allocation strategy for the target task and a weight value corresponding to the second prediction allocation strategy based on the state space, the preset reward function, and the action space constructed by the computing node; The resource allocation strategy for the target task is determined according to the weight value corresponding to the first prediction allocation strategy and the weight value corresponding to the second prediction allocation strategy.

7. A task scheduling device, characterized in that: The device comprises: A request receiving module, used for receiving a task processing request for a target task sent by a terminal device; A first construction module, configured to determine, in response to the task processing request, a computing node for processing the task, and construct first graph structure data according to a link bandwidth between the computing nodes; The second construction module is used to split the target task into a plurality of subtasks with dependency relationships, and construct directed graph structure data according to the dependency relationships between the subtasks; a strategy generation module, configured to determine the resource allocation strategy for the target task according to a pre-trained strategy determination model, based on a preset reward function, an action space constructed by the computing node, and a state space containing information required to determine the resource allocation strategy, wherein the state space contains feature information of the first graph structure data, feature information of the directed graph structure data, and feature information of the intermediate resource allocation strategy generated during the resource allocation strategy generation process; The task allocation module is used to send the subtasks to corresponding computing nodes for processing according to the resource allocation strategy.

8. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the task scheduling method according to any one of claims 1 to 6 when executed by the processor.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the task scheduling method according to any one of claims 1 to 6 when executed by a processor.

Images