Task allocation method and device, equipment, storage medium and program product

Abstract

The invention relates to a task allocation method and device, equipment, a storage medium and a program product. The method is used in task allocation nodes of an edge computing cluster, the edge computing cluster comprises a plurality of nodes, and the method comprises the steps of obtaining state information of each node and task information of a target to-be-allocated task; inputting the state information and the task information into a strategy network, and determining a target node for processing the target to-be-allocated task, so that the target node processes the target to-be-allocated task; wherein network parameters of the strategy network are updated online according to a reinforcement learning algorithm, and a reward function corresponding to the reinforcement learning algorithm is determined according to service quality of candidate nodes, energy consumption information of the candidate nodes, priorities of tasks to be distributed and strategy oscillation punishment. The strategy oscillation penalty is used for representing the distance between the current moment output and the last moment output of the strategy network. By adopting the method, the reliability of task allocation can be improved.

Description

Technical Field

[0001] This application relates to the field of task allocation technology, and in particular to a task allocation method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the development of cloud computing and edge computing, task allocation and resource management have become key issues in ensuring the efficiency and response quality of computing systems. Especially in edge computing clusters composed of low-resource devices, how to achieve efficient, low-energy-consumption task allocation with guaranteed service quality has become an important direction in edge intelligence research.

[0003] In existing technologies, target nodes can be determined from various nodes in a cluster based on load balancing algorithms. However, the reliability of task allocation needs to be improved when using existing methods. Summary of the Invention

[0004] Therefore, it is necessary to provide a task allocation method, apparatus, device, storage medium, and program product that can improve the reliability of task allocation in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a task allocation method for task allocation nodes in an edge computing cluster, wherein the edge computing cluster includes multiple nodes, and the method includes:

[0006] Obtain the status information of each node and the task information of the target tasks to be assigned;

[0007] Input the status information and task information into the policy network to determine the target node for processing the target task to be assigned, so that the target node can process the target task to be assigned.

[0008] The network parameters of the policy network are updated online according to the reinforcement learning algorithm. The reward function corresponding to the reinforcement learning algorithm is determined based on the service quality of the candidate nodes, the energy consumption information of the candidate nodes, the priority of the tasks to be assigned, and the policy oscillation penalty. The policy oscillation penalty is used to characterize the distance between the current output of the policy network and the output of the previous time step.

[0009] In one embodiment, the process of the reinforcement learning algorithm updating the network parameters of the policy network online includes:

[0010] The interaction sampling steps are executed iteratively multiple times to determine multiple sets of interaction data. Each set of interaction data includes the current state information, allocation strategy, reward, and the next state information of each node.

[0011] The network parameters of the policy network are updated based on the interaction data and the loss function;

[0012] The t-th interactive sampling step includes:

[0013] The current state information of each node and the task information of the task to be assigned are input into the policy network to determine the assignment policy; where, when t equals 1, the current state information is the preset state information, and when t is greater than 1, the current state information is the state information of the next moment determined in the (t-1)th interaction sampling step.

[0014] Update the current state information of each node and determine the state information of each node at the next moment;

[0015] The reward is determined based on the reward function.

[0016] In one embodiment, determining the reward based on the reward function includes:

[0017] The service quality and energy consumption information of the candidate nodes are determined based on the state information of the candidate nodes at the next moment corresponding to the allocation strategy.

[0018] The priority and corresponding priority weight of the tasks to be assigned are determined based on the task information of the tasks to be assigned.

[0019] The strategy oscillation penalty is determined based on the allocation strategy and the historical allocation strategy from the previous moment.

[0020] The reward is determined based on the difference between the service quality, the first parameter, and the second parameter. The first parameter is the product of energy consumption information and priority weight, and the second parameter is the product of strategy oscillation penalty and preset balance factor.

[0021] In one embodiment, the status information includes CPU utilization, bandwidth utilization, allocated energy consumption, number of completed tasks, and total number of tasks. The service quality and energy consumption information of the candidate nodes are determined based on the status information of the candidate nodes at the next moment corresponding to the allocation strategy, including:

[0022] The service quality is determined based on the ratio of the number of tasks completed by candidate nodes to the total number of tasks.

[0023] Energy consumption information is determined based on the candidate node's CPU energy consumption, bandwidth energy consumption, target allocation energy consumption, and node idle energy consumption.

[0024] Among them, CPU energy consumption is determined based on CPU utilization and the first energy consumption increment coefficient; bandwidth energy consumption is determined based on bandwidth utilization and the second energy consumption increment coefficient; target allocation energy consumption is determined based on the node type of candidate nodes, the number of nodes in the edge computing cluster, and the preset allocation energy consumption table. The preset allocation energy consumption table is used to include multiple sets of correspondences between node type, the number of nodes in the edge computing cluster, and allocation energy consumption.

[0025] In one embodiment, the task allocation method further includes:

[0026] Get the task set, which includes multiple tasks to be assigned;

[0027] Determine the feature vector of each task to be assigned based on the task information of each task to be assigned.

[0028] Cluster analysis is performed on each feature vector to determine the priority of each task to be assigned and its corresponding priority weight.

[0029] In one embodiment, the reinforcement learning algorithm is a proximal policy optimization algorithm or an independent proximal policy optimization algorithm.

[0030] Secondly, this application also provides a task allocation device, disposed in a task allocation node of an edge computing cluster, the edge computing cluster including multiple nodes, the device comprising:

[0031] The acquisition module is used to acquire the status information of each node and the task information of the target task to be assigned;

[0032] The determination module is used to input various status information and task information into the policy network to determine the target node for processing the target task to be assigned, so that the target node can process the target task to be assigned.

[0033] The network parameters of the policy network are updated online according to the reinforcement learning algorithm. The reward function corresponding to the reinforcement learning algorithm is determined based on the service quality of the candidate nodes, the energy consumption information of the candidate nodes, the priority of the tasks to be assigned, and the policy oscillation penalty. The policy oscillation penalty is used to characterize the distance between the current output of the policy network and the output of the previous time step.

[0034] In one embodiment, the task allocation device further includes a training module, which iteratively executes multiple interaction sampling steps to determine multiple sets of interaction data. Each set of interaction data includes the current state information of each node, the allocation strategy, the reward, and the next state information of each node. The network parameters of the policy network are updated based on the interaction data and the loss function. The t-th interaction sampling step includes: inputting the current state information of each node and the task information of the task to be allocated into the policy network to determine the allocation strategy; wherein, when t equals 1, the current state information is the preset state information, and when t is greater than 1, the current state information is the next state information determined in the (t-1)-th interaction sampling step; updating the current state information of each node and determining the next state information of each node; and determining the reward according to the reward function.

[0035] In one embodiment, the training module is specifically used to determine the service quality and energy consumption information of the candidate nodes based on the state information of the candidate nodes at the next moment corresponding to the allocation strategy; determine the priority and corresponding priority weight of the task to be allocated based on the task information of the task to be allocated; determine the strategy oscillation penalty based on the allocation strategy and the historical allocation strategy at the previous moment; and determine the reward based on the difference between the service quality, the first parameter, and the second parameter, where the first parameter is the product of the energy consumption information and the priority weight, and the second parameter is the product of the strategy oscillation penalty and the preset balance factor.

[0036] In one embodiment, the status information includes CPU utilization, bandwidth utilization, allocated energy consumption, number of completed tasks, and total number of tasks. The training module is specifically used to determine the quality of service based on the ratio of the number of completed tasks of candidate nodes to the total number of tasks; and to determine energy consumption information based on the CPU energy consumption, bandwidth energy consumption, target allocated energy consumption, and node idle energy consumption of candidate nodes. Specifically, CPU energy consumption is determined based on CPU utilization and a first energy consumption increment coefficient; bandwidth energy consumption is determined based on bandwidth utilization and a second energy consumption increment coefficient; and target allocated energy consumption is determined based on the node type of candidate nodes, the number of nodes in the edge computing cluster, and a preset allocated energy consumption table. The preset allocated energy consumption table includes multiple sets of correspondences between node type, the number of nodes in the edge computing cluster, and allocated energy consumption.

[0037] In one embodiment, the training module is further configured to acquire a task set, which includes multiple tasks to be assigned; determine the feature vector of each task to be assigned based on the task information of each task to be assigned; and perform cluster analysis on each feature vector to determine the priority of each task to be assigned and the corresponding priority weight.

[0038] In one embodiment, the reinforcement learning algorithm is a proximal policy optimization algorithm or an independent proximal policy optimization algorithm.

[0039] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the methods described in the first aspect above.

[0040] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the methods described in the first aspect above.

[0041] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in any of the first aspects above.

[0042] The aforementioned task allocation method, apparatus, equipment, storage medium, and program product acquire the status information of each node and the task information of the target task to be allocated; input the status information and task information into the policy network to determine the target node for processing the target task to be allocated, so that the target node can process the target task to be allocated; wherein, the network parameters of the policy network are updated online according to the reinforcement learning algorithm, and the reward function corresponding to the reinforcement learning algorithm is determined according to the service quality of the candidate node, the energy consumption information of the candidate node, the priority of the task to be allocated, and the policy oscillation penalty, and the policy oscillation penalty is used to characterize the distance between the current output of the policy network and the previous output. In the above method, a multi-objective joint reward function is used, with energy consumption allocation as the main objective, policy output oscillation as the constraint, and Quality of Service (QoS) as the guarantee objective. Compared with the limitations of traditional task allocation which only focuses on completion time or load balancing, the policy network based on the above multi-objective joint reward function can ensure system responsiveness while minimizing energy consumption and achieving differentiated allocation of service levels during the task allocation process. In other words, it can obtain an allocation strategy that takes into account energy saving, response stability, and service differentiation. For example, by introducing a policy oscillation penalty term, the large fluctuations of the policy in continuous time are limited, enhancing the stability of scheduling behavior; considering the priority of tasks to be allocated, the response performance of high-level tasks can be prioritized under conditions of resource shortage or high node load, effectively improving the reliability of task allocation. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating a task allocation method in one embodiment;

[0045] Figure 2 This is a schematic diagram of the system architecture of a task allocation method in one embodiment;

[0046] Figure 3 This is a flowchart illustrating the steps for updating network parameters of a policy network in one embodiment.

[0047] Figure 4 This is a flowchart illustrating the steps for determining the reward based on the reward function in one embodiment;

[0048] Figure 5 This is a flowchart illustrating the steps for determining the service quality and energy consumption information of candidate nodes in one embodiment.

[0049] Figure 6 This is a flowchart illustrating the task allocation method in another embodiment;

[0050] Figure 7 This is a structural block diagram of a task allocation method apparatus in one embodiment;

[0051] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] Edge computing is a computing architecture that moves computing resources from centralized cloud servers to network edge devices (such as gateways, routers, or session boundary controllers). It aims to shorten data transmission paths, reduce latency, alleviate cloud load, and improve real-time processing capabilities and service quality. With the development of cloud computing and edge computing, task allocation and resource management have become key issues in ensuring the efficiency and response quality of computing systems. Especially in edge computing clusters composed of low-resource devices, how to achieve efficient, low-energy-consumption task allocation with guaranteed service quality has become an important direction in edge intelligence research.

[0054] In existing technologies, target nodes can be determined from various nodes in a cluster based on load balancing algorithms. However, the reliability of task allocation needs to be improved when using existing methods.

[0055] In view of this, this application provides a task allocation method that can improve the reliability of task allocation. The task allocation method provided in this application can be executed by a task allocation device, which can be implemented by software, hardware, or a combination of both. It can be embedded in or independent of the processor in a computer device, or stored in the memory of the computer device. In the following method embodiments, the execution entity is a node in an edge computing cluster (which can be called a task allocation node) as an example. The node in the edge computing cluster is a computer device, which can be a server or a terminal device. This application does not limit the specific type of computer device.

[0056] In one exemplary embodiment, such as Figure 1As shown, a task allocation method is provided for task allocation nodes in an edge computing cluster, wherein the edge computing cluster includes multiple nodes, and includes the following steps 101 to 102. Wherein:

[0057] Step 101: Obtain the status information of each node and the task information of the target task to be assigned.

[0058] For nodes in an edge computing cluster, the node status information can include the node's resource status and historical load. The resource status can include CPU utilization, bandwidth utilization, network status, number of processes / threads, etc. The historical load can include the total number of tasks and the number of tasks completed by the node in the past preset time period. The total number of tasks is the number of times the node has been selected as a target node to process assigned tasks, and the number of tasks completed is the number of times the node has successfully completed the assigned tasks.

[0059] Optionally, in an edge computing cluster, each node communicates with each other to send its own status information to other nodes in the cluster, thereby enabling the task allocation node to obtain the status information of each node in the edge computing cluster; or, each node can communicate with the cloud node in the edge computing cluster to send its own status information to the cloud node, so that the cloud node can send the status information of each node in the edge computing cluster to the task allocation node.

[0060] Optionally, there may be one or more task allocation nodes, and this application embodiment does not limit this.

[0061] For example, the task allocation node can be any one or any multiple nodes in the edge computing cluster, or all nodes in the edge computing cluster can act as task allocation nodes to allocate tasks.

[0062] The task information of the target task to be assigned can be the task's attribute information. For example, the task information may include the priority of the target task to be assigned, the maximum allowable response latency, the estimated actual runtime, the application type label, the task request frequency, and at least one of the user level or QoS requirements.

[0063] Step 102: Input the state information and task information into the policy network to determine the target node for processing the target task to be assigned, so that the target node can process the target task to be assigned; wherein, the network parameters of the policy network are updated online according to the reinforcement learning algorithm, and the reward function corresponding to the reinforcement learning algorithm is determined according to the service quality of the candidate node, the energy consumption information of the candidate node, the priority of the task to be assigned, and the policy oscillation penalty. The policy oscillation penalty is used to characterize the distance between the current output of the policy network and the output of the previous time step.

[0064] Optionally, after obtaining the status information of each node in the edge computing cluster, the task allocation node can input the target task to be allocated and the status information of each node into the policy network, determine the target node according to the allocation policy output by the policy network, and allocate the target task to the target node so that the target node can perform business processing on the target task to be allocated.

[0065] Policy networks can be obtained by updating their parameters online using reinforcement learning algorithms, based on any neural network model framework. Reinforcement learning is a machine learning method that allows an agent to learn optimal behavioral policies by interacting with its environment. It guides better actions by perceiving the state of the environment and responding to actions, thereby obtaining greater rewards. The rewards can serve as feedback signals to characterize the performance of the scheduling policy output by the policy network.

[0066] During the update of policy network parameters, the reward function corresponding to the reinforcement learning algorithm is determined based on the service quality of candidate nodes, the energy consumption information of candidate nodes, the priority of tasks to be assigned, and the policy oscillation penalty. Based on the above reward function, when determining target nodes according to the policy network obtained after updating parameters, energy saving, stability, and responsiveness can be taken into account, and a reliable scheduling policy can be obtained for task allocation.

[0067] In this context, candidate nodes are the nodes corresponding to the allocation strategies output by the policy network based on the old network parameters during the network parameter update training process.

[0068] For example, the reward function can be represented by the following expression:

[0069]

[0070] in, This represents the service quality of candidate node i. This represents the priority weight corresponding to the priority of the task r to be assigned. This represents the energy consumption information generated when scheduling the task r to be assigned to candidate node i, where P represents the policy oscillation penalty. The preset balance factor can be adjusted. The energy consumption information is weighted by task priority; higher-priority tasks should be penalized for higher energy consumption, thus guiding the policy network to schedule critical tasks to nodes with higher energy efficiency.

[0071] Optionally, reinforcement learning algorithms include value-based algorithms, policy-based algorithms, and algorithms that combine value and policy. For more information on reinforcement learning algorithms, please refer to the relevant technical documentation, which will not be elaborated upon here.

[0072] In this embodiment, the reinforcement learning algorithm is described using either Proximal Policy Optimization (PPO) or Independent Proximal Policy Optimization (IPPO) as an example. IPPO, as a multi-agent reinforcement learning algorithm, equips each agent with an independent policy network and evaluation network, and trains independently using the proximal policy optimization algorithm. That is, each of the N agents in the system has its own policy and value function, and optimizes independently during training. This is a completely decentralized method and exhibits good performance in partially observable, competitive, or communication-constrained scenarios.

[0073] In this embodiment, each task to be assigned, upon arriving at the edge computing cluster, is controlled by its corresponding independent intelligent agent. The agent runs on a task allocation node and performs task allocation based on a policy network deployed within that node. For example, as... Figure 2 The diagram shows a system architecture for updating network parameters and allocating tasks based on the IPPO algorithm.

[0074] The above task allocation method obtains the state information of each node and the task information of the target task to be allocated; it inputs the state information and task information into the policy network to determine the target node to process the target task, so that the target node can process the target task; wherein, the network parameters of the policy network are updated online according to the reinforcement learning algorithm, and the reward function corresponding to the reinforcement learning algorithm is determined based on the service quality of candidate nodes, the energy consumption information of candidate nodes, the priority of the task to be allocated, and the policy oscillation penalty, which is used to characterize the distance between the current output of the policy network and the output of the previous time step. In the above method, by using a multi-objective joint reward function with energy consumption allocation as the main objective, policy output oscillation as the constraint, and service quality (QoS) as the guarantee objective, compared with the limitations of traditional task allocation that only focuses on completion time or load balancing, the policy network based on the above multi-objective joint reward function can ensure system responsiveness while minimizing energy consumption and achieving service level differentiation allocation during the task allocation process. That is, it can obtain an allocation strategy that takes into account energy saving, response stability, and service differentiation, effectively improving the reliability of task allocation.

[0075] The following is an exemplary description of the process of updating the network parameters of the policy network based on the independent near-end policy optimization algorithm. In an exemplary embodiment, such as Figure 3 As shown, optionally, the process of updating the network parameters of the policy network online using a reinforcement learning algorithm includes the following steps 301 to 302. Wherein:

[0076] Step 301: Iteratively execute multiple interaction sampling steps to determine multiple sets of interaction data. Each set of interaction data includes the current state information, allocation strategy, reward, and the next state information of each node.

[0077] The t-th interaction sampling step includes: inputting the current state information of each node and the task information of the task to be assigned into the policy network to determine the allocation policy; wherein, when t equals 1, the current state information is the preset state information, and when t is greater than 1, the current state information is the state information of the next moment determined in the (t-1)-th interaction sampling step; updating the current state information of each node and determining the state information of each node at the next moment; and determining the reward according to the reward function.

[0078] Optionally, multiple interactive sampling steps are performed iteratively to collect data and obtain an experience replay pool. The experience replay pool includes a set of interactive data obtained from each interactive sampling step. For the t-th interactive sampling step, or at time t, the interactive data can be represented as... ,in, This refers to the current state information. The allocation strategy for the output of the policy network can be based on... Determine candidate nodes, This indicates the reward corresponding to the allocation strategy. This indicates the state information of each node at the next moment after the allocation strategy is executed.

[0079] Optionally, the status information includes CPU utilization, bandwidth utilization, allocated energy consumption, number of completed tasks, and total number of tasks, as well as the status information of each node at the current moment. It can be represented as:

[0080]

[0081] in, This represents the CPU utilization of node i at time t. This represents the bandwidth utilization of node i at time t. This represents the target energy consumption of node i at time t. The target energy consumption can represent the actual energy consumption of the policy network model on the node. This includes the number of tasks completed by node i at time t and the total number of tasks.

[0082] The iteration can be stopped after the preset number of iterations is met, and an experience backtracking pool can be obtained based on the interaction data obtained in each interaction sampling step.

[0083] Step 302: Update the network parameters of the policy network based on the interaction data and loss function.

[0084] Optionally, for each set of interactive data, the current state information is recorded. and the state information at the next moment Input into the evaluation network, and determine the state based on the output of the evaluation network. Value estimation and state Value estimation ,in, Indicates the state The expected long-term cumulative reward that can be obtained by following the current strategy is as follows: Indicates the state The expected long-term cumulative reward can be obtained by following the current strategy. Then, based on the reward... and estimated value and Determine the advantage function corresponding to this interaction data. and target value function .

[0085] Among them, the dominant function It is possible to weigh immediate rewards The changes in future value can be determined using the following recursive formula:

[0086]

[0087] in, As a discount factor, As the attenuation factor, The advantage function for the next time step is obtained by working backward from the last set of interaction data to obtain the advantage function for each set of interaction data.

[0088] Target value function The objective of evaluating network updates can be determined using the following formula:

[0089]

[0090] For a set of interaction functions, determine their current state information. Allocation strategy Advantage function and target value function As a set of training samples, a small batch of training samples is determined from the experience revisit pool in the manner described above, and then the network parameters of the evaluation network and the policy network are updated based on the small batch of training samples.

[0091] For example, for a training sample, the action probability of the current policy can be determined based on the policy network. Then, based on the action probability of the current strategy... The action probabilities of the old strategy The ratio of ) determines the strategy ratio, and the strategy ratio This can be expressed by the following formula:

[0092]

[0093] Then, the loss is determined based on the policy ratio, the advantage function, and the loss function corresponding to the PPO algorithm. The loss is then backpropagated to update the parameters of the policy network.

[0094] The PPO algorithm achieves stable and efficient training results by limiting the policy update magnitude. The PPO algorithm uses proximal ratio pruning loss to limit the policy update magnitude. The proximal ratio pruning loss is defined as follows:

[0095]

[0096] Meanwhile, for a training sample, the value loss is calculated according to the value loss function, and the parameters of the evaluation network are updated by backpropagation.

[0097] In one exemplary embodiment, such as Figure 4 As shown, optionally, the above-described method of determining the reward based on the reward function includes steps 401 to 404. Wherein:

[0098] Step 401: Determine the service quality and energy consumption information of the candidate nodes based on the next state information of the candidate nodes corresponding to the allocation strategy.

[0099] Optionally, during training, the policy network determines the reward from a set of interaction data. At that time, the allocation can be determined based on the service quality of the candidate node, the energy consumption information of the candidate node, the priority of the task to be allocated, and the policy oscillation penalty.

[0100] The service quality of a candidate node can be determined based on the load in its state information at the next moment. For example, the service quality can be determined based on the ratio of the number of completed tasks to the total number of tasks, or the service quality can be determined by weighting the ratio of the number of completed tasks to the total number of tasks.

[0101] The energy consumption information of a candidate node can be determined based on the sum of CPU energy consumption, bandwidth energy consumption, target allocation energy consumption, and node idle energy consumption. CPU energy consumption is determined based on the CPU utilization rate in its status information, and bandwidth energy consumption is determined based on the bandwidth utilization rate in its status information.

[0102] Step 402: Determine the priority and corresponding priority weight of the task to be assigned based on the task information of the task to be assigned.

[0103] Optionally, the priority of the task to be assigned can be determined first based on the task information, and then a preset weight table can be queried based on the priority to determine the priority weight corresponding to that priority. For example, if the priorities of the tasks to be assigned include high priority, medium priority, and low priority, the weight table includes the correspondence between each priority and its priority weight.

[0104] Among them, priority and priority weight are positively correlated. The higher the priority, the greater its priority weight, so as to reflect the importance of the task to be assigned in the reward function.

[0105] In one possible implementation, the task information of the task to be assigned includes the priority of the task to be assigned. The priority can be obtained from the task information of the task to be assigned, and then the corresponding priority weight can be determined.

[0106] In another possible implementation, the task information of the task to be assigned can be input into a pre-trained classification model, the priority of the task to be assigned can be determined based on the output of the classification model, and then the corresponding priority weight can be determined.

[0107] In another possible implementation, a task set can be obtained, which includes multiple tasks to be assigned; the feature vector of each task to be assigned can be determined based on the task information of each task to be assigned; cluster analysis can be performed on each feature vector to determine the priority of each task to be assigned and the corresponding priority weight.

[0108] For each task to be assigned, features can be extracted from its task information to determine its corresponding feature vector. Then, each feature is normalized and encoded (e.g., using One-Hot encoding) to obtain a unified-dimensional task feature representation, forming a task vector set. Finally, the task vector set is clustered using a clustering algorithm to determine the priority of each task to be assigned.

[0109] It can be based on algorithms such as K-means (KMeans), density-based spatial clustering of applications with noise (DBSCAN), and spectral clustering.

[0110] For example, when clustering a task vector set using the KMeans algorithm, the number of cluster centers K=3 can be set to automatically divide tasks into three categories: high priority, medium priority, and low priority. Each task is automatically assigned to its corresponding level based on the distance between cluster centers and the average feature value, without the need for explicit labels. The KMeans clustering model can be incrementally learned and updated periodically in the task log to adapt to changes in business needs or task distribution, thereby enhancing the generalization ability and long-term stability of the clustering model.

[0111] Among these methods, determining task priorities through clustering can avoid the subjectivity and maintenance costs associated with manually setting task service levels (such as SLA levels).

[0112] Step 403: Determine the strategy oscillation penalty based on the allocation strategy and the historical allocation strategy of the previous moment.

[0113] Optionally, the historical allocation strategy of the previous moment can be obtained, that is, the allocation strategy determined in the (t-1)th interactive sampling step.

[0114] The strategy oscillation penalty can be determined using the following formula. :

[0115]

[0116] in, This represents the allocation strategy, specifically the allocation strategy determined in the t-th interactive sampling step. This represents the historical allocation strategy, i.e., the allocation strategy determined in the (t-1)th interactive sampling step. The scheduling instability caused by drastic policy changes can be constrained by policy oscillation penalties.

[0117] At this point, the reward function can be expressed as:

[0118]

[0119] Step 404: Determine the reward based on the service quality and the difference between the first parameter and the second parameter. The first parameter is the product of energy consumption information and priority weight, and the second parameter is the product of strategy oscillation penalty and preset balance factor.

[0120] Optionally, after determining service quality, priority weights, energy consumption information, and policy oscillation penalties, these can be substituted into the reward function to obtain the reward. .

[0121] The above process determines the service quality and energy consumption information of candidate nodes based on their next-time state information according to the allocation strategy; it determines the priority and corresponding priority weight of the tasks to be allocated based on their task information; it determines the strategy oscillation penalty based on the allocation strategy and the historical allocation strategy from the previous time step; and it determines the reward based on the difference between the service quality, the first parameter, and the second parameter, where the first parameter is the product of energy consumption information and priority weight, and the second parameter is the product of the strategy oscillation penalty and a preset balance factor. Through this strategy network based on the aforementioned multi-objective joint reward function, the system's responsiveness can be guaranteed while minimizing energy consumption and achieving differentiated allocation of service levels during task allocation. This results in an allocation strategy that balances energy efficiency, response stability, and service differentiation, effectively improving the reliability of task allocation.

[0122] In one exemplary embodiment, the status information includes CPU utilization, bandwidth utilization, allocated energy consumption, number of completed tasks, and total number of tasks, such as... Figure 5 As shown, optionally, the determination of the service quality and energy consumption information of the candidate nodes based on the next-time state information of the candidate nodes corresponding to the allocation strategy includes the following steps 501 to 502. Wherein:

[0123] Step 501: Determine the service quality based on the ratio of the number of tasks completed by candidate nodes to the total number of tasks.

[0124] Alternatively, service quality can be represented by the following formula:

[0125]

[0126] in, This represents the number of tasks completed by the candidate nodes. This represents the total number of tasks for the candidate nodes.

[0127] Step 502: Determine energy consumption information based on the candidate node's CPU energy consumption, bandwidth energy consumption, target allocation energy consumption, and node idle energy consumption.

[0128] Among them, CPU energy consumption is determined based on CPU utilization and the first energy consumption increment coefficient; bandwidth energy consumption is determined based on bandwidth utilization and the second energy consumption increment coefficient; target allocation energy consumption is determined based on the node type of candidate nodes, the number of nodes in the edge computing cluster, and the preset allocation energy consumption table. The preset allocation energy consumption table is used to include multiple sets of correspondences between node type, the number of nodes in the edge computing cluster, and allocation energy consumption.

[0129] First, the process of determining the first energy consumption increment coefficient and the second energy consumption increment coefficient will be explained.

[0130] The energy consumption increment coefficients resulting from CPU and bandwidth were determined through regression modeling of measured data. Real-time CPU and bandwidth energy consumption of each node were measured at different load levels (e.g., CPU utilization of 0%, 25%, 50%, 75%, and 100%) to obtain sample pairs (CPU utilization and real-time CPU energy consumption, bandwidth utilization and real-time bandwidth energy consumption). Then, a linear regression model was fitted with real-time CPU energy consumption as the dependent variable and CPU utilization as the independent variable to obtain the first increment coefficient. Similarly, a linear regression model was fitted with real-time bandwidth energy consumption as the dependent variable and bandwidth utilization as the independent variable to obtain the second increment coefficient.

[0131] Next, the process of determining the preset energy consumption allocation table is explained. Here, energy consumption allocation can be the actual operating cost of the policy network on the nodes. Each node in edge computing clusters of different sizes can be preloaded with the policy network and subjected to multiple rounds of inference testing using dummy input data. During the testing process, performance analysis tools (such as perf) are used to collect CPU / memory / energy consumption metrics, recording average CPU time, memory usage, and unit power consumption. These are then normalized to obtain the cost value. After performance analysis, the correspondence between different node types and energy consumption in edge computing clusters of various sizes is obtained.

[0132] By querying the preset energy consumption allocation table based on the current scale of the edge computing cluster (including the number of nodes) and the node type of the candidate nodes, the target energy consumption allocation corresponding to the candidate nodes can be obtained.

[0133] The energy consumption information of candidate nodes can be expressed by the following formula:

[0134]

[0135] in, For idle energy consumption of nodes, The first energy consumption increment coefficient, For CPU utilization, For bandwidth utilization, This is the second energy consumption increment coefficient. Allocate energy consumption to the target.

[0136] As an optional implementation method, such as Figure 6 As shown, the task allocation method provided in this application embodiment may include the following specific steps:

[0137] Step 601: Obtain the task set, which includes multiple tasks to be assigned.

[0138] Step 602: Determine the feature vector of each task to be assigned based on the task information of each task to be assigned.

[0139] Step 603: Perform cluster analysis on each feature vector to determine the priority of each task to be assigned and the corresponding priority weight.

[0140] Step 604: Obtain the status information of each node and the task information of the target task to be assigned.

[0141] Step 605: Input the current status information of each node and the task information of the task to be assigned into the policy network to determine the allocation policy and the candidate nodes corresponding to the allocation policy.

[0142] When t equals 1, the current state information is the preset state information; when t is greater than 1, the current state information is the state information of the next moment determined in the (t-1)th interactive sampling step.

[0143] Step 606: Update the current state information of each node and determine the state information of each node at the next moment.

[0144] Step 607: Determine the service quality based on the ratio of the number of tasks completed by candidate nodes to the total number of tasks.

[0145] Step 608: Determine the CPU energy consumption of the candidate node based on the CPU utilization rate of the candidate node and the first energy consumption increment coefficient.

[0146] Step 609: Determine the bandwidth energy consumption of the candidate node based on the bandwidth utilization rate and the second energy consumption increment coefficient of the candidate node.

[0147] Step 610: Determine the target energy allocation based on the node type of the candidate nodes, the number of nodes in the edge computing cluster, and the preset energy allocation table.

[0148] The preset energy consumption allocation table includes multiple sets of correspondences between node type, number of nodes in the edge computing cluster, and energy consumption allocation.

[0149] Step 611: Determine energy consumption information based on the candidate node's CPU energy consumption, bandwidth energy consumption, target allocation energy consumption, and node idle energy consumption.

[0150] Step 612: Determine the strategy oscillation penalty based on the allocation strategy and the historical allocation strategy of the previous moment.

[0151] Step 613: Determine the reward based on the service quality and the difference between the first parameter and the second parameter. The first parameter is the product of energy consumption information and priority weight, and the second parameter is the product of strategy oscillation penalty and preset balance factor.

[0152] Step 614: Determine a set of interactive data based on the current state information, allocation strategy, reward, and next-time state information of each node.

[0153] Step 615: Determine whether the number of iterations meets the preset number. If yes, proceed to step 605; otherwise, proceed to step 616.

[0154] Step 616: Update the network parameters of the policy network based on the interaction data and loss function.

[0155] Step 617: Input the status information and task information into the policy network to determine the target node for processing the target task to be assigned, so that the target node can process the target task to be assigned.

[0156] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0157] Based on the same inventive concept, this application also provides a task allocation device for implementing the task allocation method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more task allocation device embodiments provided below can be found in the limitations of the task allocation method described above, and will not be repeated here.

[0158] In one exemplary embodiment, such as Figure 7 As shown, a task allocation device 700 is provided, which is disposed in the task allocation node of an edge computing cluster. The edge computing cluster includes multiple nodes, including: an acquisition module 701 and a determination module 702. Wherein:

[0159] The acquisition module 701 is used to acquire the status information of each node and the task information of the target task to be assigned;

[0160] The determination module 702 is used to input various status information and task information into the policy network to determine the target node for processing the target task to be assigned, so that the target node can process the target task to be assigned.

[0161] The network parameters of the policy network are updated online according to the reinforcement learning algorithm. The reward function corresponding to the reinforcement learning algorithm is determined based on the service quality of the candidate nodes, the energy consumption information of the candidate nodes, the priority of the tasks to be assigned, and the policy oscillation penalty. The policy oscillation penalty is used to characterize the distance between the current output of the policy network and the output of the previous time step.

[0162] In one embodiment, the task allocation device further includes a training module, which iteratively executes multiple interaction sampling steps to determine multiple sets of interaction data. Each set of interaction data includes the current state information of each node, the allocation strategy, the reward, and the next state information of each node. The network parameters of the policy network are updated based on the interaction data and the loss function. The t-th interaction sampling step includes: inputting the current state information of each node and the task information of the task to be allocated into the policy network to determine the allocation strategy; wherein, when t equals 1, the current state information is the preset state information, and when t is greater than 1, the current state information is the next state information determined in the (t-1)-th interaction sampling step; updating the current state information of each node and determining the next state information of each node; and determining the reward according to the reward function.

[0163] In one embodiment, the training module is specifically used to determine the service quality and energy consumption information of the candidate nodes based on the state information of the candidate nodes at the next moment corresponding to the allocation strategy; determine the priority and corresponding priority weight of the task to be allocated based on the task information of the task to be allocated; determine the strategy oscillation penalty based on the allocation strategy and the historical allocation strategy at the previous moment; and determine the reward based on the difference between the service quality, the first parameter, and the second parameter, where the first parameter is the product of the energy consumption information and the priority weight, and the second parameter is the product of the strategy oscillation penalty and the preset balance factor.

[0164] In one embodiment, the status information includes CPU utilization, bandwidth utilization, allocated energy consumption, number of completed tasks, and total number of tasks. The training module is specifically used to determine the quality of service based on the ratio of the number of completed tasks of candidate nodes to the total number of tasks; and to determine energy consumption information based on the CPU energy consumption, bandwidth energy consumption, target allocated energy consumption, and node idle energy consumption of candidate nodes. Specifically, CPU energy consumption is determined based on CPU utilization and a first energy consumption increment coefficient; bandwidth energy consumption is determined based on bandwidth utilization and a second energy consumption increment coefficient; and target allocated energy consumption is determined based on the node type of candidate nodes, the number of nodes in the edge computing cluster, and a preset allocated energy consumption table. The preset allocated energy consumption table includes multiple sets of correspondences between node type, the number of nodes in the edge computing cluster, and allocated energy consumption.

[0165] In one embodiment, the training module is further configured to acquire a task set, which includes multiple tasks to be assigned; determine the feature vector of each task to be assigned based on the task information of each task to be assigned; and perform cluster analysis on each feature vector to determine the priority of each task to be assigned and the corresponding priority weight.

[0166] In one embodiment, the reinforcement learning algorithm is a proximal policy optimization algorithm or an independent proximal policy optimization algorithm.

[0167] Each module in the aforementioned task allocation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0168] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a task allocation method.

[0169] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0170] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps described in any of the above method embodiments.

[0171] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps described in any of the above method embodiments.

[0172] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps described in any of the above method embodiments.

[0173] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0175] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A task allocation method, characterized in that, In a task allocation node for an edge computing cluster, wherein the edge computing cluster includes multiple nodes, the method includes: Obtain the status information of each node and the task information of the target task to be assigned; The status information and task information are input into the policy network to determine the target node for processing the target task to be assigned, so that the target node can process the target task to be assigned. The network parameters of the policy network are updated online according to the reinforcement learning algorithm. The reward function corresponding to the reinforcement learning algorithm is determined based on the service quality of the candidate nodes, the energy consumption information of the candidate nodes, the priority of the tasks to be assigned, and the policy oscillation penalty. The policy oscillation penalty is used to characterize the distance between the current output of the policy network and the output of the previous time step.

2. The method according to claim 1, characterized in that, The process of updating the network parameters of the policy network online using a reinforcement learning algorithm includes: The interaction sampling steps are executed iteratively multiple times to determine multiple sets of interaction data. Each set of interaction data includes the current state information, allocation strategy, reward, and the next state information of each node. The network parameters of the policy network are updated based on the interaction data and loss function described above; The t-th interactive sampling step includes: The current state information of each node and the task information of the task to be assigned are input into the policy network to determine the allocation policy; wherein, when t equals 1, the current state information is the preset state information, and when t is greater than 1, the current state information is the state information of the next moment determined in the (t-1)th interaction sampling step. Update the current state information of each node and determine the state information of each node at the next moment; The reward is determined based on the reward function.

3. The method according to claim 2, characterized in that, Determining the reward based on the reward function includes: The service quality and energy consumption information of the candidate node are determined based on the state information of the candidate node at the next moment corresponding to the allocation strategy. The priority and corresponding priority weight of the task to be assigned are determined based on the task information of the task to be assigned. The oscillation penalty of the allocation strategy is determined based on the allocation strategy and the historical allocation strategy of the previous moment; The reward is determined based on the difference between the service quality, the first parameter, and the second parameter, where the first parameter is the product of the energy consumption information and the priority weight, and the second parameter is the product of the strategy oscillation penalty and the preset balance factor.

4. The method according to claim 3, characterized in that, The status information includes CPU utilization, bandwidth utilization, allocated energy consumption, number of completed tasks, and total number of tasks. Determining the service quality and energy consumption information of the candidate nodes based on their status information at the next moment according to the allocation strategy includes: The service quality is determined based on the ratio of the number of tasks completed by the candidate nodes to the total number of tasks. The energy consumption information is determined based on the CPU energy consumption, bandwidth energy consumption, target allocation energy consumption, and node idle energy consumption of the candidate nodes; The CPU energy consumption is determined based on the CPU utilization rate and a first energy consumption increment coefficient; the bandwidth energy consumption is determined based on the bandwidth utilization rate and a second energy consumption increment coefficient; the target allocation energy consumption is determined based on the node type of the candidate node, the number of nodes in the edge computing cluster, and a preset allocation energy consumption table, wherein the preset allocation energy consumption table includes multiple sets of correspondences between node type, the number of nodes in the edge computing cluster, and allocation energy consumption.

5. The method according to claim 3, characterized in that, The method further includes: Obtain a task set, which includes multiple tasks to be assigned; Determine the feature vector of each of the tasks to be assigned based on the task information of each task to be assigned. Cluster analysis is performed on each of the feature vectors to determine the priority of each task to be assigned and its corresponding priority weight.

6. The method according to any one of claims 1 to 5, characterized in that, The reinforcement learning algorithm is a proximal policy optimization algorithm or an independent proximal policy optimization algorithm.

7. A task allocation device, characterized in that, The device is configured in a task allocation node of an edge computing cluster, wherein the edge computing cluster includes multiple nodes, and the device includes: The acquisition module is used to acquire the status information of each node and the task information of the target task to be assigned; The determination module is used to input the status information and the task information into the policy network to determine the target node for processing the target task to be assigned, so that the target node can process the target task to be assigned. The network parameters of the policy network are updated online according to the reinforcement learning algorithm. The reward function corresponding to the reinforcement learning algorithm is determined based on the service quality of the candidate nodes, the energy consumption information of the candidate nodes, the priority of the tasks to be assigned, and the policy oscillation penalty. The policy oscillation penalty is used to characterize the distance between the current output of the policy network and the output of the previous time step.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images