A method for offloading computing network tasks based on hypergraph partitioning

Through hypergraph partitioning and deep reinforcement learning, optimized computing power network task offloading has solved the problems of high energy consumption and poor adaptability in the existing technology, and achieved more efficient resource management and energy consumption reduction.

CN118113367BActive Publication Date: 2024-11-01TIANJIN UNIV
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202410074605.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-18
Publication Date
2024-11-01
Estimated Expiration
2044-01-18

AI Technical Summary

Technical Problem

The existing computing power network task offloading technology has the problems of high energy consumption and poor adaptability, especially due to the unsatisfactory task offloading results caused by coarse-grained analysis and simple chart characterization, as well as the difficulty in dispersed management and scheduling of heterogeneous resources.

Method used

Hypergraph partitioning is used to characterize heterogeneous tasks and resources, and through a deep reinforcement learning method based on hypergraphs, combined with a two-layer attention mechanism, task offload decisions are optimized to reduce energy consumption and delays.

Benefits of technology

Improve the generalization performance and stability of task offloading, realize more effective resource orchestration and energy consumption management, and significantly reduce the final energy consumption and time delay.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118113367B_ABST
    Figure CN118113367B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for offloading computing power network tasks based on hypergraph partitioning, which belongs to the technical field of computing power network task offloading, and includes the following steps: step 1, establishing a hypergraph representation model for the tasks of the task offloading system; step 2, establishing a delay and energy consumption calculation model in the task offloading system; step 3, establishing the optimization target of the delay and energy consumption calculation model; step 4, using a deep reinforcement learning method based on hypergraph partitioning and a double-layer attention mechanism to learn in the task offloading system and train a task offloading prediction network. The present invention provides a method for offloading computing power network tasks based on hypergraph partitioning, introduces a hypergraph to characterize fine-grained heterogeneous tasks and heterogeneous resources, and solves the energy consumption problem in task offloading based on deep reinforcement learning of the hypergraph.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computing power network task offloading, and in particular, relates to a method for computing power network task offloading based on hypergraph partitioning. Background Art

[0002] With the rapid development of artificial intelligence technology, large models have gradually become a hot topic in today's era. The training of large models depends on massive data and powerful computing power support. CPN is a new architecture that deeply integrates computing and networking. It is a new information infrastructure that takes computing as the center and the network as the foundation to provide computing resource services. For large computing tasks, its training can be accelerated by task offloading. Task offloading determines how to offload computing tasks to the resource pool for further intensive computing and ensure the quality of computing resource services. However, the energy consumption of CPN has become a core issue. It is speculated that data centers will gradually become one of the largest sources of energy consumption in the world. The task offloading problem is closely related to energy consumption, and the important factors that lead to increased energy consumption include coarse-grained analysis of heterogeneous tasks and heterogeneous resources, and task offloading decisions with poor generalization and extreme reliance on professional mathematical models. The specific problems currently faced by task offloading are as follows:

[0003] 1. Coarse-grained analysis and simple graph representation of computing tasks. The computing tasks in CPN are heterogeneous and complex. The CPN framework generally treats tasks as similar entities. However, many computing tasks can actually be decomposed or reused. Coarse-grained analysis of computing tasks may lead to unsatisfactory task offloading effects and increase corresponding energy consumption. Some work divides tasks into directed acyclic graphs in a fine-grained manner. However, for heterogeneous tasks and complex multi-task relationships, simple graph structure representation is already somewhat weak and cumbersome.

[0004] 2. Decentralized management of computing resources. As the number of resources in CPN grows, it becomes extremely difficult to identify and effectively schedule appropriate resources from a huge pool, leading to conflicts and uncertainties in task offloading. How to organize heterogeneous computing resources and unify them into an organic whole is an important issue in CPN.

[0005] 3. Current task offloading methods usually use heuristic algorithms or neural networks for task allocation. However, due to the lack of information on the representation of the task side and the resource side, even algorithms with strong generalization are prone to instability in task offloading. Existing methods are difficult to adapt to unexpected interference or situations that do not occur.

[0006] To address the above problems, the present invention introduces a hypergraph to characterize fine-grained heterogeneous tasks and heterogeneous resources, and uses deep reinforcement learning technology based on the hypergraph to solve the energy consumption problem in task offloading. Summary of the invention

[0007] The purpose of the present invention is to provide a method for offloading computing network tasks based on hypergraph partitioning, so as to solve the problems of high energy consumption and poor adaptability of task offloading in the prior art.

[0008] To achieve the above object, the present invention provides a method for offloading computing network tasks based on hypergraph partitioning, comprising the following steps:

[0009] Step 1: Establish a hypergraph representation model for the tasks of the task offloading system;

[0010] Step 2: Establish a delay and energy consumption calculation model in the task offloading system;

[0011] Step 3: Establish the optimization target of the delay and energy consumption calculation model;

[0012] Step 4: Use the deep reinforcement learning method based on hypergraph partitioning and double-layer attention mechanism to learn in the task offloading system and train the task offloading prediction network.

[0013] Preferably, the hypergraph representation model in step 1 is a directed hypergraph containing task information and task relationships. in represents the task node that carries task information, ε={e1,e2,…,e j ,…,e m} represents a directed hyperedge of multiple associations, n and m represent the number of tasks and hyperedges respectively, and the information of the i-th task is expressed as: {C i ,inData i ,outData i ,type i}, where C i Indicates computing power requirements, inData i ,outData i Respectively represent the data size of the task input and output, type i represents the task type; assuming that the set of task types is represented as Assume there are k tasks, and the information contained in the jth directed hyperedge is {pred j ,eW j ,t j ,eType j}, pred j Indicates the need to j The set of predecessor tasks that need to be completed before the task starts executing, eW jRepresents the number of tasks from the predecessor set to t j The data transmission time required for the task, eType j Indicates whether the task nodes included in this hyperedge are recommended to be offloaded to different resource pools. The adjacent matrix H of the task hypergraph is represented as a multidimensional matrix:

[0014]

[0015] H ij1 =eType j .

[0016] Preferably, the specific process of establishing the delay and energy consumption calculation model in the task offloading system in step 2 is as follows:

[0017] S21. Establish a CPN resource pool environment;

[0018] S22. Calculate the delay and energy consumption according to the given offloading decision.

[0019] Preferably, the resource pool environment of the CPN established in S21 is specifically as follows: the resource pool in the CPN is a set of virtual machines VMs that are arranged into a unified manner and can process a specified task type, and is used Tasks executed on the local system instead of being selected to be offloaded to the resource pool in the CPN will be regarded as being offloaded to the resource pool No. 0. The local system is regarded as the resource pool No. 0. The virtual machines in the resource pool handle different task types. The computing capacity is expressed as CP u,v Represents the computing power of the virtual machine in the u-th resource pool that processes the v-th task type.

[0020] Preferably, the specific process of calculating the delay and energy consumption according to the given unloading decision in S22 is as follows:

[0021] Assume that the uninstall decision given by the system is A 1:n ={a i |i∈{1,2,…,n}, where Indicates that the i-th task will be offloaded to a i In the resource pool, once the offloading decision is made, the tasks start to execute in the specified order. i , assuming a i =u, then the time required for unloading is:

[0022]

[0023]

[0024]

[0025] Among them, T i,u Indicates the total time, and Represent the transmission time and execution time respectively, r ul and r dl They represent the transmission rates of the data upload channel and download channel, respectively, i Indicates computing power requirements, CP u,typei Indicates that the uth resource pool processes the type i The computing power of the virtual machine for each task type;

[0026] For decomposable tasks, the overall time calculation follows the topological structure of the tasks. The earliest time for each task to start execution is expressed as:

[0027]

[0028] In the formula, CT k Indicates the earliest completion time of the predecessor node k, k belongs to task t i Any of the predecessors, Indicates that the kth task is offloaded to the corresponding decision a k time, pre(·) represents the time of task t i All predecessor tasks:

[0029] pre(t i )={k|k∈pred j ,t j =t i ,j∈{1,2,…,m}}

[0030] Arrange the task nodes in the hypergraph into a sequence according to the earliest execution time of each task. After the predecessor task is completed, the task nodes are gradually unloaded to the corresponding resource pool;

[0031] For VMs in the resource pool, there are two states during task offloading: one is the waiting state when there is no task running, and the other is the running state when there is a task. The two states can be switched to each other. u,v ∈{0,1} represents the state of the vth task type running on the VM of the uth CPN resource pool, with 0 representing the waiting state and 1 representing the running state; AT u,v Indicates the expected available time of the VM. When the VM is not running tasks, WAT u,v is equal to the current time, otherwise it is equal to the time WAT of the previous waiting state u,v Plus the task t currently running on it i Uninstall time:

[0032]

[0033] In order to calculate the estimated completion time of each task, a waiting queue q is established for each VM. u,v Abbreviated as qw, the offloaded tasks are queued according to their positions in the task sequence. Assume that task t i The position assigned to the waiting queue is p i ∈{1,…,qlen}, qlen represents the queue length, tasks in the task queue represent tasks that are still waiting to be executed, and the expected completion time PFT i It can be expressed as:

[0034]

[0035] In the formula, It indicates the offloading time required for the jth task in the qw waiting queue, including the task transmission time and execution time, which are all offloaded to the u node. represents the execution time. Since it is local, no transmission time is required. v represents the vth virtual machine corresponding to u, that is, the virtual machine that processes the vth task.

[0036] The earliest start time of each task is calculated based on the topological structure of the hypergraph task:

[0037]

[0038] Among them, PFT k represents the expected completion time of k, where k is the predecessor task. represents the data transmission and processing time required from task k to task i,

[0039]

[0040] Assume that t e It indicates that there is no practical exit task, which indicates that all tasks are completed. Then the total time of the overall task unloading is expressed as:

[0041]

[0042] Similar to latency calculation, tasks consume energy during transmission and execution. and

[0043]

[0044]

[0045] Among them, TP i,u represents the transmission power, represents the effective switch capacitance of the u-th resource pool, CP u,v represents the computing power of the virtual machine in the u-th resource pool processing the v-th task type, so the total energy consumption of the task execution is expressed as:

[0046]

[0047] Since the calculation of energy consumption does not need to consider the topological relationship between the tasks represented by the hypergraph, the total energy consumption is expressed as:

[0048]

[0049] In the formula, Indicates that the task offloading decision corresponding to task i is a i .

[0050] Preferably, the optimization goal of the delay and energy consumption calculation model in step 3 is to minimize the overall delay and energy consumption under the condition that all tasks can be offloaded. The specific expression is as follows:

[0051]

[0052] In the formula, ω t represents the weight of the total delay, ω e Represents the weight of total energy consumption.

[0053] Preferably, the specific process of learning in the task offloading system using the deep reinforcement learning method based on hypergraph partitioning and double-layer attention mechanism in step 4 is as follows:

[0054] S41, performing hypergraph partition preprocessing on the hypergraph representing the task, and partitioning the hypergraph through KaHypar;

[0055] S42. Establish Markov decision making in reinforcement learning. The specific process is as follows:

[0056] Based on reinforcement learning, the task offloading process is modeled as an MDP. Defining the state space in an MDP Action Space And the reward function The MDP state of offloading the i-th task is represented as the current hypergraph embedding And the offloading decision of the first i tasks:

[0057]

[0058] The action space is represented as the offloading decision a of the i-th task i , the reward function is expressed as the increment of the optimization target P:

[0059]

[0060] The reward function encourages learning in the direction of reducing energy consumption and latency. The unloading strategy function π is in state s i Next select action a i The probability of generating an overall unloading decision is expressed as:

[0061]

[0062] S43. Train a deep neural network based on double-layer attention. The specific process is as follows:

[0063] Assume that the embedding information of each node in the hypergraph can be expressed as Before entering the encoder, the embedding information is encoded using the first layer of self-attention The calculation method is as follows:

[0064]

[0065] α i,j is the attention score:

[0066]

[0067] sim(i,j)=query i key j

[0068] query i represents the query vector obtained by multiplying the i-th task embedding and the learnable weight matrix, value j represents the value vector obtained by multiplying the i-th task embedding and the learnable weight matrix, key j represents the key vector obtained by multiplying the i-th task embedding and the learnable weight matrix, sim(i,j) represents the correlation between the i-th task and the j-th task, and the self-attention encoding is input into the encoder. Both the encoder and the decoder use LSTM networks. The final hidden state of the encoder is used as the context vector input of the decoder, and then a layer of attention mechanism is added to optimize the memory unit c and hidden state in the LSTM network of the decoder calculate:

[0069]

[0070]

[0071]

[0072] In the formula, represents the attention scores of the i-th task and the j-th task in the second layer of attention network, represents the hidden state of the j-th encoder, represents the hidden state of the i-1th decoder, cos(i,j) represents the cosine similarity between the i-th task and the j-th task in the second layer of attention network;

[0073] The attention score calculation uses cosine similarity to capture the direction information, and the hidden state of the decoder is calculated as:

[0074]

[0075] where f(·) is a nonlinear network layer, c i represents the memory unit in the LSTM network of the decoder corresponding to the i-th task, a i-1 represents the offloading decision of i-1 tasks, and the final offloading strategy function π(a i |s i ) and the value function v(s) that predicts the total reward i ) is obtained through hidden states and fully connected layers;

[0076] The training of the network needs to be combined with the PPO algorithm, assuming that the old strategy function is based on the previous For a single learning task t i The trajectory τ is obtained by sampling, which can be used to update the new policy function (a i |s i θ i ),in and θ i These are the old policy parameters used in PPO and the policy parameters that need to be updated. Network updates need to rely on the calculation of the loss function. The first is the clipping similarity loss of PPO:

[0077]

[0078]

[0079]

[0080] in It means limiting the probability distribution ratio to [1-∈, 1+∈], is the generalized advantage estimation function, represents the ratio of the new and old strategies corresponding to the jth task, represents the parameters of the old policy function for the i-th task, π(a j |s j θ j) represents the new policy parameters for the i-th task, γ∈[0,1] is the discount factor used to calculate the cumulative error, λ is the parameter for controlling the deviation and variance, and the MSE loss of the value function is added:

[0081]

[0082]

[0083] In the formula, v π represents the network value function, s j represents the MDP state of the jth task, represents the actual estimated value function, r j+k represents the MDP reward of the j+kth task;

[0084] So the overall loss function is expressed as:

[0085]

[0086] Preferably, the training of the task offloading prediction network is continuously optimized by parameter gradient ascent, and the specific expression is as follows:

[0087]

[0088] Where ω represents the learning rate, and the cosine learning rate adjustment scheme is adopted. It represents the gradient of the loss function calculation. After z steps, the old policy function will be replaced. The gradient calculation uses the Adam first-order approximation.

[0089] Therefore, the present invention adopts the above-mentioned method of computing network task offloading based on hypergraph partitioning, which has the following beneficial effects:

[0090] ① Use hypergraphs, which are more generalizable than graph structures, to represent heterogeneous computing tasks. For decomposable tasks, we use directed hypergraphs to replace directed acyclic graphs. Hypergraphs can represent complex multi-association relationships. In addition to representing the topological relationship between computing tasks, they can also represent other complex relationships such as task type, computing demand relationship, and offloading type. Generally speaking, using hypergraphs to represent computing tasks can consider more comprehensive and realistic task relationships, thereby improving the final task offloading effect and providing stronger generalization performance;

[0091] ② For computing tasks represented by hypergraphs, we proposed a new hypergraph-based deep reinforcement learning scheme to realize the offloading of computing tasks, and used the hypergraph partitioning method to preprocess the hypergraph as the initial input state. Existing networks cannot meet the above requirements because they are more targeted at task models represented by simplified graph structures. The offloading scheme of the present invention can flexibly adjust the offloading decision according to the computing task requirements in different scenarios, thereby achieving a stable offloading optimization effect and reducing the final offloading energy consumption. Compared with traditional offloading schemes, the scheme we proposed can reach convergence faster and achieve better task offloading optimization effects;

[0092] ③In addition, we added a double layer of attention to the network structure in deep reinforcement learning. The self-attention mechanism can well learn the multiple correlations between tasks in the hypergraph. We conducted a large number of simulation experiments to evaluate the performance of this method. Compared with various baselines, this model can greatly accelerate the convergence speed and achieve the optimal energy consumption reduction effect.

[0093] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 It is a hypergraph task representation graph of the present invention;

[0095] Figure 2 It is a diagram of the reinforcement learning process of task offloading of the present invention.

[0096] Figure 3 This is a diagram of the seq2seq network structure based on double-layer attention in the present invention.

[0097] Figure 4 It is a training comparison chart of the present invention.

[0098] Figure 5 It is a diagram showing the influence of different numbers of tasks on the results of the present invention. DETAILED DESCRIPTION

[0099] The following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0100] See also Figure 1-5 , a method for offloading computing network tasks based on hypergraph partitioning, comprising the following steps:

[0101] Step 1: Establish a hypergraph representation model for the tasks of the task offloading system; the hypergraph representation model is a directed hypergraph containing task information and task relationships. in represents the task node that carries task information, ε={e1,e2,…,e j ,…,e m} represents a directed hyperedge of multiple associations, n and m represent the number of tasks and hyperedges respectively, and the information of the i-th task is expressed as: {C i ,inData i ,outData i ,type i}, where C i Indicates computing power requirements, inData i ,outData i Respectively represent the data size of the task input and output, type i represents the task type; assuming that the set of task types is represented as Assume there are k tasks, and the information contained in the jth directed hyperedge is {pred j ,eW j ,t j ,eType j}, pred j Indicates the need to j The set of predecessor tasks that need to be completed before the task starts executing, eW j Represents the number of tasks from the predecessor set to t j The data transmission time required for the task, eType j Indicates whether the task nodes included in this hyperedge are recommended to be offloaded to different resource pools. The adjacent matrix H of the task hypergraph is represented as a multidimensional matrix:

[0102]

[0103] H ij1 =eType j .

[0104] Step 2: Establish a delay and energy consumption calculation model in the task offloading system; the specific establishment process is as follows:

[0105] S21, establish a resource pool environment of CPN; specifically: the resource pool in CPN is a collection of virtual machines VMs that are arranged into a unified system and can handle specified task types, and is used Tasks executed on the local system instead of being selected to be offloaded to the resource pool in the CPN will be regarded as being offloaded to the resource pool No. 0. The local system is regarded as the resource pool No. 0. The virtual machines in the resource pool handle different task types. The computing capacity is expressed as CPu,v Represents the computing power of the virtual machine in the u-th resource pool that processes the v-th task type.

[0106] S22: Calculate the delay and energy consumption according to the given unloading decision. The specific process is as follows:

[0107] Assume that the uninstall decision given by the system is A 1:n ={a i |i∈{1,2,…,n}, where Indicates that the i-th task will be offloaded to a i In the resource pool, once the offloading decision is made, the tasks start to execute in the specified order. i , assuming a i =u, then the time required for unloading is:

[0108]

[0109]

[0110]

[0111] Among them, T i,u Indicates the total time, and represents the transmission time and execution time respectively, ul and r dl They represent the transmission rates of the data upload channel and download channel, respectively, i Indicates the computing power requirement, Indicates that the uth resource pool processes the type i The computing power of the virtual machine for each task type;

[0112] For decomposable tasks, the overall time calculation follows the topological structure of the tasks. The earliest time for each task to start execution is expressed as:

[0113]

[0114] In the formula, CT k Indicates the earliest completion time of the predecessor node k, k belongs to task t i Any of the predecessors, Indicates that the kth task is offloaded to the corresponding decision a k time, pre(·) represents the time of task t i All predecessor tasks:

[0115] pre(t i )={k|k∈pred j ,t j =ti ,j∈{1,2,…,m}}

[0116] like Figure 1 As shown in Figure 1, the task nodes in the hypergraph can be arranged into a sequence according to the earliest execution time of each task, and they will be gradually unloaded to the corresponding resource pool after the completion of the predecessor task;

[0117] For VMs in the resource pool, there are two states during task offloading: one is the waiting state when there is no task running, and the other is the running state when there is a task. The two states can be switched to each other. u,v ∈{0,1} represents the state of the vth task type running on the VM of the uth CPN resource pool, with 0 representing the waiting state and 1 representing the running state; AT u,v Indicates the expected available time of the VM. When the VM is not running tasks, WAT u,v is equal to the current time, otherwise it is equal to the time WAT of the previous waiting state u,v Plus the task t currently running on it i Uninstall time:

[0118]

[0119] In order to calculate the estimated completion time of each task, a waiting queue q is established for each VM. u,v Abbreviated as qw, the offloaded tasks are queued according to their positions in the task sequence. Assume that task t i The position assigned in the waiting queue is pi∈{1,…,qlen}, qlen represents the queue length, and the tasks in the task queue represent tasks that are still waiting to be executed. The expected completion time PFT i It can be expressed as:

[0120]

[0121] In the formula, It indicates the offloading time required for the jth task in the qw waiting queue, including the task transmission time and execution time, which are all offloaded to the u node. represents the execution time. Since it is local, no transmission time is required. v represents the vth virtual machine corresponding to u, that is, the virtual machine that processes the vth task.

[0122] The earliest start time of each task is calculated based on the topological structure of the hypergraph task:

[0123]

[0124] Among them, PFT krepresents the expected completion time of k, where k is the predecessor task. represents the data transmission and processing time required from task k to task i,

[0125]

[0126] Assume that t e It indicates that there is no practical exit task, which indicates that all tasks are completed. Then the total time of the overall task unloading is expressed as:

[0127]

[0128] Similar to latency calculation, tasks consume energy during transmission and execution. and

[0129]

[0130]

[0131] Among them, TP i,u represents the transmission power, represents the effective switch capacitance of the u-th resource pool, CP u,v represents the computing power of the virtual machine in the u-th resource pool processing the v-th task type, so the total energy consumption of the task execution is expressed as:

[0132]

[0133] Since the calculation of energy consumption does not need to consider the topological relationship between the tasks represented by the hypergraph, the total energy consumption is expressed as:

[0134]

[0135] In the formula, Indicates that the task offloading decision corresponding to task i is a i .

[0136] Step 3: Establish the optimization goal of the delay and energy consumption calculation model. The optimization goal of the delay and energy consumption calculation model is to minimize the overall delay and energy consumption under the condition that all tasks can be offloaded. The specific expression is as follows:

[0137]

[0138] In the formula, ω t represents the weight of the total delay, ω e Represents the weight of total energy consumption.

[0139] Step 4: Use the deep reinforcement learning method based on hypergraph partitioning and double-layer attention mechanism to learn in the task offloading system and train the task offloading prediction network. The specific process is as follows:

[0140] S41, performing hypergraph partition preprocessing on the hypergraph representing the task, and partitioning the hypergraph through KaHypar;

[0141] S42. Establish Markov decision making in reinforcement learning. The specific process is as follows:

[0142] Based on reinforcement learning, the task offloading process is modeled as an MDP. Defining the state space in an MDP Action Space And the reward function The MDP state of offloading the i-th task is represented as the current hypergraph embedding And the offloading decision of the first i tasks:

[0143]

[0144] The action space is represented as the offloading decision a of the i-th task i , the reward function is expressed as the increment of the optimization target P:

[0145]

[0146] The reward function encourages learning in the direction of reducing energy consumption and latency. The unloading strategy function π is in state s i Next select action a i The probability of generating an overall unloading decision is expressed as:

[0147]

[0148] The process of reinforcement learning is as follows Figure 2 As shown in the figure, as the unloading process continues, the corresponding state, reward and action will continue to change. The policy function and the overall reward function can be learned through some deep neural networks to achieve the effect of optimizing the target. The reinforcement learning algorithm we use is the PPO algorithm, which is a classic policy gradient algorithm.

[0149] S43. Train a deep neural network based on double-layer attention. The specific process is as follows:

[0150] Our deep neural network uses the seq2seq architecture, which is an encoder-decoder neural network structure. We changed the internal network so that it can learn the information of the hypergraph. The specific architecture is as follows Figure 3As shown, the encoder and decoder networks use the classic time series prediction network LSTM. The embedding information of the hypergraph can use the adjacent matrix and task attribute information processed by the fully connected layer. Assume that the embedding information of each node in the hypergraph can be expressed as Before entering the encoder, the embedding information is encoded using the first layer of self-attention The calculation method is as follows:

[0151]

[0152] α i,j is the attention score:

[0153]

[0154] sim(i,j)=query i key j

[0155] query i represents the query vector obtained by multiplying the i-th task embedding and the learnable weight matrix, value j represents the value vector obtained by multiplying the i-th task embedding and the learnable weight matrix, key j represents the key vector obtained by multiplying the i-th task embedding and the learnable weight matrix, sim(i,j) represents the correlation between the i-th task and the j-th task, and the self-attention encoding is input into the encoder. Both the encoder and the decoder use LSTM networks. The final hidden state of the encoder is used as the context vector input of the decoder, and then a layer of attention mechanism is added to optimize the memory unit c and hidden state in the LSTM network of the decoder calculate:

[0156]

[0157]

[0158]

[0159] In the formula, represents the attention scores of the i-th task and the j-th task in the second layer of attention network, represents the hidden state of the j-th encoder, represents the hidden state of the i-1th decoder, cos(i,j) represents the cosine similarity between the i-th task and the j-th task in the second layer of attention network;

[0160] The attention score calculation uses cosine similarity to capture the direction information, and the hidden state of the decoder is calculated as:

[0161]

[0162] where f(·) is a nonlinear network layer, c i represents the memory unit in the LSTM network of the decoder corresponding to the i-th task, a i-1 represents the offloading decision of i-1 tasks, and the final offloading strategy function π(a i |s i ) and the value function v(s) that predicts the total reward i ) is obtained through hidden states and fully connected layers;

[0163] The training of the network needs to be combined with the PPO algorithm, assuming that the old strategy function is based on the previous For a single learning task t i The trajectory τ is obtained by sampling, which can be used to update the new policy function (a i |s i θ i ),in and θ i These are the old policy parameters used in PPO and the policy parameters that need to be updated. Network updates need to rely on the calculation of the loss function. The first is the clipping similarity loss of PPO:

[0164]

[0165]

[0166]

[0167] in It means limiting the probability distribution ratio to [1-∈, 1+∈], is the generalized advantage estimation function, represents the ratio of the new and old strategies corresponding to the jth task, represents the parameters of the old policy function for the i-th task, π(a j |s j θ j ) represents the new policy parameters for the i-th task, γ∈[0,1] is the discount factor used to calculate the cumulative error, λ is the parameter for controlling the deviation and variance, and the MSE loss of the value function is added:

[0168]

[0169]

[0170] In the formula, v π represents the network value function, s j represents the MDP state of the jth task, represents the actual estimated value function, r j+k represents the MDP reward of the j+kth task;

[0171] So the overall loss function is expressed as:

[0172]

[0173] The training of the task offloading prediction network is continuously optimized through parameter gradient ascent. The specific expression is as follows:

[0174]

[0175] Where ω represents the learning rate, and the cosine learning rate adjustment scheme is adopted. It represents the gradient of the loss function calculation. After z steps, the old policy function will be replaced. The gradient calculation uses the Adam first-order approximation. Through the above training method, we can obtain a highly generalized task offloading prediction network, which can capture the relationship between tasks in many complex situations and allocate the correct computing requirements to each task.

[0176] Example

[0177] To evaluate the effectiveness of our proposed method, we compared several different deep reinforcement learning algorithms, mainly TRPO, A-PPO with single-layer attention, and PPO without attention mechanism. Figure 4 As shown in the first figure, the convergence speed of our proposed method is significantly faster than other algorithms and can obtain the best reward. Compared with the conventional PPO algorithm, our reward is increased by about 20.08%. Only our algorithm uses the initial hypergraph partition preprocessing, and other network inputs are randomized. Compared with the A-PPO algorithm, which converges faster, the convergence speed is still significantly accelerated by 53.40%. It can save a lot of training time and cost. Figure 4 The second figure is a comparison of the deep reinforcement learning method based on the hypergraph structure and the deep reinforcement learning method based on the ordinary graph structure. It can be found that the algorithm based on the hypergraph we proposed significantly speeds up the convergence of the algorithm. Although the final rewards are not much different, the structure based on the hypergraph representation and the preprocessing of the hypergraph partition can significantly speed up the convergence of the training. In addition to the comparison of network training effects, we also studied the impact of the number of tasks on the system's latency, energy consumption, and rewards, such as Figure 5As shown in the figure, the increase in the number of tasks will increase the delay and energy consumption related to task execution and data transmission, which will lead to calculation delays for all algorithms, increased energy consumption and a decrease in rewards. However, as the number of tasks increases, the downward trend of rewards is also decreasing. This is because the increase in the number of tasks will lead to more multi-association relationships, and the advantages of hypergraph partitioning can be better utilized to capture more internal connections.

[0178] Therefore, the present invention adopts the above-mentioned method of computing network task offloading based on hypergraph partitioning, introduces a hypergraph to characterize fine-grained heterogeneous tasks and heterogeneous resources, and solves the energy consumption problem in task offloading based on deep reinforcement learning of the hypergraph.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for offloading computing network tasks based on hypergraph partitioning, characterized in that: The following steps are involved: Step 1: Establish a hypergraph representation model for the tasks of the task offloading system; Step 2: Establish a delay and energy consumption calculation model in the task offloading system; Step 3: Establish the optimization target of the delay and energy consumption calculation model; Step 4: Use the deep reinforcement learning method based on hypergraph partitioning and double-layer attention mechanism to learn in the task offloading system and train the task offloading prediction network; The hypergraph representation model in step 1 is a directed hypergraph containing task information and task relationships. in represents the task node that carries task information, ε={e1,e2,…,e j ,…,e m } represents a directed hyperedge of multiple associations, n and m represent the number of tasks and hyperedges respectively, and the information of the i-th task is expressed as: {C i ,inData i ,outData i ,type i }, where C i Indicates computing power requirements, inData i ,outData i Respectively represent the data size of the task input and output, type i represents the task type; assuming that the set of task types is represented as Assume there are k tasks, and the information contained in the jth directed hyperedge is {pred j ,eW j ,t j ,eType j }, pred j Indicates that it is necessary to j The set of predecessor tasks that need to be completed before the task starts executing, eW j Represents the number of tasks from the predecessor set to t j The data transmission time required for the task, eType j Indicates whether the task nodes included in this hyperedge are recommended to be offloaded to different resource pools. The adjacent matrix H of the task hypergraph is represented as a multidimensional matrix: H ij1 =eType j ; The specific process of establishing the delay and energy consumption calculation model in the task offloading system in step 2 is as follows: S21. Establish a CPN resource pool environment; S22, calculating the delay and energy consumption according to the given offloading decision; The resource pool environment of CPN established in S21 is specifically as follows: the resource pool in CPN is a collection of virtual machines VMs that are arranged into a unified system and can process specified task types. Tasks executed on the local system instead of being selected to be offloaded to the resource pool in the CPN will be regarded as being offloaded to the resource pool No.

0. The local system is regarded as the resource pool No.

0. The virtual machines in the resource pool handle different task types. The computing capacity is expressed as CP u,v represents the computing power of the virtual machine in the u-th resource pool that processes the v-th task type; The specific process of calculating the latency and energy consumption according to the given offloading decision in S22 is as follows: Assume that the uninstall decision given by the system is A 1:n ={a i |i∈{1,2,…,n}, where Indicates that the i-th task will be offloaded to a i In the resource pool, once the offloading decision is made, the tasks start to execute in the specified order. i , assuming a i =u, then the time required for unloading is: Among them, T i,u Indicates the total time, and represents the transmission time and execution time respectively, ul and r dl They represent the transmission rates of the data upload channel and download channel, respectively, i Indicates the computing power requirement, Indicates that the uth resource pool processes the type i The computing power of the virtual machine for each task type; For decomposable tasks, the overall time calculation follows the topological structure of the tasks. The earliest time for each task to start execution is expressed as: In the formula, CT k Indicates the earliest completion time of the predecessor node k, k belongs to task t i Any of the predecessors, Indicates that the kth task is offloaded to the corresponding decision a k time, pre(·) represents the time of task t i All predecessor tasks: pre(t i )={k|k∈pred j ,t j =t i ,j∈{1,2,…,m}} Arrange the task nodes in the hypergraph into a sequence according to the earliest execution time of each task. After the predecessor task is completed, the task nodes are gradually unloaded to the corresponding resource pool; For VMs in the resource pool, there are two states during task offloading: one is the waiting state when there is no task running, and the other is the running state when there is a task. The two states can be switched to each other. u,v ∈{0,1} represents the state of the vth task type running on the VM of the uth CPN resource pool, with 0 representing the waiting state and 1 representing the running state; AT u,v Indicates the expected available time of the VM. When the VM is not running tasks, WAT u,v is equal to the current time, otherwise it is equal to the time WAT of the previous waiting state u,v Plus the task t currently running on it i Uninstall time: In order to calculate the estimated completion time of each task, a waiting queue q is established for each VM. u,v Abbreviated as qw, the offloaded tasks are queued according to their position in the task sequence. Assume that task t i The position assigned to the waiting queue is p i ∈{1,…,qlen}, qlen represents the queue length, tasks in the task queue represent tasks that are still waiting to be executed, and the expected completion time PFT i It can be expressed as: In the formula, It indicates the offloading time required for the jth task in the qw waiting queue, including the task transmission time and execution time, which are all offloaded to the u node. represents the execution time. Since it is local, no transmission time is required. v represents the vth virtual machine corresponding to u, that is, the virtual machine that processes the vth task. The earliest start time of each task is calculated based on the topological structure of the hypergraph task: Among them, PFT k represents the expected completion time of k, where k is the predecessor task. represents the data transmission and processing time required from task k to task i, Assume that t e It indicates that there is no practical exit task, which indicates that all tasks are completed. Then the total time of the overall task unloading is expressed as: Similar to latency calculation, tasks consume energy during transmission and execution. and Among them, TP i,u represents the transmission power, represents the effective switch capacitance of the u-th resource pool, CP u,v represents the computing power of the virtual machine in the u-th resource pool processing the v-th task type, so the total energy consumption of the task execution is expressed as: Since the calculation of energy consumption does not need to consider the topological relationship between the tasks represented by the hypergraph, the total energy consumption is expressed as: In the formula, Indicates that the task offloading decision corresponding to task i is a i ; The optimization goal of the latency and energy consumption calculation model in step 3 is to minimize the overall latency and energy consumption under the condition that all tasks can be offloaded. The specific expression is as follows: In the formula, ω t represents the weight of the total delay, ω e represents the weight of total energy consumption; The specific process of learning in the task offloading system using the deep reinforcement learning method based on hypergraph partitioning and double-layer attention mechanism in step 4 is as follows: S41, performing hypergraph partition preprocessing on the hypergraph representing the task, and partitioning the hypergraph through KaHypar; S42. Establish Markov decision making in reinforcement learning. The specific process is as follows: Based on reinforcement learning, the task offloading process is modeled as an MDP. Defining the state space in an MDP Action Space And the reward function The MDP state of offloading the i-th task is represented as the current hypergraph embedding And the offloading decision of the first i tasks: The action space is represented as the offloading decision a of the i-th task i , the reward function is expressed as the increment of the optimization target P: The reward function encourages learning in the direction of reducing energy consumption and latency. The unloading strategy function π is in state s i Next select action a i The probability of generating an overall unloading decision is expressed as: S43. Train a deep neural network based on double-layer attention. The specific process is as follows: Assume that the embedding information of each node in the hypergraph can be expressed as Before entering the encoder, the embedding information is encoded using the first layer of self-attention The calculation method is as follows: α i,j is the attention score: sim(i,j)=query i key j query i represents the query vector obtained by multiplying the i-th task embedding and the learnable weight matrix, value j represents the value vector obtained by multiplying the i-th task embedding and the learnable weight matrix, key j represents the key vector obtained by multiplying the i-th task embedding and the learnable weight matrix, sim(i,j) represents the correlation between the i-th task and the j-th task, and the self-attention encoding is input into the encoder. Both the encoder and the decoder use LSTM networks. The final hidden state of the encoder is used as the context vector input of the decoder, and then a layer of attention mechanism is added to optimize the memory unit c and hidden state in the LSTM network of the decoder calculate: In the formula, represents the attention scores of the i-th task and the j-th task in the second layer of attention network, represents the hidden state of the j-th encoder, represents the hidden state of the i-1th decoder, cos(i,j) represents the cosine similarity between the i-th task and the j-th task in the second layer of attention network; The attention score calculation uses cosine similarity to capture the direction information, and the hidden state of the decoder is calculated as: where f(·) is a nonlinear network layer, c i represents the memory unit in the LSTM network of the decoder corresponding to the i-th task, a i-1 represents the offloading decision of i-1 tasks, and the final offloading strategy function π(a i |s i ) and the value function v(s) that predicts the total reward i ) is obtained through hidden states and fully connected layers; The training of the network needs to be combined with the PPO algorithm, assuming that the old strategy function is based on the previous For a single learning task t i The trajectory τ is obtained by sampling, which can be used to update the new policy function (a i |s i θ i ),in and θ i These are the old policy parameters used in PPO and the policy parameters that need to be updated. Network updates need to rely on the calculation of the loss function. The first is the clipping similarity loss of PPO: in It means limiting the probability distribution ratio to [1-∈, 1+∈], is the generalized advantage estimation function, represents the ratio of the new and old strategies corresponding to the jth task, represents the parameters of the old policy function for the i-th task, π(a j |s j θ j ) represents the new policy parameters for the i-th task, γ∈[0,1] is the discount factor used to calculate the cumulative error, λ is the parameter for controlling the deviation and variance, and the MSE loss of the value function is added: In the formula, v π represents the network value function, s j represents the MDP state of the jth task, represents the actual estimated value function, r j+k represents the MDP reward of the j+kth task; So the overall loss function is expressed as:

2. According to the method of claim 1, the method is characterized in that: The training of the task offloading prediction network is continuously optimized through parameter gradient ascent. The specific expression is as follows: Where ω represents the learning rate, and the cosine learning rate adjustment scheme is adopted. It represents the gradient of the loss function calculation. After z steps, the old policy function will be replaced. The gradient calculation uses the Adam first-order approximation.

Citation Information

Patent Citations

  • A computing unloading scheduling method based on depth reinforcement learning

    CN109257429A

  • Optimization method for minimizing energy consumption and time delay sum in edge calculation

    CN109600178A

  • DRL-based edge video target detection task unloading method and system

    CN115022319A

  • Cloud edge collaborative element reinforcement learning calculation unloading method based on wide attention mechanism

    CN116009990A

  • Multi-user dependency task unloading method based on edge calculation and deep reinforcement learning

    CN116431326A