Edge computing unloading method and device, electronic equipment and storage medium

Abstract

The invention discloses an edge computing unloading method and device, electronic equipment and a storage medium, and is used for solving the problems that an existing edge computing unloading method is low in safety and poor in task unloading strategy. The method comprises the following steps: according to the privacy level of the mobile equipment in the t time slot and a constructed cost function for processing subtasks of executable tasks of the mobile equipment in the t time slot, constructing a task unloading problem with minimum cost and maximum privacy level; a Lyapunov optimization theory is adopted to convert a task unloading problem into a single time slot task unloading strategy optimization problem, and an optimization objective function is obtained; and modeling the optimization objective function as a Markov decision process, and determining an optimal task unloading strategy according to a preset deep reinforcement learning algorithm.

Description

Technical Field

[0001] The present application relates to the technical field of edge computing, and in particular, to an edge computing offloading method, apparatus, electronic device, and storage medium. Background Art

[0002] With the wide application of Mobile Edge Computing (MEC) technology, offloading the computing tasks generated by users' mobile devices to a mobile edge computing server for processing improves the processing efficiency of the computing tasks of mobile devices. At the same time, it also brings new privacy risks. Especially when processing sensitive data on the edge computing server, it is vulnerable to hacker attacks. The increasingly diverse attack means make data protection more complex.

[0003] In related technologies, in order to improve the security of edge computing offloading, one way is to adopt user privacy protection based on the traditional greedy algorithm. By gradually selecting the current optimal privacy protection strategy, it can evaluate privacy risks in real time and respond quickly, reducing the risk of data leakage. However, the greedy algorithm selects the current optimal solution at each step, lacking a global perspective. It only considers the current state and ignores the long-term effects and subsequent decisions, which may lead to the overall solution not being globally optimal, resulting in a contradiction between short-term interests and long-term privacy protection goals. Moreover, the greedy algorithm may not be able to effectively handle complex constraints, resulting in an inability to achieve an ideal privacy protection effect. In a dynamic environment, it is also difficult for this algorithm to quickly adapt to changing user needs and privacy threats, requiring more calculations and adjustments. Thus, the security of edge computing offloading is relatively low.

[0004] Another way is user privacy protection based on the genetic algorithm, which adapts to changing privacy threats and user needs by simulating natural selection and genetic mechanisms. However, the genetic algorithm usually requires a relatively large number of iterations to find the optimal solution, with a slow convergence speed, affecting the efficiency of real-time privacy protection. Moreover, the genetic algorithm highly depends on parameter settings, such as population size, mutation rate, and crossover rate, etc. Inappropriate parameters will lead to poor effects. Although the genetic algorithm has exploratory properties, it may still fall into local optimal solutions in complex problems and is difficult to ensure obtaining the global optimal solution. Due to the genetic algorithm using a random search mechanism, the accuracy of the solution is not high. Thus, the task offloading strategy is not good, and it is difficult to ensure effectiveness under strict privacy protection requirements. Summary of the Invention

[0005] In order to solve the problems of low security and poor task offloading strategy in existing edge computing offloading methods, embodiments of the present application provide an edge computing offloading method, apparatus, electronic device, and storage medium.

[0006] In a first aspect, an embodiment of the present application provides an edge computing offloading method, including:

[0007] Obtain the number of subtasks included in the task generated by the mobile device in the t time slot, the number of subtasks of the task that the mobile device can execute, the transmission rate between the mobile device and the mobile edge computing server, the bandwidth of the mobile edge computing server, and the available bandwidth of all mobile edge computing servers;

[0008] Determine the privacy level of the mobile device in the t time slot according to the number of subtasks included in the task generated by the mobile device in the t time slot, the number of subtasks of the task that the mobile device can execute, and the transmission rate between the mobile device and the mobile edge computing server, where the privacy level of the mobile device represents the privacy level of the mobile device;

[0009] Construct a cost function for processing the subtasks of the task that the mobile device can execute in the t time slot according to the number of subtasks of the task that the mobile device can execute in the t time slot, the bandwidth of the mobile edge computing server, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, the task offloading decision parameter of the mobile edge computing server, and the bandwidth allocation ratio parameter of the mobile edge computing server;

[0010] Construct a task offloading problem with minimized cost and maximized privacy level according to the privacy level of the mobile device in the t time slot and the cost function for processing the subtasks of the task that the mobile device can execute;

[0011] Use the Lyapunov optimization theory to transform the task offloading problem into a single-time slot task offloading strategy optimization problem, and obtain an optimized objective function;

[0012] Model the optimized objective function as a Markov decision process, and determine the optimal task offloading strategy according to a preset deep reinforcement learning algorithm.

[0013] In an implementation manner, determining the privacy level of the mobile device in the t time slot according to the number of subtasks included in the task generated by the mobile device in the t time slot, the number of subtasks of the task that the mobile device can execute, and the transmission rate between the mobile device and the mobile edge computing server specifically includes:

[0014] Determine the user usage pattern privacy level of the mobile device in the t time slot according to the number of subtasks included in the task generated by the mobile device in the t time slot, the number of subtasks of the task that the mobile device can execute, and the transmission rate between the mobile device and the mobile edge computing server;

[0015] Determine the location privacy level of the mobile device in the t time slot according to the number of subtasks of the tasks executable by the mobile device in the t time slot and the transmission rate between the mobile device and the mobile edge computing server;

[0016] Determine the weighted sum of the user usage pattern privacy level and the location privacy level of the mobile device in the t time slot as the privacy level of the mobile device in the t time slot.

[0017] In one implementation, the task offloading decision parameter of the mobile device characterizes whether to offload tasks to the mobile edge computing server for execution, and the task offloading decision parameter of the mobile edge computing server characterizes the proportion of tasks offloaded by the mobile device executed by the mobile edge computing server and the proportion of tasks offloaded by the mobile device offloaded to the cloud server for execution;

[0018] Construct a cost function for processing the subtasks of the tasks executable by the mobile device in the t time slot according to the number of subtasks of the tasks executable by the mobile device in the t time slot, the bandwidth of the mobile edge computing server, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, the task offloading decision parameter of the mobile edge computing server, and the bandwidth allocation ratio parameter of the mobile edge computing server, specifically including:

[0019] Determine the computing cost function for processing the subtasks of the tasks executable by the mobile device in the t time slot according to the number of subtasks of the tasks executable by the mobile device in the t time slot, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server;

[0020] Determine the communication cost function for processing the subtasks of the tasks executable by the mobile device in the t time slot according to the bandwidth of the mobile edge computing server in the t time slot, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, and the bandwidth allocation ratio parameter of the mobile edge computing server;

[0021] Determine the cost function for processing the subtasks of the tasks executable by the mobile device in the t time slot according to the computing cost function and the communication cost function.

[0022] In one implementation, construct a task offloading problem that minimizes cost and maximizes privacy level according to the privacy level of the mobile device in the t time slot and the cost function for processing the subtasks of the tasks executable by the mobile device, specifically including:

[0023] Construct a task offloading problem that maximizes cost and maximizes privacy level as:

[0024]

[0025] Among them, P i (t) represents the privacy level of the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, and n represents the number of mobile devices;

[0026] C i (t) represents the cost function of processing the subtasks included in the task generated by the i-th mobile device in the t-th time slot;

[0027] represents the trade-off coefficient between the privacy level and the cost function; T represents the number of time slots;

[0028] α i (t) represents the task offloading decision parameter of the i-th mobile device in the t-th time slot;

[0029] η j (t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server in the t-th time slot, where j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0030] γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server in the t-th time slot, representing the proportion of the subtasks included in the task offloaded by the i-th mobile device executed by the j-th mobile edge computing server in the t-th time slot;

[0031] T i (t) represents the processing delay of processing the subtasks included in the task generated by the i-th mobile device in the t-th time slot, K i represents the task processing delay constraint;

[0032] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, represents the minimum transmission rate between the mobile device and the mobile edge computing server;

[0033] represents the transmission rate between the j-th mobile edge computing server and the cloud server in the t-th time slot, represents the minimum transmission rate between the mobile edge computing server and the cloud server;

[0034] Q i (t) represents the task queue length of the i-th mobile device in the t-th time slot. The task queue of the i-th mobile device is used to store the tasks generated by the i-th mobile device in each time slot, Represents the maximum length of the task queue of the \(i\)-th mobile device.

[0035] In one embodiment, the user usage pattern privacy level of the mobile device at the \(t\) time slot is determined according to the number of subtasks included in the task generated by the mobile device at the \(t\) time slot, the number of subtasks that the mobile device can execute the task, and the transmission rate between the mobile device and the mobile edge computing server. Specifically, it includes:

[0036] Calculate the user usage pattern privacy level of the \(i\)-th mobile device at the \(t\) time slot through the following formula:

[0037]

[0038] where \(P\) i,U (t) represents the user usage pattern privacy level of the \(i\)-th mobile device at the \(t\) time slot, \(i = 1, 2, \cdots, n\), and \(n\) represents the number of mobile devices;

[0039] D i (t) represents the number of subtasks included in the task generated by the \(i\)-th mobile device at the \(t\) time slot;

[0040] A i (t) represents the number of subtasks that the \(i\)-th mobile device can execute the task at the \(t\) time slot;

[0041] represents the transmission rate between the \(i\)-th mobile device and the \(j\)-th mobile edge computing server at the \(t\) time slot, \(j = 1, 2, \cdots, m\), and \(m\) represents the number of mobile edge computing servers;

[0042] r thr represents the transmission rate threshold from the \(i\)-th mobile device to the \(j\)-th mobile edge computing server;

[0043] \(\prod(\cdot)\) is an indicator function. If holds, then If does not hold, then

[0044] In one embodiment, the location privacy level of the mobile device at the \(t\) time slot is determined according to the number of subtasks that the mobile device can execute the task at the \(t\) time slot and the transmission rate between the mobile device and the mobile edge computing server. Specifically, it includes:

[0045] Calculate the location privacy level of the \(i\)-th mobile device at the \(t\) time slot through the following formula:

[0046]

[0047] Among them, P i,L (t) represents the location privacy level of the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, and n represents the number of mobile devices;

[0048] A i (t) represents the number of subtasks that the i-th mobile device can execute tasks in the t-th time slot;

[0049] ∏(·) is an indicator function. If A i (t) > 0 holds, then ∏(A i (t) > 0) = 1. If A i (t) > 0 does not hold, then ∏(A i (t) > 0) = 0;

[0050] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, where j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0051] r thr represents the transmission rate threshold from the i-th mobile device to the j-th mobile edge computing server;

[0052] If holds, then If does not hold, then

[0053] In one implementation, the weighted sum of the user usage pattern privacy level and the location privacy level of the mobile device in the t-th time slot is determined as the privacy level of the mobile device in the t-th time slot, which specifically includes:

[0054] Calculate the privacy level of the i-th mobile device in the t-th time slot through the following formula:

[0055] P i (t) = λ1P i,U (t) + λ2P i,L (t)

[0056] Among them, P i (t) represents the privacy level of the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, and n represents the number of mobile devices;

[0057] P i,U(t) represents the privacy level of the usage pattern of the user of the i-th mobile device in the t-th time slot, and λ1 represents the weight of the privacy level of the usage pattern of the user of the i-th mobile device in the t-th time slot;

[0058] P i,L (t) represents the location privacy level of the i-th mobile device in the t-th time slot, and λ2 represents the weight of the location privacy level of the i-th mobile device in the t-th time slot.

[0059] In one implementation, according to the number of subtasks of the tasks executable by the mobile device in the t-th time slot, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server, the computing cost function for processing the subtasks of the tasks executable by the mobile device in the t-th time slot is determined, specifically including:

[0060] The computing cost function for processing the subtasks of the tasks executable by the i-th mobile device in the t-th time slot is determined by the following formula:

[0061]

[0062] Among them, represents the computing cost function for processing the subtasks of the tasks executable by the i-th mobile device in the t-th time slot, i = 1, 2, ……, n, and n represents the number of mobile devices;

[0063] α i (t) represents the task offloading decision parameter of the i-th mobile device in the t-th time slot. If α i (t) = 0, then the subtasks included in the tasks generated and processed locally by the i-th mobile device are processed. If α i (t) = 1, then the subtasks included in the tasks generated by the i-th mobile device are offloaded to the j-th mobile edge computing server;

[0064] γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server in the t-th time slot, representing the proportion of the subtasks included in the tasks offloaded by the i-th mobile device and executed by the j-th mobile edge computing server in the t-th time slot; 1 - γ ij (t) represents the proportion of the subtasks included in the tasks offloaded by the i-th mobile device and offloaded by the j-th mobile edge computing server to the cloud server in the t-th time slot, j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0065] A i (t) represents the number of subtasks of the tasks executable by the i-th mobile device in the t-th time slot;

[0066] β1 represents the cost per bit of computing data for the j-th mobile edge computing server in the t-th time slot;

[0067] β2 represents the cost per bit of computing data for the cloud server in the t-th time slot.

[0068] In one implementation, according to the bandwidth of the mobile edge computing server in the t-th time slot, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, and the bandwidth allocation ratio parameter of the mobile edge computing server, determine the communication cost function for processing the subtasks of the executable tasks of the mobile device in the t-th time slot, specifically including:

[0069] Determine the communication cost function for processing the subtasks of the executable tasks of the i-th mobile device in the t-th time slot through the following formula:

[0070]

[0071] where, represents the communication cost function for processing the subtasks of the executable tasks of the i-th mobile device in the t-th time slot, i = 1, 2, ……, n, and n represents the number of mobile devices;

[0072] α i (t) represents the task offloading decision parameter of the i-th mobile device in the t-th time slot. If α i (t) = 0, then the subtasks included in the tasks generated locally by the i-th mobile device are processed. If α i (t) = 1, then the subtasks included in the tasks generated by the i-th mobile device are offloaded to the j-th mobile edge computing server;

[0073] represents the rental cost per hertz of bandwidth;

[0074] η j (t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server in the t-th time slot, j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0075] B e (t) represents the available bandwidth of all mobile edge computing servers;

[0076] B j represents the bandwidth of the j-th mobile edge computing server in the t-th time slot.

[0077] In one implementation, according to the computing cost function and the communication cost function, determine the cost function for processing the subtasks of the executable tasks of the mobile device in the t-th time slot, specifically including:

[0078] The cost function of the subtask for processing the executable task of the \(i\)th mobile device in the \(t\)th time slot is determined by the following formula:

[0079]

[0080] where \(C\) i (t) represents the cost function of the subtask for processing the executable task of the \(i\)th mobile device in the \(t\)th time slot, \(i = 1, 2,\cdots, n\), and \(n\) represents the number of mobile devices;

[0081] represents the computing cost function of the subtask for processing the executable task of the \(i\)th mobile device in the \(t\)th time slot;

[0082] represents the communication cost function of the subtask for processing the executable task of the \(i\)th mobile device in the \(t\)th time slot.

[0083] In one implementation, obtaining the transmission rate between the mobile device and the mobile edge computing server in the \(t\)th time slot specifically includes:

[0084] Obtaining the transmission power of the mobile device and the channel gain between the mobile device and the mobile edge computing server in the \(t\)th time slot;

[0085] Determining the interference noise between the mobile device and the mobile edge computing server in the \(t\)th time slot according to the transmission power of the mobile device and the channel gain between the mobile device and the mobile edge computing server in the \(t\)th time slot;

[0086] Determining the transmission rate between the mobile device and the mobile edge computing server in the \(t\)th time slot according to the bandwidth of the mobile edge computing server in the \(t\)th time slot, the interference noise between the mobile device and the mobile edge computing server, the transmission power of the mobile device, and the channel gain between the mobile device and the mobile edge computing server.

[0087] In one implementation, before constructing the task offloading problem of minimizing cost and maximizing privacy level, it further includes:

[0088] Obtaining the data processing rate of the mobile device and the available computing resources of the mobile edge computing server in the \(t\)th time slot;

[0089] Determining the local processing delay of the mobile device for processing the subtasks of the executable tasks of the mobile device according to the number of subtasks of the executable tasks of the mobile device in the \(t\)th time slot, the task offloading decision parameter of the mobile device, and the data processing rate of the mobile device.

[0090] Determine the processing delay of the subtasks of the mobile device executable task processed by the mobile edge computing server according to the available computing resources of the mobile edge computing server and the number of subtasks of the mobile device executable task in the t time slot;

[0091] Determine the transmission delay of offloading the subtasks of the mobile device executable task from the mobile device to the mobile edge computing server according to the number of subtasks of the mobile device executable task in the t time slot and the transmission rate between the mobile device and the mobile edge computing server;

[0092] Obtain the transmission rate between the mobile edge computing server and the cloud server in the t time slot;

[0093] Determine the transmission delay of offloading the subtasks of the mobile device executable task from the mobile edge computing server to the cloud server according to the number of subtasks of the mobile device executable task in the t time slot and the transmission rate between the mobile edge computing server and the cloud server;

[0094] Obtain the download delay of downloading the processing result from the mobile edge computing server to the mobile device in the t time slot;

[0095] According to the local processing delay of the subtasks of the mobile device executable task processed by the mobile device in the t time slot, the processing delay of the subtasks of the mobile device executable task processed by the mobile edge computing server, the transmission delay of offloading the subtasks of the mobile device executable task from the mobile device to the mobile edge computing server, the transmission delay of offloading the subtasks of the mobile device executable task from the mobile edge computing server to the cloud server, the download delay of downloading the processing result from the mobile edge computing server to the mobile device, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server, determine the processing delay of processing the subtasks included in the mobile device executable task in the t time slot.

[0096] In one implementation, obtaining the transmission rate between the mobile edge computing server and the cloud server in the t time slot specifically includes:

[0097] Obtain the transmission power of the mobile edge computing server in the t time slot, the transmission power of other mobile edge computing servers except the mobile edge computing server, and the channel gain between the mobile edge computing server and the cloud server;

[0098] Determine the interference noise between the other mobile edge computing servers and the cloud server in the t-th time slot according to the transmission power of the other mobile edge computing servers except the mobile edge computing server in the t-th time slot and the channel gain between the mobile edge computing server and the cloud server;

[0099] Determine the transmission rate between the mobile edge computing server and the cloud server in the t-th time slot according to the transmission power of the mobile edge computing server in the t-th time slot, the channel gain between the mobile edge computing server and the cloud server, the interference noise between the other mobile edge computing servers and the cloud server, the available bandwidth of all mobile edge computing servers, and the bandwidth allocation ratio parameter of the mobile edge computing server.

[0100] In one implementation, according to the local processing delay of the subtasks of the mobile device executable task processed by the mobile device in the t-th time slot, the processing delay of the subtasks of the mobile device executable task processed by the mobile edge computing server, the transmission delay of unloading the subtasks of the mobile device executable task from the mobile device to the mobile edge computing server, the transmission delay of unloading the subtasks of the mobile device executable task from the mobile edge computing server to the cloud server, the download delay of downloading the processing result from the mobile edge computing server to the mobile device, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server, determine the processing delay of processing the subtasks included in the mobile device executable task in the t-th time slot, specifically including:

[0101] Calculate the processing delay of processing the subtasks included in the task generated by the i-th mobile device in the t-th time slot through the following formula:

[0102]

[0103] where, T i (t) represents the processing delay of processing the subtasks included in the task generated by the i-th mobile device in the t-th time slot, i = 1, 2, ……, n, and n represents the number of mobile devices;

[0104] α i (t) represents the task offloading decision parameter of the i-th mobile device in the t-th time slot;

[0105] γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server in the t-th time slot, characterizing the proportion of the subtasks included in the task unloaded by the i-th mobile device executed by the j-th mobile edge computing server in the t-th time slot; 1 - γ ij$(t)$ represents the proportion of subtasks included in the task unloaded by the $j$-th mobile edge computing server from the $i$-th mobile device to the cloud server during the $t$-th time slot, where $j = 1, 2, \ldots, m$, and $m$ represents the number of mobile edge computing servers;

[0106] represents the local processing delay of the $i$-th mobile device processing the subtasks of the tasks executable by the $i$-th mobile device during the $t$-th time slot;

[0107] represents the transmission delay of the subtasks of the tasks executable by the $i$-th mobile device unloaded from the $i$-th mobile device to the $j$-th mobile edge computing server during the $t$-th time slot;

[0108] represents the processing delay of the $j$-th mobile edge computing server processing the subtasks of the tasks executable by the $i$-th mobile device during the $t$-th time slot;

[0109] represents the download delay of downloading the processing result from the $j$-th mobile edge computing server to the $i$-th mobile device during the $t$-th time slot;

[0110] represents the transmission delay of the subtasks of the tasks executable by the $i$-th mobile device unloaded from the $j$-th mobile edge computing server to the cloud server during the $t$-th time slot.

[0111] In one implementation, the optimization objective function is:

[0112]

[0113]

[0114] where $E$ represents the mathematical expectation;

[0115] Q i $(t)$ represents the task queue length of the $i$-th mobile device during the $t$-th time slot. The task queue of the $i$-th mobile device is used to store the tasks generated by the $i$-th mobile device in each time slot. represents the maximum length of the task queue of the $i$-th mobile device, where $i = 1, 2, \ldots, n$, and $n$ represents the number of mobile devices;

[0116] represents the queue length of the processing result returned by the $i$-th mobile device during the $t$-th time slot;

[0117] D i $(t)$ represents the number of subtasks included in the task generated by the $i$-th mobile device during the $t$-th time slot;

[0118] P i P(t) represents the privacy level of the i-th mobile device in the t-th time slot;

[0119] C i C(t) represents the cost function for processing the subtasks included in the task generated by the i-th mobile device in the t-th time slot;

[0120] represents the trade-off coefficient between the privacy level and the cost function; T represents the number of time slots;

[0121] α i α(t) represents the task offloading decision parameter of the i-th mobile device in the t-th time slot;

[0122] η j η(t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server in the t-th time slot, where j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0123] γ ij γ(t) represents the task offloading decision parameter of the j-th mobile edge computing server in the t-th time slot, characterizing the proportion of the subtasks included in the task offloaded by the i-th mobile device executed by the j-th mobile edge computing server in the t-th time slot;

[0124] T i T(t) represents the processing delay for processing the subtasks included in the task generated by the i-th mobile device in the t-th time slot, K i represents the task processing delay constraint;

[0125] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, represents the minimum transmission rate between the mobile device and the mobile edge computing server;

[0126] represents the transmission rate between the j-th mobile edge computing server and the cloud server in the t-th time slot, represents the minimum transmission rate between the mobile edge computing server and the cloud server;

[0127] V represents the Lyapunov trade-off factor.

[0128] In one implementation, the optimization objective function is modeled as a Markov decision process, specifically including:

[0129] The state space is set as:

[0130]

[0131] Among them, s(t) represents the state space at the t-th time slot;

[0132] D i (t) represents the number of subtasks included in the task generated by the i-th mobile device at the t-th time slot;

[0133] R j (t) represents the available computing resources of the j-th mobile edge computing server at the t-th time slot;

[0134] B e (t) represents the available bandwidth of all mobile edge computing servers;

[0135] represents the channel gain between the j-th mobile edge computing server and the cloud server;

[0136] The action space is set as:

[0137] a(t) = {α i (t), γ ij (t), η j (t)}

[0138] Among them, a(t) represents the action space at the t-th time slot;

[0139] The reward function is set as:

[0140]

[0141] Among them, R(t) represents the reward function at the t-th time slot;

[0142] The long-term average reward value is set as:

[0143]

[0144] Among them, R represents the long-term average reward value.

[0145] In one embodiment, the optimal task offloading strategy is determined according to a preset deep reinforcement learning algorithm, which specifically includes:

[0146] The preset deep reinforcement learning algorithm is used to solve the Markov decision process to obtain a task offloading strategy prediction model;

[0147] The optimal task offloading strategy is determined according to the task offloading strategy prediction model.

[0148] In a second aspect, an edge computing offloading device provided by an embodiment of the present application includes:

[0149] A first acquisition module, configured to acquire the number of subtasks included in the tasks generated by the mobile device in the t time slot, the number of subtasks of the tasks executable by the mobile device, the transmission rate between the mobile device and the mobile edge computing server, the bandwidth of the mobile edge computing server, and the available bandwidth of all mobile edge computing servers;

[0150] A first determination module, configured to determine the privacy level of the mobile device in the t time slot according to the number of subtasks included in the tasks generated by the mobile device in the t time slot, the number of subtasks of the tasks executable by the mobile device, and the transmission rate between the mobile device and the mobile edge computing server, where the privacy level of the mobile device represents the privacy level of the mobile device;

[0151] A first construction module, configured to construct a cost function for processing the subtasks of the tasks executable by the mobile device in the t time slot according to the number of subtasks of the tasks executable by the mobile device in the t time slot, the bandwidth of the mobile edge computing server, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, the task offloading decision parameter of the mobile edge computing server, and the bandwidth allocation ratio parameter of the mobile edge computing server;

[0152] A second construction module, configured to construct a task offloading problem with minimized cost and maximized privacy level according to the privacy level of the mobile device in the t time slot and the cost function for processing the subtasks of the tasks executable by the mobile device;

[0153] A processing module, configured to convert the task offloading problem into a single-time-slot task offloading strategy optimization problem by using the Lyapunov optimization theory to obtain an optimized objective function;

[0154] A second determination module, configured to model the optimized objective function as a Markov decision process and determine an optimal task offloading strategy according to a preset deep reinforcement learning algorithm.

[0155] In one embodiment, the first determination module is specifically configured to determine the privacy level of the user usage pattern of the mobile device at the t time slot according to the number of subtasks included in the task generated by the mobile device at the t time slot, the number of subtasks of the task executable by the mobile device, and the transmission rate between the mobile device and the mobile edge computing server; determine the location privacy level of the mobile device at the t time slot according to the number of subtasks of the task executable by the mobile device at the t time slot and the transmission rate between the mobile device and the mobile edge computing server; and determine the weighted sum of the privacy level of the user usage pattern of the mobile device and the location privacy level of the mobile device at the t time slot as the privacy level of the mobile device at the t time slot.

[0156] In one embodiment, the task offloading decision parameter of the mobile device characterizes whether to offload the task to the mobile edge computing server for execution, and the task offloading decision parameter of the mobile edge computing server characterizes the proportion of the task offloaded by the mobile device executed by the mobile edge computing server and the proportion of the task offloaded by the mobile device offloaded to the cloud server for execution;

[0157] The first construction module is specifically configured to determine the computational cost function for processing the subtasks of the task executable by the mobile device at the t time slot according to the number of subtasks of the task executable by the mobile device at the t time slot, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server; determine the communication cost function for processing the subtasks of the task executable by the mobile device at the t time slot according to the bandwidth of the mobile edge computing server at the t time slot, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, and the bandwidth allocation ratio parameter of the mobile edge computing server; and determine the cost function for processing the subtasks of the task executable by the mobile device at the t time slot according to the computational cost function and the communication cost function.

[0158] In one embodiment, the second construction module is specifically configured to construct the task offloading problem of maximizing cost and maximizing privacy level as:

[0159]

[0160] where P i (t) represents the privacy level of the i-th mobile device at the t time slot, i = 1, 2, ……, n, and n represents the number of mobile devices;

[0161] C i (t) represents the cost function for processing the subtasks included in the task generated by the i-th mobile device at the t time slot;

[0162] represents the trade - off coefficient between the privacy level and the cost function; T represents the number of time slots;

[0163] α i (t) represents the task offloading decision parameter of the i - th mobile device at the t - th time slot;

[0164] η j (t) represents the bandwidth allocation ratio parameter of the j - th mobile edge computing server at the t - th time slot, j = 1, 2, ……, m, where m represents the number of mobile edge computing servers;

[0165] γ ij (t) represents the task offloading decision parameter of the j - th mobile edge computing server at the t - th time slot, characterizing the proportion of subtasks included in the task offloaded by the j - th mobile edge computing server for the i - th mobile device at the t - th time slot;

[0166] T i (t) represents the processing delay of subtasks included in the task generated by the i - th mobile device at the t - th time slot, K i represents the task processing delay constraint;

[0167] represents the transmission rate between the i - th mobile device and the j - th mobile edge computing server at the t - th time slot, represents the minimum transmission rate between the mobile device and the mobile edge computing server;

[0168] represents the transmission rate between the j - th mobile edge computing server and the cloud server at the t - th time slot, represents the minimum transmission rate between the mobile edge computing server and the cloud server;

[0169] Q i (t) represents the task queue length of the i - th mobile device at the t - th time slot. The task queue of the i - th mobile device is used to store the tasks generated by the i - th mobile device in each time slot, represents the maximum length of the task queue of the i - th mobile device.

[0170] In one implementation, the first determination module is specifically configured to calculate the user usage pattern privacy level of the i - th mobile device at the t - th time slot through the following formula:

[0171]

[0172] where P i,U(t) represents the privacy level of the usage pattern of the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, and n represents the number of mobile devices;

[0173] D i (t) represents the number of subtasks included in the task generated by the i-th mobile device in the t-th time slot;

[0174] A i (t) represents the number of subtasks of the task that the i-th mobile device can execute in the t-th time slot;

[0175] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, where j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0176] r thr represents the transmission rate threshold from the i-th mobile device to the j-th mobile edge computing server;

[0177] ∏(·) is an indicator function. If holds, then If does not hold, then

[0178] In one implementation, the first determination module is specifically configured to calculate the location privacy level of the i-th mobile device in the t-th time slot through the following formula:

[0179]

[0180] where P i,L (t) represents the location privacy level of the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, and n represents the number of mobile devices;

[0181] A i (t) represents the number of subtasks of the task that the i-th mobile device can execute in the t-th time slot;

[0182] ∏(·) is an indicator function. If A i (t)>0 holds, then ∏(A i (t)>0)=1. If A i (t)>0 does not hold, then ∏(A i (t)>0)=0;

[0183] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, where j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0184] r thr represents the transmission rate threshold from the i-th mobile device to the j-th mobile edge computing server;

[0185] If holds, then If does not hold, then

[0186] In one implementation, the first determination module is specifically configured to calculate the privacy level of the i-th mobile device in the t-th time slot through the following formula:

[0187] P i (t) = λ1P i,U (t) + λ2P i,L (t)

[0188] where P i (t) represents the privacy level of the i-th mobile device in the t-th time slot, i = 1, 2, ……, n, and n represents the number of mobile devices;

[0189] P i,U (t) represents the user usage pattern privacy level of the i-th mobile device in the t-th time slot, and λ1 represents the weight of the user usage pattern privacy level of the i-th mobile device in the t-th time slot;

[0190] P i,L (t) represents the location privacy level of the i-th mobile device in the t-th time slot, and λ2 represents the weight of the location privacy level of the i-th mobile device in the t-th time slot.

[0191] In one implementation, the first construction module is specifically configured to determine the computational cost function of the subtasks for processing the executable tasks of the i-th mobile device in the t-th time slot through the following formula:

[0192]

[0193] where represents the computational cost function of the subtasks for processing the executable tasks of the i-th mobile device in the t-th time slot, i = 1, 2, ……, n, and n represents the number of mobile devices;

[0194] α i (t) represents the task offloading decision parameter of the i-th mobile device in the t-th time slot. If α i (t) = 0, then the subtasks included in the tasks generated by local processing on the i-th mobile device. If α iIf γ(t) = 1, then the subtasks included in the task generated by the i-th mobile device are offloaded to the j-th mobile edge computing server;

[0195] γ ij γ(t) represents the task offloading decision parameter of the j-th mobile edge computing server in the t-th time slot, characterizing the proportion of the subtasks included in the task offloaded by the j-th mobile edge computing server from the i-th mobile device in the t-th time slot; 1 - γ ij (t) characterizes the proportion of the subtasks included in the task offloaded by the j-th mobile edge computing server from the i-th mobile device that are offloaded to the cloud server in the t-th time slot, j = 1, 2, ……, m, where m represents the number of mobile edge computing servers;

[0196] A i A(t) represents the number of subtasks of the task that can be executed by the i-th mobile device in the t-th time slot;

[0197] β1 represents the cost per bit of computing data of the j-th mobile edge computing server in the t-th time slot;

[0198] β2 represents the cost per bit of computing data of the cloud server in the t-th time slot.

[0199] In one implementation, the first construction module is specifically configured to determine the communication cost function for processing the subtasks of the task that can be executed by the i-th mobile device in the t-th time slot through the following formula:

[0200]

[0201] Among them, represents the communication cost function for processing the subtasks of the task that can be executed by the i-th mobile device in the t-th time slot, i = 1, 2, ……, n, where n represents the number of mobile devices;

[0202] α i α(t) represents the task offloading decision parameter of the i-th mobile device in the t-th time slot. If α i (t) = 0, then the subtasks included in the task generated locally by the i-th mobile device are processed. If α i (t) = 1, then the subtasks included in the task generated by the i-th mobile device are offloaded to the j-th mobile edge computing server;

[0203] represents the rental cost per hertz of bandwidth;

[0204] η j(t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server in the t-th time slot, where j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0205] B e (t) represents the available bandwidth of all the mobile edge computing servers;

[0206] B j represents the bandwidth of the j-th mobile edge computing server in the t-th time slot.

[0207] In one embodiment, the first construction module is specifically configured to determine the cost function of the subtask for processing the executable task of the i-th mobile device in the t-th time slot through the following formula:

[0208]

[0209] where C i (t) represents the cost function of the subtask for processing the executable task of the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, and n represents the number of mobile devices;

[0210] represents the computing cost function of the subtask for processing the executable task of the i-th mobile device in the t-th time slot;

[0211] represents the communication cost function of the subtask for processing the executable task of the i-th mobile device in the t-th time slot.

[0212] In one embodiment, the first acquisition module is specifically configured to acquire the transmission power of the mobile device and the channel gain between the mobile device and the mobile edge computing server in the t-th time slot; determine the interference noise between the mobile device and the mobile edge computing server in the t-th time slot according to the transmission power of the mobile device and the channel gain between the mobile device and the mobile edge computing server in the t-th time slot; and determine the transmission rate between the mobile device and the mobile edge computing server in the t-th time slot according to the bandwidth of the mobile edge computing server, the interference noise between the mobile device and the mobile edge computing server, the transmission power of the mobile device, and the channel gain between the mobile device and the mobile edge computing server.

[0213] In one embodiment, the device further includes:

[0214] A second acquisition module, configured to acquire the data processing rate of the mobile device and the available computing resources of the mobile edge computing server at the t-th time slot before constructing a task offloading problem with minimized construction cost and maximized privacy level;

[0215] A third determination module, configured to determine the local processing delay of the mobile device for processing the subtasks of the tasks executable by the mobile device according to the number of subtasks of the tasks executable by the mobile device at the t-th time slot, the task offloading decision parameters of the mobile device, and the data processing rate of the mobile device;

[0216] A fourth determination module, configured to determine the processing delay of the mobile edge computing server for processing the subtasks of the tasks executable by the mobile device according to the available computing resources of the mobile edge computing server at the t-th time slot and the number of subtasks of the tasks executable by the mobile device;

[0217] A fifth determination module, configured to determine the transmission delay of offloading the subtasks of the tasks executable by the mobile device from the mobile device to the mobile edge computing server according to the number of subtasks of the tasks executable by the mobile device at the t-th time slot and the transmission rate between the mobile device and the mobile edge computing server;

[0218] A third acquisition module, configured to acquire the transmission rate between the mobile edge computing server and the cloud server at the t-th time slot;

[0219] A sixth determination module, configured to determine the transmission delay of offloading the subtasks of the tasks executable by the mobile device from the mobile edge computing server to the cloud server according to the number of subtasks of the tasks executable by the mobile device at the t-th time slot and the transmission rate between the mobile edge computing server and the cloud server;

[0220] A fourth acquisition module, configured to acquire the download delay of downloading the processing result from the mobile edge computing server to the mobile device at the t-th time slot;

[0221] A seventh determination module, configured to determine, according to the local processing delay of the subtasks of the mobile device executable task processed by the mobile device in the t time slot, the processing delay of the subtasks of the mobile device executable task processed by the mobile edge computing server, the transmission delay of offloading the subtasks of the mobile device executable task from the mobile device to the mobile edge computing server, the transmission delay of offloading the subtasks of the mobile device executable task from the mobile edge computing server to the cloud server, the download delay of downloading the processing result from the mobile edge computing server to the mobile device, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server, the processing delay of processing the subtasks included in the mobile device executable task in the t time slot.

[0222] In one embodiment, the third acquisition module is specifically configured to acquire the transmission power of the mobile edge computing server in the t time slot, the transmission power of other mobile edge computing servers except the mobile edge computing server, and the channel gain between the mobile edge computing server and the cloud server; determine the interference noise between the other mobile edge computing server and the cloud server in the t time slot according to the transmission power of other mobile edge computing servers except the mobile edge computing server in the t time slot and the channel gain between the mobile edge computing server and the cloud server; determine the transmission rate between the mobile edge computing server and the cloud server in the t time slot according to the transmission power of the mobile edge computing server in the t time slot, the channel gain between the mobile edge computing server and the cloud server, the interference noise between the other mobile edge computing server and the cloud server, the available bandwidth of all mobile edge computing servers, and the bandwidth allocation ratio parameter of the mobile edge computing server.

[0223] In one embodiment, the seventh determination module is specifically configured to calculate the processing delay of processing the subtasks included in the task generated by the i-th mobile device in the t time slot through the following formula:

[0224]

[0225] where, T i (t) represents the processing delay of processing the subtasks included in the task generated by the i-th mobile device in the t time slot, i = 1, 2,..., n, and n represents the number of mobile devices;

[0226] α i (t) represents the task offloading decision parameter of the i-th mobile device in the t time slot;

[0227] γ ij$(t)$ represents the task offloading decision parameter of the $j$-th mobile edge computing server in the $t$-th time slot, characterizing the proportion of subtasks included in the task offloaded by the $i$-th mobile device to the $j$-th mobile edge computing server in the $t$-th time slot; $1 - \gamma$ ij $(t)$ characterizes the proportion of subtasks included in the task offloaded by the $i$-th mobile device to the $j$-th mobile edge computing server that are offloaded to the cloud server in the $t$-th time slot, $j = 1, 2, \ldots, m$, where $m$ represents the number of mobile edge computing servers;

[0228] represents the local processing delay of the subtasks of the tasks executable by the $i$-th mobile device processed by the $i$-th mobile device in the $t$-th time slot;

[0229] represents the transmission delay of the subtasks of the tasks executable by the $i$-th mobile device offloaded from the $i$-th mobile device to the $j$-th mobile edge computing server in the $t$-th time slot;

[0230] represents the processing delay of the subtasks of the tasks executable by the $i$-th mobile device processed by the $j$-th mobile edge computing server in the $t$-th time slot;

[0231] represents the download delay of the processing results downloaded from the $j$-th mobile edge computing server to the $i$-th mobile device in the $t$-th time slot;

[0232] represents the transmission delay of the subtasks of the tasks executable by the $i$-th mobile device offloaded from the $j$-th mobile edge computing server to the cloud server in the $t$-th time slot.

[0233] In one implementation, the optimization objective function is:

[0234]

[0235] where $E$ represents the mathematical expectation;

[0236] $Q$ i $(t)$ represents the task queue length of the $i$-th mobile device in the $t$-th time slot, and the task queue of the $i$-th mobile device is used to store the tasks generated by the $i$-th mobile device in each time slot, represents the maximum length of the task queue of the $i$-th mobile device, $i = 1, 2, \ldots, n$, where $n$ represents the number of mobile devices;

[0237] represents the queue length of the processing results returned by the $i$-th mobile device in the $t$-th time slot;

[0238] D i D(t) represents the number of subtasks included in the task generated by the i-th mobile device in the t-th time slot;

[0239] P i P(t) represents the privacy level of the i-th mobile device in the t-th time slot;

[0240] C i C(t) represents the cost function of processing the subtasks included in the task generated by the i-th mobile device in the t-th time slot;

[0241] represents the trade-off coefficient between the privacy level and the cost function; T represents the number of time slots;

[0242] α i α(t) represents the task offloading decision parameter of the i-th mobile device in the t-th time slot;

[0243] η j η(t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server in the t-th time slot, j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0244] γ ij γ(t) represents the task offloading decision parameter of the j-th mobile edge computing server in the t-th time slot, representing the proportion of the subtasks included in the task offloaded by the i-th mobile device executed by the j-th mobile edge computing server in the t-th time slot;

[0245] T i T(t) represents the processing delay of processing the subtasks included in the task generated by the i-th mobile device in the t-th time slot, K i represents the task processing delay constraint;

[0246] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, represents the minimum transmission rate between the mobile device and the mobile edge computing server;

[0247] represents the transmission rate between the j-th mobile edge computing server and the cloud server in the t-th time slot, represents the minimum transmission rate between the mobile edge computing server and the cloud server;

[0248] V represents the Lyapunov trade-off factor.

[0249] In one implementation manner, the second determination module is specifically configured to:

[0250] The state space is set as:

[0251]

[0252] where s(t) represents the state space at the t-th time slot;

[0253] D i (t) represents the number of subtasks included in the task generated by the i-th mobile device at the t-th time slot;

[0254] R j (t) represents the available computing resources of the j-th mobile edge computing server at the t-th time slot;

[0255] B e (t) represents the available bandwidth of all mobile edge computing servers;

[0256] represents the channel gain between the j-th mobile edge computing server and the cloud server;

[0257] The action space is set as:

[0258] a(t) = {α i (t), γ ij (t), η j (t)}

[0259] where a(t) represents the action space at the t-th time slot;

[0260] The reward function is set as:

[0261]

[0262] where R(t) represents the reward function at the t-th time slot;

[0263] The long-term average reward value is set as:

[0264]

[0265] where R represents the long-term average reward value.

[0266] In one implementation manner, the second determination module is specifically configured to solve the Markov decision process by using the preset deep reinforcement learning algorithm to obtain a task offloading policy prediction model; and determine an optimal task offloading policy according to the task offloading policy prediction model.

[0267] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the edge computing offloading method described in the present application is implemented.

[0268] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps in the edge computing offloading method described in the present application are implemented.

[0269] The beneficial effects of the present application are as follows:

[0270] The edge computing offloading method, device, electronic device and storage medium provided by the embodiments of the present application obtain the number of subtasks included in the tasks generated by the mobile device (MD) in the t time slot, the number of subtasks of the tasks executable by the mobile device, the transmission rate between the mobile device and the mobile edge computing server, the bandwidth of the mobile edge computing server, and the available bandwidth of all mobile edge computing servers; determine the privacy level of the mobile device in the t time slot according to the number of subtasks included in the tasks generated by the mobile device in the t time slot, the number of subtasks of the tasks executable by the mobile device, and the transmission rate between the mobile device and the mobile edge computing server, where the privacy level of the mobile device represents the privacy level of the mobile device; construct a cost function for processing the subtasks of the tasks executable by the mobile device in the t time slot according to the number of subtasks of the tasks executable by the mobile device in the t time slot, the bandwidth of the mobile edge computing server, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, the task offloading decision parameter of the mobile edge computing server, and the bandwidth allocation ratio parameter of the mobile edge computing server; construct a task offloading problem that minimizes cost and maximizes privacy level according to the privacy level of the mobile device in the t time slot and the cost function for processing the subtasks of the tasks executable by the mobile device; use the Lyapunov optimization theory to transform the task offloading problem into a single-time slot task offloading strategy optimization problem to obtain an optimized objective function; model the optimized objective function as a Markov decision process, and determine the optimal task offloading strategy according to a preset deep reinforcement learning algorithm. In the embodiments of the present application, the privacy level of the mobile device and the cost of processing the tasks generated by the mobile device are comprehensively considered. According to the privacy level of the mobile device and the cost function for processing the subtasks of the tasks executable by the mobile device, a task offloading problem that minimizes cost and maximizes privacy level is constructed. Since the objective of this task offloading problem is a long-term objective and cannot be directly solved, to solve this task offloading problem, the present application uses the Lyapunov optimization theory to decouple the original problem into a task offloading strategy optimization problem within a single time slot to obtain an optimized objective function to ensure the stability of the system. Considering that the transformed single-time slot task offloading strategy optimization problem has non-convex characteristics, the present application further introduces a Markov decision process to model the optimized objective function and solves it according to a preset deep reinforcement learning algorithm to obtain the optimal task offloading strategy. Thus, while ensuring the stability of the system, the total cost is minimized and the maximization of user privacy protection is achieved. While improving the security of edge computing offloading, the accuracy of the task offloading strategy is improved.

[0271] Other features and advantages of the present application will be set forth in the following description, and in part will be obvious from the description, or may be learned by practice of the present application. The objectives and other advantages of the present application may be realized and attained by the structure particularly pointed out in the written description, claims, as well as the drawings. Description of the Drawings

[0272] The drawings described herein are for further understanding of the present application, and form a part of the present application. The schematic embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation to the present application. In the drawings:

[0273] Figure 1 It is a schematic diagram of an application scenario of the edge computing offloading method provided by an embodiment of the present application;

[0274] Figure 2 It is a schematic flowchart of the edge computing offloading method provided by an embodiment of the present application;

[0275] Figure 3 It is a schematic flowchart of obtaining the transmission rate between a mobile device and a mobile edge computing server at time slot t provided by an embodiment of the present application;

[0276] Figure 4 It is a schematic flowchart of determining the privacy level of a mobile device at time slot t provided by an embodiment of the present application;

[0277] Figure 5 It is a schematic flowchart of constructing a cost function of subtasks for processing executable tasks of a mobile device at time slot t provided by an embodiment of the present application;

[0278] Figure 6 It is a schematic flowchart of determining the processing delay of subtasks included in an executable task of a mobile device at time slot t provided by an embodiment of the present application;

[0279] Figure 7 It is a schematic flowchart of obtaining the transmission rate between a mobile edge computing server and a cloud server at time slot t provided by an embodiment of the present application;

[0280] Figure 8 It is a schematic flowchart of determining an optimal task offloading policy provided by an embodiment of the present application;

[0281] Figure 9 It is a schematic structural diagram of an edge computing offloading device provided by an embodiment of the present application;

[0282] Figure 10 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0283] To solve the problems of low security and poor task offloading strategies in existing edge computing offloading methods, the embodiments of the present application provide an edge computing offloading method, device, electronic device, and storage medium.

[0284] The following describes the preferred embodiments of the present application with reference to the accompanying drawings of the specification. It should be understood that the preferred embodiments described herein are only for the purpose of illustrating and explaining the present application, and are not used to limit the present application. And without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0285] In this article, it should be understood that among the technical terms involved in the present application:

[0286] 1. Mobile Edge Computing: It is a distributed computing method based on the mobile communication network. By pushing computing resources and services to the network edge, close to the terminal device and the user, it can achieve fast response, low latency, and high reliability.

[0287] 2. Deep Neural Network (DNN): It is a class of neural network models with a multi-layer structure in the field of machine learning. Through the connection and training of multiple layers of neurons (or nodes), it extracts high-level features from the input data, and then completes complex non-linear problems.

[0288] First, refer to Figure 1, which is a schematic diagram of an application scenario of the edge computing offloading method provided by an embodiment of the present application. It is a cloud-edge collaboration scenario. The edge computing system may include: a cloud service layer, an edge layer, and an end-user layer. Among them, the cloud service layer includes a cloud server (Cloud Server, CS) 101, and the cloud server 101 is used for the allocation, scheduling, and processing of the overall computing resources of the tasks offloaded by the edge layer. The edge layer includes m mobile edge computing servers (i.e., MEC servers) 102, and the end-user layer includes n mobile devices 103. Each mobile device 103 includes two cache queues: a task queue 104 and a return queue 105. The task queue 104 is used to store the tasks generated by the mobile device 103, and the return queue 105 is used to store the returned task processing result data. The mobile edge computing server 102 is deployed at a position closer to the mobile device 103 (i.e., a position closer to the data source) to shorten the data transmission delay with the mobile device 103 and improve the response speed. After the cloud server 101 determines the task offloading policy, it issues the task offloading policy to the corresponding mobile device 103 and mobile edge computing server 102, so that the mobile device 103 and the mobile edge computing server 102 execute the task offloading policy. This application scenario may further include a base station 106 and an agent 107. The agent 107 can replace the cloud server for the allocation and scheduling of the overall computing resources. After the agent 107 determines the task offloading policy, it sends the task offloading policy to the cloud server 101 through the base station 106. The cloud server 101 issues the task offloading policy to the corresponding mobile device 103 and mobile edge computing server 102, so that the mobile device 103 and the mobile edge computing server 102 execute the task offloading policy. At this time, the cloud server 101 can be only used to process the tasks offloaded by the mobile edge computing server 102, thereby saving the computing resources of the cloud server and improving the task processing efficiency. The embodiments of the present application do not limit this. In each time slot t, if the mobile edge computing server 102 receives the tasks offloaded by the mobile device 103, the mobile edge computing server 102 can offload some or all of the tasks to the cloud server 101 according to the dynamic network environment according to the task offloading policy, and the cloud server 101 processes them to save the computing resources of the mobile edge computing server 102 through the wireless network connection.

[0289] In the embodiments of the present application, the mobile device 103 may be a mobile terminal device, and may include, but is not limited to, smart phones, tablet computers, etc. The embodiments of the present application do not limit this.

[0290] Based on the above application scenario, the following will refer to the attached Figures 2 to 8To describe the exemplary embodiments of the present application in more detail, it should be noted that the above application scenarios are only shown for the convenience of understanding the spirit and principle of the present application, and the embodiments of the present application are not limited by any of them. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0291] As Figure 2 shown, it is a schematic flowchart of the implementation process of the edge computing offloading method provided by the embodiment of the present application. This edge computing offloading method can be applied to the above cloud server 101 or the agent 107. Hereinafter, only the application to the cloud server 101 will be taken as an example for illustration, and it may specifically include the following steps:

[0292] S21. Obtain the number of subtasks included in the task generated by the mobile device in the t time slot, the number of subtasks that the mobile device can execute for the task, the transmission rate between the mobile device and the mobile edge computing server, the bandwidth of the mobile edge computing server, and the available bandwidth of all mobile edge computing servers.

[0293] Specifically, each mobile device can generate a task within the time slot t. This task can be, but is not limited to, a DNN task (i.e., a DNN training task). A task can be divided into multiple subtasks. Taking the DNN task as an example, the training task of each layer of the DNN model can be used as a subtask, that is: the number of layers included in the DNN model is the number of subtasks, and the number of subtasks included in the task generated by the i-th mobile device can be denoted as D i (t) (that is: the computing load of the i-th mobile device), i = 1, 2,..., n, N = {1, 2,..., n} represents the set of mobile devices, and n is the number of mobile devices. M = {1, 2,..., m} represents the set of mobile edge computing servers, and m is the number of mobile edge computing servers. At the t time slot, the number of subtasks that the i-th mobile device can execute for the task is A i (t). The data result set of each processing result returned to the queue can be represented. The size of the inference result data is the same as the output data size of the last layer of the DNN model. Define as the inference result set that the mobile device can download.

[0294] In implementation, after the mobile device generates a task at the t time slot, it can actively report to the cloud server the number of subtasks included in the task and the number of subtasks that the mobile device can execute for the task at the t time slot. The cloud server can also actively obtain from the mobile device the number of subtasks included in the task generated by the mobile device at the t time slot and the number of subtasks that the mobile device can execute for the task at the t time slot according to the time slot. The embodiments of the present application do not limit this.

[0295] In implementation, it can be in accordance with as Figure 3The process shown obtains the transmission rate between the mobile device and the mobile edge computing server in time slot t, including the following steps:

[0296] S31. Obtain the transmission power of the mobile device in time slot t and the channel gain between the mobile device and the mobile edge computing server.

[0297] Specifically, the transmission power of the i-th mobile device can be denoted as The channel gain of the connection between the mobile device and the mobile edge computing server remains constant within the same time slot t. The channel gain between the i-th mobile device and the j-th mobile edge computing server can be denoted as

[0298] S32. Determine the interference noise between the mobile device and the mobile edge computing server in time slot t according to the transmission power of the mobile device in time slot t and the channel gain between the mobile device and the mobile edge computing server.

[0299] Specifically, the interference noise between the i-th mobile device and the j-th mobile edge computing server comes from the connections of other mobile devices on the same channel. The interference noise between the i-th mobile device and the j-th mobile edge computing server in time slot t can be calculated by the following formula:

[0300]

[0301] Where represents the interference noise between the i-th mobile device and the j-th mobile edge computing server in time slot t, i = 1, 2, ……, n, n is the number of mobile devices, j = 1, 2, ……, m, m is the number of mobile edge computing servers;

[0302] represents the transmission power of the i-th mobile device in time slot t;

[0303] represents the channel gain between the i-th mobile device and the j-th mobile edge computing server in time slot t.

[0304] S33. Determine the transmission rate between the mobile device and the mobile edge computing server in time slot t according to the bandwidth of the mobile edge computing server in time slot t, the interference noise between the mobile device and the mobile edge computing server, the transmission power of the mobile device, and the channel gain between the mobile device and the mobile edge computing server.

[0305] Specifically, the transmission rate between the i-th mobile device and the j-th mobile edge computing server in time slot t can be calculated by the following formula:

[0306]

[0307] Among them, represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot;

[0308] B j represents the bandwidth of the j-th mobile edge computing server in the t-th time slot;

[0309] σ 2 represents the average Gaussian white noise.

[0310] In implementation, the cloud server can obtain the bandwidth of the mobile edge computing server and the available bandwidth of all mobile edge computing servers according to time slots. The available bandwidth of all mobile edge computing servers can be denoted as B e (t).

[0311] S22. Determine the privacy level of the mobile device in the t-th time slot according to the number of subtasks included in the task generated by the mobile device in the t-th time slot, the number of subtasks that the mobile device can execute the task, and the transmission rate between the mobile device and the mobile edge computing server.

[0312] Among them, the privacy level of the mobile device characterizes the privacy level of the mobile device.

[0313] In implementation, it can be determined according to the process as Figure 4 shown to determine the privacy level of the mobile device in the t-th time slot, including the following steps:

[0314] S41. Determine the privacy level of the user usage pattern of the mobile device in the t-th time slot according to the number of subtasks included in the task generated by the mobile device in the t-th time slot, the number of subtasks that the mobile device can execute the task, and the transmission rate between the mobile device and the mobile edge computing server.

[0315] In specific implementation, the user usage pattern can include but is not limited to the following patterns: the time period when tasks arrive, the behavior of the user's mobile device executing DNN tasks, etc. Unreliable mobile edge computing servers and cloud servers will record the number of tasks generated by each mobile device. The number of subtasks D included in the task generated by the i-th mobile device i(t), if the channel condition is continuously quantified, then after the time slot t > 0, the number of tasks in the task queue and the return queue of the i-th mobile device will become 0. In this case, the number of subtasks contained in the tasks generated by the i-th mobile device is the same as the number of subtasks contained in the tasks actually offloaded to the mobile edge computing server. That is, the mobile device offloads all tasks to the mobile edge computing server for processing, which will cause the mobile edge computing to monitor the mobile device by recording the arrival intensity of tasks. Using these historical records, an attacker can further infer the user's usage pattern, resulting in privacy leakage.

[0316] Specifically, the privacy level of the user usage pattern of the i-th mobile device in the t time slot can be calculated by the following formula:

[0317]

[0318] Among them, P i,U (t) represents the privacy level of the user usage pattern of the i-th mobile device in the t time slot, i = 1, 2, ……, n, and n represents the number of mobile devices;

[0319] D i (t) represents the number of subtasks contained in the tasks generated by the i-th mobile device in the t time slot;

[0320] A i (t) represents the number of subtasks of the tasks that the i-th mobile device can execute in the t time slot;

[0321] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t time slot, j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0322] r thr represents the transmission rate critical value from the i-th mobile device to the j-th mobile edge computing server;

[0323] ∏(·) is an indicator function. If holds, then If does not hold, then

[0324] The higher the privacy level of the user usage pattern of the mobile device, the less likely the user's usage pattern privacy will be leaked. In order to enhance the privacy level P of the user usage pattern i,U(t), that is, to reduce the risk of privacy leakage, it is necessary to deliberately increase the difference between the number of tasks generated by the mobile device and the number of tasks it offloads to the mobile edge computing server. That is to say, even though the channel condition is good enough, some subtasks of the current tasks still need to be processed on the local mobile device to expand the difference between the two.

[0325] S42. Determine the location privacy level of the mobile device in the t time slot according to the number of subtasks of the tasks executable by the mobile device in the t time slot and the transmission rate between the mobile device and the mobile edge computing server.

[0326] When the channel condition is good enough, the mobile device may choose to offload tasks to the mobile edge computing server for execution. However, when the channel condition is very poor, the mobile device tends to execute locally or keep the tasks in the cache queue. In this case, the attacker cannot monitor the tasks of the mobile terminal through the mobile edge computing server, but this also indicates that the mobile device is relatively far from the mobile edge computing server because the channel condition quality is related to the distance between the mobile device and the mobile edge server. In addition, if multiple mobile edge computing servers cooperate, the attacker can more accurately determine the user's location. Therefore, the general task offloading strategy will lead to the leakage of the user's location information.

[0327] In specific implementation, the location privacy level of the i-th mobile device in the t time slot can be calculated by the following formula:

[0328]

[0329] Among them, P i,L (t) represents the location privacy level of the i-th mobile device in the t time slot, i = 1, 2,..., n, and n represents the number of mobile devices;

[0330] A i (t) represents the number of subtasks of the tasks executable by the i-th mobile device in the t time slot;

[0331] ∏(·) is an indicator function. If A i (t) > 0 holds, then ∏(A i (t) > 0) = 1. If A i (t) > 0 does not hold (that is: A i (t) ≤ 0), then ∏(A i (t) > 0) = 0;

[0332] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t time slot, j = 1, 2,..., m, and m represents the number of mobile edge computing servers;

[0333] r thr represents the transmission rate threshold from the i-th mobile device to the j-th mobile edge computing server;

[0334] If holds, then If does not hold (i.e., then

[0335] S43. Determine the weighted sum of the user usage pattern privacy level and the location privacy level of the mobile device at time slot t as the privacy level of the mobile device at time slot t.

[0336] To avoid the leakage of user location privacy, although the channel condition is very poor, the mobile device still needs to offload tasks to the mobile edge computing server.

[0337] Specifically, the privacy level of the i-th mobile device at time slot t can be calculated by the following formula:

[0338] P i (t) = λ1P i,U (t) + λ2P i,L (t)

[0339] where, P i (t) represents the privacy level of the i-th mobile device at time slot t, i = 1, 2,..., n, and n represents the number of mobile devices;

[0340] P i,U (t) represents the user usage pattern privacy level of the i-th mobile device at time slot t, and λ1 represents the weight of the user usage pattern privacy level of the i-th mobile device at time slot t;

[0341] P i,L (t) represents the location privacy level of the i-th mobile device at time slot t, and λ2 represents the weight of the location privacy level of the i-th mobile device at time slot t, and λ1 + λ2 = 1.

[0342] In implementation, λ1 and λ2 can be set according to actual needs. For example, λ1 = 0.6 and λ2 = 0.4 can be set, and the embodiments of the present application do not limit this. P i (t) The larger the value, the smaller the risk of privacy leakage.

[0343] S23. Construct a cost function for processing the subtasks of the tasks executable by the mobile device in time slot t based on the number of subtasks of the tasks executable by the mobile device in time slot t, the bandwidth of the mobile edge computing server, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, the task offloading decision parameter of the mobile edge computing server, and the bandwidth allocation ratio parameter of the mobile edge computing server.

[0344] In specific implementation, the task offloading decision parameter of the mobile device represents whether to offload the task to the mobile edge computing server for execution, and the task offloading decision parameter of the mobile edge computing server represents the proportion of the tasks offloaded by the mobile device that the mobile edge computing server executes and the proportion of the tasks offloaded by the mobile device that are offloaded to the cloud server for execution.

[0345] Set the task offloading decision parameter of the i-th mobile device to α i (t), where α i (t) is a binary variable that can take values of 0 or 1, indicating whether the A i (t) subtasks included in the tasks executable by the i-th mobile device in time slot t are offloaded to the j-th mobile edge computing server for execution in time slot t, that is:

[0346]

[0347] Set the task offloading decision parameter of the j-th mobile edge computing server in time slot t to γ ij (t), where γ ij (t) represents the proportion of the subtasks included in the tasks offloaded by the i-th mobile device that the j-th mobile edge computing server executes in time slot t. That is, when the subtasks of the tasks generated by the i-th mobile device are offloaded to the j-th mobile edge server for processing, the proportion of the subtasks included in the tasks offloaded by the i-th mobile device that the j-th mobile edge server executes, and 1 - γ ij (t) represents the proportion of the subtasks included in the tasks offloaded by the i-th mobile device that the j-th mobile edge server offloads to the cloud server for execution.

[0348] Set the bandwidth allocation ratio parameter of the j-th mobile edge computing server among the m mobile edge computing servers in time slot t to η j (t),

[0349] α i γ ij (t) and η j (t) are three variables.

[0350] In implementation, it can be carried out as follows Figure 5The described process constructs a cost function for processing subtasks of tasks executable by a mobile device in a t time slot, including the following steps:

[0351] S51. Determine the computational cost function for processing subtasks of tasks executable by the mobile device in the t time slot according to the number of subtasks of tasks executable by the mobile device in the t time slot, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server.

[0352] In specific implementation, the computational cost function for processing the subtasks of the i-th task executable by the mobile device in the t time slot can be determined by the following formula:

[0353]

[0354] Wherein, represents the computational cost function for processing the subtasks of the i-th task executable by the mobile device in the t time slot, i = 1, 2,..., n, and n represents the number of mobile devices;

[0355] α i (t) represents the task offloading decision parameter of the i-th mobile device in the t time slot. If α i (t) = 0, then the subtasks included in the task generated and processed locally by the i-th mobile device. If α i (t) = 1, then the subtasks included in the task generated by the i-th mobile device are offloaded to the j-th mobile edge computing server;

[0356] γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server in the t time slot, characterizing the proportion of the subtasks included in the task executed by the j-th mobile edge computing server for the i-th mobile device offloading in the t time slot; 1 - γ ij (t) characterizes the proportion of the subtasks included in the task offloaded by the i-th mobile device by the j-th mobile edge computing server and offloaded to the cloud server in the t time slot, j = 1, 2,..., m, and m represents the number of mobile edge computing servers;

[0357] A i (t) represents the number of subtasks of the task executable by the i-th mobile device in the t time slot;

[0358] β1 represents the cost per bit of computing data of the j-th mobile edge computing server in the t time slot;

[0359] β2 represents the cost per bit of computing data of the cloud server in the t time slot.

[0360] β1 and β2 can be set according to requirements, and the embodiments of the present application do not limit this.

[0361] S52. Determine the communication cost function of the subtasks for processing the executable tasks of the mobile device at time slot t according to the bandwidth of the mobile edge computing server at time slot t, the available bandwidth of all mobile edge computing servers, the task offloading decision parameters of the mobile device, and the bandwidth allocation ratio parameters of the mobile edge computing server.

[0362] Specifically, the communication cost function of the subtasks for processing the executable tasks of the i-th mobile device at time slot t can be determined by the following formula:

[0363]

[0364] where represents the communication cost function of the subtasks for processing the executable tasks of the i-th mobile device at time slot t, i = 1, 2, ……, n, and n represents the number of mobile devices;

[0365] α i (t) represents the task offloading decision parameter of the i-th mobile device at time slot t. If α i (t) = 0, then the subtasks included in the tasks generated locally by the i-th mobile device are processed. If α i (t) = 1, then the subtasks included in the tasks generated by the i-th mobile device are offloaded to the j-th mobile edge computing server;

[0366] represents the rental cost per hertz of the bandwidth;

[0367] η j (t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server at time slot t, j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0368] B e (t) represents the available bandwidth of all mobile edge computing servers;

[0369] B j represents the bandwidth of the j-th mobile edge computing server at time slot t.

[0370] S53. Determine the cost function of the subtasks for processing the executable tasks of the mobile device at time slot t according to the computing cost function and the communication cost function.

[0371] Specifically, the cost function of the subtasks for processing the executable tasks of the i-th mobile device at time slot t can be determined by the following formula:

[0372]

[0373] Among them, C i (t) represents the cost function of processing the subtasks of the executable task of the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, and n represents the number of mobile devices;

[0374] represents the computing cost function of processing the subtasks of the executable task of the i-th mobile device in the t-th time slot;

[0375] represents the communication cost function of processing the subtasks of the executable task of the i-th mobile device in the t-th time slot.

[0376] S24. Construct a task offloading problem that minimizes cost and maximizes privacy level according to the privacy level of the mobile device in the t-th time slot and the cost function of processing the subtasks of the executable task of the mobile device.

[0377] Before constructing a task offloading problem that minimizes cost and maximizes privacy level, it is also necessary to obtain the processing delay of the subtasks included in the task generated by the mobile device in the t-th time slot, and use this processing delay as a constraint condition for the task offloading problem.

[0378] In specific implementation, it can be determined according to the process as Figure 6 shown to obtain the processing delay of the subtasks included in the executable task of the mobile device in the t-th time slot, including the following steps:

[0379] S61. Obtain the data processing rate of the mobile device and the available computing resources of the mobile edge computing server in the t-th time slot.

[0380] In specific implementation, the data processing rate of the i-th mobile device is v i,loc (t), and the data processing rates of different mobile devices depend on their respective hardware configurations. The available computing resources of the j-th mobile edge computing server in the t-th time slot are denoted as R j (t), and the number of CPU cycles required to process one bit of data is c.

[0381] S62. Determine the local processing delay of the mobile device in processing the subtasks of the executable task of the mobile device according to the number of subtasks of the executable task of the mobile device in the t-th time slot, the task offloading decision parameters of the mobile device, and the data processing rate of the mobile device.

[0382] In specific implementation, the number of floating-point operations required for the i-th mobile device to execute the layer l of the DNN task (i.e., the subtask corresponding to the l-th layer) is f i,l (1 ≤ l ≤ L i ), L iThe number of layers for the i-th mobile device to execute the DNN task, f i,l can be used to measure the computational complexity of the DNN model. The floating-point operation count of the conventional convolutional layer in the DNN model with Bias parameters can be calculated by the following formula, where the Bias parameter is an optional parameter used to add a bias term in the convolutional operation:

[0383] FLOPs conv =[2*(K W *K H *C I )*C o )*W*H

[0384] where, FLOPs conv represents the floating-point operation count of the conventional convolutional layer in the DNN model with Bias parameters calculated by the mobile device;

[0385] K W represents the width of the convolutional kernel, K H represents the height of the convolutional sum, C I represents the number of input channels, C o represents the number of output channels;

[0386] W represents the width of the feature map required for input when training the DNN model, and H represents the height of the feature map required for input when training the DNN model.

[0387] The floating-point operation count of the conventional fully connected layer in the DNN model with Bias parameters can be calculated by the following formula:

[0388] FLOPs fc =[2*(n I *n O )+n O

[0389] where, FLOPs fc represents the floating-point operation count of the conventional fully connected layer in the DNN model with Bias parameters calculated by the mobile device;

[0390] n I represents the number of input neurons, n O represents the number of output neurons.

[0391] Calculating FLOPs fc is also calculating the operation count of multiplication and addition in the DNN model.

[0392] The floating-point operation count of the grouped convolutional layer in the DNN model can be calculated by the following formula:

[0393]

[0394] Among them, FLOPs GC represents the number of floating-point operations in the grouped convolution layer of the DNN model calculated by the mobile device;

[0395] G represents the number of groups.

[0396] Furthermore, the computational latency generated by layer l of the DNN task executed on the i-th mobile device at time slot t can be calculated by the following formula

[0397]

[0398] Among them, v i,Loc (t) represents the data processing rate of the i-th mobile device, measured in floating-point operations per second of the task;

[0399] f i,l represents the number of floating-point operations required for layer l of the DNN task executed by the i-th mobile device. If layer l is a conventional convolutional layer in the DNN model with Bias parameters, then f i,l = FLOPs conv , if layer l is a conventional fully-connected layer in the DNN model with Bias parameters, then f i,l = FLOPs fc , if layer l is a grouped convolutional layer in the DNN model, then f i,l = FLOPs GC .

[0400] Furthermore, the local processing latency of the sub-task of the task executable by the i-th mobile device processed at time slot t can be calculated by the following formula:

[0401]

[0402] Among them, represents the local processing latency of the sub-task of the task executable by the i-th mobile device processed at time slot t.

[0403] S63. Determine the processing latency of the sub-task of the task executable by the mobile device processed by the mobile edge computing server according to the available computing resources of the mobile edge computing server and the number of sub-tasks of the task executable by the mobile device at time slot t.

[0404] If the task executable by the i-th mobile device at time slot t contains A i(t) sub - tasks are offloaded to the j - th mobile edge computing server for execution. Then the processing delay includes the transmission delay from the i - th mobile device to the j - th mobile edge computing server, the computing delay of the j - th mobile edge computing server, and the download delay of the inference result from the j - th mobile edge computing server to the i - th mobile device. During the task - processing, there is a layer - by - layer division strategy for tasks. The DNN task is divided into multiple sub - tasks, and then a suitable splitting ratio is selected to divide these sub - tasks into two parts, which are processed by the j - th mobile edge computing server and the cloud server respectively. The tasks of the i - th mobile device unloaded and processed by the j - th mobile edge computing server and the cloud server contain A i (t) The splitting ratios of the sub - tasks are: γ ij (t) and 1 - γ ij (t), specifically as follows:

[0405]

[0406] If all the A i (t) sub - tasks contained in the task executable by the i - th mobile device are processed by the j - th mobile edge computing server, then the processing delay of the sub - tasks of the task executable by the i - th mobile device processed by the j - th mobile edge computing server can be calculated by the following formula:

[0407]

[0408] Among them, represents the processing delay of the sub - tasks of the task executable by the i - th mobile device processed by the j - th mobile edge computing server at time slot t;

[0409] c represents the number of CPU cycles required for the j - th mobile edge computing server to process one bit of data;

[0410] A i (t) represents the number of sub - tasks of the task executable by the i - th mobile device at time slot t;

[0411] R j (t) represents the available computing resources of the j - th mobile edge computing server at time slot t.

[0412] The computing power of the cloud server is much greater than that of the mobile edge computing server and the local mobile device. And, during the whole task - processing, a significant bottleneck may come from the task data transmission process. The transmission delay includes the transmission delay from the local mobile device to the mobile edge computing server, the transmission delay from the mobile edge computing server to the cloud server, and the download delay of the inference result from the mobile edge computing server to the mobile device. In contrast, the computing delay of the cloud server and the transmission delay from the cloud server to the mobile device can be ignored.

[0413] S64. Determine the transmission delay for offloading subtasks of the tasks executable by the mobile device to the mobile edge computing server from the mobile device, based on the number of subtasks of the tasks executable by the mobile device in the t time slot and the transmission rate between the mobile device and the mobile edge computing server.

[0414] In specific implementation, the transmission delay for offloading subtasks of the tasks executable by the i-th mobile device to the j-th mobile edge computing server can be calculated using the following formula:

[0415]

[0416] where, represents the transmission delay for offloading subtasks of the tasks executable by the i-th mobile device to the j-th mobile edge computing server in the t time slot;

[0417] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t time slot.

[0418] S65. Obtain the transmission rate between the mobile edge computing server and the cloud server in the t time slot.

[0419] In specific implementation, the transmission rate between the mobile edge computing server and the cloud server in the t time slot can be obtained according to the process shown in Figure 7 as follows, including the following steps:

[0420] S71. Obtain the transmission power of the mobile edge computing server in the t time slot, the transmission power of other mobile edge computing servers except the mobile edge computing server, and the channel gain between the mobile edge computing server and the cloud server.

[0421] In specific implementation, the transmission power of the j-th mobile edge computing server is p j , and the transmission power of other mobile edge computing servers except the j-th mobile edge computing server is p k , k≠j, k∈M. The channel state between the mobile edge computing server and the cloud server changes in real time. The channel gain between the j-th mobile edge computing server and the cloud server is

[0422] S72. Determine the interference noise between other mobile edge computing servers and the cloud server in the t time slot, based on the transmission power of other mobile edge computing servers except the mobile edge computing server in the t time slot and the channel gain between the mobile edge computing server and the cloud server.

[0423] During specific implementation, the interference noise between other mobile edge computing servers except the j-th mobile edge computing server and the cloud server in the t-th time slot can be calculated through the following formula:

[0424]

[0425] Wherein, represents the interference noise between other mobile edge computing servers except the j-th mobile edge computing server and the cloud server in the t-th time slot;

[0426] p k represents the transmission power of other mobile edge computing servers except the j-th mobile edge computing server in the t-th time slot, k≠j, k∈M;

[0427] represents the channel gain between the j-th mobile edge computing server and the cloud server in the t-th time slot.

[0428] S73. Determine the transmission rate between the mobile edge computing server and the cloud server in the t-th time slot according to the transmission power of the mobile edge computing server in the t-th time slot, the channel gain between the mobile edge computing server and the cloud server, the interference noise between other mobile edge computing servers and the cloud server, the available bandwidth of all mobile edge computing servers, and the bandwidth allocation ratio parameter of the mobile edge computing server.

[0429] During specific implementation, the transmission rate between the j-th mobile edge computing server and the cloud server in the t-th time slot can be calculated through the following formula:

[0430]

[0431] represents the transmission rate between the j-th mobile edge computing server and the cloud server in the t-th time slot;

[0432] η j (t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server in the t-th time slot;

[0433] B e (t) represents the available bandwidth of all mobile edge computing servers;

[0434] p j represents the transmission power of the j-th mobile edge computing server in the t-th time slot;

[0435] represents the channel gain between the j-th mobile edge computing server and the cloud server in the t-th time slot;

[0436] Denote the interference noise between other mobile edge computing servers and the cloud server except the j-th mobile edge computing server in the t-th time slot;

[0437] σ 2 Denote the average Gaussian white noise.

[0438] S66. Determine the transmission delay for offloading subtasks of the executable tasks of the mobile device from the mobile edge computing server to the cloud server according to the number of subtasks of the executable tasks of the mobile device in the t-th time slot and the transmission rate between the mobile edge computing server and the cloud server.

[0439] Specifically, the transmission delay for offloading the subtasks of the i-th executable task of the mobile device from the j-th mobile edge server to the cloud server in the t-th time slot can be calculated by the following formula:

[0440]

[0441] where, Denote the transmission delay for offloading the subtasks of the i-th executable task of the mobile device from the j-th mobile edge server to the cloud server in the t-th time slot;

[0442] A i (t) denote the number of subtasks of the i-th executable task of the mobile device in the t-th time slot;

[0443] Denote the transmission rate between the j-th mobile edge computing server and the cloud server in the t-th time slot.

[0444] S67. Obtain the download delay for downloading the processing result from the mobile edge computing server to the mobile device in the t-th time slot.

[0445] Specifically, the cloud server obtains the transmission power of the mobile edge computing server and the channel gain between the mobile device and the mobile edge computing server in the t-th time slot; according to the bandwidth of the mobile edge computing server, the interference noise between the mobile device and the mobile edge computing server, the transmission power of the mobile edge computing server, and the channel gain between the mobile device and the mobile edge computing server in the t-th time slot, determine the download rate for downloading the processing result from the mobile edge computing server to the mobile device in the t-th time slot; according to the number of processing results that the mobile device can download, the download rate for downloading the processing result from the mobile edge computing server to the mobile device, and the number of mobile devices, determine the download delay for downloading the processing result from the mobile edge computing server to the mobile device in the t-th time slot.

[0446] Specifically, the download rate for downloading the processing result from the j-th mobile edge computing server to the i-th mobile device in the t-th time slot can be calculated by the following formula:

[0447]

[0448] Among them, represents the download rate of downloading the processing result from the j-th mobile edge computing server to the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, n is the number of mobile devices, and j = 1, 2, ……, m, m is the number of mobile edge computing servers;

[0449] B j represents the bandwidth of the j-th mobile edge computing server in the t-th time slot;

[0450] p j represents the transmission power of the j-th mobile edge computing server in the t-th time slot;

[0451] represents the channel gain between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot;

[0452] represents the interference noise between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot;

[0453] σ 2 represents the average Gaussian white noise.

[0454] Furthermore, the download delay of downloading the processing result from the j-th mobile edge computing server to the i-th mobile device in the t-th time slot can be calculated by the following formula:

[0455]

[0456] Among them, represents the download delay of downloading the processing result from the j-th mobile edge computing server to the i-th mobile device in the t-th time slot;

[0457] represents the number of processing results that can be downloaded by the i-th mobile device in the t-th time slot, D C (t) is the set of processing results that can be downloaded by the mobile device in the t-th time slot;

[0458] represents the download rate of downloading the processing result from the j-th mobile edge computing server to the i-th mobile device in the t-th time slot.

[0459] S68. Determine the processing delay of the subtasks included in the mobile device executable task at time slot t according to the local processing delay of the subtasks of the mobile device executable task processed by the mobile device at time slot t, the processing delay of the subtasks of the mobile device executable task processed by the mobile edge computing server, the transmission delay of offloading the subtasks of the mobile device executable task from the mobile device to the mobile edge computing server, the transmission delay of offloading the subtasks of the mobile device executable task from the mobile edge computing server to the cloud server, the download delay of downloading the processing result from the mobile edge computing server to the mobile device, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server.

[0460] When specifically implemented, the processing delay of the subtasks included in the task generated by the i-th mobile device at the time slot t can be calculated by the following formula:

[0461]

[0462] where, T i (t) represents the processing delay of the subtasks included in the task generated by the i-th mobile device at the time slot t, i = 1, 2, ……, n, and n represents the number of mobile devices;

[0463] α i (t) represents the task offloading decision parameter of the i-th mobile device at the time slot t;

[0464] γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server at the time slot t, which characterizes the proportion of the subtasks included in the task offloaded by the i-th mobile device and executed by the j-th mobile edge computing server at the time slot t; 1 - γ ij (t) characterizes the proportion of the subtasks included in the task offloaded by the i-th mobile device and offloaded by the j-th mobile edge computing server to the cloud server at the time slot t, j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0465] represents the local processing delay of the subtasks of the i-th mobile device executable task processed by the i-th mobile device at the time slot t;

[0466] represents the transmission delay of offloading the subtasks of the i-th mobile device executable task from the i-th mobile device to the j-th mobile edge computing server at the time slot t;

[0467] Denote the processing delay of the subtask of the executable task of the $i$-th mobile device processed by the $j$-th mobile edge computing server in the $t$-th time slot;

[0468] Denote the download delay of downloading the processing result from the $j$-th mobile edge computing server to the $i$-th mobile device in the $t$-th time slot;

[0469] Denote the transmission delay of offloading the subtask of the executable task of the $i$-th mobile device from the $j$-th mobile edge computing server to the cloud server in the $t$-th time slot.

[0470] Before constructing the task offloading problem with minimized construction cost and maximized privacy level, it also includes establishing a queue model.

[0471] In specific implementation, the subtasks included in the task generated by the $i$-th mobile device in the $t$-th time slot are stored in the task queue $Q$ i (t) to wait for allocation, $Q$ i (t) represents the length of the task queue of the $i$-th mobile device in the $t$-th time slot. The task queue of the $i$-th mobile device is used to store the tasks generated by the $i$-th mobile device in each time slot. At the beginning of the time slot $t$, the set of task queue lengths can be expressed as: $Q(t)=\{Q_1(t),Q_2(t),\cdots,Q$ n (t)\}, Denote the length of the queue of the processing result returned by the $i$-th mobile device in the $t$-th time slot. The processing result return queue of the $i$-th mobile device is used to store the returned processing results corresponding to the tasks offloaded by the $i$-th mobile device. At the end of the time slot $t$, the set of processing result return queues can be expressed as:

[0472] The length of the task queue $Q$ i (t) can be dynamically changed as:

[0473] $Q$ i (t + 1)=\max\{Q$ i (t)+D$ i (t)-A$ i (t),0\}, i\in N, N = \{1,2,\cdots,n\}$

[0474] where, $Q$ i (t + 1) represents the length of the task queue of the $i$-th mobile device in the $(t + 1)$-th time slot;

[0475] $Q$ i (t) represents the length of the task queue of the $i$-th mobile device in the $t$-th time slot;

[0476] $D$ i(t) represents the number of subtasks included in the task generated by the i-th mobile device in the t-th time slot;

[0477] A i (t) represents the number of subtasks of the task that the i-th mobile device can execute in the t-th time slot.

[0478] Processing result return queue The dynamic change of can be expressed as:

[0479]

[0480] Among them, represents the queue length of the processing result returned by the i-th mobile device in the (t + 1)-th time slot;

[0481] represents the queue length of the processing result returned by the i-th mobile device in the t-th time slot;

[0482] represents the number of processing results returned by the i-th mobile device in the t-th time slot, D R (t) represents the data result set of the processing result return queue in the t-th time slot; represents the number of downloadable processing results of the i-th mobile device in the t-th time slot, D C (t) is the set of downloadable processing results of the mobile device in the t-th time slot.

[0483] Then, the buffer queue state of the i-th mobile device in the t-th time slot can be expressed as:

[0484]

[0485] Among them, represents the buffer queue state of the i-th mobile device in the t-th time slot; T represents the number of time slots;

[0486] E represents the mathematical expectation;

[0487] Q i (t) represents the task queue length of the i-th mobile device in the t-th time slot;

[0488] represents the queue length of the processing result returned by the i-th mobile device in the t-th time slot;

[0489] represents the maximum length of the task queue of the i-th mobile device.

[0490] When is bounded, the system can be considered stable.

[0491] To minimize costs and maximize privacy protection, the embodiments of the present application construct a task offloading problem P1 with minimized costs and maximized privacy levels according to the privacy level of the mobile device at time slot t and the cost function of processing the subtasks of the tasks executable by the mobile device as follows:

[0492]

[0493] where P i (t) represents the privacy level of the i-th mobile device at the time slot t, i = 1, 2, ……, n, and n represents the number of mobile devices;

[0494] C i (t) represents the cost function of processing the subtasks included in the tasks generated by the i-th mobile device at the time slot t;

[0495] represents the trade-off coefficient between the privacy level and the cost function; T represents the number of time slots;

[0496] α i (t) represents the task offloading decision parameter of the i-th mobile device at the time slot t;

[0497] η j (t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server at the time slot t, j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0498] γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server at the time slot t, characterizing the proportion of the subtasks included in the tasks offloaded by the i-th mobile device executed by the j-th mobile edge computing server at the time slot t;

[0499] T i (t) represents the processing delay of processing the subtasks included in the tasks generated by the i-th mobile device at the time slot t, and K i represents the task processing delay constraint;

[0500] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server at the time slot t, represents the minimum transmission rate between the mobile device and the mobile edge computing server;

[0501] represents the transmission rate between the j-th mobile edge computing server and the cloud server at the time slot t, represents the minimum transmission rate between the mobile edge computing server and the cloud server;

[0502] Q i (t) represents the task queue length of the i-th mobile device in the t-th time slot. The task queue of the i-th mobile device is used to store the tasks generated by the i-th mobile device in each time slot. represents the maximum length of the task queue of the i-th mobile device.

[0503] During implementation, The value of can be set according to requirements, and the embodiments of the present application do not limit this.

[0504] S25. Use the Lyapunov optimization theory to transform the task offloading problem into a single-time-slot task offloading strategy optimization problem, and obtain the optimized objective function.

[0505] During specific implementation, since the computing tasks and wireless channel conditions of local mobile devices change over time, it is difficult for local mobile devices to make task offloading strategies. In addition, the task offloading strategies in different time slots are mutually influential, and the goal of the task offloading problem P1 is a long-term goal. Therefore, problem P1 cannot be directly solved. To solve this task offloading problem, the embodiments of the present application use the Lyapunov optimization theory to decouple the original problem into a task offloading strategy optimization problem within a single time slot to ensure the stability of the system.

[0506] First, define the Lyapunov function L(Θ(t)) as:

[0507]

[0508] Θ(t) represents the queue backlog matrix composed of the task queue lengths of all mobile devices in the t-th time slot; L(Θ(t)) represents the queue backlog state, and the smaller its value, the less task backlog and the more stable the system. Furthermore, based on the long-term average overhead (i.e., cost) of the system and the queue, the embodiments of the present application define the drift-plus-penalty function Δ V (Θ(t)) as:

[0509]

[0510] V represents the Lyapunov trade-off factor, and its value is greater than or equal to 0 and can be set according to requirements. The embodiments of the present application do not limit this.

[0511] To ensure the stability of the queue backlog state L(Θ(t)), the embodiments of the present application need to obtain the minimum value of Δ V (Θ(t)). According to the proof principle of the Lyapunov optimization method, the upper limit of the drift-plus-penalty function can be obtained as:

[0512]

[0513] wherein, is the maximum number of subtasks included in the task generated by the i-th mobile device.

[0514] By removing the constant term in the above formula, the optimized problem P2 of the task offloading strategy within a single time slot after transformation is obtained, that is, the optimized objective function is:

[0515]

[0516] wherein, E represents the mathematical expectation;

[0517] Q i (t) represents the length of the task queue of the i-th mobile device at the t-th time slot. The task queue of the i-th mobile device is used to store the tasks generated by the i-th mobile device in each time slot. represents the maximum length of the task queue of the i-th mobile device, i = 1, 2,..., n, and n represents the number of mobile devices;

[0518] represents the length of the queue of the processing result returned by the i-th mobile device at the t-th time slot;

[0519] D i (t) represents the number of subtasks included in the task generated by the i-th mobile device at the t-th time slot;

[0520] P i (t) represents the privacy level of the i-th mobile device at the t-th time slot;

[0521] C i (t) represents the cost function for processing the subtasks included in the task generated by the i-th mobile device at the t-th time slot;

[0522] represents the trade-off coefficient between the privacy level and the cost function; T represents the number of time slots;

[0523] α i (t) represents the task offloading decision parameter of the i-th mobile device at the t-th time slot;

[0524] η j (t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server at the t-th time slot, j = 1, 2,..., m, and m represents the number of mobile edge computing servers;

[0525] γ ij(t) represents the task offloading decision parameter of the j-th mobile edge computing server in the t-th time slot, characterizing the proportion of subtasks included in the task offloaded by the j-th mobile edge computing server for the i-th mobile device in the t-th time slot;

[0526] T i (t) represents the processing delay of the subtasks included in the task generated by the i-th mobile device in the t-th time slot, K i represents the task processing delay constraint;

[0527] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, represents the minimum transmission rate between the mobile device and the mobile edge computing server;

[0528] represents the transmission rate between the j-th mobile edge computing server and the cloud server in the t-th time slot, represents the minimum transmission rate between the mobile edge computing server and the cloud server;

[0529] V represents the Lyapunov trade-off factor.

[0530] S26. Model the optimization objective function as a Markov decision process, and determine the optimal task offloading strategy according to the preset deep reinforcement learning algorithm.

[0531] By analyzing the optimization objective function and its related constraint conditions, it can be seen that the converted single-time-slot task offloading strategy optimization problem is a complex problem combining discrete variables and continuous variables, and it is difficult to solve using traditional methods. Therefore, the embodiment of this application introduces a Markov decision process to model the optimization objective function.

[0532] Specifically, in the t-th time slot, by monitoring the dynamic environment of the system, information such as the number of subtasks included in the task generated by the mobile device (i.e., the computing load of the mobile device), the available computing resources of the mobile edge computing server, the available bandwidth of all mobile edge computing servers, and the channel state between the mobile edge computing server and the cloud server can be obtained. The state space can be set as:

[0533]

[0534] where s(t) represents the state space in the t-th time slot;

[0535] D i (t) represents the number of subtasks included in the task generated by the i-th mobile device in the t-th time slot;

[0536] Rj (t) represents the available computing resources of the j-th mobile edge computing server in the t-th time slot;

[0537] B e (t) represents the available bandwidth of all mobile edge computing servers;

[0538] represents the channel gain between the j-th mobile edge computing server and the cloud server.

[0539] In the action space, at the t-th time slot, the cloud server (or agent) can execute the action a(t) according to the current state s(t): the task offloading decision of the mobile device, the task offloading decision of the mobile edge computing server, and the bandwidth allocation ratio decision of the mobile edge computing server. Therefore, the action space can be set as:

[0540] a(t) = {α i (t), γ ij (t), η j (t)}

[0541] where a(t) represents the action space at the t-th time slot;

[0542] The goal of the embodiment of this application is to minimize the system cost and maximize the privacy protection. Therefore, at the t-th time slot, after the cloud server (or agent) executes the action a(t) according to the current state s(t), the generated reward function can be set as:

[0543]

[0544] where R(t) represents the reward function at the t-th time slot.

[0545] Then, define the long-term average reward value as:

[0546]

[0547] where R represents the long-term average reward value.

[0548] Furthermore, determine the optimal task offloading strategy according to the preset deep reinforcement learning algorithm. Among them, the preset deep reinforcement learning algorithm can but is not limited to using the improved TD3 algorithm, and the embodiment of this application does not limit this.

[0549] Specifically, it can be determined according to the process as Figure 8 described to determine the optimal task offloading strategy, including the following steps:

[0550] S81. Solve the Markov decision process using the preset deep reinforcement learning algorithm to obtain the task offloading strategy prediction model.

[0551] In the embodiments of the present application, the preset deep reinforcement learning algorithm may, but is not limited to, adopt the TD3 algorithm, and the embodiments of the present application do not limit this. The preset deep reinforcement learning model includes: a main network, a target network, and an embedding layer. The embedding layer is connected between the main network and the target network. The main network includes a main Actor network, a first main Critic network, and a second main Critic network. The target network includes a target Actor network, a first target Critic network, and a second target Critic network. The dual-output head Actor network is used to process continuous actions and discrete actions in the mixed action space. For continuous actions, the Tanh activation function can be used to constrain the output range. For discrete actions, the Softmax activation function can be adopted to generate a probability distribution, so as to realize the probability selection of discrete actions. At the same time, the Critic network can accept the input of the mixed action space, convert the discrete action into a continuous representation through the introduced embedding layer, and input it into the network together with the continuous action to improve the efficiency of action evaluation. In order to optimize the discrete action selection of the Actor network, the cross-entropy loss function based on the probability distribution can be adopted to update the task offloading strategy, which not only optimizes the action selection process but also improves the efficiency of task offloading strategy learning. In view of the exploration complexity of the mixed action space, the embodiments of the present application design different exploration strategies for continuous actions and discrete actions. For continuous actions, the method of adding noise can be adopted for exploration to ensure the effectiveness of exploration. For discrete actions, exploration can be carried out by sampling the probability distribution and random selection to enhance the diversity and coverage of the exploration process.

[0552] The parameters of the main Actor network in the main network are The parameters of the first main Critic network are θ1, and the parameters of the second main Critic network are θ2; the parameters of the target Actor network in the target network are The parameters of the first target Critic network are θ1 ′ , and the second target Critic network is θ2 ′ . By updating the parameters in the main network and the target network of the preset deep reinforcement model, an offloading strategy prediction model is obtained. The offloading strategy prediction model is used to predict the corresponding action information based on the state information of the edge computing system. The action information includes the task offloading decision parameters of the mobile device, the task offloading decision parameters of the mobile edge computing server, and the bandwidth allocation ratio parameters of the mobile edge computing server.

[0553] During implementation, in the main Actor network is used to generate the action a(t), which can be expressed as:

[0554]

[0555] Among them, ε is noise, which is a small Gaussian distribution noise used to smooth the target offloading strategy to enhance the exploration ability of the offloading strategy prediction model.

[0556] For the discrete action space, the softmax(a(t)) activation function can be used to normalize the obtained action output and convert it into a probability distribution vector. Then, sampling is performed according to this probability distribution vector to determine the corresponding action. The specific calculation method is as follows:

[0557]

[0558] a i (t) represents the i-th action.

[0559] After the cloud server or the agent executes the action a(t), the system state is updated from s(t) to s(t + 1), and the corresponding reward value R(t) is obtained and stored in the experience pool for updating the network parameters of the preset deep reinforcement learning model (i.e., TD3). The parameters of the main Actor network can be updated using deterministic policy gradients.

[0560] Specifically, the parameters of the main Actor network are updated as follows:

[0561]

[0562] Among them, is the current value of Q, y = 2, and y is the target value for calculating the Q value by the target Critic network. The specific representation of y is as follows:

[0563]

[0564] Among them, R is the long-term average reward value, k is the discount factor, and the value of κ is [0, 1]. is the action generated by the target Actor network based on the state s(t).

[0565] After updating the parameters of the preset deep reinforcement learning model, an offloading strategy prediction model is obtained.

[0566] S82. Determine the optimal task offloading strategy according to the task offloading strategy prediction model.

[0567] In specific implementation, the cloud server or the agent receives a task offloading request sent by the mobile device. The task offloading request carries the number of subtasks included in the task generated by the mobile device, obtains the status information of the edge computing system. The status information of the edge computing system includes the number information of the subtasks included in the task generated by the mobile device, the available computing resource information of the mobile edge computing server, the available bandwidth information for indexing the mobile edge computing server, and the channel status information between the mobile edge computing server and the cloud server. Input the status information of the edge computing system into the offloading policy prediction model to obtain the task offloading policy of the mobile device. The task offloading policy includes the predicted task offloading decision of the mobile device, the task offloading decision of the mobile edge computing server, and the bandwidth allocation ratio decision of the mobile edge computing server. Furthermore, execute the subtasks of the task generated by the mobile device according to the predicted task offloading policy of the mobile device.

[0568] The edge computing offloading method provided by the embodiment of the present application obtains the number of subtasks included in the tasks generated by the mobile device in the t time slot, the number of subtasks of the tasks that the mobile device can execute, the transmission rate between the mobile device and the mobile edge computing server, the bandwidth of the mobile edge computing server, and the available bandwidth of all mobile edge computing servers; determines the privacy level of the mobile device in the t time slot according to the number of subtasks included in the tasks generated by the mobile device in the t time slot, the number of subtasks of the tasks that the mobile device can execute, and the transmission rate between the mobile device and the mobile edge computing server, where the privacy level of the mobile device represents the privacy level of the mobile device; constructs a cost function for processing the subtasks of the tasks that the mobile device can execute in the t time slot according to the number of subtasks of the tasks that the mobile device can execute in the t time slot, the bandwidth of the mobile edge computing server, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, the task offloading decision parameter of the mobile edge computing server, and the bandwidth allocation ratio parameter of the mobile edge computing server; constructs a task offloading problem that minimizes the cost and maximizes the privacy level according to the privacy level of the mobile device in the t time slot and the cost function for processing the subtasks of the tasks that the mobile device can execute; uses the Lyapunov optimization theory to transform the task offloading problem into a single time slot task offloading strategy optimization problem to obtain an optimized objective function; models the optimized objective function as a Markov decision process, and determines the optimal task offloading strategy according to a preset deep reinforcement learning algorithm. In the embodiment of the present application, the privacy level of the mobile device and the cost of processing the tasks generated by the mobile device are comprehensively considered. According to the privacy level of the mobile device and the cost function for processing the subtasks of the tasks that the mobile device can execute, a task offloading problem that minimizes the cost and maximizes the privacy level is constructed. Since the objective of this task offloading problem is a long-term objective and cannot be directly solved, to solve this task offloading problem, the present application uses the Lyapunov optimization theory to decouple the original problem into a task offloading strategy optimization problem within a single time slot to obtain an optimized objective function to ensure the stability of the system. Considering that the converted single time slot task offloading strategy optimization problem has a non-convex characteristic, the present application further introduces a Markov decision process to model the optimized objective function and solves it according to a preset deep reinforcement learning algorithm to obtain the optimal task offloading strategy. Thus, while ensuring the stability of the system, the total cost is minimized and the maximization of user privacy protection is achieved, and while improving the security of edge computing offloading, the accuracy of the task offloading strategy is improved.

[0569] Based on the same inventive concept, the embodiment of the present application also provides an edge computing offloading device. Since the principle of solving problems by the above edge computing offloading device is similar to that of the above edge computing offloading method, the implementation of the above device can refer to the implementation of the method, and the repeated parts will not be described again.

[0570] As Figure 9As shown, it is a schematic structural diagram of an edge computing offloading device provided by an embodiment of the present application, which can be applied to a cloud server or an agent as shown in Figure 1 shown. The edge computing offloading device may include:

[0571] A first acquisition module 91, configured to acquire the number of subtasks included in the task generated by the mobile device in the t time slot, the number of subtasks that the mobile device can execute for the task, the transmission rate between the mobile device and the mobile edge computing server, the bandwidth of the mobile edge computing server, and the available bandwidth of all mobile edge computing servers;

[0572] A first determination module 92, configured to determine the privacy level of the mobile device in the t time slot according to the number of subtasks included in the task generated by the mobile device in the t time slot, the number of subtasks that the mobile device can execute for the task, and the transmission rate between the mobile device and the mobile edge computing server. The privacy level of the mobile device represents the privacy level of the mobile device;

[0573] A first construction module 93, configured to construct a cost function for processing the subtasks of the task that the mobile device can execute in the t time slot according to the number of subtasks that the mobile device can execute for the task in the t time slot, the bandwidth of the mobile edge computing server, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, the task offloading decision parameter of the mobile edge computing server, and the bandwidth allocation ratio parameter of the mobile edge computing server;

[0574] A second construction module 94, configured to construct a task offloading problem of minimizing cost and maximizing privacy level according to the privacy level of the mobile device in the t time slot and the cost function for processing the subtasks of the task that the mobile device can execute;

[0575] A processing module 95, configured to convert the task offloading problem into a single-time slot task offloading strategy optimization problem by using the Lyapunov optimization theory to obtain an optimized objective function;

[0576] A second determination module 96, configured to model the optimized objective function as a Markov decision process and determine an optimal task offloading strategy according to a preset deep reinforcement learning algorithm.

[0577] In one implementation, the first determination module 92 is specifically configured to determine the privacy level of the user usage pattern of the mobile device in the t time slot according to the number of subtasks included in the task generated by the mobile device in the t time slot, the number of subtasks of the tasks executable by the mobile device, and the transmission rate between the mobile device and the mobile edge computing server; determine the location privacy level of the mobile device in the t time slot according to the number of subtasks of the tasks executable by the mobile device in the t time slot and the transmission rate between the mobile device and the mobile edge computing server; and determine the weighted sum of the privacy level of the user usage pattern of the mobile device and the location privacy level of the mobile device in the t time slot as the privacy level of the mobile device in the t time slot.

[0578] In one implementation, the task offloading decision parameter of the mobile device characterizes whether to offload the task to the mobile edge computing server for execution, and the task offloading decision parameter of the mobile edge computing server characterizes the proportion of the tasks offloaded by the mobile device executed by the mobile edge computing server and the proportion of the tasks offloaded by the mobile device offloaded to the cloud server for execution;

[0579] The first construction module 93 is specifically configured to determine the computational cost function for processing the subtasks of the tasks executable by the mobile device in the t time slot according to the number of subtasks of the tasks executable by the mobile device in the t time slot, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server; determine the communication cost function for processing the subtasks of the tasks executable by the mobile device in the t time slot according to the bandwidth of the mobile edge computing server in the t time slot, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, and the bandwidth allocation ratio parameter of the mobile edge computing server; and determine the cost function for processing the subtasks of the tasks executable by the mobile device in the t time slot according to the computational cost function and the communication cost function.

[0580] In one implementation, the second construction module 94 is specifically configured to construct the task offloading problem of maximizing cost and maximizing privacy level as:

[0581]

[0582] where P i (t) represents the privacy level of the i-th mobile device in the t time slot, i = 1, 2,..., n, and n represents the number of mobile devices;

[0583] C i (t) represents the cost function for processing the subtasks included in the task generated by the i-th mobile device in the t time slot;

[0584] represents the trade-off coefficient between the privacy level and the cost function; T represents the number of time slots;

[0585] α i (t) represents the task offloading decision parameter of the i-th mobile device at the t-th time slot;

[0586] η j (t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server at the t-th time slot, j = 1, 2, ……, m, where m represents the number of mobile edge computing servers;

[0587] γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server at the t-th time slot, characterizing the proportion of subtasks included in the task offloaded by the j-th mobile edge computing server for the i-th mobile device at the t-th time slot;

[0588] T i (t) represents the processing delay of the subtasks included in the task generated by the i-th mobile device at the t-th time slot, K i represents the task processing delay constraint;

[0589] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server at the t-th time slot, represents the minimum transmission rate between the mobile device and the mobile edge computing server;

[0590] represents the transmission rate between the j-th mobile edge computing server and the cloud server at the t-th time slot, represents the minimum transmission rate between the mobile edge computing server and the cloud server;

[0591] Q i (t) represents the task queue length of the i-th mobile device at the t-th time slot, and the task queue of the i-th mobile device is used to store the tasks generated by the i-th mobile device in each time slot, represents the maximum length of the task queue of the i-th mobile device.

[0592] In one implementation manner, the first determination module 92 is specifically configured to calculate the user usage pattern privacy level of the i-th mobile device at the t-th time slot through the following formula:

[0593]

[0594] where P i,U(t) represents the privacy level of the usage pattern of the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, and n represents the number of mobile devices;

[0595] D i (t) represents the number of subtasks included in the task generated by the i-th mobile device in the t-th time slot;

[0596] A i (t) represents the number of subtasks of the task that the i-th mobile device can execute in the t-th time slot;

[0597] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, where j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0598] r thr represents the transmission rate threshold from the i-th mobile device to the j-th mobile edge computing server;

[0599] ∏(·) is an indicator function. If holds, then If does not hold, then

[0600] In one implementation, the first determination module 92 is specifically configured to calculate the location privacy level of the i-th mobile device in the t-th time slot through the following formula:

[0601]

[0602] where P i,L (t) represents the location privacy level of the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, and n represents the number of mobile devices;

[0603] A i (t) represents the number of subtasks of the task that the i-th mobile device can execute in the t-th time slot;

[0604] ∏(·) is an indicator function. If A i (t) > 0 holds, then ∏(A i (t) > 0) = 1. If A i (t) > 0 does not hold, then ∏(A i (t) > 0) = 0;

[0605] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, where j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0606] r thr represents the transmission rate threshold value from the i-th mobile device to the j-th mobile edge computing server;

[0607] If holds, then If does not hold, then

[0608] In one embodiment, the first determination module 92 is specifically configured to calculate the privacy level of the i-th mobile device in the t time slot through the following formula:

[0609] P i (t) = λ1P i,U (t) + λ2P i,L (t)

[0610] where P i (t) represents the privacy level of the i-th mobile device in the t time slot, i = 1, 2,..., n, and n represents the number of mobile devices;

[0611] P i,U (t) represents the user usage pattern privacy level of the i-th mobile device in the t time slot, and λ1 represents the weight of the user usage pattern privacy level of the i-th mobile device in the t time slot;

[0612] P i,L (t) represents the location privacy level of the i-th mobile device in the t time slot, and λ2 represents the weight of the location privacy level of the i-th mobile device in the t time slot.

[0613] In one embodiment, the first construction module 93 is specifically configured to determine the computational cost function of the subtasks for processing the executable tasks of the i-th mobile device in the t time slot through the following formula:

[0614]

[0615] where represents the computational cost function of the subtasks for processing the executable tasks of the i-th mobile device in the t time slot, i = 1, 2,..., n, and n represents the number of mobile devices;

[0616] α i (t) represents the task offloading decision parameter of the i-th mobile device in the t time slot. If α i (t) = 0, then the subtasks included in the tasks generated by local processing on the i-th mobile device. If α iIf \(\alpha^{(t)} = 1\), then the subtasks included in the task generated by the \(i\)-th mobile device are offloaded to the \(j\)-th mobile edge computing server;

[0617] \(\gamma\) ij \(\gamma^{(t)}\) represents the task offloading decision parameter of the \(j\)-th mobile edge computing server in the \(t\)-th time slot, characterizing the proportion of the subtasks included in the task offloaded by the \(j\)-th mobile edge computing server from the \(i\)-th mobile device in the \(t\)-th time slot; \(1 - \gamma\) ij \((1 - \gamma)^{(t)}\) characterizes the proportion of the subtasks included in the task offloaded by the \(j\)-th mobile edge computing server from the \(i\)-th mobile device that are offloaded to the cloud server in the \(t\)-th time slot, \(j = 1, 2, \ldots, m\), where \(m\) represents the number of mobile edge computing servers;

[0618] A i \(A^{(t)}\) represents the number of subtasks of the task that can be executed by the \(i\)-th mobile device in the \(t\)-th time slot;

[0619] \(\beta_1\) represents the cost per bit of computing data of the \(j\)-th mobile edge computing server in the \(t\)-th time slot;

[0620] \(\beta_2\) represents the cost per bit of computing data of the cloud server in the \(t\)-th time slot.

[0621] In one implementation, the first construction module 93 is specifically configured to determine the communication cost function for processing the subtasks of the task that can be executed by the \(i\)-th mobile device in the \(t\)-th time slot through the following formula:

[0622]

[0623] Wherein, represents the communication cost function for processing the subtasks of the task that can be executed by the \(i\)-th mobile device in the \(t\)-th time slot, \(i = 1, 2, \ldots, n\), where \(n\) represents the number of mobile devices;

[0624] \(\alpha\) i \(\alpha^{(t)}\) represents the task offloading decision parameter of the \(i\)-th mobile device in the \(t\)-th time slot. If \(\alpha\) i \(\alpha^{(t)} = 0\), then the subtasks included in the task generated locally by the \(i\)-th mobile device are processed. If \(\alpha\) i \(\alpha^{(t)} = 1\), then the subtasks included in the task generated by the \(i\)-th mobile device are offloaded to the \(j\)-th mobile edge computing server;

[0625] represents the rental cost per hertz of bandwidth;

[0626] \(\eta\) j(t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server in the t-th time slot, where j = 1, 2, ……, m, and m represents the number of mobile edge computing servers;

[0627] B e (t) represents the available bandwidth of all the mobile edge computing servers;

[0628] B j represents the bandwidth of the j-th mobile edge computing server in the t-th time slot.

[0629] In one embodiment, the first construction module 93 is specifically configured to determine the cost function of the subtask for processing the executable task of the i-th mobile device in the t-th time slot through the following formula:

[0630]

[0631] where C i (t) represents the cost function of the subtask for processing the executable task of the i-th mobile device in the t-th time slot, where i = 1, 2, ……, n, and n represents the number of mobile devices;

[0632] represents the computing cost function of the subtask for processing the executable task of the i-th mobile device in the t-th time slot;

[0633] represents the communication cost function of the subtask for processing the executable task of the i-th mobile device in the t-th time slot.

[0634] In one embodiment, the first acquisition module 91 is specifically configured to acquire the transmission power of the mobile device and the channel gain between the mobile device and the mobile edge computing server in the t-th time slot; determine the interference noise between the mobile device and the mobile edge computing server in the t-th time slot according to the transmission power of the mobile device and the channel gain between the mobile device and the mobile edge computing server in the t-th time slot; determine the transmission rate between the mobile device and the mobile edge computing server in the t-th time slot according to the bandwidth of the mobile edge computing server, the interference noise between the mobile device and the mobile edge computing server, the transmission power of the mobile device, and the channel gain between the mobile device and the mobile edge computing server.

[0635] In one embodiment, the device further includes:

[0636] A second acquisition module, configured to acquire the data processing rate of the mobile device and the available computing resources of the mobile edge computing server at the t-th time slot before constructing a task offloading problem with minimized construction cost and maximized privacy level;

[0637] A third determination module, configured to determine the local processing delay of the mobile device for processing the subtasks of the tasks executable by the mobile device according to the number of subtasks of the tasks executable by the mobile device at the t-th time slot, the task offloading decision parameters of the mobile device, and the data processing rate of the mobile device;

[0638] A fourth determination module, configured to determine the processing delay of the mobile edge computing server for processing the subtasks of the tasks executable by the mobile device according to the available computing resources of the mobile edge computing server at the t-th time slot and the number of subtasks of the tasks executable by the mobile device;

[0639] A fifth determination module, configured to determine the transmission delay of offloading the subtasks of the tasks executable by the mobile device from the mobile device to the mobile edge computing server according to the number of subtasks of the tasks executable by the mobile device at the t-th time slot and the transmission rate between the mobile device and the mobile edge computing server;

[0640] A third acquisition module, configured to acquire the transmission rate between the mobile edge computing server and the cloud server at the t-th time slot;

[0641] A sixth determination module, configured to determine the transmission delay of offloading the subtasks of the tasks executable by the mobile device from the mobile edge computing server to the cloud server according to the number of subtasks of the tasks executable by the mobile device at the t-th time slot and the transmission rate between the mobile edge computing server and the cloud server;

[0642] A fourth acquisition module, configured to acquire the download delay of downloading the processing result from the mobile edge computing server to the mobile device at the t-th time slot;

[0643] A seventh determination module, configured to determine a processing delay of sub-tasks included in the task executable by the mobile device in the t time slot according to a local processing delay of the sub-tasks of the task executable by the mobile device processed by the mobile device in the t time slot, a processing delay of the sub-tasks of the task executable by the mobile device processed by the mobile edge computing server, a transmission delay of offloading the sub-tasks of the task executable by the mobile device from the mobile device to the mobile edge computing server, a transmission delay of offloading the sub-tasks of the task executable by the mobile device from the mobile edge computing server to the cloud server, a download delay of downloading a processing result from the mobile edge computing server to the mobile device, a task offloading decision parameter of the mobile device, and a task offloading decision parameter of the mobile edge computing server.

[0644] In an implementation manner, the third acquisition module is specifically configured to acquire a transmission power of the mobile edge computing server in the t time slot, a transmission power of other mobile edge computing servers except the mobile edge computing server, and a channel gain between the mobile edge computing server and the cloud server; determine an interference noise between the other mobile edge computing server and the cloud server in the t time slot according to the transmission power of the other mobile edge computing servers except the mobile edge computing server in the t time slot and the channel gain between the mobile edge computing server and the cloud server; determine a transmission rate between the mobile edge computing server and the cloud server in the t time slot according to the transmission power of the mobile edge computing server in the t time slot, the channel gain between the mobile edge computing server and the cloud server, the interference noise between the other mobile edge computing server and the cloud server, available bandwidths of all the mobile edge computing servers, and a bandwidth allocation ratio parameter of the mobile edge computing server.

[0645] In an implementation manner, the seventh determination module is specifically configured to calculate a processing delay of sub-tasks included in the task generated by the i-th mobile device in the t time slot through the following formula:

[0646]

[0647] where, T i (t) represents the processing delay of sub-tasks included in the task generated by the i-th mobile device in the t time slot, i = 1, 2,..., n, and n represents the number of mobile devices;

[0648] α i (t) represents the task offloading decision parameter of the i-th mobile device in the t time slot;

[0649] γ ij$(t)$ represents the task offloading decision parameter of the $j$-th mobile edge computing server in the $t$-th time slot, characterizing the proportion of subtasks included in the task offloaded by the $i$-th mobile device to the $j$-th mobile edge computing server in the $t$-th time slot; $1 - \gamma$ ij $(t)$ characterizes the proportion of subtasks included in the task offloaded by the $i$-th mobile device to the $j$-th mobile edge computing server that are offloaded to the cloud server in the $t$-th time slot, $j = 1, 2, \ldots, m$, where $m$ represents the number of mobile edge computing servers;

[0650] represents the local processing delay of the subtasks of the executable tasks of the $i$-th mobile device processed by the $i$-th mobile device in the $t$-th time slot;

[0651] represents the transmission delay of the subtasks of the executable tasks of the $i$-th mobile device offloaded from the $i$-th mobile device to the $j$-th mobile edge computing server in the $t$-th time slot;

[0652] represents the processing delay of the subtasks of the executable tasks of the $i$-th mobile device processed by the $j$-th mobile edge computing server in the $t$-th time slot;

[0653] represents the download delay of the processing results downloaded from the $j$-th mobile edge computing server to the $i$-th mobile device in the $t$-th time slot;

[0654] represents the transmission delay of the subtasks of the executable tasks of the $i$-th mobile device offloaded from the $j$-th mobile edge computing server to the cloud server in the $t$-th time slot.

[0655] In one implementation, the optimization objective function is:

[0656]

[0657] where $E$ represents the mathematical expectation;

[0658] $Q$ i $(t)$ represents the task queue length of the $i$-th mobile device in the $t$-th time slot. The task queue of the $i$-th mobile device is used to store the tasks generated by the $i$-th mobile device in each time slot, represents the maximum length of the task queue of the $i$-th mobile device, $i = 1, 2, \ldots, n$, where $n$ represents the number of mobile devices;

[0659] represents the queue length of the processing results returned by the $i$-th mobile device in the $t$-th time slot;

[0660] D i D(i)(t) represents the number of subtasks included in the task generated by the i-th mobile device in the t-th time slot;

[0661] P i P(i)(t) represents the privacy level of the i-th mobile device in the t-th time slot;

[0662] C i C(i)(t) represents the cost function for processing the subtasks included in the task generated by the i-th mobile device in the t-th time slot;

[0663] represents the trade-off coefficient between the privacy level and the cost function; T represents the number of time slots;

[0664] α i α(i)(t) represents the task offloading decision parameter of the i-th mobile device in the t-th time slot;

[0665] η j η(j)(t) represents the bandwidth allocation ratio parameter of the j-th mobile edge computing server in the t-th time slot, j = 1, 2, ……, m, where m represents the number of mobile edge computing servers;

[0666] γ ij γ(j)(t) represents the task offloading decision parameter of the j-th mobile edge computing server in the t-th time slot, representing the proportion of the subtasks included in the task offloaded by the i-th mobile device executed by the j-th mobile edge computing server in the t-th time slot;

[0667] T i T(i)(t) represents the processing delay for processing the subtasks included in the task generated by the i-th mobile device in the t-th time slot, K i represents the task processing delay constraint;

[0668] represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, represents the minimum transmission rate between the mobile device and the mobile edge computing server;

[0669] represents the transmission rate between the j-th mobile edge computing server and the cloud server in the t-th time slot, represents the minimum transmission rate between the mobile edge computing server and the cloud server;

[0670] V represents the Lyapunov trade-off factor.

[0671] In one implementation manner, the second determination module 96 is specifically configured to:

[0672] The state space is set as:

[0673]

[0674] where s(t) represents the state space at the t-th time slot;

[0675] D i (t) represents the number of subtasks included in the task generated by the i-th mobile device at the t-th time slot;

[0676] R j (t) represents the available computing resources of the j-th mobile edge computing server at the t-th time slot;

[0677] B e (t) represents the available bandwidth of all mobile edge computing servers;

[0678] represents the channel gain between the j-th mobile edge computing server and the cloud server;

[0679] The action space is set as:

[0680] a(t) = {α i (t), γ ij (t), η j (t)}

[0681] where a(t) represents the action space at the t-th time slot;

[0682] The reward function is set as:

[0683]

[0684] where R(t) represents the reward function at the t-th time slot;

[0685] The long-term average reward value is set as:

[0686]

[0687] where R represents the long-term average reward value.

[0688] In one implementation manner, the second determination module 96 is specifically configured to solve the Markov decision process by using the preset deep reinforcement learning algorithm to obtain a task offloading policy prediction model; and determine an optimal task offloading policy according to the task offloading policy prediction model.

[0689] Based on the same technical concept, an embodiment of the present application further provides an electronic device 1000, refer to Figure 10As shown, the electronic device 1000 is used to implement the edge computing offloading method described in the above method embodiments. The electronic device 1000 in this embodiment may include: a memory 1001, a processor 1002, and a computer program stored in the memory and executable on the processor, such as an edge computing offloading program. When the processor executes the computer program, it implements the steps in the above various edge computing offloading method embodiments.

[0690] In the embodiments of the present application, the specific connection medium between the above-mentioned memory 1001 and processor 1002 is not limited. In the embodiments of the present application Figure 10 it is shown that the memory 1001 and the processor 1002 are connected through a bus 1003. The bus 1003 is represented by a thick line in Figure 10 and the connection manners between other components are only for illustrative purposes and are not to be taken as limiting. The bus 1003 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 10 only a thick line is used to represent it in, but it does not mean that there is only one bus or one type of bus.

[0691] The memory 1001 can be a volatile memory, such as a random-access memory (RAM); the memory 1001 can also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), or the memory 1001 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1001 can be a combination of the above memories.

[0692] The processor 1002 is used to implement the edge computing offloading method provided in the embodiments of the present application.

[0693] The embodiments of the present application also provide a computer-readable storage medium, storing computer-executable instructions required to be executed by the above-mentioned processor, which includes a program for executing the operations required to be executed by the above-mentioned processor.

[0694] In some possible implementation manners, various aspects of the edge computing offloading method provided in the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on an electronic device, the program code is used to cause the electronic device to execute the steps in the edge computing offloading method according to various exemplary embodiments of the present application described above in this specification.

[0695] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, apparatuses, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0696] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (apparatuses), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0697] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0698] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0699] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0700] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these modifications and variations.

Claims

1. An edge computing offloading method, characterized in that: include: Obtain the number of subtasks contained in the task generated by the mobile device in time slot t, the number of subtasks of the task executable by the mobile device, the transmission rate between the mobile device and the mobile edge computing server, the bandwidth of the mobile edge computing server, and the available bandwidth of all mobile edge computing servers; Determine the privacy level of the mobile device at the time slot t according to the number of subtasks contained in the task generated by the mobile device at the time slot t, the number of subtasks of the task executable by the mobile device, and the transmission rate between the mobile device and the mobile edge computing server, wherein the privacy level of the mobile device represents the privacy level of the mobile device; Constructing a cost function for processing the subtasks of the task executable by the mobile device in the t time slot according to the number of subtasks of the task executable by the mobile device in the t time slot, the bandwidth of the mobile edge computing server, the available bandwidth of all the mobile edge computing servers, the task offloading decision parameter of the mobile device, the task offloading decision parameter of the mobile edge computing server and the bandwidth allocation ratio parameter of the mobile edge computing server; According to the privacy level of the mobile device at the time slot t and the cost function of processing the subtasks of the task executable by the mobile device, construct a task offloading problem of minimizing the cost and maximizing the privacy level; The task offloading problem is transformed into a single time slot task offloading strategy optimization problem by using Lyapunov optimization theory, and the optimization objective function is obtained. The optimization objective function is modeled as a Markov decision process, and the optimal task offloading strategy is determined according to a preset deep reinforcement learning algorithm.

2. The method according to claim 1, characterized in that Determining the privacy level of the mobile device in the t time slot according to the number of subtasks contained in the task generated by the mobile device in the t time slot, the number of subtasks of the task executable by the mobile device, and the transmission rate between the mobile device and the mobile edge computing server, specifically includes: Determine the privacy level of the user usage pattern of the mobile device at the time slot t according to the number of subtasks contained in the task generated by the mobile device at the time slot t, the number of subtasks of the task executable by the mobile device, and the transmission rate between the mobile device and the mobile edge computing server; Determine the location privacy level of the mobile device at the time slot t according to the number of subtasks of the task that can be executed by the mobile device at the time slot t and the transmission rate between the mobile device and the mobile edge computing server; A weighted sum of the user usage pattern privacy level of the mobile device and the location privacy level of the mobile device at the t time slot is determined as the privacy level of the mobile device at the t time slot.

3. The method according to claim 1, characterized in that The task offloading decision parameter of the mobile device represents whether to offload the task to the mobile edge computing server for execution, and the task offloading decision parameter of the mobile edge computing server represents the proportion of the tasks offloaded by the mobile device executed by the mobile edge computing server and the proportion of the tasks offloaded by the mobile device to be offloaded to the cloud server for execution; According to the number of subtasks of the task executable by the mobile device in the t time slot, the bandwidth of the mobile edge computing server, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, the task offloading decision parameter of the mobile edge computing server and the bandwidth allocation ratio parameter of the mobile edge computing server, a cost function for processing the subtasks of the task executable by the mobile device in the t time slot is constructed, specifically including: Determine a computational cost function for processing the subtasks of the task executable by the mobile device at the t time slot according to the number of subtasks of the task executable by the mobile device at the t time slot, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server; Determine a communication cost function for processing a subtask of the task executable by the mobile device in the t time slot according to the bandwidth of the mobile edge computing server in the t time slot, the available bandwidth of all the mobile edge computing servers, the task offloading decision parameter of the mobile device, and the bandwidth allocation ratio parameter of the mobile edge computing server; A cost function for processing a subtask of the mobile device executable task at the t time slot is determined based on the computational cost function and the communication cost function.

4. The method according to claim 1, characterized in that According to the privacy level of the mobile device at the time slot t and the cost function of processing the subtasks of the task executable by the mobile device, a task offloading problem of minimizing the cost and maximizing the privacy level is constructed, which specifically includes: The task offloading problem of maximizing construction cost and privacy level is: Among them, P i (t) represents the privacy level of the i-th mobile device at the t-th time slot, i=1, 2, ..., n, and n represents the number of mobile devices; C i (t) represents the cost function of processing the subtasks contained in the task generated by the i-th mobile device in the t-th time slot; represents the trade-off coefficient between the privacy level and the cost function; T represents the number of time slots; α i (t) represents the task offloading decision parameter of the i-th mobile device at the t-th time slot; η j (t) represents the bandwidth allocation ratio parameter of the jth mobile edge computing server in the t time slot, j=1, 2, ..., m, m represents the number of mobile edge computing servers; γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server at the t-th time slot, representing the proportion of subtasks included in the task offloaded by the i-th mobile device executed by the j-th mobile edge computing server at the t-th time slot; T i (t) represents the processing delay of the subtasks contained in the task generated by the i-th mobile device in the t-th time slot, K i Represents the task processing delay constraint; represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server at the t-th time slot, Indicates the minimum transmission rate between the mobile device and the mobile edge computing server; represents the transmission rate between the j-th mobile edge computing server and the cloud server at the t-time slot, represents the minimum transmission rate between the mobile edge computing server and the cloud server; Q i (t) represents the length of the task queue of the i-th mobile device in the t-th time slot, and the task queue of the i-th mobile device is used to store the tasks generated by the i-th mobile device in each time slot, Indicates the maximum length of the task queue of the i-th mobile device.

5. The method according to claim 2, characterized in that Determining the privacy level of the user usage pattern of the mobile device in the t time slot according to the number of subtasks contained in the task generated by the mobile device in the t time slot, the number of subtasks of the task executable by the mobile device, and the transmission rate between the mobile device and the mobile edge computing server, specifically includes: The privacy level of the user usage pattern of the i-th mobile device at the time slot t is calculated by the following formula: Among them, P i,U (t) represents the privacy level of the user usage pattern of the i-th mobile device at the t-th time slot, i=1, 2, ..., n, and n represents the number of mobile devices; D i (t) represents the number of subtasks contained in the task generated by the i-th mobile device at the t time slot; A i (t) represents the number of subtasks of the task that can be executed by the i-th mobile device in the t-th time slot; represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, j=1, 2, ..., m, m represents the number of mobile edge computing servers; r thr represents a critical value of the transmission rate from the i-th mobile device to the j-th mobile edge computing server; ∏(·) is an indicator function, if If established, like If not established, 6. The method according to claim 2, characterized in that Determining the location privacy level of the mobile device in the t time slot according to the number of subtasks of the task that can be executed by the mobile device in the t time slot and the transmission rate between the mobile device and the mobile edge computing server, specifically includes: The location privacy level of the i-th mobile device at the time slot t is calculated by the following formula: Among them, P i,L (t) represents the location privacy level of the i-th mobile device at the t-th time slot, i=1, 2, ..., n, and n represents the number of mobile devices; A i (t) represents the number of subtasks of the task that can be executed by the i-th mobile device in the t-th time slot; ∏(·) is the indicator function. If A i (t)>0 holds, then ∏(A i (t)>0)=1, if A i (t)>0 does not hold, then ∏(A i (t)>0)=0; represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server in the t-th time slot, j=1, 2, ..., m, m represents the number of mobile edge computing servers; r thr represents a critical value of the transmission rate from the i-th mobile device to the j-th mobile edge computing server; like If established, like If not established, 7. The method according to claim 2, 4, 5 or 6, characterized in that Determining the weighted sum of the user usage pattern privacy level of the mobile device and the location privacy level of the mobile device at the t time slot as the privacy level of the mobile device at the t time slot specifically includes: The privacy level of the i-th mobile device at the time slot t is calculated by the following formula: P i (t)=λ1P i,U (t)+λ2P i,L (t) Among them, P i (t) represents the privacy level of the i-th mobile device at the t-th time slot, i=1, 2, ..., n, and n represents the number of mobile devices; P i,U (t) represents the privacy level of the user usage pattern of the i-th mobile device at the t-th time slot, and λ1 represents the weight of the privacy level of the user usage pattern of the i-th mobile device at the t-th time slot; P i,L (t) represents the location privacy level of the i-th mobile device at the t-th time slot, and λ2 represents the weight of the location privacy level of the i-th mobile device at the t-th time slot.

8. The method according to claim 3, characterized in that Determining a computational cost function for processing the subtasks of the task executable by the mobile device in the t time slot according to the number of subtasks of the task executable by the mobile device in the t time slot, the task offloading decision parameter of the mobile device, and the task offloading decision parameter of the mobile edge computing server, specifically includes: The computational cost function for processing the subtask of the i-th mobile device executable task at the t time slot is determined by the following formula: in, represents a computational cost function for processing a subtask of the task executable by the i-th mobile device in the t-th time slot, i=1, 2, ..., n, where n represents the number of mobile devices; α i (t) represents the task offloading decision parameter of the i-th mobile device at the t time slot, if α i (t) = 0, then the subtasks contained in the task generated are processed locally on the i-th mobile device. If α i (t)=1, then the subtasks contained in the task generated by the i-th mobile device are offloaded to the j-th mobile edge computing server; γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server at the t-th time slot, representing the proportion of subtasks contained in the task offloaded by the i-th mobile device executed by the j-th mobile edge computing server at the t-th time slot; 1-γ ij (t) represents the proportion of subtasks included in the task offloaded from the i-th mobile device by the j-th mobile edge computing server at the t-th time slot to the cloud server, j=1, 2, ..., m, where m represents the number of mobile edge computing servers; A i (t) represents the number of subtasks of the task that can be executed by the i-th mobile device in the t-th time slot; β1 represents the cost per bit of data calculation by the j-th mobile edge computing server at the t-th time slot; β2 represents the cost per bit of data calculated by the cloud server in the t time slot.

9. The method according to claim 3, characterized in that Determining a communication cost function for processing a subtask of an executable task of the mobile device in the t time slot according to the bandwidth of the mobile edge computing server in the t time slot, the available bandwidth of all mobile edge computing servers, the task offloading decision parameter of the mobile device, and the bandwidth allocation ratio parameter of the mobile edge computing server, specifically includes: The communication cost function for processing the subtask of the task executable by the i-th mobile device in the t-time slot is determined by the following formula: in, represents a communication cost function for processing a subtask of the task executable by the i-th mobile device in the t-th time slot, i=1, 2, ..., n, where n represents the number of mobile devices; α i (t) represents the task offloading decision parameter of the i-th mobile device at the t time slot, if α i (t) = 0, then the subtasks contained in the task generated are processed locally on the i-th mobile device. If α i (t)=1, then the subtasks contained in the task generated by the i-th mobile device are offloaded to the j-th mobile edge computing server; ω represents the rental fee per Hz of bandwidth; η j (t) represents the bandwidth allocation ratio parameter of the jth mobile edge computing server in the t time slot, j=1, 2, ..., m, m represents the number of mobile edge computing servers; B e (t) represents the available bandwidth of all mobile edge computing servers; B j Represents the bandwidth of the j-th mobile edge computing server in the t time slot.

10. The method according to claim 3, 4, 8 or 9, characterized in that Determining a cost function for processing a subtask of the mobile device executable task in the t time slot according to the computation cost function and the communication cost function specifically includes: The cost function for processing the subtask of the i-th mobile device executable task at the t time slot is determined by the following formula: Among them, C i (t) represents a cost function for processing a subtask of the task executable by the i-th mobile device in the t-th time slot, i=1, 2, ..., n, and n represents the number of mobile devices; A computational cost function representing processing a subtask of the i-th mobile device executable task at the t-th time slot; Denotes a communication cost function for processing a subtask of the i-th mobile device executable task at the t time slot.

11. The method according to claim 1, characterized in that Obtaining the transmission rate between the mobile device and the mobile edge computing server in time slot t, specifically including: Acquire the transmission power of the mobile device at the time slot t, and the channel gain between the mobile device and the mobile edge computing server; Determine the interference noise between the mobile device and the mobile edge computing server at the time slot t according to the transmission power of the mobile device at the time slot t and the channel gain between the mobile device and the mobile edge computing server; The transmission rate between the mobile device and the mobile edge computing server in the t time slot is determined based on the bandwidth of the mobile edge computing server in the t time slot, the interference noise between the mobile device and the mobile edge computing server, the transmission power of the mobile device, and the channel gain between the mobile device and the mobile edge computing server.

12. The method according to claim 4, characterized in that Before framing the task offloading problem of minimizing cost and maximizing privacy level, we also include: Acquire the data processing rate of the mobile device and the available computing resources of the mobile edge computing server at the time slot t; Determining a local processing delay of the mobile device for processing the subtasks of the task executable by the mobile device in the time slot t, according to the number of subtasks of the task executable by the mobile device, the task offloading decision parameter of the mobile device, and the data processing rate of the mobile device; Determining a processing delay of the mobile edge computing server for processing the subtask of the task executable by the mobile device according to the available computing resources of the mobile edge computing server and the number of subtasks of the task executable by the mobile device in the time slot t; Determine, according to the number of subtasks of the task executable by the mobile device in the time slot t and the transmission rate between the mobile device and the mobile edge computing server, a transmission delay for offloading the subtasks of the task executable by the mobile device from the mobile device to the mobile edge computing server; Obtaining the transmission rate between the mobile edge computing server and the cloud server in the t time slot; Determine, based on the number of subtasks of the task executable by the mobile device in the time slot t and the transmission rate between the mobile edge computing server and the cloud server, a transmission delay for offloading the subtasks of the task executable by the mobile device from the mobile edge computing server to the cloud server; Obtaining a download delay for downloading a processing result from the mobile edge computing server to the mobile device in the t time slot; The processing delay of processing the subtasks included in the mobile device executable task in the t time slot is determined based on the local processing delay of the mobile device in processing the subtasks of the mobile device executable task in the t time slot, the processing delay of the mobile edge computing server in processing the subtasks of the mobile device executable task, the transmission delay of unloading the subtasks of the mobile device executable task from the mobile device to the mobile edge computing server, the transmission delay of unloading the subtasks of the mobile device executable task from the mobile edge computing server to the cloud server, the download delay of downloading the processing results from the mobile edge computing server to the mobile device, the task offloading decision parameters of the mobile device and the task offloading decision parameters of the mobile edge computing server.

13. The method according to claim 12, characterized in that Obtaining the transmission rate between the mobile edge computing server and the cloud server in the time slot t specifically includes: Acquire the transmission power of the mobile edge computing server at the time slot t, the transmission power of other mobile edge computing servers except the mobile edge computing server, and the channel gain between the mobile edge computing server and the cloud server; Determine the interference noise between the other mobile edge computing servers and the cloud server in the t time slot according to the transmission power of other mobile edge computing servers except the mobile edge computing server in the t time slot and the channel gain between the mobile edge computing server and the cloud server; The transmission rate between the mobile edge computing server and the cloud server in the t time slot is determined based on the transmission power of the mobile edge computing server in the t time slot, the channel gain between the mobile edge computing server and the cloud server, the interference noise between the other mobile edge computing servers and the cloud server, the available bandwidth of all mobile edge computing servers and the bandwidth allocation ratio parameter of the mobile edge computing server.

14. The method according to claim 12, characterized in that According to the local processing delay of the mobile device in processing the subtask of the mobile device executable task in the t time slot, the processing delay of the mobile edge computing server in processing the subtask of the mobile device executable task, the transmission delay of unloading the subtask of the mobile device executable task from the mobile device to the mobile edge computing server, the transmission delay of unloading the subtask of the mobile device executable task from the mobile device to the cloud server, the download delay of downloading the processing result from the mobile edge computing server to the mobile device, the task offloading decision parameter of the mobile device and the task offloading decision parameter of the mobile edge computing server, the processing delay of processing the subtask included in the mobile device executable task in the t time slot is determined, specifically including: The processing delay of the subtasks contained in the task generated by the i-th mobile device in the t time slot is calculated by the following formula: Among them, T i (t) represents the processing delay of the subtasks contained in the task generated by the i-th mobile device in the t-th time slot, i=1, 2, ..., n, n represents the number of mobile devices; α i (t) represents the task offloading decision parameter of the i-th mobile device at the t-th time slot; γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server at the t-th time slot, representing the proportion of subtasks contained in the task offloaded by the i-th mobile device executed by the j-th mobile edge computing server at the t-th time slot; 1-γ ij (t) represents the proportion of subtasks included in the task offloaded from the i-th mobile device by the j-th mobile edge computing server at the t-th time slot to the cloud server, j=1, 2, ..., m, where m represents the number of mobile edge computing servers; represents the local processing delay of the i-th mobile device processing the subtask of the task executable by the i-th mobile device in the t-th time slot; represents the transmission delay of offloading the subtask of the task executable by the i-th mobile device from the i-th mobile device to the j-th mobile edge computing server at the t-th time slot; represents the processing delay of the j-th mobile edge computing server processing the subtask of the task executable by the i-th mobile device in the t-th time slot; represents the download delay of downloading the processing result from the j-th mobile edge computing server to the i-th mobile device in the t-th time slot; Represents the transmission delay of offloading the subtask of the i-th mobile device executable task from the j-th mobile edge computing server to the cloud server in the t-th time slot.

15. The method according to claim 1 or 4, characterized in that The optimization objective function is: Where E represents mathematical expectation; Q i (t) represents the length of the task queue of the i-th mobile device in the t-th time slot, and the task queue of the i-th mobile device is used to store the tasks generated by the i-th mobile device in each time slot, represents the maximum length of the task queue of the i-th mobile device, i=1, 2, ..., n, and n represents the number of mobile devices; represents the queue length of the processing result returned by the i-th mobile device at the t time slot; D i (t) represents the number of subtasks contained in the task generated by the i-th mobile device at the t time slot; P i (t) represents the privacy level of the i-th mobile device at the t-th time slot; C i (t) represents the cost function of processing the subtasks contained in the task generated by the i-th mobile device in the t-th time slot; represents the trade-off coefficient between the privacy level and the cost function; T represents the number of time slots; α i (t) represents the task offloading decision parameter of the i-th mobile device at the t-th time slot; η j (t) represents the bandwidth allocation ratio parameter of the jth mobile edge computing server in the t time slot, j=1, 2, ..., m, m represents the number of mobile edge computing servers; γ ij (t) represents the task offloading decision parameter of the j-th mobile edge computing server at the t-th time slot, representing the proportion of subtasks included in the task offloaded by the i-th mobile device executed by the j-th mobile edge computing server at the t-th time slot; T i (t) represents the processing delay of the subtasks contained in the task generated by the i-th mobile device in the t-th time slot, K i Represents the task processing delay constraint; represents the transmission rate between the i-th mobile device and the j-th mobile edge computing server at the t-th time slot, Indicates the minimum transmission rate between the mobile device and the mobile edge computing server; represents the transmission rate between the j-th mobile edge computing server and the cloud server at the t-time slot, represents the minimum transmission rate between the mobile edge computing server and the cloud server; V represents the Lyapunov trade-off factor.

16. The method according to claim 15, characterized in that The optimization objective function is modeled as a Markov decision process, specifically including: The state space is set up as: Wherein, s(t) represents the state space at the time slot t; D i (t) represents the number of subtasks contained in the task generated by the i-th mobile device at the t time slot; R j (t) represents the available computing resources of the j-th mobile edge computing server at the t-th time slot; B e (t) represents the available bandwidth of all mobile edge computing servers; represents the channel gain between the j-th mobile edge computing server and the cloud server; The action space is set as: a(t)={a i (t),c ij (t), the j (t)} Wherein, a(t) represents the action space at the time slot t; The reward function is set as: Wherein, R(t) represents the reward function at the time slot t; The long-term average reward value is set to: Among them, R represents the long-term average reward value.

17. The method according to claim 1 or 16, characterized in that The optimal task offloading strategy is determined based on the preset deep reinforcement learning algorithm, including: The preset deep reinforcement learning algorithm is used to solve the Markov decision process to obtain a task offloading strategy prediction model; An optimal task offloading strategy is determined according to the task offloading strategy prediction model.

18. An edge computing offloading device, characterized in that: include: A first acquisition module is used to acquire the number of subtasks contained in the task generated by the mobile device in the time slot t, the number of subtasks of the task executable by the mobile device, the transmission rate between the mobile device and the mobile edge computing server, the bandwidth of the mobile edge computing server, and the available bandwidth of all mobile edge computing servers; A first determination module is used to determine the privacy level of the mobile device at the time slot t according to the number of subtasks contained in the task generated by the mobile device at the time slot t, the number of subtasks of the task executable by the mobile device, and the transmission rate between the mobile device and the mobile edge computing server, wherein the privacy level of the mobile device represents the privacy level of the mobile device; A first construction module is used to construct a cost function for processing the subtasks of the task executable by the mobile device in the t time slot according to the number of subtasks of the task executable by the mobile device in the t time slot, the bandwidth of the mobile edge computing server, the available bandwidth of all the mobile edge computing servers, the task offloading decision parameter of the mobile device, the task offloading decision parameter of the mobile edge computing server and the bandwidth allocation ratio parameter of the mobile edge computing server; A second building module is used to build a task offloading problem of minimizing cost and maximizing privacy level according to the privacy level of the mobile device at the time slot t and the cost function of processing the subtasks of the task executable by the mobile device; A processing module, used for converting the task offloading problem into a single time slot task offloading strategy optimization problem by using Lyapunov optimization theory, and obtaining an optimization objective function; The second determination module is used to model the optimization objective function as a Markov decision process and determine the optimal task offloading strategy according to a preset deep reinforcement learning algorithm.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the edge computing offloading method as described in any one of claims 1 to 17 is implemented.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the edge computing offloading method as described in any one of claims 1 to 17 are implemented.

Images