A differential privacy protection method for anti-reasoning attack of task offloading strategy in edge computing environment

Abstract

This invention discloses a differential privacy protection method for task offloading strategies in edge computing environments, resistant to inference attacks, belonging to the field of edge computing and privacy protection. The method first constructs an offloading decision model framework based on Markov Decision Process (MDP) in mobile edge computing scenarios, encompassing system models, transmission models, computation models, and user task models. Then, based on this framework, an initial offloading strategy is generated, and a privacy loss function is simultaneously constructed to quantify the privacy risks caused by inference attacks. Subsequently, the privacy loss and reward value of the strategy in the current time slot are calculated, and the smoothness sensitivity of the reward value is solved. The strategy is perturbed through an exponential mechanism. Finally, iterative convergence is judged by fixed time slots; if a preset maximum iteration period is reached, the final strategy is output; otherwise, the perturbation steps are repeated. This invention effectively reduces the privacy leakage risk of task offloading strategies, is applicable to deep reinforcement learning-driven offloading scenarios, and ensures the security of offloading strategies in edge computing environments.

Description

Technical Field

[0001] This invention relates to the field of edge computing and privacy protection, and in particular to a differential privacy protection method against inference attacks for task offloading strategies in edge computing environments, applicable to privacy protection of task offloading strategies using deep reinforcement learning in edge computing environments. Background Technology

[0002] With the rapid development of mobile communication and IoT technologies, the number of mobile devices and IoT devices has surged, and computationally intensive and low-latency applications (such as facial recognition and autonomous driving) are becoming increasingly common. To meet users' high demands for service quality, edge computing (MEC) has been proposed. MEC provides computing and storage support through edge servers located close to users, offering flexible computing expansion services with low latency, low energy consumption, and enhanced privacy and security. Task offloading is a key process in edge computing, essentially an optimization problem, typically with latency, energy consumption, and cost as optimization objectives. It is generally modeled using mixed-integer nonlinear programming (MINLP), Markov decision processes (MDP), or game theory models. Deep reinforcement learning (DRL), due to its self-learning capabilities and adaptability, has become the mainstream method for task offloading strategies.

[0003] However, when learning user task offloading strategies, DRL models also "remember" user preferences and offloading targets, leading to privacy risks. In highly dynamic and open edge computing environments, malicious attackers can infer users' offloading strategies and preferences through algorithms such as Inverse Reinforcement Learning (IRL), and then use this information to carry out malicious actions, such as inducing users to offload tasks to untrusted servers, threatening user privacy and task security. Existing research mainly focuses on the privacy protection of offloading data, such as using secure multi-party computation, homomorphic encryption, and differential privacy techniques to prevent attackers from directly obtaining offloading data. However, research on privacy protection for offloading strategies is still insufficient, especially in task offloading scenarios based on deep reinforcement learning. The characteristics of these models make strategy preferences easier for attackers to infer and exploit, becoming a significant bottleneck for the further development of edge computing. Therefore, an efficient privacy protection mechanism is needed to ensure the quality of task offloading services while preventing attackers from obtaining users' offloading strategies and preferences through model inference, fundamentally improving user privacy and the security of task offloading. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention proposes a differential privacy protection method against inference attacks for task offloading strategies in edge computing environments. This method mainly includes:

[0005] S1: Construct an offloading decision model framework based on Markov Decision Process (MDP) in mobile edge computing scenarios, including: system model, transmission model, computing model and user task model;

[0006] S2: Construct an offloading decision model based on Markov process (MDP) to obtain the initial offloading strategy, and at the same time, construct a privacy loss function;

[0007] S3: Calculate the current time slot The privacy loss and reward value of the offloading strategy are calculated, the smoothness sensitivity of the current reward value is obtained, and the differential privacy method with an exponential mechanism is used to perturb the current offloading strategy.

[0008] S4: Make a judgment with a fixed time slot as the iteration period. If the preset maximum iteration period is reached, output the final unloading strategy. Otherwise, jump to step S3 to execute.

[0009] Advantages and effects of the present invention:

[0010] 1) This invention is applicable to highly dynamic and open edge computing environments. Addressing the privacy leakage risks associated with task offloading strategies, it proposes a differential privacy protection method resistant to inference attacks. By constructing a privacy model and introducing a differential privacy mechanism, it effectively prevents attackers from using algorithms such as inverse reinforcement learning (IRL) to infer user offloading strategies and preferences, comprehensively protecting user privacy. This method, combined with the characteristics of deep reinforcement learning models, provides a general privacy protection scheme, widely applicable to edge computing scenarios such as connected vehicles and industrial IoT. Its modular design also ensures good practicality and scalability, providing reliable support for the secure application of edge computing.

[0011] 2) This invention utilizes Markov Decision Processes (MDPs) to construct an offloading decision framework. In a dynamic environment, it employs staged smoothing sensitivity calculations and dynamic noise intensity adjustments to ensure the differential privacy protection mechanism adapts to changes in system resources, achieving precise perturbation of the offloading strategy and effectively balancing privacy protection and task offloading performance. By constructing a privacy model and introducing a differential privacy mechanism, this invention effectively prevents privacy leaks caused by inference attacks. While protecting user privacy, it maximizes the performance and service quality of the task offloading strategy, achieving a dual optimization of privacy protection and system performance. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of an edge computing scenario according to the present invention.

[0013] Figure 2 This is a schematic diagram of a differential privacy protection method against inference attacks for a task offloading strategy in an edge computing environment, as described in an embodiment of the present invention.

[0014] Figure 3 This is a schematic diagram of a differential privacy protection method against inference attacks in an edge computing environment, based on a preferred embodiment of the present invention, which is a task offloading strategy. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Figure 1 This is a schematic diagram of an edge computing scenario according to the present invention, such as... Figure 1 As shown, an edge computing environment includes multiple edge nodes, IoT devices, and mobile terminals to support the computing needs of scenarios such as smart transportation, autonomous driving, and industrial IoT. When a device generates a task, it requests the edge server to offload computing. However, the task offloading process may leak device privacy and offloading preferences. Especially under malicious attacks, attackers can infer the user's offloading strategy, posing privacy risks. Existing methods mostly focus on data privacy protection. This embodiment does not consider data privacy in detail, but focuses only on the device's offloading strategy. It proposes a differential privacy protection method for task offloading strategies in an edge computing environment that is resistant to inference attacks, in order to improve the privacy security of task offloading.

[0017] Figure 2 This is a schematic diagram of a differential privacy protection method against inference attacks in an edge computing environment, based on a task offloading strategy according to the present invention. Figure 2 As shown, the method includes:

[0018] This invention provides a differential privacy protection method against inference attacks for task offloading strategies in edge computing environments, such as... Figure 2 As shown, the method includes:

[0019] S1. Construct an offloading decision model framework based on Markov Decision Process (MDP) in mobile edge computing scenarios, including: system model, transmission model, computing model and user task model;

[0020] In this embodiment of the invention, step S1, which involves constructing the system model, transmission model, computation model, and user task model, specifically includes:

[0021] S11. Establish the system model and user task model for the mobile edge computing scenario. The system model includes a set of mobile devices (MDs) and edge nodes (ENs), denoted as follows: as well as Divide time into A specific segment consisting of time slots is denoted as Each time slot lasts Each MD maintains a local computation and transmission queue, with tasks queued in a first-in, first-out (FIFO) manner. Each EN corresponds to I queues, each associated with a MD. The user task model defines the task. In time slot Arrival at MD The task index, its size is ,in , Indicates the number of tasks Represents the task deadline, a binary variable. Indicates whether the task is processed locally or sent. Indicates whether the task is unloaded to a specific EN.

[0022] S12. In the task transmission queue, tasks are assigned to appropriate edge nodes (ENs) and transmitted via the network interface of mobile devices (MDs). The MD is configured in a time slot. The transmission rate with EN j is , The time slot indicating whether a task is discarded or completed transmission. The number of time slots that need to wait in the queue before a task can be transmitted is defined as:

[0023]

[0024] in, If MD is in the time slot Assign tasks The task will then proceed with the transmission. Complete the transmission or discard.

[0025]

[0026] in The number of time slots required for a task to move from MDi to ENj is defined as:

[0027]

[0028] make Let the energy consumption for the task transmission from MDi to ENj be:

[0029]

[0030] in, Let denot be the communication power consumption of MD i in time slot t.

[0031] S13. If the task is assigned to the local computing queue, then set the processing rate of MD i to [value]. ,definition This is the time slot during which the task will be processed or discarded. (Setting) The number of time slots a task waits in the local computation queue is defined as:

[0032]

[0033] in, If MD is in the time slot If the task is scheduled to perform local computation, then the task will be... The calculation is completed or the data is discarded.

[0034]

[0035] in The number of time slots required for a task to complete processing from MDi is defined as:

[0036]

[0037] make For the local calculation of energy consumption of MD i, we have:

[0038]

[0039] in, This represents the processing power consumption of MD i.

[0040] S14. If the task is assigned to the edge node computing queue, then set the processing rate of EN j to be... ,definition This is the time when the task is processed or discarded on EN j. (Setting) The number of time slots for a task to wait in the edge computing queue is defined as:

[0041]

[0042] in, If MDi transfers the task to ENj for computation in time slot t, the task will then... The calculation is completed or the data is discarded.

[0043]

[0044] in, The number of time slots required for a task to complete processing from MDi is defined as:

[0045]

[0046] make For the energy consumption of the calculation at EN j in the task, we have:

[0047]

[0048] in, This represents the processing power consumption of MDj.

[0049] S2. Construct an offloading decision model based on Markov process (MDP) to obtain the initial offloading strategy, and at the same time, construct a privacy loss function;

[0050] In this embodiment of the invention, an offloading decision model based on a Markov process (MDP) is constructed to obtain an initial offloading strategy. Simultaneously, a privacy loss function is constructed, specifically including:

[0051] S21. Based on the system model, transmission model, computation model, and user task model established in the above steps, the offloading decision framework of the Markov Decision Process (MDP) is constructed as follows:

[0052] State space:

[0053]

[0054] Wherein, the state vector It consists of task size, local computation queue length, transmission queue length, edge node queue length, and historical data. Indicates in time slot The queue length of MD i tasks on ENs This is historical data.

[0055] Action space:

[0056]

[0057] in, This indicates whether the task is computed locally or offloaded to the edge node EN j.

[0058] Excitation function:

[0059]

[0060] in, This represents the reward for either discarding or completing the task; R>0 indicates a fixed reward for task completion. For latency consumption, For energy consumption, , The weights are time delay and energy consumption, respectively.

[0061]

[0062] If the task is discarded, then .

[0063]

[0064] If the task is discarded, then .

[0065] S22: Based on the Bellman optimal equation of the Markov decision process, and combining the task's state, actions, and rewards, the unloading strategy is obtained:

[0066]

[0067] in, The optimal strategy is... This is the discount factor.

[0068] S23. Construct a privacy model to infer device offloading strategies based on system states and assess privacy losses, specifically including:

[0069] S231. An attacker with prior knowledge uses the user's uninstallation experience... The reward function is obtained by inferring the reward function through inverse reinforcement learning. The result of the attack is defined as follows:

[0070]

[0071] in Indicates the attacker is The time slot infers the reward function of the current model through inverse reinforcement learning based on historical experience trajectories. , This indicates that the attacker deduced The probability of.

[0072] Evaluate the results of the attack and define the function. The correctness of the reward function inferred by the opponent compared to the actual reward function of the user is defined as follows:

[0073]

[0074] in They are respectively in The cumulative reward of a trajectory under the reward function, where Until the trajectory ends. Similarly, there are... ,in As a discount factor, The maximum difference value is awarded.

[0075] S232. An attacker with prior knowledge uses the user's uninstallation experience... By using inverse reinforcement learning, the unloading strategy is inferred, and the unloading strategy is obtained. The results of the attack are defined as follows:

[0076]

[0077] in Indicates the attacker is The time slots infer the unloading strategy of the current model through inverse reinforcement learning based on historical experience trajectories. , This indicates that the attacker deduced The probability of.

[0078] Evaluate the results of the attack and define the function. The correctness of the relationship between the adversary's inferred uninstallation strategy and the user's actual uninstallation strategy is defined as follows:

[0079]

[0080] Among them respectively strategy The following behavioral distribution, The KL divergence is the ratio of the two distributions. The smaller the KL divergence, the closer the distributions are. This represents the maximum value of the KL divergence.

[0081] S233. Privacy loss is quantified as the expected value of a correct attack (i.e., successfully inferring the model's reward function and offloading strategy). Privacy loss is defined as follows:

[0082]

[0083] in, express Privacy loss due to the policy of constantly uninstalling.

[0084] S3. Calculate the current time slot The privacy loss and reward value of the offloading strategy are calculated, the smoothness sensitivity of the current reward value is obtained, and the differential privacy method with an exponential mechanism is used to perturb the current offloading strategy.

[0085] In this embodiment of the invention, the current time slot is calculated. The privacy loss and reward values ​​of the offloading policy are calculated, the smoothness sensitivity of the current reward value is obtained, and a differential privacy method with an exponential mechanism is used to perturb the current offloading policy, specifically including:

[0086] S31. Calculate the sensitivity to staged smoothing.

[0087] Let's assume (abbreviated as) The maximum value of ) is The minimum value is Then global sensitivity Local sensitivity can then be expressed as:

[0088]

[0089] in If the value is the second largest, then the sensitivity at a distance of K is:

[0090]

[0091] in If the modified (k+1)th and (k+2)th largest values ​​are given, then the smoothing sensitivity is:

[0092]

[0093] in for Privacy budget at all times.

[0094] To better illustrate that the smoothness sensitivity mechanism of the present invention satisfies the definition of ε-differential privacy, this embodiment analyzes and proves the smoothness sensitivity mechanism of the present invention as follows:

[0095] First, we prove that smoothness sensitivity is an upper bound of local sensitivity, as follows:

[0096] When k=0, we have Substitute k=0 into ,have ,therefore For local sensitivity The upper boundary.

[0097] Secondly, it is proved that the smoothness sensitivity satisfies the smoothness property, as follows:

[0098] consider and have:

[0099]

[0100]

[0101] For any k, the sensitivity at distance k satisfy:

[0102]

[0103] because The k+1 neighborhood contains The k-neighborhood.

[0104] Therefore, the change in smoothness sensitivity satisfies:

[0105]

[0106] and ,but

[0107]

[0108] therefore:

[0109]

[0110] S32. Combining privacy loss and smoothing sensitivity, a differential privacy method with an exponential mechanism is used to perturb the offloading strategy.

[0111]

[0112] in, Indicates the selection of state and action combination The probability of.

[0113] To better illustrate that the exponential mechanism of the present invention satisfies the definition of ε-differential privacy, this embodiment analyzes and proves the exponential mechanism of the present invention as follows:

[0114] To simplify the proof process, the exponential mechanism is abbreviated to the following form:

[0115]

[0116] in accordance with - Differential privacy is defined as follows:

[0117]

[0118] Split into two items, there are

[0119]

[0120] because The most changes are ,Right now:

[0121]

[0122] Therefore, we have:

[0123]

[0124] For any , satisfy:

[0125]

[0126] Therefore, we have:

[0127]

[0128]

[0129] Combining the two items, we have:

[0130]

[0131] Therefore, the exponential mechanism satisfies - Differential privacy.

[0132] S4. Make a judgment with a fixed time slot as the iteration period. If the preset maximum iteration period is reached, output the final unloading strategy. Otherwise, jump to step S3 to execute.

[0133] The overall differential privacy mechanism of this invention consists of a combination of exponential differential privacy mechanisms at different stages. By calculating the privacy loss and smoothness sensitivity at different stages, the perturbation intensity is dynamically adjusted to accelerate model convergence and maximize model utility while ensuring privacy.

[0134] To better illustrate the differential privacy mechanism of this invention, - The definition of differential privacy is explained in this embodiment, which analyzes and proves the differential privacy mechanism of the present invention as follows:

[0135] Privacy budgets at different stages The privacy budget for the overall iteration cycle is defined as:

[0136]

[0137] According to the differential privacy combinatorial theorem,

[0138] There are T independent differential privacy mechanisms, each All satisfied of - Differential privacy.

[0139] Therefore, the combination mechanism satisfy - Differential privacy.

[0140] therefore satisfy - Differential privacy.

[0141] The embodiments of the present invention can effectively balance the efficiency of task unloading strategy and the privacy protection of unloading strategy, and can dynamically adapt to changes in the task unloading environment.

[0142] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include ROM, RAM, disk, or optical disk, etc.

[0143] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A differential privacy protection method against inference attacks for task offloading strategies in edge computing environments, characterized in that, The method includes: S1: Construct an offloading decision model framework based on Markov Decision Process (MDP) in mobile edge computing scenarios, including: system model, transmission model, computing model and user task model; S2: Construct an offloading decision model based on Markov process (MDP) to obtain the initial offloading strategy, and at the same time, construct a privacy loss function; S3: Calculate the current time slot The privacy loss and reward value of the offloading strategy are calculated, the smoothness sensitivity of the current reward value is obtained, and the differential privacy method with an exponential mechanism is used to perturb the current offloading strategy. S4: Make a judgment with a fixed time slot as the iteration period. If the preset maximum iteration period is reached, output the final unloading strategy; otherwise, jump to step S3 to execute. Step S1 contains the following: S11: The system model contains a set of mobile devices (MDs) and edge nodes (ENs), denoted as follows: as well as Divide time into A specific segment consisting of time slots is denoted as Each time slot lasts Each MD maintains a local computation and transmission queue, and tasks are queued in a first-in-first-out manner. Each EN corresponds to I queues, which are associated with each MD. S12: User Task Model Defines Tasks In time slot Arrival at MD The task index, its size is ,in , Indicates the number of tasks. Represents the task deadline, a binary variable. Indicates whether the task is processed locally or sent. Indicates whether the task is unloaded to a specific EN; S13: The transmission model is defined as follows: ; ; in, The time slot indicating whether a task is discarded or completed transmission. Indicates the number of time slots required to complete the transmission. This indicates the waiting time in the transmission queue during the task. The energy consumption for task transmission from MD i to EN j. For MDi in time slot Communication power consumption; S14: Computational models can be divided into local computing models and edge computing models. The local computing model is as follows: ; ; in, This indicates the time slot in which a task is discarded or completes its computation. This indicates the number of time slots required to complete the calculation. This indicates the waiting time in the calculation queue during the task. Energy consumption for local computing tasks. This represents the CPU power consumption of MD i; The edge computing model is as follows: ; ; in, This indicates the time slot in which a task is discarded or completes its computation. This represents the number of time slots the task waits in the edge computing queue. This indicates the number of time slots required to complete the calculation. Calculate the energy consumption of the task for edge nodes. This represents the CPU power consumption of ENj.

2. The differential privacy protection method against inference attacks for a task offloading strategy in an edge computing environment according to claim 1, characterized in that, Step S2 is as follows: S21: The offloading decision framework of a Markov decision process (MDP) is represented as follows: State space: ; Wherein, the state vector It consists of task size, local computation queue length, transmission queue length, edge node queue length, and historical data. Indicates in time slot The queue length of MD i tasks on ENs This is historical data; ; in, Indicates whether the task is computed locally or offloaded to the edge node EN j; ; in, This indicates the reward for either the task being discarded or the calculation being completed. This refers to a fixed reward for completing the task. For latency consumption, For energy consumption, , The weights for latency and energy consumption are respectively. S22: Based on the Bellman optimal equation of the Markov decision process, and combining the task's state, actions, and rewards, the unloading strategy is obtained: ; in, The optimal strategy is... Discount factor; S23: Calculate the privacy loss of the current offloading strategy using the privacy loss function as follows: ; in, express Privacy losses due to constantly uninstalling policies Indicates the attacker is At that moment, with The reward function is deduced based on this premise. probability, This indicates the correctness of the reward function inferred by the opponent compared to the user's actual reward function. Indicates the attacker is At that moment, with Based on this premise, the uninstallation strategy is deduced. The probability, This indicates the correctness of the uninstallation strategy inferred by the adversary compared to the user's actual uninstallation strategy.

3. The differential privacy protection method against inference attacks for a task offloading strategy in an edge computing environment according to claim 1, characterized in that, Step S3 is as follows: S31: Obtaining the offloading decision framework based on Markov Decision Process (MDP) The reward value for each time slot is calculated, and the smoothness sensitivity of the current reward value is determined as follows: ; in, Represents the state action function at time t The smoothed upper bound of the value, Represents distance k Sensitivity to value Privacy budget at time t; S32: Combining this smoothness sensitivity with privacy loss, an exponential mechanism is used to perturb the current offloading strategy, as follows: ; in, This indicates the selection of a state and action combination. The probability of.

Images