Anti-inference attack differential privacy protection method for task unloading strategy in edge computing environment

Abstract

The invention discloses an anti-inference attack differential privacy protection method for a task unloading strategy in an edge computing environment, and belongs to the field of edge computing and privacy protection. The method comprises the following steps: constructing an unloading decision model framework based on a Markov decision process (MDP) in a mobile edge computing scene, a system model, a transmission model, a calculation model and a user task model; constructing a privacy model for the inference attack, speculating an unloading strategy of equipment through a system state, and evaluating privacy loss; on the basis, calculating stage smooth sensitivity, and disturbing an unloading strategy by adopting a differential privacy method of an exponential mechanism in combination with privacy loss; and dynamically adjusting the noise intensity according to the privacy loss and smooth sensitivity of the whole iteration period until the model converges. According to the method, the risk of leakage of the task unloading strategy is reduced, the method is suitable for the task unloading strategy of deep reinforcement learning, and the safety of the task unloading strategy in the edge computing environment can be effectively protected.

Description

Technical Field

[0001] The present invention relates to the field of edge computing and privacy protection, and in particular to a differential privacy protection method for a task offloading strategy in an edge computing environment against inference attacks, which is suitable for privacy protection of a task offloading strategy using deep reinforcement learning in an edge computing environment. Background Art

[0002] With the rapid development of mobile communications and the Internet of Things (IoT) technologies, the number of mobile devices and IoT devices has skyrocketed, and computationally intensive, low-latency applications (such as facial recognition and autonomous driving) have become increasingly popular. To meet users' high demands for quality of service, edge computing (MEC) has been proposed. This technology uses edge servers close to users to provide computing and storage support, offering flexible computing expansion services with low latency, low energy consumption, and enhanced privacy and security. Task offloading is a key process in edge computing. Essentially, it is an optimization problem, typically optimized for latency, energy consumption, and cost. It is typically modeled as a mixed integer nonlinear programming (MINLP), Markov decision process (MDP), or game theory. Deep reinforcement learning (DRL), due to its self-learning capabilities and adaptability, has become a mainstream approach to task offloading strategies.

[0003] However, when learning user task offloading policies, DRL models also "remember" user preferences and offloading targets, leading to privacy leakage risks. In highly dynamic and open edge computing environments, malicious attackers can infer users' offloading policies and preferences through algorithms such as inverse reinforcement learning (IRL). They can then exploit this information to carry out malicious actions, such as inducing users to offload tasks to untrusted servers, threatening user privacy and task security. Existing research has primarily focused on protecting the privacy of offloaded data, using techniques such as secure multi-party computation, homomorphic encryption, and differential privacy to prevent attackers from directly accessing offloaded data. However, research on privacy protection of offloading policies remains insufficient, especially in task offloading scenarios based on deep reinforcement learning. The model's characteristics make policy preferences more susceptible to inference and exploitation by attackers, presenting a significant bottleneck to the further development of edge computing. Therefore, an efficient privacy protection mechanism is needed to prevent attackers from obtaining users' offloading policies and preferences through model inference while ensuring the quality of task offloading services, thereby fundamentally improving user privacy and the security of task offloading. Summary of the Invention

[0004] In order to solve the problems existing in the prior art, the present invention proposes a differential privacy protection method against inference attacks for task offloading strategies in an edge computing environment. The method mainly includes:

[0005] Establish system models, transmission models, computational models, and user task models in mobile edge computing scenarios, and construct an offloading decision framework based on the Markov decision process (MDP);

[0006] Build a privacy model for inference attacks, infer the device's uninstallation strategy based on the system status, and evaluate the privacy loss;

[0007] On this basis, the phased smoothing sensitivity is calculated, and the offloading strategy is protected with local differential privacy in combination with privacy loss. An exponential mechanism is used to perturb the offloading strategy.

[0008] The noise intensity is dynamically adjusted according to the privacy loss and smoothing sensitivity of the overall iteration cycle until the model converges.

[0009] Advantages and effects of the present invention:

[0010] 1) The present invention is applicable to highly dynamic and open edge computing environments. Aiming at the privacy leakage risks faced by task offloading strategies, a differential privacy protection method that is resistant to inference attacks is proposed. By constructing a privacy model and introducing a differential privacy mechanism, it effectively prevents attackers from using algorithms such as inverse reinforcement learning (IRL) to infer user offloading strategies and preferences, thereby comprehensively protecting user privacy and security. This method combines the characteristics of deep reinforcement learning models to provide a general privacy protection solution that is widely applicable to edge computing scenarios such as the Internet of Vehicles and the Industrial Internet of Things. It also has good practicality and scalability through modular design, providing reliable support for the secure application of edge computing.

[0011] 2) This invention utilizes a Markov decision process (MDP) to construct an offloading decision framework. In a dynamic environment, it uses phased smoothing of sensitivity calculations and dynamic adjustment of noise intensity to ensure that the differential privacy protection mechanism can adapt to changes in system resources, achieve precise perturbations to offloading strategies, and effectively balance privacy protection with task offloading performance. By constructing a privacy model and introducing a differential privacy mechanism, this invention effectively prevents the privacy leakage risks brought about by inference attacks. While protecting user privacy, it maximizes the performance and service quality of the task offloading strategy, achieving dual optimization of privacy protection and system efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 Schematic diagram of the edge computing scenario of the present invention

[0013] Figure 2 This is a flowchart of a differential privacy protection method against inference attacks for a task offloading strategy in an edge computing environment according to an embodiment of the present invention;

[0014] Figure 3 This is a flow chart of a differential privacy protection method against inference attacks for a task offloading strategy in an edge computing environment in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0016] Figure 1 This is a schematic diagram of the edge computing scenario of the present invention, as shown in Figure 1 As shown, the edge computing environment includes multiple edge nodes, IoT devices, and mobile terminals, supporting the computing needs of scenarios such as smart transportation, autonomous driving, and the Industrial Internet of Things. When a device generates a task, it requests the edge server to offload the computation. However, this task offloading process may leak device privacy and offloading preferences. In particular, under malicious attacks, attackers can infer the user's offloading strategy, posing a privacy risk. Existing methods focus primarily on data privacy protection. This example bypasses this consideration and focuses solely on the device's offloading strategy. We propose a differential privacy protection method for task offloading strategies in edge computing environments that is resistant to inference attacks, thereby enhancing the privacy security of task offloading.

[0017] Figure 2 This is a flowchart of a differential privacy protection method for anti-inference attack of a task offloading strategy in an edge computing environment of the present invention. Figure 2 As shown, the method includes:

[0018] The present invention provides a differential privacy protection method against inference attacks for task offloading strategies in edge computing environments. Figure 2 As shown, the method includes:

[0019] S1. Construct an offloading decision model framework based on Markov decision process (MDP) in the mobile edge computing scenario, including system model, transmission model, calculation model and user task model;

[0020] In this embodiment of the present invention, in step S1, a system model, a transmission model, a calculation model, and a user task model are constructed to build an offloading decision framework based on a Markov decision process (MDP), specifically including:

[0021] S11. Establish a system model and user task model for the mobile edge computing scenario. The system model includes a set of mobile devices (MDs) and edge nodes (ENs), denoted as I = {1, 2, ..., i} and J = {1, 2, ..., j} respectively. The time is divided into a specific segment consisting of t time slots, denoted as T = {1, 2, ..., t}. Each time slot lasts for τ seconds. Each MD maintains a local computing and sending queue, and tasks are queued in a first-in-first-out manner. Each EN corresponds to I queues, which are associated with each MD. The user task model defines task z i (t) is the task index that arrives at MD i∈I at time slot t∈T, and its size is λ i (t)∈{0}∪Λ={λ1,λ2,...,λ θ}, where θ represents the number of tasks, Δ i (t) represents the task deadline, and the binary variable x i (t) indicates whether the task is processed locally or sent, y i,j (t) Indicates whether the task is offloaded to a specific EN.

[0022] S12, in the task transmission queue, the task is dispatched to the appropriate edge node (EN) and transmitted through the network interface of the mobile device (MD). The transmission rate of MD and ENj in time slot t is set to r i,j (t), Indicates the time slot when the task is discarded or the transmission is completed, Indicates the number of time slots a task must wait in the queue before being transmitted, defined as:

[0023]

[0024] in, If MD schedules task z in time slot t i (t) is transmitted, the task will be Complete the transfer or discard.

[0025]

[0026] in represents the number of time slots required for the task to go from MD i to ENj, which is defined as:

[0027]

[0028] make is the energy consumption of task transmission from MD i to ENj, then:

[0029]

[0030] in, is the communication power consumption of MD i in time slot t.

[0031] S13. If the task is assigned to the local computing queue, set the processing rate of MD i to f i ,definition The time at which the task will be processed or discarded in this time slot. The number of time slots that a task waits in the local computation queue is defined as:

[0032]

[0033] in, If MD schedules a task for local computation at time slot t, the task will be Complete the calculation or be discarded.

[0034]

[0035] in represents the number of time slots required for the task to complete processing from MD i, and is defined as:

[0036]

[0037] make is the local computing energy consumption of MD i, then:

[0038]

[0039] in, is the processing power consumption of MD i.

[0040] S14. If the task is assigned to the edge node computing queue, set the processing rate of ENj to definition The time when the task is processed or discarded on ENj. The number of time slots that a task waits in the local computation queue is defined as:

[0041]

[0042] in, If MD i transfers a task to ENj to perform computation at time slot t, the task will be Complete the calculation or be discarded.

[0043]

[0044] in represents the number of time slots required for the task to complete processing from MD i, and is defined as:

[0045]

[0046] make is the computational energy consumption in ENj during the task, then:

[0047]

[0048] in, is the processing power consumption of MDj.

[0049] S15. Based on the system model, transmission model, calculation model, and user task model established in the above steps, a Markov decision process (MDP) offloading decision framework is constructed as follows:

[0050] State Space:

[0051]

[0052] Among them, the state vector S i (t) consists of task size, local computing queue length, transmission queue length, edge node queue length, and historical data. represents the queue length of MD i task on ENs at time slot t-1, T tra (t) is historical data.

[0053] Action Space:

[0054] a i (t)=(x i (t)y i (t))

[0055] Among them, a i (t) indicates whether the task is computed locally or offloaded to the edge node ENj.

[0056] Activation function:

[0057]

[0058] Among them, q i (s i (t),a i (t)) represents the reward for the task being discarded or completing the calculation, R>0 represents the fixed reward for task completion, D i (s i (t),a i (t) is the delay consumption, E i (s i (t),a i (t)) is the energy consumption, ω1, ω2.

[0059] are the weights of latency and energy consumption respectively.

[0060]

[0061] If the task is discarded, then D i (s i (t),a i (t)=0.

[0062]

[0063] If the task is discarded, then E i (s i (t),a i (t)=0.

[0064] Uninstall strategy:

[0065]

[0066] in, is the optimal strategy, and γ∈(0,1] is the discount factor.

[0067] S2. Build a privacy model for inference attacks, infer the device's uninstallation strategy based on the system status, and evaluate the privacy loss;

[0068] In an embodiment of the present invention, a privacy model is constructed to infer the device's uninstallation strategy based on the system status and evaluate the privacy loss, specifically including:

[0069] S21. The attacker with prior knowledge uses the user's uninstall experience T tra (t), the reward function is inferred through inverse reinforcement learning to obtain the reward function. The result of the attack is defined as follows:

[0070] h R (R infer |t)=Pr{d R (t)= <t,R infer >|T tra (t)}

[0071] where d R (t)= <t,R infer > indicates that the attacker infers the reward function R of the current model through inverse reinforcement learning based on the historical experience trajectory at time slot t infer , h R (R infer |t) indicates that the attacker inferred d R (t)= <t,R infer >probability.

[0072] To evaluate the attack results, define the function,E R (R,R infer) represents the correctness between the adversary’s inferred reward function and the user’s actual reward function, and is defined as follows:

[0073]

[0074] Among them G R , In R, R infer The cumulative reward of a trajectory under the reward function, where G R =R t +γR t+1 +γ 2 R t+2 ..., until the trajectory ends. Similarly, Where γ is the discount factor, e R is the maximum reward difference value.

[0075] The inference event means that at time t, the attacker infers the uninstallation strategy of the current model based on the previous experience trajectory and reward function.

[0076] S22. The attacker with prior knowledge uses the user's uninstall experience T tra (t), through inverse reinforcement learning, the unloading strategy is inferred to obtain the unloading strategy π infer The results of the attack are defined as follows:

[0077] h π (π infer |t)=Pr{d π (t)= <t,π infer >|T tra (t)}

[0078] To evaluate the attack results, we define a function that represents the correctness between the adversary's inferred uninstallation and the user's actual uninstallation strategy, which is defined as follows:

[0079]

[0080] Where P(t tra ), Q(t tra ) are strategies π, π infer The behavior distribution under is the KL divergence. The smaller the KL divergence, the closer the two distributions are. π is the maximum KL divergence.

[0081] S23. Privacy loss is quantified as the expectation of a correct attack (i.e., successfully inferring the model's reward function and uninstallation policy). Privacy loss is defined as follows:

[0082]

[0083] Among them, ρ t represents the privacy loss of the uninstallation strategy at time t.

[0084] S3. On this basis, the phased smoothing sensitivity is calculated, and the differential privacy method with an exponential mechanism is used to perturb the offloading strategy in combination with privacy loss.

[0085] In an embodiment of the present invention, the phased smoothing sensitivity is calculated, and the differential privacy method of the exponential mechanism is used to perturb the offloading strategy in combination with the privacy loss, specifically including:

[0086] S31. Calculate stage smoothing sensitivity

[0087] Let's assume that Q(s i (t),α i (t))(abbreviated as ) is R max , the minimum value is R min , then the global sensitivity GS f =R max -R min . Then the local sensitivity can be expressed as:

[0088]

[0089] where R n-1 is the second largest value, then the sensitivity when the distance is K is:

[0090]

[0091] where R k+1 ,R k+2 are the modified k+1th and k+2th maximum values, then the smoothing sensitivity is:

[0092]

[0093] in is the privacy budget at time t.

[0094] To better illustrate that the smoothed sensitivity mechanism of the present invention satisfies the definition of ε-differential privacy, this embodiment analyzes and proves the smoothed sensitivity mechanism of the present invention as follows:

[0095] First, we prove that the smoothing sensitivity is the upper bound of the local sensitivity as follows:

[0096] When k = 0, we have Substitute k = 0 into have therefore Local sensitivity The upper bound of .

[0097] Secondly, it is proved that the smoothing sensitivity satisfies the smoothness property as follows:

[0098] consider and have:

[0099]

[0100]

[0101] For any k, the sensitivity of distance k satisfy:

[0102]

[0103] because The k+1 neighborhood of k-neighborhood of .

[0104] Therefore, the smooth sensitivity change satisfies:

[0105]

[0106] and but

[0107]

[0108] therefore:

[0109]

[0110] S32. Combining privacy loss and smoothing sensitivity, a differential privacy method with an exponential mechanism is used to implement a disturbance offloading strategy.

[0111]

[0112] Among them, P(s i (t),a i (t)) represents the selection state and action combination s i (t),a i The probability of (t).

[0113] To better illustrate that the index mechanism of the present invention satisfies the definition of ε-differential privacy, this example analyzes and proves the index mechanism of the present invention as follows:

[0114] To simplify the proof process, the exponential mechanism is abbreviated as follows:

[0115]

[0116] According to ε t -The definition of differential privacy is:

[0117]

[0118] Split the two items, there are

[0119]

[0120] Because Q(s i (t),a i (t)) changes by at most ΔQ, that is:

[0121]

[0122] So, we have:

[0123]

[0124] For any a' i (t), Q(s i (t),a i (t)) satisfies:

[0125]

[0126] Therefore, we have:

[0127]

[0128] Combining the two, we have:

[0129]

[0130] Therefore, the exponential mechanism satisfies ε t -Differential privacy

[0131] S4. Dynamically adjust the noise intensity according to the privacy loss and smoothing sensitivity of the overall iteration cycle until the model converges.

[0132] The overall differential privacy mechanism of the present invention is composed of a combination of differential privacy exponential mechanisms at different stages. By calculating the privacy loss and smoothing sensitivity at different stages, the perturbation intensity is dynamically adjusted to accelerate the convergence of the model, thereby maximizing the model utility while ensuring privacy.

[0133] To better illustrate that the differential privacy mechanism of the present invention satisfies the definition of ε-differential privacy, this embodiment analyzes and proves the differential privacy mechanism of the present invention as follows:

[0134] The privacy budget at different stages is The privacy budget of the overall iteration cycle is defined as:

[0135]

[0136] According to the differential privacy combination theorem,

[0137] There are T independent differential privacy mechanisms, each P(a i (t)|s i (t)) all satisfy T tra ε of (t) t -Differential privacy.

[0138] So the combination mechanism P(a(t)|s(t))={P(a i (1)|s i (1)),...,P(a i (t)|s i (t))} satisfies -Differential privacy.

[0139] Therefore, P(a(t)|s(t)) satisfies ε-differential privacy.

[0140] The embodiments of the present invention can effectively balance the efficiency of the task offloading strategy and the privacy protection of the offloading strategy, and can dynamically adapt to changes in the task offloading environment.

[0141] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: ROM, RAM, disk or CD, etc.

[0142] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A differential privacy protection method against inference attacks for task offloading strategy in edge computing environment, characterized in that: The method comprises: Construct an offloading decision model framework based on Markov decision process (MDP) in the mobile edge computing scenario, including system model, transmission model, calculation model and user task model; Build a privacy model for inference attacks, infer the device's uninstallation strategy based on the system status and evaluate the privacy loss; On this basis, the phased smoothing sensitivity is calculated, and the differential privacy method with exponential mechanism is used to perturb the offloading strategy in combination with privacy loss. The noise intensity is dynamically adjusted according to the privacy loss and smoothing sensitivity of the overall iteration cycle until the model converges.

2. According to claim 1, the method for protecting differential privacy against inference attacks of a task offloading strategy in an edge computing environment is characterized in that: A system model for mobile edge computing scenarios is established. The system model includes a set of mobile devices (MDs) and edge nodes (ENs), denoted by I = {1, 2, ..., i} and J = {1, 2, ..., j}, respectively. The time is divided into specific segments consisting of t time slots, denoted by T = {1, 2, ..., t}. Each time slot lasts for τ seconds. Each MD maintains a local computing and sending queue, and tasks are queued according to first-in-first-out. Each EN corresponds to I queues, which are associated with each MD.

3. According to claim 1, the method for protecting differential privacy against inference attacks of a task offloading strategy in an edge computing environment is characterized in that: Establish a user task model in the mobile edge computing scenario and define task z i (t) is the task index that arrives at MDi∈I at time slot t∈T, and its size is λ i (t)∈{0}∪Λ={λ1,λ2,...,λ θ }, where θ represents the number of tasks, Δ i (t) represents the task deadline, and the binary variable x i (t) indicates whether the task is processed locally or sent, y i,j (t) Indicates whether the task is offloaded to a specific EN.

4. According to claim 1, the method for protecting differential privacy against inference attacks of a task offloading strategy in an edge computing environment is characterized in that: The system transmission model is expressed as: in, represents the time slot when the task is discarded or the transmission is completed, represents the number of time slots required to complete the transmission, Indicates the waiting time in the transmission queue of the task. is the energy consumption of task transfer from MDi to ENj, is the communication power consumption of MD i in time slot t.

5. According to claim 1, the method for protecting differential privacy against inference attacks of a task offloading strategy in an edge computing environment is characterized in that: The calculation model is expressed as: Local computing model: in, represents the time slot when the task is abandoned or the computation is completed, represents the number of time slots required to complete the calculation, Indicates the waiting time of the task in the computing queue. is the energy consumption of local computing tasks, It is the CPU power consumption of MD i. Edge computing model: in, represents the time slot when the task is discarded or the computation is completed. The task reaches the server at time t'. Indicates the time slots when other tasks in the queue complete their computation or are discarded. represents the number of time slots required to complete the calculation, Calculate the energy consumption of tasks for edge nodes, is the CPU power consumption of ENj.

6. According to claim 1, the method for protecting differential privacy against inference attacks of a task offloading strategy in an edge computing environment is characterized in that: The offloading decision framework of Markov decision process (MDP) is expressed as: State Space: Among them, the state vector S i (t) consists of task size, local computing queue length, transmission queue length, edge node queue length, and historical data. represents the queue length of MD i task on ENs at time slot t-1, T tra (t) is historical data. Action Space: a i (t)=(x i (t)y i (t)) Among them, a i (t) indicates whether the task is computed locally or offloaded to the edge node ENj. Activation function: Among them, q i (s i (t),a i (t)) represents the reward for the task being discarded or completing the calculation, R>0 represents the fixed reward for task completion, D i (s i (t),a i (t) is the delay consumption, E i (s i (t),a i (t)) is the energy consumption, ω1, ω2 are the weights of delay and energy consumption respectively. Uninstall strategy: in, is the optimal strategy, and γ∈(0,1] is the discount factor.

7. According to claim 1, the method for protecting differential privacy against inference attacks of a task offloading strategy in an edge computing environment is characterized in that: The privacy model is Among them, ρ t represents the privacy loss of the uninstallation strategy at time t, h R (R infer |t) indicates that the attacker is at time t, with T tra (t) is used as a premise to infer the reward function R infer Probability, E R (R,R infer ) represents the correctness between the adversary’s inferred reward function and the user’s actual reward function, h π (π infer |t) indicates that the attacker is at time t, with T tra (t) is used as a premise to infer the unloading strategy π infer The probability of E π (π,π infer ) represents the correctness between the uninstallation strategy inferred by the adversary and the actual uninstallation strategy of the user.

8. The method for protecting differential privacy against inference attacks of task offloading strategies in edge computing environments according to claim 1 is characterized in that: The reward value of the t time slot is obtained through the Markov decision process (MDP)-based unloading decision framework in claim 5, and the smoothed sensitivity of the current reward is calculated. Subsequently, the smoothed sensitivity is combined with the privacy loss in claim 6, and an exponential mechanism is used to perturb the unloading strategy.

9. According to claim 7, the method for protecting differential privacy against inference attacks of a task offloading strategy in an edge computing environment is characterized in that: The smoothing sensitivity is calculated by: in, represents the state-action function Q(s at time t i (t),a i (t)) value, A k (Q(s i (t),a i (t))) represents Q(s) at distance k i (t),a i (t)) value sensitivity, is the privacy budget at time t. The ways in which the exponential mechanism perturbs the unloading strategy include: Among them, P(s i (t),a i (t)) represents the selection of state and action combination s i (t),a i The probability of (t).

10. According to claim 1, the method for protecting differential privacy against inference attacks of a task offloading strategy in an edge computing environment is characterized in that: According to the offloading decision framework of Markov decision process (MDP), the privacy loss and smoothing sensitivity at different stages are calculated, and the noise intensity is dynamically adjusted in the iterative cycle until the model converges.

Images