Internet of Things dynamic spectrum access method and system based on zero sample machine forgetting

Abstract

The invention provides a dynamic spectrum access method based on zero sample machine forgetting. According to the method, a deep Q network teacher model is constructed to carry out spectrum access training, a generative adversarial network is adopted to generate high-quality pseudo data, and a gating knowledge distillation mechanism is designed to realize selective forgetting. The core of the method is that machine forgetting can be realized without accessing original training data, and the defects of a traditional method in the aspect of privacy protection are effectively overcome. The method comprises the following steps: firstly, defining a state space containing a channel, a user demand and a geographic position, and designing a reward function considering basic reward, channel utilization efficiency, user satisfaction and interference influence; on the basis, model training is realized through a teacher-student network architecture and a knowledge distillation technology, and the effectiveness of the method is verified by adopting multiple evaluation indexes. The method maintains good spectrum access performance while protecting user privacy, is high in calculation efficiency, is easy to deploy, and is suitable for dynamic spectrum management in an Internet of Things environment.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things and wireless communication technology, and in particular to a dynamic spectrum access method based on zero-sample machine forgetting. The method achieves the goal of improving spectrum utilization efficiency while protecting user privacy through deep reinforcement learning and machine forgetting technology.

[0002] The present invention is mainly applied to dynamic spectrum management in the IoT environment, including but not limited to: vehicle networking communication in intelligent transportation systems, device interconnection in smart cities, device collaboration in industrial IoT, and smart home scenarios. Through innovative machine learning methods, the present invention provides a new technical path to resolve the contradiction between spectrum resource shortage and user privacy protection in the IoT environment. Technical Background

[0003] With the commercial deployment of 5G technology and the rapid development of IoT technology, the number of smart devices has exploded. According to statistics, the number of IoT-connected devices worldwide is expected to exceed 75 billion by 2025. The access of these devices has brought tremendous pressure to wireless communication networks, especially the increasing demand for spectrum resources. The traditional static spectrum allocation method can no longer meet the growing communication needs. How to improve the efficiency of spectrum utilization has become an important research topic in the current wireless communication field.

[0004] Dynamic spectrum access (DSA) technology plays an important role in the Internet of Things as a key technology to improve spectrum utilization efficiency. Traditional DSA technology mainly relies on machine learning and deep learning methods to predict and manage spectrum allocation to optimize spectrum utilization efficiency. In recent years, with the development of deep reinforcement learning technology, spectrum access methods based on deep Q network (DQN) have shown good performance and can effectively learn the optimal spectrum access strategy in complex environments.

[0005] However, these machine learning-based methods face severe challenges in practical applications. In terms of performance, traditional DSA methods focus too much on spectrum sensing and spectrum sharing strategies, ignoring the balance between energy efficiency and long-term spectrum utilization efficiency. In a multi-user competition environment, spectrum access conflicts still exist. In addition, existing algorithms are not adaptable enough in dynamically changing wireless environments, and the model converges slowly, making it difficult to ensure real-time performance.

[0006] In terms of privacy protection, these technologies face greater challenges when they encounter strict data privacy and protection regulations, such as the EU's General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA). Existing machine learning models need to collect and process a large amount of user behavior data to train spectrum management models. These data may contain sensitive information such as user location information and usage habits. Not only is there a risk of privacy leakage during model training, but when users request to delete or withdraw their data, the data that forms the basis of model training also needs to be removed.

[0007] In order to meet the challenge of data privacy protection, machine unlearning technology has emerged. This technology aims to make the trained machine learning model "forget" the learned data. There are currently three main machine unlearning methods: complete retraining method, approximate forgetting method and incremental update method. The complete retraining method is to retrain the model after deleting the specified data from the training set, but this method has high computational overhead and low efficiency, and is difficult to implement in large-scale models. The approximate forgetting method simulates the retraining effect by optimizing the model parameters. Although the computational efficiency is high, the accuracy is difficult to guarantee, and there may be data residual problems. The incremental update method only updates the model parameters related to the data to be forgotten, but it needs to save the intermediate state during the training process, and the storage overhead is large.

[0008] These existing methods all have some common problems. The first is the dependence on the original training data. Most methods require access to the original training data, but in actual applications, the original data may have been deleted or inaccessible, which goes against the original intention of data privacy protection. The second is the performance loss problem. The forgetting process may cause the model performance to drop significantly. It is difficult to strike a balance between protecting privacy and maintaining performance, which affects the generalization ability of the model. Finally, there is the problem of implementation complexity. The existing methods are complex to implement, difficult to deploy, require large computing resources, and are difficult to run on resource-constrained IoT devices.

[0009] Therefore, a new technical solution is urgently needed to solve the above problems. This paper proposes a dynamic spectrum access method based on zero-sample machine forgetting. Through the teacher-student network strategy and knowledge distillation technology, machine forgetting is achieved without directly using training data, providing a new solution for IoT spectrum management. This method has the following main contributions:

[0010] (1) A machine forgetting method is proposed that does not require access to the original training data. The gated knowledge distillation mechanism ensures that the model maintains its core performance while forgetting the specified data, fundamentally solving the problem of data privacy protection.

[0011] (2) We designed a novel teacher-student network architecture, which generates high-quality pseudo data through a generative adversarial network, achieves effective knowledge transfer, and strikes a good balance between protecting privacy and maintaining performance.

[0012] (3) We developed an algorithmic framework that is computationally efficient and easy to deploy, which significantly reduces computational and storage overheads, enabling this method to run effectively on resource-constrained IoT devices and has broad practical value. Summary of the invention

[0013] The purpose of the present invention is to provide a dynamic spectrum access method based on zero-sample machine forgetting, aiming to solve the following technical problems:

[0014] 1. Solve the shortcomings of traditional dynamic spectrum access methods in protecting user privacy;

[0015] 2. Solve the problem that existing machine forgetting methods rely on original training data;

[0016] 3. Solve the problem of model performance degradation during machine forgetting;

[0017] 4. Provide a computationally efficient and easy-to-deploy machine forgetting solution.

[0018] To achieve the above object, the technical solution provided by the present invention mainly includes the following steps:

[0019] 1. Construction of Deep Q Network Teacher Model

[0020] A deep Q network (DQN) teacher model is constructed for training the dynamic spectrum access task. The state space of the model is defined as:

[0021] S={(c,d,l)|c∈C,d∈D,l∈L}

[0022] Where C is the channel set, D is the user demand set, and L is the geographical location set. The state vector contains three key features:

[0023] ○Channel number c: indicates the available communication channel

[0024] ○ User demand d: indicates the user's bandwidth demand

[0025] ○Geographic location l: indicates the user's spatial location information

[0026] The teacher model is trained using the ε-greedy strategy. When the random number r < ε, the action is randomly selected; when the random number r ≥ ε, the action with the largest Q value is selected:

[0027] A t = argmax a∈A Q(St , a)

[0028] The reward function of the model is designed as:

[0029] R t+1 =B(C t , A t )+α·U(C t , A t )+β·V(d,A t )-γ·W(l,A t )

[0030] Where B(C t , A t ) is the basic reward, U(C t , A t ) is the channel utilization efficiency, V(d, A t ) is user satisfaction, W(l, A t ) is the interference effect.

[0031] The specific calculation method of each reward is as follows:

[0032] (1) Basic reward B (C_t, A_t):

[0033]

[0034] (2) Channel utilization efficiency U(C_t, A_t):

[0035] U(C t , A t ) = η b ·B e (C t , A t )·(1-α c ·C f (C t ))

[0036] where η b is the basic efficiency coefficient, B e is the channel basic efficiency, C f is the congestion factor, α c is the congestion weight.

[0037] (3) User satisfaction V(d, A_t):

[0038]

[0039] Where BW(A t ) is the bandwidth of the selected channel.

[0040] (4) Interference influence W(1, A_t):

[0041] W(l,A t )=∑ i∈N(l) w i ·I(A t , A i )

[0042] Where N(l) is the neighbor set of position 1, w i is the interference weight, I(A t , A i ) is the channel interference function.

[0043] 2. Data Generation Based on Generative Adversarial Networks

[0044] The present invention uses a generative adversarial network (GAN) to construct a generator G for generating pseudo data for knowledge transfer. The loss function of the generator is designed as:

[0045] L G (φ)=COS(T(x p ), s(x p ;θ))

[0046] where x p is the generated pseudo data, T and S are the outputs of the teacher model and the student model respectively.

[0047] 3. Gated Knowledge Distillation Mechanism

[0048] Design a gating mechanism F to filter the generated data, and its filtering conditions include:

[0049] (1) Eigenvalue check:

[0050]

[0051] (2) Data transmission conditions:

[0052] X pass ={x p |F(x p )=1,x p ∈X p}

[0053] When it comes to geolocation features, the gating mechanisms are categorized as follows:

[0054] When 1 is a discrete value:

[0055] D f ={s∈S:l∈V f}

[0056] D r ={s∈S:l∈V r}

[0057] When 1 is a continuous value:

[0058] D f ={s∈S:l<θ}

[0059] D r ={s∈S:l≥θ}

[0060] Where D f is the forgotten data set, D r To retain the data set, θ is the preset threshold.

[0061] 4. Student model training

[0062] The training process of the student model includes:

[0063] (1) Generate data filtering:

[0064] X train ={x|F(x)=1,x∈G(z;φ)}

[0065] (2) Loss calculation:

[0066]

[0067] Where α and β are weight coefficients. The calculation method of attention loss is:

[0068]

[0069] in

[0070] The student model parameters are updated using the gradient descent method:

[0071]

[0072] where η θ is the learning rate.

[0073] 5. Model Validation

[0074] To verify the effectiveness of the model, the present invention designs three evaluation indicators:

[0075] (1) Performance evaluation of reserved dataset:

[0076]

[0077] (2) Verification of forgetting effect:

[0078]

[0079] (3) Privacy protection capability assessment:

[0080] Pprivacy =1-AUC(MIA(S))

[0081] Where MIA represents membership inference attack and AUC is the area under the ROC curve.

[0082] 6. Beneficial effects

[0083] The technical solution provided by the present invention has the following beneficial effects:

[0084] (1) The present invention has a significant effect in privacy protection. Machine forgetting can be achieved without accessing the original training data. The gated knowledge distillation mechanism effectively prevents privacy data leakage. At the same time, it can effectively resist privacy attacks such as member reasoning attacks, fundamentally ensuring user data security.

[0085] (2) The present invention has excellent model performance. It maintains a performance level comparable to that of the teacher model on the retained dataset, but significantly reduces the model prediction accuracy on the forgotten dataset, successfully achieving a good balance between privacy protection and performance maintenance. This selective forgetting feature ensures that the model will not affect normal business functions while protecting user privacy.

[0086] (3) The present invention has a low deployment cost. Due to the use of efficient algorithm design, the computational complexity is low and it is suitable for running on resource-constrained IoT devices. At the same time, this solution does not need to save the original training data, which greatly reduces the storage overhead. In addition, the training process is simple and intuitive, easy to implement and deploy, and is conducive to technology promotion.

[0087] (4) The present invention has broad application prospects. It can be directly applied to various IoT dynamic spectrum access scenarios to solve the privacy protection problem in spectrum resource management. The core idea of ​​this technical solution can be extended to other machine learning application fields that require privacy protection, providing a new research idea for the development of privacy protection technology, which has important theoretical value and practical significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] The present invention includes the following drawings:

[0089] Figure 1 This is the overall architecture diagram of the dynamic spectrum access method based on zero-sample machine forgetting of the present invention, which mainly includes the following core components:

[0090] ○Teacher model: pre-trained Spectrum Access DQN model, used as the source model for knowledge transfer

[0091] ○Student model: The target model that needs to be machine-forgotten, learning the core knowledge of the Teacher model through knowledge distillation

[0092] ○Generator: used to generate high-quality pseudo data and avoid access to original training data

[0093] ○Filter: Filter the generated pseudo data to ensure data quality

[0094] ○Attention difference module: calculates the output difference between the Teacher and Student models to guide the selective forgetting process

[0095] ○ Mean absolute and Cosine similarity loss functions: used to evaluate and optimize the knowledge transfer effect between models

[0096] Figure 2 This is the training flow chart of the deep Q learning network of the present invention, showing the complete process of interaction between the reinforcement learning agent and the environment:

[0097] ○ Environment: Contains information such as spectrum resource status, user requests, etc., receives the agent's actions and returns rewards and new status

[0098] ○Memory Unit (Experience Replay Unit): Stores the experience tuple (x_t, a_(t+1), r_(t+1), x_(t+1)) of the agent’s interaction with the environment, and provides training samples through Mini-Batch sampling

[0099] ○Q-Network: Evaluates the value of state-action pairs and outputs Q-values ​​for action selection

[0100] ○Target Network: A cloned version of the Q network, used to calculate the target Q value and improve training stability

[0101] ○ Loss Function: Calculates the error between the predicted Q value and the target Q value, and optimizes the network parameters through gradient descent

[0102] Figure 3 This is a comparison diagram of the experimental performance of the present invention, including three sub-diagrams:

[0103] ○(a) shows the comparison of the convergence steps between multi-agent and single-agent DQN, reflecting the learning efficiency

[0104] ○(b) Comparison of the performance of the two methods in terms of reward value, reflecting the decision quality

[0105] ○(c) shows the comparison of the spectrum access success rate between the two methods, verifying the actual application effect

[0106] Figure 4This is the result diagram of the zero-sample machine forgetting training of the present invention, including multiple groups of comparisons:

[0107] ○ In the reserved data set Performance on , including convergence steps, rewards, and success rates

[0108] ○In the forgotten dataset Performance on the test to verify the forgetting effect

[0109] ○ Convergence comparison under different training strategies reflects the stability of the method

[0110] Figure 5 This is a diagram of the interaction between the agent and the environment in reinforcement learning, which explains:

[0111] ○The process of the agent observing the state of the environment and making action decisions

[0112] ○ The environment gives rewards and new states based on the agent’s actions

[0113] ○ Agents optimize their decision-making strategies through continuous interaction

[0114] Figure 6 This is the ROC curve of the federated learning model, showing:

[0115] ○ The privacy protection ability of the model under membership inference attacks

[0116] ○The ROC curve is close to the upper left corner, indicating good classification ability

[0117] ○AUC value is 0.79, reflecting the overall performance level

[0118] Figure 7 This is the ROC curve of the machine forgetting model, which shows:

[0119] ○The model’s performance in privacy protection

[0120] ○The ROC curve shape is similar to the federated learning model

[0121] ○ The AUC value is 0.69, showing stronger privacy protection capabilities

[0122] Figure 8 This is a comparison chart of ablation experiment results, which contains three sub-pictures:

[0123] ○ The top row shows the training effect when using cosine similarity as the loss function

[0124] ○ The lower row shows the experimental results after adjusting the hyperparameters of the attention mechanism

[0125] ○ Each set of results includes a comparison of three dimensions: number of steps, rewards, and success rate DETAILED DESCRIPTION

[0126] 1. Experimental environment configuration

[0127] The specific implementation environment of the present invention is as follows:

[0128] (1) Hardware environment configuration

[0129] The processor uses GPU acceleration, the memory configuration is 8GB or more, and the storage device is SSD 256GB or more.

[0130] (2) Software environment configuration

[0131] The operating system supports Linux / Windows, the development language uses Python 3.7 and above, and the deep learning framework uses PyTorch 1.8 and above.

[0132] (3) DQN teacher model parameter configuration

[0133] The memory pool size is set to 500*5, the mini-batch size is 32, the target network update frequency is 100, the ε-greedy strategy parameter is 0.9, the discount factor γ is 0.9, the learning rate is set to 0.1, the activation function is ReLU, and the optimizer is Adam.

[0134] (4) Zero-shot forgetting training parameter configuration

[0135] The random noise vector dimension is 100, the state feature dimension is 3, the generator iterates 1 time per round, the student model iterates 4 or 10 times per round, the exploration probability is 0.1, the generation batch size is 32, the number of training rounds is 100, the attention loss hyperparameter is 250, and the gradient clipping threshold is 5.

[0136] 2. Experimental scene setting

[0137] Spectrum environment simulation:

[0138] ○Total number of channels: 10

[0139] ○ Initial occupied channels: 5

[0140] ○Each channel feature contains: (c, d, 1) three dimensions

[0141] ○Channel state transition probability matrix:

[0142]

[0143] Dataset division:

[0144] ○ Divide the dataset based on geographic location feature 1:

[0145] ■When l is a discrete value:

[0146] D f ={s∈S:l∈V f}

[0147] D r ={s∈S:l∈V r}

[0148] ■When l is a continuous value:

[0149] D f ={s∈S:l<θ}

[0150] D r ={s∈S:l≥θ}

[0151] ○ Where θ is the preset threshold, usually 0.5

[0152] 1. Reward function design:

[0153] R(s t , a t )=B(C t , A t )+αU(C t , A t )+βV(d,A t )-γW(l,A t )

[0154] in:

[0155] ○B(C t , A t ) is the basic reward

[0156] ○U(C t , A t ) is the channel utilization

[0157] ○V(d,A t ) is user satisfaction

[0158] ○W(l,A t ) is the interference effect

[0159] 3. Experimental Procedure

[0160] Teacher model training phase

[0161] First, the deep Q network is initialized and the state space and action space parameters are set. An experience replay pool is created to store the interaction data between the agent and the environment, and the ε-greedy strategy is used to strike a balance between exploration and exploitation. The agent obtains rewards by interacting with the environment and stores the experience data in the replay pool. The system randomly samples batch data from the replay pool to update the network parameters, and regularly updates the target network to improve training stability. Repeat the above training process until the model converges.

[0162] Zero-shot forgetting training phase

[0163] In this stage, the generator network and the student DQN model are initialized. The generator generates random noise and pseudo data samples, and the data is quality-filtered through a gating mechanism. The filtered high-quality data is input into the teacher model and the student model at the same time, and the difference between the outputs of the two models is calculated. The student model parameters are updated in combination with the MSE loss and the attention loss, and the appropriate number of iterations is selected according to the specific training strategy. The training process is repeated until the student model reaches the expected performance index.

[0164] Model evaluation phase

[0165] In this phase, we comprehensively evaluate the performance of the student model. We first verify the model performance on the retained dataset, and then test the forgetting effect on the forgotten dataset. We evaluate the privacy protection ability of the model through membership inference attacks.

[0166] And compare and analyze the performance differences under different training strategies. Finally, comprehensively evaluate the advantages and disadvantages of the model in various indicators.

[0167] Parameter optimization stage

[0168] Comprehensive optimization of model performance is carried out, including adjusting the hyperparameters of the attention mechanism, optimizing the loss function design of the generator, and testing the combined effects of different training strategies to ultimately determine the optimal parameter configuration scheme.

[0169] 4. Experimental Results Analysis

[0170] (1) Multi-agent vs. single-agent performance comparison

[0171] Performance evaluation indicators include convergence speed Accumulated Rewards And the success rate

[0172] The experimental results show that although the number of steps of the multi-agent system is initially high, it quickly decreases and stabilizes at a low level. The average reward of the multi-agent system rises rapidly in the early stage and remains stable. The access success rate is very stable and close to 100% in the later stage of training. Compared with the single-agent method, the multi-agent method performs better in learning efficiency and stability.

[0173] (2) Analysis of Zero-Sampling Forgetting Effect

[0174] Key evaluation metrics include performance on the reserved dataset Forgetting effect verification And privacy protection capability P privacy =1-AUC(MIA(S)).

[0175] The experimental results show that cosine similarity as a loss function does not work well on the reserved dataset, while the strategy of MAE combined with the attention mechanism performs best, with fewer steps and higher success rate on the reserved dataset, and shows obvious forgetting effect on the forgotten dataset. In terms of training strategy, strategy 1 (generator training 1 time + student model training 10 times) works best, while excessively increasing the number of student model training times (20 times) will lead to model instability and overfitting.

[0176] (3) Ablation Experiment Results

[0177] The loss function design includes MSE loss Loss of attention And the total loss L total =αL MSE +βL AT .

[0178] Experiments have found that using the maximum cosine similarity as the generator loss function will cause the model to fail to converge, while the MAE loss function is more stable. In terms of the attention mechanism, when the hyperparameter is adjusted to 500, although it can converge quickly on the retained dataset, it also shows a convergence trend on the forgotten dataset, indicating that the forgetting effect is not ideal. The optimal configuration was finally determined to be MAE loss + appropriate attention weight + training strategy 1, which achieves effective privacy protection while maintaining model performance.

[0179] 5. Application Suggestions

[0180] (1) Parameter tuning suggestions

[0181] It is recommended to flexibly adjust the ε-greedy strategy parameters according to the specific application scenario and reasonably set the attention loss weight to balance the retention and forgetting effects. At the same time, it is necessary to consider the actual computing resource constraints and choose a suitable training strategy.

[0182] (2) Deployment considerations

[0183] During the deployment process, it is necessary to ensure that the hardware resources meet the model operation requirements, regularly evaluate the model performance and forgetting effect, and update the model parameters in a timely manner according to environmental changes.

[0184] (3) Expanded application direction

[0185] This method has broad application prospects and can be extended to other IoT scenarios. It is suitable for various deep reinforcement learning tasks and can be effectively combined with existing privacy protection technologies.

Claims

1. A dynamic spectrum access method based on zero-sample machine forgetting, characterized in that , including the following steps: (1) Construct a deep Q network teacher model for training dynamic spectrum access tasks, where the state vector is: S t =(c,d,l) It includes three features: channel number c, user demand d and geographical location l; (2) Construct the generator G based on the generative adversarial network, and its loss function is: L G (φ)=COS(T(x p ),S(x p ;i)) where x p is the generated pseudo data, T and S are the outputs of the teacher model and the student model respectively; (3) Setting the gating mechanism F to filter the generated data: Among them C f is the set of categories that need to be forgotten; (4) Train the student model, whose loss function includes mean square error loss and attention loss: Where λ is the weight coefficient of attention loss.

2. The method according to claim 1, characterized in that , the reward function of the deep Q network is: R t+1 =B(C t ,A t )+α·U(C t ,A t )+β·V(d,A t )-γ·W(l,A t ) Where B(C t ,A t ) is the basic reward, U(C t ,A t ) is the channel utilization efficiency, V(d,A t ) is user satisfaction, W(l,A t ) is the interference effect.

3. The method according to claim 1, characterized in that , the attention loss L AT The calculation method is: in 4. The method according to claim 1, characterized in that , the gating mechanism classifies data based on geographic location feature l: When l is a discrete value: D f ={s∈S:l∈V f } D r ={s∈S:l∈V r } When l is a continuous value: D f ={s∈S:l<θ} D r ={s∈S:l≥θ} Where D f For the forgotten data set, D r To retain the data set, θ is the preset threshold.

5. The method according to claim 1, characterized in that ,The parameters of the student model are updated using the gradient descent method: where η θ is the learning rate.

6. The method according to claim 1, characterized in that ,The state space of the deep Q-network teacher model is defined as: S={(c,d,l)|c∈C,d∈D,l∈L} Where C is the channel set, D is the user demand set, and L is the geographical location set.

7. The method according to claim 1, characterized in that , the calculation method of the mean square error loss is: Where y is the teacher model output, is the output of the student model, and n is the number of samples.

8. The method according to claim 1, characterized in that ,The training of the teacher model adopts the ε-greedy strategy: When the random number r < ε, the action is selected randomly; When the random number r ≥ ε, select the action with the largest Q value: A t =argmax a∈A Q(S t ,a)。 9. The method according to claim 2, characterized in that , the basic reward B(C t ,A t ) is defined as:

10. The method according to claim 2, characterized in that , the channel utilization efficiency U(C t ,A t ) is calculated as: U(C t ,A t )=η b ·B e (C t ,A t )·(1-a c ·C f (C t )) where η b is the basic efficiency coefficient, B e is the channel basic efficiency, C f is the congestion factor, α c is the congestion weight.

11. The method according to claim 2, characterized in that , the user satisfaction V(d,A t ) is calculated as: Where BW(A t ) is the bandwidth of the selected channel.

12. The method according to claim 2, characterized in that , the interference affects W(l,A t ) is calculated as: Where N(l) is the set of neighbors at position l, w i is the interference weight, I(A t ,A i ) is the channel interference function.

13. The method according to claim 1, characterized in that ,The filtering conditions of the gating mechanism include: (1) Eigenvalue check: (2) Data transmission conditions: X pass ={x p |F(x p )=1,x p ∈X p }。 14. The method according to claim 1, characterized in that ,The training process of the student model includes: (1) Generate data filtering: X train ={x|F(x)=1,x∈G(z;φ)} (2) Loss calculation: Where α and β are weight coefficients.

15. The method according to claim 1, characterized in that , also includes the verification step: (1) Performance evaluation of reserved dataset: (2) Verification of forgetting effect: (3) Privacy protection capability assessment: P privacy =1-AUC(MIA(S)) Where MIA represents membership inference attack and AUC is the area under the ROC curve.

Images