A member inference attack defense method and system based on strong member information prediction

Abstract

The present invention belongs to the field of machine learning technology and discloses a member inference attack defense method and system based on strong member information prediction. The method obtains a real training data set and divides it into a model training data set and a test data set that do not overlap with each other; the training data set is divided into two training data subsets; a teacher model is trained in the test data set, and two student sub-models are distilled out of the two training data subsets using the teacher model; the data in the two training data subsets are relabeled; and in the model prediction process, the defender uses the strong member information prediction model before the attacker attacks, determines the strong member information in the model prediction output, removes it, and retains the remaining content as the final output of the model. By determining and removing the strong member information in the model prediction results, the present invention prevents the member privacy leakage problem that exists in the model prediction process, and on this basis retains good model performance.

Description

Technical Field

[0001] The present invention belongs to the field of machine learning technology, and in particular relates to a member inference attack defense method and system based on strong member information prediction. Background Art

[0002] With the rapid development of big data technology and the gradual increase in computing power, machine learning has made significant progress and has been widely applied in various fields, demonstrating outstanding performance. Thanks to the increasing popularity of artificial intelligence (AI) services and the development of cloud computing, Machine Learning as a Service (MLaaS) is gradually maturing. MLaaS is a general term for automated and semi-automated cloud platforms, such as Baidu's PaddlePaddle and Amazon ML, which are often used to solve fundamental problems such as data preprocessing, model training, and future prediction. Based on this service, any customer with machine learning requirements can submit their own datasets to the service, meeting their needs for quickly training and deploying machine learning models, even without prior knowledge of machine learning methods.

[0003] While AI brings us immense convenience, it also presents increasingly prominent information security concerns, primarily manifesting in: 1) the risk of training data privacy leaks during model prediction services; and 2) the risk of data leaks during the model prediction process. These risks not only jeopardize the security of individual users' sensitive data but can also result in significant financial losses for AI service providers. Currently, machine learning models face security threats such as model extraction attacks, model inversion attacks, and membership inference attacks. This research focuses on defending against membership inference attacks in machine learning models under a black-box model. A membership inference attack involves an attacker, given a model and a specific sample, attempting to accurately infer whether the sample exists in the target model's training set. Because the training data of some AI models contains sensitive personal information, such as biometrics and medical data, insecure models can pose a threat to personal privacy when subjected to membership inference attacks.

[0004] Although many scholars have proposed various protection strategies against membership inference attacks, it is often difficult to balance model performance and security. Shokri et al. proposed hiding part of the model prediction information, retaining only the largest k categories in the prediction vector output by the model, or increasing the entropy of the model output vector by adding a temperature coefficient to the Softmax function, hoping to reduce the real output obtained by the attacker from the target model, thereby reducing the risk of attack. However, after relaxing the attack conditions, Salem et al. achieved a good attack effect by using only the largest three categories in the output vector in the attack. Shokri et al. [1] also believed that overfitting of the target model is an important cause of membership inference attacks, so they also proposed using L2 regularization technology to alleviate the overfitting phenomenon of the model, hoping to reduce the risk of membership inference attacks by improving the generalization ability of the model. Although L2 regularization technology can reduce the overfitting of the model to a certain extent, the excessive punishment of the model to defend against membership inference attacks will cause serious damage to the model performance. Abadi et al. proposed the DP-SGD algorithm based on differential privacy, and made the model training process meet differential privacy under rigorous mathematical proof. Although DP-SGD is not a defense strategy against membership inference attacks, experiments have shown that models trained using the DP-SGD algorithm can effectively defend against membership inference attacks. However, the large amount of noise added to the gradient can significantly degrade model performance. Nasr et al. proposed adding the attack model's gain as an adversarial regularizer to the objective function during model training, using a mini-max strategy to train a target classifier that can defend against membership inference attacks. However, due to the large assumptions about the attacker's attack methods, the actual defense effect of the model is poor. Jia et al. used adversarial examples to defend against membership inference attacks. By adding carefully designed noise to the predicted output of the sample on the target model, they transformed it into an adversarial example that can mislead the attack model without changing the target model's predicted label.

[0005] The existing defense members' inference attack strategies always come at the expense of model performance or have deployment difficulties.

[0006] Through the above analysis, the problems and defects of the existing technology are as follows:

[0007] Existing strategies for defending against membership inference attacks often sacrifice model performance or present deployment difficulties, making it difficult to strike a balance between model security and performance. For example, the L2 regularization term, used to defend against membership inference attacks, significantly degrades model performance. The DP-SGD algorithm, while providing rigorous mathematical proof, results in unacceptable performance loss. Adversarial regularization methods can only defend against attacks pre-defined by the defender and fail to provide comprehensive model security. Modifying model outputs to predict adversarial examples significantly increases the model's computational load and significantly increases model deployment overhead. Summary of the Invention

[0008] In response to the problems existing in the prior art, the present invention provides a member inference attack defense method and system based on strong member information prediction.

[0009] The present invention is implemented as follows: a method for defending against membership inference attacks based on strong membership information prediction includes: if a test sample x comes from the model's training set, since the D1 and D2 datasets are disjoint, it can only exist in one of the datasets. Assuming that it comes from the D2 dataset, the strong membership information prediction model m will pre-infer f2(x) as the strong membership information in the model output. The aggregation function will remove the content of f2(x) before the attacker obtains the prediction result and return f1(x) to the attacker.

[0010] Essentially, the defender proactively conducts a membership inference attack before the attacker launches their attack. Because the defender's background knowledge is always greater than the attacker's, they can train a model with stronger membership inference attack capabilities, known as a strong membership information prediction model. Therefore, the defender can always eliminate the discrepancies in the model's predicted output caused by membership and non-membership, making it difficult for the attacker to execute an effective membership inference attack.

[0011] Step 1: The defender obtains the real dataset D and divides it into two non-overlapping data subsets as the model training dataset D train And the test dataset D test ;

[0012] Step 2: Train the dataset D train , and then divided into two non-overlapping training data subsets D1 and D2;

[0013] Step 3: The defender uses the training dataset D train Train a teacher model f on teacher , and using the teacher model f teacher Two student sub-models f1 and f2 are distilled from datasets D1 and D2 respectively. The two student sub-models f1 and f2 serve as reference models for each other.

[0014] Step 4: The defender designs new labels for the data in datasets D1 and D2 to determine their membership information; the data label in D1 is 1, and the data label in D2 is 0;

[0015] Step 5: The data in datasets D1 and D2 and their prediction outputs in the reference model in step 3 are concatenated and used as the input of the strong membership information prediction model to train the strong membership information prediction model;

[0016] Step 6: During the model prediction process, the defender uses the strong member information prediction model before the attacker attacks, determines the strong member information in the model prediction output, removes it, and uses the remaining content as the final output of the model.

[0017] Step 7: The defender uses the aggregation function to remove member information from the model output and uses the remaining content as the final output of the model.

[0018] Furthermore, in step three, the original data set is used to train the teacher model, and is used to distill the student sub-model learned on the independent and identically distributed subset, so that the performance of the two reference models is close to that of the teacher model.

[0019] Furthermore, the training process of the strong member information prediction model in step 6 includes:

[0020] (1) Design new labels for the data in the datasets D1 and D2 to determine their membership information; that is, the data label in D1 is 1, and the data label in D2 is 0;

[0021] (2) To ensure that the strong membership information prediction model can memorize the training samples, we concatenate the labeled training data and the output of the two reference models as the input of the strong membership information prediction model, so that the model has a better "memory effect" on the training samples, and use the BCELoss loss function to calculate the loss value;

[0022] (3) Update the model parameters using the gradient descent algorithm, using the Adam optimizer, until the strong membership information prediction model converges.

[0023] Furthermore, in step seven, during the model prediction phase, the defender uses the trained strong member information prediction model to first determine the strong member information in the model output before the attacker carries out the attack, and promptly removes and hides it, so that the attacker cannot obtain the information used to conduct an effective member inference attack, rendering it ineffective.

[0024] Furthermore, in step 4, in order to avoid the strong membership information prediction model from incorrectly predicting the membership of data that is not in the training set, which leads to a stronger membership leakage caused by the target model prediction output, the output dimension of the membership information prediction model is set to 1. When determining the membership information of a sample, one-hot encoding is used to uniquely determine the possible membership of the current sample.

[0025] Furthermore, in step seven, data distillation is used, and the strong member information prediction model is used to remove the strong member information in the model prediction output, so that the model achieves a better trade-off in terms of performance and security.

[0026] Another object of the present invention is to provide a member inference attack defense system based on strong member information prediction, the member inference attack defense system based on strong member information prediction comprising:

[0027] Data acquisition module, used to obtain real data set D;

[0028] The dataset partitioning module divides the real dataset D into two non-intersecting data subsets as the model training dataset D train And the test dataset D test , the training data set D train , and then divided into two non-overlapping training data subsets D1 and D2;

[0029] The teacher model building module is used to train the dataset D train Train a teacher model f teacher , and using the teacher model f teacher Distill two student sub-models f1 and f2 in datasets D1 and D2 respectively;

[0030] A strong membership information prediction model construction module is used to construct a strong membership information prediction model m based on two reference models to predict the membership information of the current sample x;

[0031] The model training module is used to splice the data in the datasets D1 and D2 and their outputs in the reference model as the input of the strong member information prediction model, and train the strong member information prediction model;

[0032] The model prediction module is used to use the strong member information prediction model before the attacker attacks, determine the strong member information in the model prediction output, remove it, and use the remaining content as the final output of the model.

[0033] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0034] First, the technical solution of the present invention brings about some creative technical effects after solving the problem.

[0035] The specific description is as follows:

[0036] 1. The privacy protection scheme of this invention involves pre-emptively performing a more powerful membership inference attack on the model's predicted output, identifying and removing strong membership information. This fundamentally addresses the problem of membership information leakage caused by the difference in the predicted output distribution between members and non-members. Considering the disjoint partitioning of the dataset, we employ knowledge distillation to address the degradation in model performance caused by the dataset being reduced directly due to data partitioning in existing defense strategies. This allows the model to achieve a better balance between security and performance.

[0037] 2. This invention improves the security of the model and, to a certain extent, solves the problem of member membership information leakage caused by the difference between the training accuracy and the test accuracy of the model due to overfitting.

[0038] 3. The present invention takes into account the close relationship between membership inference attacks and model overfitting phenomena. In order to enable the strong membership information prediction model to accurately capture the membership information of the training samples, a new label is redesigned for the current sample to determine its membership. During the training process, it is spliced ​​with the output in the corresponding reference model and used as the input of the strong membership prediction model, and it is trained to overfit. Based on the fact that a single sample can only exist in a single data set under the data partitioning, if the current sample is a training sample, the overfitted strong membership information prediction model can accurately know its true source. If it is not a training sample, there is actually no strong membership information in the model prediction output, but the strong membership prediction model can select the content that is most likely to be strong membership information for elimination, so that no matter what the membership of the current sample to be tested is, the model prediction output does not contain any signal that can be used to perform a membership inference attack.

[0039] Second, the present invention is based on a defense method for strong member information prediction and hidden member inference attacks, which aims to ensure a trade-off between model performance and security, and can be easily and quickly deployed. Intuitively, the effectiveness of the defense method for strong member information prediction and hidden member inference attacks comes from the fact that the defender can always effectively remove content with strong member information from the model output before the attacker. If the sample x to be tested comes from the D2 data set, the strong member information prediction model will pre-infer f2(x) as strong member information, and the aggregation function will remove the f2(x) content before the attacker obtains the prediction result, and return f1(x) to the attacker. Since the strong member information in the model prediction result has been removed, the attacker cannot obtain the difference in the model's prediction output on member and non-member data, and thus effectively defends against member inference attacks.

[0040] Third, as auxiliary evidence for the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0041] (1) This invention primarily addresses the issue of membership information leakage in training datasets during machine learning. It also provides a design solution for service providers to offer secure and efficient machine learning as a service.

[0042] (2) This invention re-examines the membership inference attack and makes it more benign. The intuition is that, under the premise that the defender has more sufficient background knowledge, a model with stronger membership inference attack than the attacker is designed to determine the strong membership information in the model prediction output. It also considers using an aggregation function to remove the strong membership information. The returned model prediction content will not contain the membership information of the current sample to be tested, which fundamentally defends against the membership inference attack. And through data distillation, it solves the problem of model performance degradation caused by data partitioning, making it possible to better balance model security and performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Flowchart of a member inference attack defense method based on strong member information prediction provided by an embodiment of the present invention;

[0044] Figure 2 This is the basic process of machine learning as a service in the context of embodiments of the present invention;

[0045] Figure 3 This is a basic flow chart of a member inference attack on machine learning under an embodiment of the present invention;

[0046] Figure 4 This is the overall framework diagram of the member inference attack defense system based on strong member information prediction implemented by an embodiment of the present invention. The defender first inputs the sample to be tested into two reference models respectively. If it is actually the training sample of the model, then one of f1(x) and f2(x) must be strong member information. At this time, the defender uses the strong member information prediction model to determine the strong member information and removes it during the aggregation process, so that the attacker can only obtain the model prediction output without membership information. In essence, due to our operation, the distribution difference between members and non-members in the model output is eliminated, thereby reducing the leakage of member membership information.

[0047] Figure 5 The two reference model training diagrams provided by the embodiment of the present invention are as follows: the defender firstly converts the real data set D train , divided into two non-overlapping training data subsets D1 and D2, in the training data set D train Train a teacher model f teacher , and using the teacher model f teacherDistill two student sub-models f1 and f2 from datasets D1 and D2 respectively, ensuring that the performance of the two student sub-models is close to that of the teacher model;

[0048] Figure 6 This is a basic framework diagram of the strong membership information prediction model provided by an embodiment of the present invention. To ensure that the strong membership information can efficiently capture data from the training set, the output of the sample and the current sample in the reference model are spliced ​​together during model training, enabling the strong membership information prediction model to effectively memorize the membership information of the current sample.

[0049] Figure 7 This is an experimental diagram of the defense effect of various defense strategies trained with the AlexNet model on the Fashion-MNIST dataset provided by an embodiment of the present invention;

[0050] Figure 8 This is an experimental diagram of the defense effects of various defense strategies trained with the ResNet-18 model on the CIFAR-10 dataset, provided by an embodiment of the present invention.

[0051] Figure 9 This is an experimental diagram of the defense effects of various defense strategies trained with the ResNet-18 model on the CIFAR-100 dataset, provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention. The symbol D represents a dataset, the symbol f represents a model, the symbol m represents a strong membership information prediction model, and the symbol x represents a sample to be tested.

[0053] like Figure 1 As shown, the member inference attack defense method based on strong member information prediction provided by the embodiment of the present invention includes:

[0054] S101: Obtain a real dataset D and divide it into two non-overlapping data subsets as the model training dataset D train And the test dataset D test ;

[0055] S102: training data set D train , and then divided into two non-overlapping training data subsets D1 and D2;

[0056] S103: The defender is in the training dataset D train Train a teacher model f teacher , and using the teacher model fteacher Two student sub-models f1 and f2 are distilled from datasets D1 and D2 respectively. The two student sub-models f1 and f2 serve as reference models for each other.

[0057] S104: constructing a strong membership information prediction model m for the two reference models, for predicting the membership information of the current sample x;

[0058] S105: The defender designs new labels for the data in datasets D1 and D2 to determine their membership information; the data label in D1 is 1, and the data label in D2 is 0;

[0059] S106: splicing the data in the datasets D1 and D2 and the output of the reference model in step 3 as the input of the strong member information prediction model, and training the strong member information prediction model;

[0060] S107: During the model prediction process, the defender uses the strong member information prediction model before the attacker attacks to determine the strong member information in the model prediction output, removes it, and uses the remaining content as the final output of the model.

[0061] Example 1

[0062] This embodiment provides a method for defending against attacks based on strong member information prediction and hidden member inference in the framework of the AlexNet neural network. The overall framework of the defense strategy of the present invention is shown in the figure below: Figure 2 shown.

[0063] First, the dataset with private information (Fashion-MNIST) is randomly divided into two independent and identically distributed subsets of similar size, D train , D test , and in D train The teacher model f is trained using the ResNet-18 neural network framework on the dataset t The batch size of the teacher model during training is set to 128, the learning rate is 0.05, the number of iterative training is 100, and the SGD optimizer is used.

[0064] Then D train The dataset was divided into two subsets of similar size, D1 and D2. The teacher model was then used to distill two reference models, f1 and f2, each using the AlexNet neural network framework. The reference models were trained with a batch size of 128, a learning rate of 0.001, 100 training iterations, and the Adam optimizer.

[0065] Then, new labels are designed for the data in the data sets D1 and D2 to determine their membership information. That is, the data label in D1 is 1, and the data label in D2 is 0. The training data after the modified labels and the output on its two reference models are spliced ​​as the input of the strong membership information prediction model, and the loss value is calculated. The framework of the strong membership information prediction model is composed of a simple convolutional neural network and a fully connected neural network. The model parameters are updated using the gradient descent algorithm until the strong membership information prediction model converges. The batch size of the strong membership information prediction model is set to 64, the learning rate is set to 0.0001, the number of iterations is 50, and the Adam optimizer is used.

[0066] During the model prediction phase, the test samples are fed into the reference model, and the trained strong membership information prediction model is used to identify strong membership signals. These signals are then removed, and the remaining signals are retained as the model's final output. To prevent the target model's predicted output from causing even stronger membership leakage due to the strong membership information prediction model incorrectly predicting membership for data not in the training set, the output dimension of the membership information prediction model is set to 1. When determining the membership information of a sample, one-hot encoding is used to uniquely identify the current sample's possible membership.

[0067] Finally, the model is attacked using the existing member inference attack method, and the attack accuracy is used to measure the model security. test The prediction accuracy on measures the performance of the model. Since membership inference attacks are essentially binary classification models, attackers can at least predict the correct membership information of the current sample through random guessing. Therefore, we hope that the closer the accuracy of membership inference attacks is to 50%, the higher the test accuracy and the better the defense strategy. Figure 7 Detailed experimental results for various defense strategies on the Fashion-MNIST dataset are presented. While DP-SGD provides a solid theoretical basis for protecting the privacy of individual members in the training set and achieves the best defense against membership inference attacks, it exhibits a sharp decline in model performance. L2 regularization, designed primarily to prevent model overfitting, requires a strong L2 regularization term to ensure effective defense against membership inference attacks, significantly reducing the model's actual performance.

[0068] Adversarial regularization (AdvReg) is based on a max-min strategy, adding the attack gain of the attack model as a regularization term to the loss function of the training target model. This allows the learned model to defend against membership inference attacks, but this also limits the target model to only be able to effectively defend against a specific attack model. Although BAN-MPR provides a good trade-off between model performance and security, its actual implementation will consume a lot of resources. Pre-MIA is the defense strategy proposed by this invention. Experiments show that this defense strategy can effectively resist membership inference attacks to around 50% while ensuring that model performance is almost unaffected.

[0069] Example 2

[0070] This embodiment uses a strong member information prediction and concealment defense method under the ResNet-18 neural network framework to successfully implement a method for protecting member privacy on the CIFAR-10 and CIFAR-100 datasets. The overall framework of the defense strategy of the present invention is shown in the figure below. Figure 4 shown.

[0071] The experimental steps in Example 1 were repeated with the following differences:

[0072] To ensure the universality of the defense scheme described in this patent, ResNet-18 is used as the model framework, and the CIFAR-10 and CIFAR-100 datasets are used as model training sets. Since membership inference attacks are closely related to model overfitting, the more complex the training data and the more complex the model architecture, the more likely it is to cause model overfitting, which in turn increases the risk of the model being attacked by membership inference. It shows that in the undefended state, the trained model is more likely to be successfully attacked by membership inference. The specific experimental results are as follows: Figure 8 and Figure 9 shown.

[0073] This paper experiments on the Fashion-MNIST, CIFAR-10, and CIFAR-100 public datasets, simulating them as private datasets and randomly dividing them into training and test sets to measure model performance. AlexNet and ResNet-18 neural networks were used as target model architectures. Four types of membership inference attacks designed with a specific knowledge context were used to measure the effectiveness of the proposed defense strategy against membership inference attacks.

[0074] The experimental results show that the defense strategy proposed in this invention can effectively reduce the risk of member inference attacks and ensure that the model performance is almost unaffected.

[0075] In the description of the present invention, unless otherwise specified, "plurality" means two or more; terms such as "upper," "lower," "left," "right," "inner," "outer," "front end," "rear end," "head," and "tail" indicate positions or relationships based on those shown in the accompanying drawings. These terms are intended solely to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, terms such as "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0076] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0077] The embodiments of the present invention have achieved some positive results during the development or use process, and indeed have great advantages over the existing technology. The following content describes them in conjunction with data, charts, etc. from the experimental process.

[0078] This paper conducts experiments on the Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets, using the AlexNet and ResNet-18 neural network architectures, and compares existing defense methods such as L2 regularization, DP-SGD algorithm, adversarial regularization, and BAN-MPR and LSBAN-MPR defense methods. test The prediction performance on represents the performance of its model, and four different membership inference attack methods are used to illustrate the security performance of the model. We hope that our proposed method can effectively reduce the success rate of membership inference attacks from different attack methods while ensuring the performance of the model.

[0079] Figure 7-9The experimental results we conducted show that using L2 regularization technology to defend against member inference attacks will also greatly reduce the performance of the model. II Threat, but cannot effectively defend against Attack I The defense method of pre-MIA has similar defense effect to BAN-MPR and SLBAN-MPR against member inference attacks (the difference is less than 2%). In addition, since the pre-MIA defense strategy can always exclude content with strong member information in time during model prediction, it is effective against attack I , Attack II , Attack III All three forms of attack can prevent attackers from carrying out effective attacks to a certain extent.

[0080] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A member inference attack defense method based on strong member information prediction, characterized in that: include: The effectiveness of the strong membership information prediction and hidden membership inference attack defense method comes from the fact that the defender can always effectively remove content with strong membership information from the model output before the attacker. If the test sample x comes from the D2 dataset, the strong membership information prediction model will pre-infer f2(x) as strong membership information. Before the aggregation function obtains the prediction result, the strong membership information prediction model will remove the content of f2(x) and return f1(x) to the attacker as the final output. The member inference attack defense method based on strong member information prediction includes: Step 1: Get the real data set D and divide it into two non-intersecting data subsets as the model training data set D train And the test dataset D test ; Step 2: Train the dataset D train , and then divided into two non-overlapping training data subsets D1 and D2; Step 3: The defender uses the training dataset D train Train a teacher model f teacher , and using the teacher model f teacher Two student sub-models f1 and f2 are distilled from datasets D1 and D2 respectively. The two student sub-models f1 and f2 serve as reference models for each other. Step 4: Construct a strong membership information prediction model m for the two reference models to predict the membership information of the current sample x; Step 5: The defender designs new labels for the data in datasets D1 and D2 to determine their membership information; the data label in D1 is 1, and the data label in D2 is 0; Step 6: After splicing and integrating the data in datasets D1 and D2 and their outputs in the reference model in step 3, the strong member information prediction model is trained; Step 7: During the model prediction process, the defender uses the strong member information prediction model before the attacker attacks to determine the strong member information in the model prediction output, remove it, and use the remaining content as the final output of the model.

2. The member inference attack defense method based on strong member information prediction according to claim 1, characterized in that: In step 3, the teacher model is trained using the original dataset and used to distill the student sub-models learned on the independent and identically distributed subsets, so that the performance of the two reference models is close to that of the teacher model; The training process of the strong member information prediction model in step 6 includes: Calculate the loss value; use the gradient descent algorithm to update the model parameters until the strong membership information prediction model converges.

3. The member inference attack defense method based on strong member information prediction according to claim 1, characterized in that: In the step 4, the output dimension of the membership information prediction model is set to 1, and when determining the membership information of a certain sample, one-hot encoding is used to uniquely determine the possible membership of the current sample.

4. A member inference attack defense system based on strong member information prediction for use in the member inference attack defense method based on strong member information prediction according to any one of claims 1 to 3, characterized in that: The member inference attack defense system based on strong member information prediction includes: Data acquisition module, used to obtain the real data set D, The dataset partitioning module divides the real dataset D into two non-intersecting data subsets as the model training dataset D train And the test dataset D test , the training data set D train , and then divided into two non-overlapping training data subsets D1 and D2; The teacher model building module is used to train the dataset D train Train a teacher model f teacher , and using the teacher model f teacher Distill two student sub-models f1 and f2 in datasets D1 and D2 respectively; A strong membership information prediction model construction module is used to construct a strong membership information prediction model m based on two reference models to predict the membership information of the current sample x; The model training module is used to splice the data in the datasets D1 and D2 and their outputs in the reference model as the input of the strong member information prediction model, and train the strong member information prediction model; The model prediction module is used to use the strong member information prediction model before the attacker attacks, determine the strong member information in the model prediction output, remove it, and use the remaining content as the final output of the model.

5. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the member inference attack defense method based on strong member information prediction as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the member inference attack defense method based on strong member information prediction according to any one of claims 1 to 3.

Images