Defense method and device for member reasoning attack in large model fine tuning, and medium

Abstract

The invention discloses a defense method and device for member reasoning attacks in large model fine tuning and a medium, when the accuracy difference of a target classifier on training data and verification data is larger than a threshold value, the defense method for the member reasoning attacks in large model fine tuning is executed, and the defense method comprises the steps that before gradient descent of the target classifier, the target classifier is subjected to gradient descent; according to the cardinal number change of the label set corresponding to each label between the kth batch and the (k-1) th batch, performing data fusion on samples in the label set corresponding to each label between the kth batch and the (k-1) th batch; and / or, in the gradient descending process of the target classifier, averaging gradient updating parameters of the pth layer in the target classifier, and updating each gradient element value in the gradient matrix corresponding to the pth layer by using the average value; the gradients of other hidden layers except the pth layer in the target classifier are updated based on gradient updating parameters obtained through calculation of a gradient descent method.

Description

Technical Field

[0001] The present invention relates to a security protection method in the field of machine learning, and in particular to a defense method, device, and medium against member inference attacks in large model fine-tuning. Background Art

[0002] Deep learning models, with their massive parameter sizes and powerful inference capabilities, have demonstrated exceptional performance across a wide range of complex tasks, ushering in the era of large models in artificial intelligence. Unlike traditional end-to-end models, large models utilize a "pre-training + fine-tuning" training paradigm. This approach pre-trains the model on massive amounts of unlabeled data, then fine-tunes it on labeled data for the target task. This adapts the model to the specific task, overcoming the limitations of traditional AI models in generalizing to other tasks. It is generally believed that the more complex the model's training process and structure, the higher the security risks it faces. In particular, large models, with their powerful deep learning and data memory capabilities, are susceptible to the risk of data privacy breaches.

[0003] Membership inference attacks (MIA) are a common attack method in deep learning. Their goal is to infer whether a specific data sample belongs to the training dataset of a target machine learning model (member) or not (non-member). During the target model's inference phase, the attacker can construct specific data and query the model to obtain the output of the target model (usually a classifier). Ultimately, based on this output and the attacker's background knowledge (black box, white box, or crystal box), a binary classification is performed based on the output of the target model to infer whether the data sample exists in the target model's training set. Common MIA methods fall into two main categories: "shadow model"-based membership inference attacks and metric-based membership inference attacks. These attacks employ "shadow models" that approximate the functionality of the target machine learning model and devise metrics to distinguish between member and non-member data outputs. This type of attack can indirectly pose a serious threat to data privacy, especially in applications involving sensitive datasets. For example, if a target machine learning model is associated with a specific private disease (such as cancer), the model can be used to determine appropriate drug dosages or identify the genetic basis of the disease. In this scenario, knowing whether a patient's clinical records are used to train the target model can infer whether the patient suffers from this private disease.

[0004] At present, significant progress has been made in the defense methods against membership inference attacks. The main goal is to maximize the practicality of the target model in the classification task while reducing the distinguishability and difference between the outputs of member and non-member data samples, so that the attack binary classification accuracy is as low as possible.

[0005] Existing defense methods can be divided into three main categories:

[0006] Type 1: Change the weight of the target model (from θ T arrive );

[0007] Type 2: Interference target model training data (from D T arrive );as well as,

[0008] Type 3: Modify the target model loss function (from L T arrive ).

[0009] The defense method aims to make the outputs of member and non-member data samples almost indistinguishable without affecting the accuracy of the target model task. That is, the classification accuracy of the membership inference attacker's attack binary classifier is close to random guessing (50%), so that the membership inference attack "failed" and the attacker suffered corresponding losses due to the computational costs of training the shadow model and training the attack binary classifier.

[0010] From a defense perspective, existing defense methods for membership inference attacks still need to improve their trade-off between target model availability and data privacy, and their versatility across diverse attacker background knowledge is insufficient. First, these defense methods often impact the output of the target model, resulting in performance degradation. Although some defense methods propose adjustable parameters to balance target model availability and data privacy, this trade-off remains unsatisfactory. Second, these defense methods primarily target small models. When the target model is a fine-tuned large model, the complexity of defense methods based on model modification increases significantly, leading to a sharp decline in defense effectiveness. Furthermore, some defense methods lack versatility. For example, some methods can only defend against black-box and white-box attackers, but cannot simultaneously defend against crystal box attackers. Summary of the Invention

[0011] In response to the shortcomings of the existing technology, the present invention provides a defense method, device, and medium against member inference attacks in large model fine-tuning.

[0012] In a first aspect, an embodiment of the present invention provides a method for defending against membership inference attacks during large model fine-tuning. When the difference between the accuracy of a target classifier on training data and validation data is greater than a threshold, the method for defending against membership inference attacks during large model fine-tuning is executed. The method includes:

[0013] Before the target classifier gradient descent, data fusion is performed on the samples in the label set corresponding to each label between the kth batch and the k-1th batch according to the cardinality change of the label set corresponding to each label between the kth batch and the k-1th batch;

[0014] and / or,

[0015] During the gradient descent process of the target classifier, the gradient update parameters of the p-th layer in the target classifier are averaged, and the average value is used to update each gradient element value in the gradient matrix corresponding to the p-th layer; the gradients of the hidden layers other than the p-th layer in the target classifier are updated based on the gradient update parameters calculated by the gradient descent method.

[0016] In a second aspect, an embodiment of the present invention provides an electronic device comprising a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned defense method against member inference attacks in large model fine-tuning.

[0017] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium storing a computer program, wherein the program, when executed by a processor, implements the above-mentioned defense method against member inference attacks in large model fine-tuning.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the above-mentioned defense method against member inference attacks in large model fine-tuning is implemented.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] The present invention provides a defense method against member inference attacks in large model fine-tuning, and introduces an inertia mechanism, that is, the defense method is only executed when the difference between the accuracy of the target classifier on the training data and the verification data is greater than a threshold; at the same time, the present invention is designed to perform data perturbation before the gradient descent of the target classifier and / or perform gradient modification during the gradient descent of the target classifier. The above two strategies can be dynamically adjusted and combined according to actual needs to better meet the practicality and privacy requirements of different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of a member inference attack based on a "shadow model" provided by an embodiment of the present invention;

[0023] Figure 2 A flowchart of a method for defending against membership inference attacks in large model fine-tuning provided by an embodiment of the present invention;

[0024] Figure 3 An example diagram of running a data fusion method on the MNIST dataset provided by an embodiment of the present invention;

[0025] Figure 4 An example diagram of performing gradient anonymity on layer p of a model with q layers provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] It should be noted that, unless there is any conflict, the features in the following embodiments and implementations may be combined with each other.

[0029] like Figure 1 As shown, a shadow model-based membership inference attack method, that is, a shadow model-based group ensemble membership inference attack method, specifically includes the following steps:

[0030] Step 1: Convert r shadow models and shadow dataset D S Divide into m≤r groups, each group is assigned n shadow models and shares a shadow dataset

[0031] Step 2: Each shadow dataset Cross-divided to n shadow models as their training / testing subsets Each shadow dataset and its subsets The number of data samples included is the same as the target training dataset D T The number of included data samples is the same.

[0032] Step 3: Each shadow model is tested on a separate training / testing subset The corresponding output vectors are marked as “in” / “out” for training the attack binary classifier.

[0033] Regarding the above-mentioned member reasoning attack method based on shadow model, such as Figure 2 As shown, the present invention provides a defense method for membership inference attacks in large model fine-tuning. When the difference between the accuracy of the target classifier on the training data and the verification data is greater than a threshold, the defense method for membership inference attacks in large model fine-tuning is executed. The defense method includes:

[0034] Before the target classifier gradient descent, data fusion is performed on the samples in the label set corresponding to each label between the kth batch and the k-1th batch according to the cardinality change of the label set corresponding to each label between the kth batch and the k-1th batch;

[0035] and / or,

[0036] During the gradient descent process of the target classifier, the gradient update parameters of the p-th layer in the target classifier are averaged, and the average value is used to update each gradient element value in the gradient matrix corresponding to the p-th layer; the gradients of the hidden layers other than the p-th layer in the target classifier are updated based on the gradient update parameters calculated by the gradient descent method.

[0037] Furthermore, according to the cardinality change of the label set corresponding to each label between the kth batch and the k-1th batch, the process of performing data fusion on the samples in the label set corresponding to each label between the kth batch and the k-1th batch includes:

[0038] Among them, when the cardinality of the label set corresponding to the lth label in the kth batch is less than the cardinality of the label set corresponding to the lth label in the k-1th batch, that is, Perform incremental mixing; specifically, from the label set corresponding to the lth label in the k-1th batch Randomly select multiple first sample points O and select the label set corresponding to multiple labels from the kth batch Randomly select multiple second sample points from different labels s Update the label set corresponding to multiple labels of the kth batch based on the first sample point O The second sample point in Among them, λ represents the weight coefficient.

[0039] Among them, when the cardinality of the label set corresponding to the lth label in the kth batch is greater than the cardinality of the label set corresponding to the lth label in the k-1th batch, that is, Perform descending mixing; specifically, from the label set corresponding to the lth label of the kth batch Randomly select the third sample point R and select the label set corresponding to multiple labels in the kth batch Randomly select multiple second sample points from different labels s Based on the third sample point R, the label set corresponding to the lth label of the kth batch is updated The second sample point in Among them, λ represents the weight coefficient. At the same time, the label set corresponding to the lth label in the kth batch The second sample point in After the update, the label set corresponding to the lth label in the kth batch The third sample point R is discarded.

[0040] Among them, when the cardinality of the label set corresponding to the lth label in the kth batch is equal to the cardinality of the label set corresponding to the lth label in the k-1th batch, that is, From the label set corresponding to the lth label in the kth batch The machine selects multiple fourth sample points Get the label set corresponding to multiple labels in the k-1th batch The corresponding center of mass Based on centroid Update the label set corresponding to the lth label in the kth batch The fourth sample point in Among them, λ represents the weight coefficient.

[0041] Among them, the label set corresponding to multiple labels in the k-1th batch is calculated The corresponding center of mass The expression is as follows:

[0042]

[0043] Where, The label set corresponding to multiple labels in the k-1th batch Represents the label set corresponding to multiple labels in the k-1th batch The cardinality, Represents the label set corresponding to multiple labels in the k-1th batch The samples in .

[0044] It should be noted that this example uses data fusion (DF) for defense, which can simultaneously consider mixing data sample points in terms of categories and batches. Its goal is to reduce the distribution difference between training data and test data by increasing the distribution diversity of training data. This example is divided into three different situations based on tracking the changes in the number of data sample points under the same label in two adjacent batches. Depending on the difference in the number of data sample points in the current batch under each label and the data sample points under the same label in the previous batch, the data sample points in the current batch are discarded, added, or updated. The similarity in the number of data sample points in the two adjacent batches is maintained.

[0045] Furthermore, the p-th layer is usually selected as the last linear classification layer in the target classifier. Meanwhile, it should be noted that the target classifier adopts different neural network model structures, and those skilled in the art can adjust the p-th layer according to actual conditions.

[0046] It should be noted that this example uses the Random Gradient Anonymity (RGA) strategy for defense. Its core concept is to anonymize the gradient parameter updates of a specific model layer (i.e., layer p) during the model gradient descent process. This means that all gradients of that layer are updated using the average of all gradient parameter updates on the same layer. This aims to reduce the distinguishability of model gradient updates due to different features in the data. Furthermore, the model structure for anonymization can be customized based on the characteristics of different fine-tuning methods.

[0047] At the same time, in order to adjust the trade-off between model utility, data privacy, and method efficiency of the combination of data fusion and stochastic gradient anonymization mechanism, this example also introduces an inert mechanism to monitor learning performance indicators. That is, the combined algorithm of data fusion and stochastic gradient anonymization mechanism will be activated on the batch only when the accuracy gap between the target classifier on the training data and the verification data is greater than a predefined threshold, thereby executing the combined algorithm in a controlled manner. Data fusion and stochastic gradient anonymization mechanism can work independently or collaboratively to form a comprehensive defense measure.

[0048] Example 1

[0049] This example uses the existing MNIST image dataset as an example to further illustrate the specific implementation of a defense method against membership inference attacks during large model fine-tuning. The MNIST image dataset contains grayscale images of handwritten digits from 0 to 9, with each image corresponding to a label (0-9) representing the actual digit in the image. The method specifically includes the following steps:

[0050] like Figure 3 As shown in Figure 2, examples of data fusion methods on the k-1th batch and the kth batch on the MNIST dataset are shown, where category 2 of the kth batch shows decreasing mixing, category 0 shows increasing mixing, and category 1 shows the case where the cardinality of the previous and next label sets remains unchanged.

[0051] Take the kth batch of category 2 as an example to demonstrate decreasing mixing, that is, the cardinality of category 2 in the kth batch is larger than the cardinality of category 2 in the k-1th batch. At this time, select a point from category 2 in the kth batch and then mix it with other categories in the kth batch, such as the point of category 3 in the figure, and discard the point selected from category 2 in the kth batch.

[0052] Take the kth batch of category 0 as an example to demonstrate incremental mixing, that is, the cardinality of category 0 in the kth batch is smaller than the cardinality of category 0 in the k-1th batch. At this time, select a point from category 0 in the k-1th batch and then mix it with other categories in the kth batch. In the figure, the point of category 5 is selected to update the point of category 5 in the kth batch.

[0053] Take the k-th batch of category 1 as an example to demonstrate the case where the cardinality remains unchanged. At this time, the centroid of the data points of category 1 in the k-1-th batch is calculated, and then mixed with the points of category 1 in the k-th batch to update the points of category 1 in the k-th batch.

[0054] Finally, each category in the kth batch is mixed differently according to the above three situations, and the data of the kth batch is updated, and then the model training is performed.

[0055] like Figure 4 As shown in Figure 1, when updating the gradient in each round of gradient descent during model training, the average value of the gradient value of layer p is calculated and used to replace the update of the gradient value of the layer.

[0056] Using the classic pre-trained PoolFormer model on the MNIST dataset, the attack success rate of the class vector attack method can be reduced from 71.11% to 56.37%, the attack success rate of the global loss attack method can be reduced from 73.02% to 60.84%, and the attack success rate of the white-box attack method can be reduced from 66.03% to 55.85%.

[0057] In summary, the present invention provides a defense method against membership inference attacks in large model fine-tuning, introducing an inertia mechanism, that is, the defense method is only executed when the difference between the accuracy of the target classifier on the training data and the verification data is greater than a threshold; and combined with data perturbation and gradient anonymity mechanisms, it meets the practicality and privacy requirements of different application scenarios in a controlled manner.

[0058] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned defense method for member inference attacks in large model fine-tuning. Figure 3 As shown in the figure, a hardware structure diagram of any device with data processing capability is provided for the defense method against member inference attacks in large model fine-tuning provided by the embodiment of the present invention. Figure 3 In addition to the processor, memory, and network interface shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0059] Accordingly, the present application also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the above-mentioned defense method for member inference attacks in large model fine-tuning. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities as described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium can also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.

[0060] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only.

[0061] It will be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A defense method against membership inference attacks in large model fine-tuning, characterized by: When the difference between the accuracy of the target classifier on the training data and the validation data is greater than a threshold, a defense method against membership inference attacks in large model fine-tuning is executed. The defense method includes: Before the target classifier gradient descent, data fusion is performed on the samples in the label set corresponding to each label between the kth batch and the k-1th batch according to the cardinality change of the label set corresponding to each label between the kth batch and the k-1th batch; and / or, During the gradient descent process of the target classifier, the gradient update parameters of the p-th layer in the target classifier are averaged, and the average value is used to update each gradient element value in the gradient matrix corresponding to the p-th layer; the gradients of the hidden layers other than the p-th layer in the target classifier are updated based on the gradient update parameters calculated by the gradient descent method.

2. The method for defending against membership inference attacks in large model fine-tuning according to claim 1, characterized in that: According to the cardinality change of the label set corresponding to each label between the kth batch and the k-1th batch, the process of data fusion for the samples in the label set corresponding to each label between the kth batch and the k-1th batch includes: When the cardinality of the label set corresponding to the lth label in the kth batch is less than the cardinality of the label set corresponding to the lth label in the k-1th batch, that is, From the label set corresponding to the lth label in the k-1th batch Randomly select multiple first sample points O and select the label set corresponding to multiple labels from the kth batch Randomly select multiple second sample points from different labels s Update the label set corresponding to multiple labels of the kth batch based on the first sample point O The second sample point in Among them, λ represents the weight coefficient.

3. The method for defending against membership inference attacks in large model fine-tuning according to claim 1, characterized in that: According to the cardinality change of the label set corresponding to each label between the kth batch and the k-1th batch, the process of data fusion for the samples in the label set corresponding to each label between the kth batch and the k-1th batch includes: When the cardinality of the label set corresponding to the lth label in the kth batch is greater than the cardinality of the label set corresponding to the lth label in the k-1th batch, that is, From the label set corresponding to the lth label in the kth batch Randomly select the third sample point R and select the label set corresponding to multiple labels in the kth batch Randomly select multiple second sample points from different labels s Based on the third sample point R, the label set corresponding to the lth label of the kth batch is updated The second sample point in Among them, λ represents the weight coefficient.

4. The method for defending against membership inference attacks in large model fine-tuning according to claim 2, characterized in that: When the cardinality of the label set corresponding to the lth label in the kth batch is greater than the cardinality of the label set corresponding to the lth label in the k-1th batch, the following also applies: The label set corresponding to the lth label in the kth batch The second sample point in After the update, the label set corresponding to the lth label in the kth batch The third sample point R is discarded to ensure that the cardinality of the label set corresponding to the lth label in the kth batch is consistent with the cardinality of the label set corresponding to the lth label in the k-1th batch, thereby reducing the data privacy leakage caused by the change in the cardinality of the label set corresponding to each label between the kth batch and the k-1th batch.

5. The method for defending against membership inference attacks in large model fine-tuning according to claim 1, characterized in that: According to the cardinality change of the label set corresponding to each label between the kth batch and the k-1th batch, the process of data fusion for the samples in the label set corresponding to each label between the kth batch and the k-1th batch includes: When the cardinality of the label set corresponding to the lth label in the kth batch is equal to the cardinality of the label set corresponding to the lth label in the k-1th batch, that is, From the label set corresponding to the lth label in the kth batch The machine selects multiple fourth sample points Get the label set corresponding to multiple labels in the k-1th batch The corresponding center of mass Based on centroid Update the label set corresponding to the lth label in the kth batch The fourth sample point in Among them, λ represents the weight coefficient.

6. The method for defending against membership inference attacks in large model fine-tuning according to claim 1, characterized in that: Get the label set corresponding to multiple labels in the k-1th batch The corresponding center of mass The process package is as follows: Where, The label set corresponding to multiple labels in the k-1th batch Represents the label set corresponding to multiple labels in the k-1th batch The cardinality, Represents the label set corresponding to multiple labels in the k-1th batch The samples in .

7. The method for defending against membership inference attacks in large model fine-tuning according to claim 1, characterized in that: The p-th layer is the last linear classification layer in the target classifier.

8. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the defense method against member inference attacks in large model fine-tuning as described in any one of claims 1-7 above.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the defense method against membership inference attacks in large model fine-tuning as described in any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for defending against member inference attacks in large model fine-tuning as described in any one of claims 1 to 7 is implemented.

Images