Partition learning-oriented poisoning attack defense method and device, and storage medium

Abstract

The invention relates to the field of information security technology, and relates to a segmentation learning-oriented poisoning attack defense method, an electronic device and a storage medium, and designs two different poisoning attack defense schemes according to the amount of knowledge of an attacker: a poisoning attack defense method in a white box scene and a poisoning attack defense method in a black box scene. The segmentation learning-oriented poisoning attack defense scheme can be suitable for more defense scenes and can resist poisoning attacks in more occasions, and the defense capability of the model on the poisoning attacks can be improved.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to a poisoning attack defense method, device and storage medium for segmentation learning. Background Art

[0002] Split Learning is an emerging distributed machine learning method used between the client and the server. Split learning can be divided into VANILLA type and U type according to the number of parts the model is split into. VANILLA type split learning divides the model into two parts, with the client and the server holding one part respectively; while U type split learning divides the model into three consecutive parts, with the client holding the first and third parts and the server holding the second part. Through split learning, the model is split and trained, the original data is not transferred from the server to the client, and the model is appropriately split between the client and the server, reducing the burden of training.

[0003] In U-type segmentation learning, the training steps are divided into two parts: segmentation and training. In the segmentation step, the client and the server segment the model into three consecutive parts: f1, f2, and f3. The client holds the client models f1 and f3, and the server holds the server model f2. Since the input layer (f1) and output layer (f3) of the model are both held by the client, the samples and labels of the training data are also owned by the client, which is different from VANILLA-type segmentation learning (VANILLA-type segmentation learning divides the model into two parts, and the client holds the data samples and the server holds the data labels).

[0004] In the training step, the client uses its own local data samples to input into f1 to obtain the embedded dataset E c and sent to the server, which then sends E c The server then inputs the updated f2 to the client, which then sends the output to f3 to get the model output. The server then calculates the loss between the output and the label and performs back propagation in the opposite direction of the sample data transmission, thereby updating each part of the model. Finally, the server sends the updated f2 to the client, which combines f1, f2, and f3 to form a complete model.

[0005] Poisoning attack, also known as poisoning attack, is an attack on the model training stage. It changes the model architecture, causing the model to misidentify the training data and output wrong prediction results. Backdoor attack is a special targeted poisoning attack. The attacker uses a carefully constructed malicious sample to implant a trigger into the sample, so that the model can infer normally for samples that do not contain the trigger; but for samples that contain the trigger, the model will be affected and fail to work, thereby reducing the accuracy of the model prediction results.

[0006] Existing defense schemes against segmentation learning poisoning attacks have the following problems:

[0007] 1. Existing defense schemes against segmentation learning attacks are mainly aimed at VANI LLA type segmentation learning. In this scenario, data samples and labels are held by the client and server respectively. However, in real scenarios, data samples and labels are often all held by the client, and in the real world, the server usually does not have the ability to access the data.

[0008] 2. The existing defense schemes against segmentation learning attacks have relatively simple defense scenarios, and can usually only block one of the poisoning attacks in the white-box scenario or the poisoning attacks in the black-box scenario; they cannot perform flexible defense based on the knowledge obtained by the attacking server.

[0009] Third, the purpose of current poisoning attacks is to reduce the accuracy of the model, ignoring the possibility that attackers may carry out poisoning attacks by reducing the robustness of the model (such as the ability to resist adversarial attacks). Summary of the invention

[0010] In order to improve the model's defense capability against poisoning attacks, a poisoning attack defense method, device and storage medium for segmentation learning are provided.

[0011] In order to achieve the above-mentioned object of the present invention, the present invention provides a poisoning attack defense method for segmentation learning, comprising:

[0012] Obtain a training data set for U-shaped segmentation learning and first model information of a model to be trained; the model to be trained includes three consecutive sub-models, the three sub-models are client model f1, server model f2 and client model f3; the first model information includes the architecture and parameters of the server model f2, the architecture of the client model f1, the architecture of the client model f3 and the client's embedded data set E c ;

[0013] Perform model inference according to the first model information and the training data set to obtain a trained client model f″3;

[0014] Determine the output category of the model to be trained and the probability distribution corresponding to the output category according to the client model f″3;

[0015] Construct an original probability dataset based on the output categories of the model to be trained and the probability distribution corresponding to the output categories;

[0016] Perform data processing on the samples in the original probability data set according to preset rules to obtain a poisoning probability data set;

[0017] According to the original probability data set and the poisoning probability data set, the parameters of the server model f2 are enhanced to obtain the enhanced server model f'2;

[0018] or,

[0019] Obtain the training data set and the second model information of the model to be trained; the second model information includes the architecture and parameters of the server model f2, and the client embedding data set E c ;

[0020] According to the server model f2, the client embeds the dataset E c Generate adversarial samples with the adversarial sample generator;

[0021] Use the training data set and adversarial samples to perform perturbation training on the training model to generate a poisoned sample-label set;

[0022] The parameters of the server model are enhanced according to the poisoned sample-label set to obtain the enhanced server model f'2.

[0023] Optionally, the training data set includes a number of input samples and reference labels corresponding to the input samples;

[0024] The method of performing model inference according to the first model information and the training data set to obtain a trained client model f″3 includes:

[0025] Initialize the parameters of the client model f1 to obtain the initialized client model f'1;

[0026] Initialize the parameters of the client model f3 to obtain the initialized client model f'3;

[0027] The model composed of the client model f'1, the server model f2 and the client model f'3 is recorded as the first training model, the input sample is processed using the first training model, and the predicted label of the input sample is output;

[0028] Determine a loss function of the first training model according to the predicted label and the reference label corresponding to the input sample;

[0029] The parameters of the client model f'3 are optimized according to the loss function to obtain the client model f″3.

[0030] Optionally, the loss function is calculated as

[0031]

[0032] Where n is the number of input samples in the training dataset; x j represents the input sample, y j represents the reference label corresponding to the input sample, j represents the index of the input sample in the training dataset; θ1 represents the parameters of the client model f1′, θ3 represents the parameters of the client model f3′, and θ2 is the parameter of the server model f2; f1′(x j ; θ1) represents the value of x j Input to the client model f1′ acts on the parameter θ1; l(·,·) is the loss function of the input sample;

[0033] The step of optimizing the parameters of the client model f'3 according to the loss function to obtain the client model f″3 includes:

[0034] Keeping θ2 unchanged, the loss function L(θ1,θ2,θ3) is used to optimize the parameters of the client model f1′ to obtain the optimized parameters θ1 * , use the loss function L(θ1,θ2,θ3) to optimize the parameters of the client model f3′ to obtain the optimized parameters θ3 * ;

[0035] According to the parameter θ3 * Determine the client model f″3.

[0036] Optionally, the step of determining the output category of the model to be trained and the probability distribution corresponding to the output category according to the client model f″3 includes:

[0037] Determine the output category of the model to be trained according to the type of the output result of the client model f″3;

[0038] Embed the client into dataset E c Input the server model f2 to get the server embedding dataset E s ;

[0039] Use the clustering algorithm to embed the obtained servers into the dataset E s Divide into several clusters, each cluster corresponds to an output category;

[0040] For the sample point data in the cluster family, the sample point data closest to the cluster center of the cluster family is selected and input into the client model f″3 to obtain the probability distribution of the output category corresponding to the cluster family.

[0041] Optionally, the server is embedded into the data set E using a clustering algorithm. s Divided into several cluster families, including:

[0042] S331. Embed dataset E from server s Randomly select K sample points as the initial cluster centers;

[0043] S332, embedding data set E into the server s For each sample point in , calculate the distance from each sample point to each cluster center, and assign the sample point to the cluster family with the nearest cluster center;

[0044] S333, for each cluster, calculating the mean of the sample points in the cluster, and taking the mean as the new cluster center of the cluster;

[0045] Steps S331 to S333 are repeatedly performed until the values ​​of the K cluster centers no longer change, and the cluster family to which each sample point belongs is determined based on the current classification of the sample points.

[0046] Optionally, performing data processing on samples in the original probability data set according to a preset rule to obtain a poisoning probability data set includes:

[0047] Sort the distribution probability installation values ​​in the original probability data set from large to small, record the maximum value of the distribution probability value in the original probability data set as the first distribution probability value, and record the second largest value of the distribution probability value in the original probability data set as the second distribution probability value;

[0048] Simultaneously change the numerical value of the first distribution probability value and the numerical value of the second distribution probability value: reduce the numerical value of the first distribution probability value and increase the numerical value of the second distribution probability value until the difference between the first distribution probability value and the second distribution probability value reaches a preset value;

[0049] The content of the original probability data set is modified based on the modified first distribution probability value and the modified second distribution probability value to obtain a poisoning probability data set.

[0050] Optionally, the server model f2 and the client embedding data set E c And the adversarial sample generator generates adversarial samples, including:

[0051] S81. Embed the client into dataset E c Input the server model f2 to get the server embedding dataset E s ;

[0052] S82. Client-based embedding dataset E c and server embedding dataset E sConstructing a comprehensive embedding dataset D e ={(E c ,E s )};

[0053] S83. Randomly select a sample pair (x i ,y i ), x i represents the client embedded data sample, y i For client-based embedding data sample x i Generated server embedding data samples;

[0054] S84, randomly initialize a data sample x embedded with the client i Perturbations of the same shape Embed the client into data sample x i With disturbance Add them together to get x i The corresponding initial adversarial sample

[0055] S85, will Input into server model f2, and get And calculate the loss value loss=l(y i ′,y i );

[0056] S86. Calculate the loss value loss about The gradient of Get the updated adversarial sample

[0057] S87. Based on the updated adversarial sample Update the disturbance value and get the updated disturbance value

[0058] Repeat steps S85 to S87 to iterate the disturbance value S times to obtain the disturbance value after iteration. S is a positive integer;

[0059] S88, the disturbance value With x i Add together to get the updated adversarial sample

[0060] Optionally, the method of using the training data set and the adversarial sample to perform perturbation training on the to-be-trained model to generate a poisoned sample-label set includes:

[0061] The adversarial sample Input into server model f2, and get Calculate yi ″ and y i The loss function between

[0062] Using loss function The optimal value of the most determined disturbance value Make y i ″ and y i Maximize the loss between

[0063] The optimal value Acts on the client embedded data sample x i Get the poisoned sample

[0064] Based on poisoned samples Embed data sample y with the server i Generate poisoned sample-label pairs to obtain poisoned sample-label sets.

[0065] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0066] at least one processor; and,

[0067] a memory communicatively connected to the at least one processor; wherein,

[0068] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned poisoning attack defense method for segmentation learning.

[0069] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned poisoning attack defense method for segmentation learning.

[0070] In summary, this application includes the following beneficial technical effects:

[0071] This patent designs two different poisoning attack defense schemes based on the attacker's knowledge: poisoning attack defense in white box scenarios and poisoning attack defense method in black box scenarios, which makes the scheme proposed in this patent more comprehensive and more applicable to different scenarios;

[0072] The server combines the original probability data set and the poisoned probability data set, and inputs them into the model composed of the server model f2 and the client model f3″ to fine-tune the server model f2 that the server had in the original training, and obtain the enhanced server model f2′. Since the poisoned probability data set is added, the entire data set is expanded, so that the model can still maintain correct predictions when dealing with low-confidence probabilities, and the boundaries of the probabilities predicted for each class become clearer, thereby enhancing the robustness of the model; the present application is different from the usual poisoning attack defense method that focuses on the accuracy of the detection model, but focuses on the model's ability to resist adversarial attacks (i.e., the robustness of the model). Under the scenario where the model accuracy does not change much, the robustness of the model is enhanced so that the model can resist poisoning attacks by attackers whose goal is to reduce the robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A flowchart of a poisoning attack defense method in a white-box scenario for segmentation learning provided by an embodiment of the present invention;

[0074] Figure 2 An overall framework diagram of a white-box poisoning attack method for segmentation learning provided by an embodiment of the present invention;

[0075] Figure 3 A flowchart of a poisoning attack defense method in a black box scenario for segmentation learning provided by an embodiment of the present invention;

[0076] Figure 4 An overall framework diagram of a black box poisoning attack method for segmentation learning provided by an embodiment of the present invention;

[0077] Figure 5 A schematic diagram of the structure of an electronic device for implementing the poisoning attack defense method for segmentation learning provided by an embodiment of the present invention.

[0078] Reference numerals: 10, processor; 11, memory; 12, communication bus; 13, communication interface.

[0079] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0080] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0081] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0082] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0083] Reference Figure 1 and Figure 2 , is a flow chart of a poisoning attack defense method for segmented learning provided by an embodiment of the present invention. In this embodiment, when the model is attacked by white-box poisoning during segmented learning, the poisoning attack defense method for segmented learning includes:

[0084] S1. Obtain a training data set for U-shaped segmentation learning and first model information of a model to be trained.

[0085] In this embodiment, the model to be trained may be an image classification model or an object recognition model. Specifically, the model to be trained includes three consecutive sub-models, and the three sub-models are respectively a client model f1, a server model f2 and a client model f3.

[0086] The first model information includes the architecture and parameters of the server model f2, the architecture of the client model f1, the architecture of the client model f3 and the client's embedded dataset E c ; The model to be trained is a neural network model (such as VGG, ResNet, etc.), VGG's full name is Visual Geometry Group; ResNet is also called residual network, ResNet's full name is Residual Network; The architecture of the client model / server model refers to the network structure of the sub-model, which determines the level, number of layers, number of nodes in each layer and the connection method of the neural network; The parameters of the client model / server model refer to the specific values ​​that the neural network needs to learn during the training process, which directly affect the network's mapping ability between input and output. Generally, it includes weights and biases, which determine the prediction ability of the network.

[0087] The training data set includes several input samples and reference labels corresponding to the input samples. In this embodiment, the input samples are natural image data, such as satellite images or security monitoring images. The server and the client play different roles in segmentation learning. The server is mainly responsible for providing data, models, and computing resources, while the client is responsible for processing local data, communicating with the server, and receiving model updates.

[0088] The functions of the server in segmentation learning mainly include: storing and providing the data sets required for training for sharing and use by multiple clients; in addition, the server initializes and maintains the global model, and the client obtains the initial model from the server and performs local training.

[0089] The client's functions in segmentation learning mainly include processing local data, extracting features and preprocessing, and then uploading the processed data to the server; in addition, the client uses the model downloaded from the server for local training and updates the model parameters based on the training results.

[0090] To facilitate understanding by those skilled in the art, the relationship between the server and the client is described below with reference to a specific example:

[0091] For example, in the road sign recognition task in autonomous driving, each smart car can be regarded as a client and the manufacturer can be regarded as a server. The manufacturer divides a complete image recognition model into three consecutive parts (i.e., client model f1, server model f2, and client model f3); the smart car holds the first part (i.e., client model f1) and the third part (i.e., client model f3), and the manufacturer holds the second part (i.e., server model f2). During the model training process, the smart car inputs the road sign photo it has taken into the first part (i.e., client model f1), and then obtains the output activation data 1 and sends it to the manufacturer, who inputs it into the second part (i.e., client model f1) to obtain activation data 2, and then sends activation data 2 to the smart car, which then inputs activation data 2 into the third part (i.e., client model f3) to train the model.

[0092] S2. Perform model inference based on the first model information and the training data set to obtain a trained client model f″3.

[0093] After the server has the architecture and parameters of the server model f2, the architecture of the client model f1, the architecture of the client model f3 and the training data set, it can train a model with similar functions to the client model f3 through the root mean square error MSE, thereby inferring the output category of the client model f3.

[0094] Specifically, step S2 includes:

[0095] S21, initializing the parameters of the client model f1, and recording the initialized client model f1 as the client model f'1;

[0096] S22, initializing the parameters of the client model f3, and recording the initialized client model f3 as the client model f'3;

[0097] S23, record the model composed of the client model f'1, the server model f2 and the client model f'3 as the first training model, process the input sample using the first training model, and output the predicted label of the input sample;

[0098] S24, determining a loss function of the first training model according to the predicted label and the reference label corresponding to the input sample;

[0099] The loss function is calculated as

[0100]

[0101] Where n is the number of input samples in the training dataset; x j represents the input sample, y j represents the reference label corresponding to the input sample, j represents the index of the input sample in the training dataset; θ1 represents the parameters of the client model f1′, θ3 represents the parameters of the client model f3′, and θ2 is the parameter of the server model f2; f1′(x j ; θ1) represents the value of x j The input is sent to the client model f1′ and acts on the parameter θ1; l(·,·) is the loss function of the input sample.

[0102] S25. Optimize the parameters of the client model f'3 according to the loss function to obtain the client model f″3.

[0103] The optimization method is: keep θ2 unchanged, use the loss function L(θ1,θ2,θ3) to optimize the parameters of the client model f1′ to obtain the optimized parameters θ1 * , use the loss function L(θ1,θ2,θ3) to optimize the parameters of the client model f3′ to obtain the optimized parameters θ3 * ;

[0104] According to the parameter θ3 * Determine the client model f″3.

[0105] Specifically, here, we use the root mean square error MSE to optimize the parameters θ3 of the client model f3′ and the parameters θ1 of the client model f1′;

[0106] The calculation formula of root mean square error MSE is:

[0107]

[0108] Where m is the total number of samples, z j Indicates the actual value, here z j f'3(f2(f'1(x i ;θ1);θ2);θ3);z j is the predicted value, here it is y j The smaller the MSE, the smaller the gap between the actual value and the predicted value.

[0109] Since the parameters of θ2 remain unchanged, only θ1 and θ3 need to be optimized during training, namely:

[0110]

[0111] Here, the loss function L only takes partial derivatives with respect to θ1 and θ3, but not with respect to θ2. After several rounds of optimization, the model reaches a convergence state, and the trained client model f3″ is obtained.

[0112] S3. Determine the output category of the model to be trained and the probability distribution corresponding to the output category according to the trained client model f″3.

[0113] The specific operation steps of S3 include:

[0114] S31, determining the output category of the model to be trained according to the type of the output result of the client model f″3;

[0115] S32: embed the client into the dataset E c Input the server model f2 to get the server embedding dataset E s ;

[0116] The server obtains the client embedding E sent by the client in the last round of segmentation learning training. c After that, it is input into the server model f2 to obtain the server embedding E s =f2(E c ), and form a comprehensive embedding dataset D e ={E c ,E s}.

[0117] S33, using clustering algorithm to embed the obtained server into data set E s Divide into several clusters, each cluster corresponds to an output category;

[0118] In this embodiment, K-means clustering is used to embed the server into the data set E. sAll sample points in are divided into K clusters, where K is a positive integer; the number of clusters K is the same as the number of categories in the training task of segmentation learning.

[0119] Use the clustering algorithm to embed the obtained servers into the dataset E s Divided into several cluster families, including:

[0120] S331. Embed dataset E from server s Randomly select K sample points as the initial cluster centers;

[0121] In this embodiment, for the convenience of subsequent description, the cluster center is recorded as

[0122] S332, embedding data set E into the server s For each sample point in , the distance from each sample point to each cluster center is calculated respectively, and the sample point is assigned to the cluster family with the nearest cluster center.

[0123] S333, for each cluster, calculating the mean of the sample points in the cluster, and taking the mean as the new cluster center of the cluster;

[0124] Let the sample point be e i , i represents the server embedding dataset E s The index of the internal sample point; minimize the sum of the squared Euclidean distances of all sample points in the cluster:

[0125]

[0126] Where J represents the sum of the squared Euclidean distances of the sample points within the cluster, ||·|| represents the Euclidean distance, and C k represents the index set of samples belonging to cluster family k, k∈[0,K-1]. Then for each cluster k, calculate the mean of the samples in the cluster and use it as the new cluster center Right now:

[0127]

[0128] Steps S331 to S333 are repeatedly performed until the values ​​of the K cluster centers no longer change, and the cluster family to which each sample point belongs is determined based on the current classification of the sample points.

[0129] Repeat the assignment and update steps until the cluster center No longer changes, or reaches a predetermined number of iterations.

[0130] After getting K cluster centers After that, for E s Each sample point e in i , calculate its distance from each cluster center, and find the cluster with the closest distance.

[0131] S34. For the sample point data in the cluster, select the sample point data closest to the cluster center of the cluster and input it into the client model f″3 to obtain the probability distribution of the output category corresponding to the cluster.

[0132] Select some sample points closest to the center of each cluster and record this type of sample points as E s ′, these E s ′ and their corresponding E c ′(from the comprehensive embedding dataset D e ) to form a new data set D s ={(E c ′,E s ′)}. Then E s ′ is input into the trained client model f3″ to obtain the probability distribution P = softmax(f3″(E s ′)), that is, the posterior probability of each output category, and the maximum probability is the prediction result of the model. Softmax is a normalization function that can convert a set of arbitrary real values ​​into probability values ​​between .

[0133] For example, in a 10-class classification problem, the probability distribution for each input is a 10-dimensional vector. Assuming that the maximum value is the value of index 3 (counting starts from 0, and the index range is 0 to 9), then the result predicted by the model is the third category. Then the original probability data set D is formed. o ={(E c ′,P)}.

[0134] S4. Construct an original probability data set based on the output categories of the model to be trained and the probability distribution corresponding to the output categories.

[0135] S5. Process the samples in the original probability data set according to preset rules to obtain a poisoning probability data set.

[0136] The specific steps of processing the samples in the original probability data set according to the preset rules to obtain the poisoning probability data set include:

[0137] S51, sorting the distribution probability values ​​in the original probability data set from large to small, recording the maximum value of the distribution probability value in the original probability data set as the first distribution probability value, and recording the second largest value of the distribution probability value in the original probability data set as the second distribution probability value;

[0138] S52, simultaneously changing the numerical values ​​of the first distribution probability value and the second distribution probability value: reducing the numerical value of the first distribution probability value and increasing the numerical value of the second distribution probability value, until the difference between the first distribution probability value and the second distribution probability value reaches a preset value;

[0139] S53: modify the content of the original probability data set based on the modified first distribution probability value and the modified second distribution probability value to obtain a poisoning probability data set.

[0140] After the server obtains the probability distribution P, it changes the maximum and second largest values ​​of the probability distribution P. Specifically, the highest probability is reduced, while the second highest probability is increased, so that the probability distributions of the first two are closer, while ensuring that the sum of all probabilities is still 1, thus obtaining the modified poisoning probability distribution P′ and embedding it with the corresponding data E c ′ forms a new poisoning probability dataset D p ={(E c ′,P′)}.

[0141] S6. Strengthen the parameters of the server model f2 according to the original probability data set and the poisoning probability data set to obtain a strengthened server model f'2.

[0142] Finally, the server combines the original probability dataset and the poisoned probability dataset, and inputs them into the model composed of the server model f2 and the client model f3″. By freezing the client model f3″, the server model f2 that the server had in the original training is fine-tuned to obtain the enhanced server model f2′. Since the entire dataset has been expanded, the model can still maintain correct predictions when dealing with low-confidence probabilities (i.e., poisoning probabilities, because its maximum value has become smaller), and the boundaries of the predicted probabilities for each class have become clearer, thereby enhancing the robustness of the model. Subsequently, by using some classic adversarial attack methods, such as FGSM, PGD, etc., the ability of the enhanced model to resist adversarial attacks can be tested.

[0143] FGSM (Fast Gradient Sign Method) is an algorithm for generating adversarial samples based on gradients. It is a non-targeted attack in adversarial attacks, that is, it does not require adversarial samples to be predicted to a specified category by the model, as long as they are different from the original sample predictions. FGSM calculates the gradient of the loss function, then takes the sign function value of the gradient, and adds perturbations along the direction of the gradient to maximize the loss function, thereby generating adversarial samples.

[0144] PGD ​​(Projected Gradient Descent) is an iterative algorithm for generating stronger adversarial samples. It applies small perturbations to the input data in each round through multiple iterations and retrains the model until a perturbation sufficient to deceive the model is found. PGD uses the projected gradient descent method to project the perturbation in each iteration to ensure that the perturbation is within a certain range, thereby generating stronger adversarial samples.

[0145] It should be noted that the poisoning attack defense method for segmentation learning in this embodiment is mainly aimed at protecting the model of image recognition or classification tasks. It can resist poisoning attacks aimed at reducing the robustness of the model by attackers, further improve the model's ability to resist poisoning attacks, and improve network security.

[0146] Reference Figure 3 and Figure 4 When the segmentation learning model is attacked by black box poisoning, the poisoning attack defense method for segmentation learning includes:

[0147] S7. Obtain the training data set and second model information of the model to be trained.

[0148] In the black box poisoning attack defense scenario, the type of model to be strengthened and the content of the training data set are the same as those in the white box poisoning attack defense scenario. For the sake of the brief description, they will not be repeated here.

[0149] The second model information includes the architecture and parameters of the server model f2, the client embedding dataset E c .

[0150] S8, based on the server model f2, the client embeds the dataset E c And the adversarial sample generator generates adversarial samples.

[0151] According to the server model f2, the client embeds the dataset E c And the adversarial sample generator generates adversarial samples, including:

[0152] S81. Embed the client into dataset E c Input the server model f2 to get the server embedding dataset E s ;

[0153] S82. Client-based embedding dataset E c and server embedding dataset E s Constructing a comprehensive embedding dataset D e ={(E c ,E s )};

[0154] S83. Randomly select a sample pair (x i ,y i ), x i represents the client embedded data sample, y i For client-based embedding data sample x i Generated server embedding data samples;

[0155] S84, randomly initialize a data sample x embedded with the client i Perturbations of the same shape Embed the client into data sample x i With disturbance Add them together to get x i The corresponding initial adversarial sample

[0156] The server uses the server model f2 and the comprehensive embedding dataset D e and adversarial sample generator, a perturbation value δ can be obtained. Specifically, the server first obtains e Randomly select a sample pair (x i ,y i ), where i∈[0,||D e ||-1], ||D e || represents the comprehensive embedding dataset D e The length of x is then randomly initialized. i Perturbations of the same shape Will disturb Embed data sample x with client i Add them together to get x i Corresponding adversarial examples Among them, in order to control the size of the perturbation, the adversarial sample Each value in is constrained to be in the range [-ε,ε].

[0157] S85, will Input into server model f2, and get And calculate the loss value loss=l(y i ′,y i );

[0158] Here, MSE is used as the loss function.

[0159] S86. Calculate the loss value loss about The gradient of Get the updated adversarial sample

[0160] Loss value loss about The expression of the gradient is:

[0161]

[0162] S87. Based on the updated adversarial sample Update the disturbance value and get the updated disturbance value

[0163]

[0164] Where α>0 is the learning rate (step size), and the sign(·) function is used to preserve the sign of the gradient, that is, the direction of the gradient.

[0165] Perturbation value The expression is:

[0166]

[0167] where clamp ||ε|| (·) means limiting the value (·) to the range of [-ε,ε]. Repeat steps S85 to S87, iterate the disturbance value S times, and obtain the disturbance value after iteration. S is a positive integer; in the preferred implementation of this embodiment, S is usually 10.

[0168] S88, the disturbance value With x i Add together to get the updated adversarial sample

[0169] S9. Use the training data set and adversarial samples to perform perturbation training on the training model to generate a poisoned sample-label set.

[0170] Get x i Adversarial examples Afterwards, the adversarial sample Input into server model f2, and get Calculate y i ″ and y i The loss between Here, we still use MSE as the loss function. Next, the optimization goal is to find an optimal value Make y i ″ and y i The loss between is maximized, that is:

[0171]

[0172] Then the optimal value Acting on x i On, get Will With y i Composing input-label pairs Thus, a poisoned sample-label set is obtained.

[0173] The specific steps of perturbation training include:

[0174] S91. Adversarial Samples Input into server model f2, get, calculate yi ″ and y i The loss function between

[0175] S92. Using loss function The optimal value of the most determined disturbance value Make y i ″ and y i Maximize the loss between .

[0176] Calculate y i ″ and y i The loss L(δ i s )=l(y i ″,y i ), here, MSE is still used as the loss function. Next, the optimization goal is to find an optimal value Make y i ″ and y i The loss between is maximized, that is:

[0177]

[0178] S93, the optimal value Acts on the client embedded data sample x i Get the poisoned sample

[0179] S94, based on poisoned samples Embed data sample y with the server i Generate poisoned sample-label pairs to obtain poisoned sample-label sets.

[0180] Steps S91 to S94 will be performed || D e || times, ||D e || represents the comprehensive embedding dataset D e length; the comprehensive embedding dataset D e All samples in are processed once, so we get ||D e || input-label pairs, combined into a poisoned sample-label set (||D P ||=||D e ||), ||D P || represents the comprehensive embedding dataset D P Length.

[0181] S10. Strengthen the parameters of the server model according to the poisoned sample-label set to obtain a strengthened server model f'2.

[0182] Using D Pand D e ={(E c ,E s )} jointly fine-tune f2, and the loss function L(θ2) is:

[0183]

[0184] where r = ||D P ||=||D e || represents the data set D P and D e The length of ; β∈[0.5,1] is a balancing quantity used to balance and l(f2(e c ;θ2),e s ) is the influence of the two losses; l(·,·) is the loss function of the sample, here, we use the root mean square error MSE; and (e c ,e s ) are D P and D emd By minimizing L(θ2), we get Right now Thus, the enhanced server model f2′ is obtained. Since the server model f2′ is trained using adversarial samples, the model to be trained can still maintain strong robustness when facing adversarial samples, thereby realizing the poisoning attack defense method in the black box case.

[0185] The present application also discloses an electronic device, such as Figure 5 , which is a schematic diagram of the structure of an electronic device for a poisoning attack defense method for segmented learning provided by an embodiment of the present invention. The electronic device may include at least one processor 10, a memory 11 connected to the at least one processor for communication, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a program for the poisoning attack defense method for segmented learning.

[0186] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (for example, executing a method for poisoning attack defense for segmentation learning, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device and process data.

[0187] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Med ia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of the method program for poisoning attack defense for segmentation learning, but also be used to temporarily store data that has been output or is to be output.

[0188] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0189] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visualized user interface.

[0190] Figure 5 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown, or combine certain components, or arrange the components differently. For example, although not shown, the electronic device may also include a power supply (such as a battery) for powering each component. Preferably, the power supply may be logically connected to at least one processor 10 through a power management device, so that functions such as charging management, discharging management, and power consumption management are implemented through the power management device. The power supply may also include any components such as one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0191] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure.

[0192] Furthermore, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile.

[0193] The present application embodiment provides a computer-readable storage medium, for example, including: any entity or device, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM) that can carry the computer program code. The computer-readable storage medium stores a computer program that can be loaded by a processor and execute the poisoning attack defense method for segmentation learning of the above embodiment.

[0194] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "an implementation", "a preferred implementation" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0195] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A poisoning attack defense method for segmentation learning, characterized in that: The method comprises: Obtain a training data set for U-shaped segmentation learning and first model information of a model to be trained; the model to be trained includes three consecutive sub-models, the three sub-models are client model f1, server model f2 and client model f3; the first model information includes the architecture and parameters of the server model f2, the architecture of the client model f1, the architecture of the client model f3 and the client's embedded data set E c ; Perform model inference according to the first model information and the training data set to obtain a trained client model f"3; Determine the output category of the model to be trained and the probability distribution corresponding to the output category according to the client model f"3; Construct an original probability dataset based on the output categories of the model to be trained and the probability distribution corresponding to the output categories; Perform data processing on the samples in the original probability data set according to preset rules to obtain a poisoning probability data set; According to the original probability data set and the poisoning probability data set, the parameters of the server model f2 are enhanced to obtain the enhanced server model f'2; or, Obtain the training data set and the second model information of the model to be trained; the second model information includes the architecture and parameters of the server model f2, and the client embedding data set E c ; According to the server model f2, the client embeds the dataset E c Generate adversarial samples with the adversarial sample generator; Use the training data set and adversarial samples to perform perturbation training on the training model to generate a poisoned sample-label set; The parameters of the server model are enhanced according to the poisoned sample-label set to obtain the enhanced server model f'2.

2. The poisoning attack defense method for segmentation learning according to claim 1, characterized in that: The training data set includes several input samples and reference labels corresponding to the input samples; The method of performing model inference according to the first model information and the training data set to obtain a trained client model f"3 includes: Initialize the parameters of the client model f1 to obtain the initialized client model f'1; Initialize the parameters of the client model f3 to obtain the initialized client model f'3; The model composed of the client model f'1, the server model f2 and the client model f'3 is recorded as the first training model, the input sample is processed using the first training model, and the predicted label of the input sample is output; Determine a loss function of the first training model according to the predicted label and the reference label corresponding to the input sample; The parameters of the client model f'3 are optimized according to the loss function to obtain the client model f"3.

3. The poisoning attack defense method for segmentation learning according to claim 2, characterized in that: The loss function is calculated as Where n is the number of input samples in the training dataset; x j represents the input sample, y j represents the reference label corresponding to the input sample, j represents the index of the input sample in the training dataset; θ1 represents the parameters of the client model f1′, θ3 represents the parameters of the client model f3′, and θ2 is the parameter of the server model f2; f1′(x j ; θ1) represents the value of x j Input to the client model f1′ acts on the parameter θ1; l(·,·) is the loss function of the input sample; The method of optimizing the parameters of the client model f'3 according to the loss function to obtain the client model f"3 includes: Keeping θ2 unchanged, the loss function L(θ1,θ2,θ3) is used to optimize the parameters of the client model f1′ to obtain the optimized parameters θ1 * , use the loss function L(θ1,θ2,θ3) to optimize the parameters of the client model f3′ to obtain the optimized parameters θ3 * ; According to the parameter θ3 * Determine the client model f"3.

4. The poisoning attack defense method for segmentation learning according to any one of claims 1 to 3, characterized in that: The step of determining the output category of the model to be trained and the probability distribution corresponding to the output category according to the client model f″3 includes: Determine the output category of the model to be trained according to the type of the output result of the client model f"3; Embed the client into dataset E c Input the server model f2 to get the server embedding dataset E s ; Use the clustering algorithm to embed the obtained servers into the dataset E s Divide into several clusters, each cluster corresponds to an output category; For the sample point data in the cluster family, the sample point data closest to the cluster center of the cluster family is selected and input into the client model f'3' to obtain the probability distribution of the output category corresponding to the cluster family.

5. The poisoning attack defense method for segmentation learning according to any one of claims 1 to 3, characterized in that: The server is embedded into the dataset E using the clustering algorithm. s Divided into several cluster families, including: S331. Embed dataset E from server s Randomly select K sample points as the initial cluster centers; S332, embedding data set E into the server s For each sample point in , calculate the distance from each sample point to each cluster center, and assign the sample point to the cluster family with the nearest cluster center; S333, for each cluster, calculating the mean of the sample points in the cluster, and taking the mean as the new cluster center of the cluster; Steps S331 to S333 are repeatedly performed until the values ​​of the K cluster centers no longer change, and the cluster family to which each sample point belongs is determined based on the current classification of the sample points.

6. The poisoning attack defense method for segmentation learning according to any one of claims 1 to 3, characterized in that: The data processing of the samples in the original probability data set according to the preset rules to obtain the poisoning probability data set includes: Sort the distribution probability installation values ​​in the original probability data set from large to small, record the maximum value of the distribution probability value in the original probability data set as the first distribution probability value, and record the second largest value of the distribution probability value in the original probability data set as the second distribution probability value; Simultaneously change the numerical value of the first distribution probability value and the numerical value of the second distribution probability value: reduce the numerical value of the first distribution probability value and increase the numerical value of the second distribution probability value until the difference between the first distribution probability value and the second distribution probability value reaches a preset value; The content of the original probability data set is modified based on the modified first distribution probability value and the modified second distribution probability value to obtain a poisoning probability data set.

7. The poisoning attack defense method for segmentation learning according to any one of claims 1 to 3, characterized in that: According to the server model f2, the client embeds the data set E c And the adversarial sample generator generates adversarial samples, including: S81. Embed the client into dataset E c Input the server model f2 to get the server embedding dataset E s ; S82. Client-based embedding dataset E c and server embedding dataset E s Constructing a comprehensive embedding dataset D e ={(E c ,E s )}; S83. Randomly select a sample pair (x i ,y i ), x i represents the client embedded data sample, y i For client-based embedding data sample x i Generated server embedding data samples; S84, randomly initialize a data sample x embedded with the client i Perturbations of the same shape Embed the client into data sample x i With disturbance Add them together to get x i The corresponding initial adversarial sample S85, will Input into server model f2, and get And calculate the loss value loss=l(y i ′,y i ); S86. Calculate the loss value loss about The gradient of Get the updated adversarial sample S87. Based on the updated adversarial sample Update the disturbance value and get the updated disturbance value Repeat steps S85 to S87 to iterate the disturbance value S times to obtain the disturbance value after iteration. S is a positive integer; S88, the disturbance value With x i Add together to get the updated adversarial sample 8. The poisoning attack defense method for segmentation learning according to claim 7, characterized in that: The method of using the training data set and the adversarial sample to perform perturbation training on the training model to generate a poisoned sample-label set includes: The adversarial sample Input into server model f2, and get Calculate y i ″ and y i The loss function between Using loss function The optimal value of the most determined disturbance value Make y i ″ and y i Maximize the loss between The optimal value Acts on the client embedded data sample x i Get the poisoned sample Based on poisoned samples Embed data sample y with the server i Generate poisoned sample-label pairs to obtain poisoned sample-label sets.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor (10); and, a memory (11) communicatively connected to the at least one processor (10); The memory (11) stores a computer program that can be executed by the at least one processor (10), and the computer program is executed by the at least one processor (10) so that the at least one processor (10) can execute the poisoning attack defense method for segmentation learning as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer storage medium stores a computer program; when the computer program is executed by a processor, the poisoning attack defense method for segmentation learning as described in any one of claims 1 to 8 is implemented.

Images