Method and device for processing anti-attack data based on black box model

Abstract

The present application relates to a method, apparatus, computer device, storage medium, and computer program product for processing adversarial attack data based on a black-box model. The method comprises: determining a first attacked data sample and at least one first candidate data sample for the attacked sample data; perturbing the first attacked data sample to obtain a corresponding pseudo-sample; comparing features of the pseudo-sample with the at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity; and updating the perturbation features of the pseudo-sample based on the candidate data sample with the highest feature similarity to obtain a constructed pseudo-sample. This method can improve the authenticity of the constructed pseudo-sample.

Description

Technical Field

[0001] The present application relates to the field of neural network defense technology, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for processing adversarial attack data based on a black box model. Background Art

[0002] The successful application of deep learning algorithms in recommender systems has further improved their performance and efficiency. However, current research has demonstrated that deep learning models are highly vulnerable to attacks, making it crucial to design effective attack algorithms to test the vulnerabilities of deep learning-based recommender systems. Because recommender systems process discrete data, deep learning-based attack algorithms cannot be directly applied to these systems. Currently, poisoning attacks are the most common attack algorithm used in recommender systems. These algorithms primarily inject false data into the input to degrade model performance. Therefore, designing effective black-box attack algorithms is crucial.

[0003] Common black-box attacks include replication attacks, query attacks, and substitution attacks. Substitution attacks involve a white-box model using distillation to learn the output distribution of a black-box model. The model is then attacked to obtain perturbations, which are then used to attack the black-box model. However, the perturbations generated by attacks based on the trained white-box model are less likely to be authentic. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for processing adversarial attack data based on a black box model, which can improve the authenticity of constructed pseudo samples, in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for processing adversarial attack data based on a black box model. The method comprises:

[0006] determining a first attacked data sample and at least one first candidate data sample of the attacked sample data;

[0007] Performing perturbation processing on the first attacked data sample to obtain a corresponding pseudo sample;

[0008] Performing feature comparison between the pseudo sample and the at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity;

[0009] The perturbation feature of the pseudo sample is updated according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample.

[0010] In one embodiment, performing feature comparison between the pseudo sample and the at least one first candidate data sample to obtain the candidate data sample with the highest feature similarity includes:

[0011] Acquiring characteristic attributes of the pseudo sample;

[0012] If the feature attribute is a discrete feature, and all the perturbation features of the pseudo sample do not exist in the at least one candidate data sample, determining the perturbation features to be updated that do not exist in the at least one first candidate data sample;

[0013] Determine the similarity between the sample feature of the at least one first candidate data sample and the disturbance feature to be updated, and obtain a candidate data sample with the highest feature similarity based on the similarity.

[0014] In one embodiment, the method further comprises:

[0015] Attacking the target black box model according to the constructed pseudo sample to obtain an attack result;

[0016] The reliability of the constructed pseudo sample is determined according to the attack result.

[0017] In one embodiment, the perturbation processing is performed on the first attacked data sample to obtain a corresponding pseudo sample, including: inputting the first attack data sample into a trained perturbation generation network, and outputting the corresponding pseudo sample.

[0018] In one embodiment, the training of the perturbation generation network includes:

[0019] Obtaining a second attacked data sample and at least one second candidate data sample of the attacked data sample for training the disturbance generating network;

[0020] Inputting the second attacked data sample into the perturbation generation network, and outputting a perturbation vector of the second attacked data sample after passing through the embedding layer, encoder, and decoder of the perturbation generation network;

[0021] Inputting the disturbance vector into the network layer of the substitution model to obtain a prediction result;

[0022] Comparing the prediction result with the corresponding candidate sub-data sample to obtain the contrast loss;

[0023] The encoder and decoder are trained based on the contrastive loss to obtain a trained perturbation generation network.

[0024] In one embodiment, the training of the encoder and decoder based on the contrastive loss to obtain a trained perturbation generation network includes:

[0025] Based on the contrast loss, inputting the at least one second candidate data sample into a substitution model and outputting an intermediate vector feature;

[0026] Performing similarity calculation on the perturbation vector and the intermediate vector features to obtain an average loss;

[0027] The encoder and decoder are trained based on the average loss until the average loss converges to a preset value, thereby obtaining a trained perturbation generation network.

[0028] In one embodiment, the surrogate model is determined based on the target black box model, and the surrogate model is determined by:

[0029] Obtain the original dataset and the constructed alternative model;

[0030] Inputting the original data set into the surrogate model and the target black-box model respectively to obtain a first output feature of the surrogate model and a second output feature of the target black-box model;

[0031] determining an output loss between the surrogate model and the target black-box model based on the first output feature and the second output feature;

[0032] determining a classification loss of the surrogate model based on the first output feature;

[0033] According to the classification loss and the output loss, a total loss function of the replacement model is determined to obtain a replacement model with the same function as the target black box model.

[0034] In one embodiment, obtaining a second attacked data sample and at least one second candidate data sample for the attacked data sample for training the perturbation generating network includes:

[0035] Obtain a training sample set;

[0036] Dividing the training sample set according to the attributes of each sample in the training sample set to obtain an attacked data sample set and a candidate data sample set;

[0037] Based on a recommendation algorithm, at least one second candidate data sample similar to each second attacked data sample set in the attacked data sample set is determined from the candidate data sample set.

[0038] In a second aspect, the present application also provides a device for processing attack data based on a black box model. The device comprises:

[0039] a data determination module, configured to determine a first attacked data sample and at least one first candidate data sample for the attacked sample data;

[0040] a disturbance processing module, configured to perform disturbance processing on the first attacked data sample to obtain a corresponding pseudo sample;

[0041] a feature comparison module, configured to perform feature comparison between the pseudo sample and the at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity;

[0042] The construction module is used to update the perturbation feature of the pseudo sample according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample.

[0043] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0044] determining a first attacked data sample and at least one first candidate data sample of the attacked sample data;

[0045] Performing perturbation processing on the first attacked data sample to obtain a corresponding pseudo sample;

[0046] Performing feature comparison between the pseudo sample and the at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity;

[0047] The perturbation feature of the pseudo sample is updated according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample.

[0048] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0049] determining a first attacked data sample and at least one first candidate data sample of the attacked sample data;

[0050] Performing perturbation processing on the first attacked data sample to obtain a corresponding pseudo sample;

[0051] Performing feature comparison between the pseudo sample and the at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity;

[0052] The perturbation feature of the pseudo sample is updated according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample.

[0053] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0054] determining a first attacked data sample and at least one first candidate data sample of the attacked sample data;

[0055] Performing perturbation processing on the first attacked data sample to obtain a corresponding pseudo sample;

[0056] Performing feature comparison between the pseudo sample and the at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity;

[0057] The perturbation feature of the pseudo sample is updated according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample.

[0058] The above-mentioned method, apparatus, computer device, storage medium and computer program product for processing attack data based on black-box attack determine at least one first candidate data sample corresponding to the first attacked data sample, perform perturbation processing on the first attacked data sample, and obtain a corresponding pseudo sample; by selecting the candidate data sample with the highest similarity to the pseudo sample from the at least one first candidate data sample, the perturbation feature of the pseudo sample is updated, thereby improving the authenticity of the generated pseudo sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 FIG1 is an application environment diagram of a method for processing anti-attack data based on a black box model in one embodiment;

[0060] Figure 2 1 is a flowchart of a method for processing anti-attack data based on a black box model in one embodiment;

[0061] Figure 3 Schematic diagram of the process of preparing candidate data samples in one embodiment;

[0062] Figure 4 A schematic flow chart of a method for training a perturbation generation network in one embodiment;

[0063] Figure 5 1 is a flow chart of a method for determining a substitution model in one embodiment;

[0064] Figure 6 1 is a flowchart of a method for processing anti-attack data based on a black box model in another embodiment;

[0065] Figure 7 FIG1 is a block diagram of a processing structure of anti-attack data based on a black box model in one embodiment;

[0066] Figure 8 FIG. 1 is an architectural diagram for determining a constructed pseudo sample in one embodiment;

[0067] Figure 9 1 is a block diagram of a device for processing attack data based on a black box model in one embodiment;

[0068] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0070] The present invention provides a method for processing attack data based on a black box model, which can be applied to Figure 1 In the application environment shown. The terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal determines a first attacked data sample and at least one first candidate data sample of the attacked sample data; performs perturbation processing on the first attacked data sample to obtain a corresponding pseudo sample; performs feature comparison on the pseudo sample and at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity; updates the perturbation feature of the pseudo sample according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, portable wearable devices, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0071] It can be understood that the "first" and "second" in this application are used to distinguish different objects. For example, the "first" and "second" in the "first attacked data sample" and the "second attacked data sample" are used to distinguish different attacked data samples. The "second attacked data sample" can also be the "first attacked data sample" and the "first attacked data sample" can also be the "second attacked data sample".

[0072] In one embodiment, Figure 2 As shown in the figure, a method for processing adversarial attack data based on a black box model is provided. Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0073] Step 202: Determine a first attacked data sample and at least one first candidate data sample of the attacked sample data.

[0074] The attacked data samples and candidate data samples can be determined based on the data's click-through rate. For example, historical data with a click-through rate of 0 can be identified as the attacked data sample, while historical data with a click-through rate of 1 can be identified as the attacked data sample. In different application scenarios, the attacked data samples and candidate data samples may be of different types. For example, the attacked data samples and candidate data samples may be, but are not limited to, image data. The candidate data samples corresponding to each attacked data sample may be determined using, but are not limited to, a KNN algorithm. The attacked data samples may contain multiple features of the same object. The attacked data samples can be considered abnormal data, while the candidate data samples can be considered normal data.

[0075] Specifically, historical data generated by human-computer interaction in relevant application scenarios is obtained, and the data attributes of the historical data are determined. The click-through rate of each historical data item is then determined based on the data attributes. The data is then partitioned based on the click-through rate to obtain attacked data samples and candidate data samples. For each first attacked data sample (click = 0), the top-K samples (click = 1) closest to it are found, thereby obtaining at least one first candidate data sample corresponding to each first attacked data sample.

[0076] Step 204: Perform perturbation processing on the first attacked data sample to obtain a corresponding pseudo sample.

[0077] Perturbation processing involves adding interference features to the attacked data sample to generate new features and obtain pseudo samples, which are also understood as perturbation vectors. Perturbation processing can be achieved by constructing a perturbation generation network. The perturbation generation network includes a ModuleDict embedding layer, an encoder, a decoder, and a reconstruction network.

[0078] Step 206 : perform feature comparison between the pseudo sample and at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity.

[0079] The features of the pseudo samples can be discrete features or continuous features. It is understandable that continuous features have continuity and do not need to be processed for realism.

[0080] Step 208 : updating the perturbation features of the pseudo sample according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample.

[0081] The constructed pseudo samples are used to attack the black-box system to improve its defense capability and fault tolerance. The black-box system can be, but is not limited to, a black-box recommendation system.

[0082] Specifically, in order to ensure the authenticity of the pseudo sample, the features of the pseudo sample are detected. If the features of the pseudo sample are discrete features, the KNN algorithm is used to compare all the features in the pseudo sample with at least one first candidate data sample to obtain the candidate data sample with the highest feature similarity.

[0083] In the above-mentioned method for processing adversarial attack data based on the black-box model, at least one first candidate data sample corresponding to the first attacked data sample is determined, and the first attacked data sample is perturbed to obtain a corresponding pseudo sample; by selecting the candidate data sample with the highest similarity to the pseudo sample from the at least one first candidate data sample, the perturbation feature of the pseudo sample is updated, thereby improving the authenticity of the generated pseudo sample.

[0084] When determining the candidate data sample with the highest feature similarity, it is determined whether all discrete features in the pseudo sample exist in the candidate data sample. If there are discrete features that do not exist in the candidate data sample, the feature with the highest similarity can be determined for each non-existent discrete feature in the candidate data sample, and these non-existent discrete features can be replaced to obtain a constructed pseudo sample. In one embodiment, Figure 3 As shown, a method for determining a candidate data sample with the highest feature similarity is provided, comprising the following steps:

[0085] Step 302: Acquire characteristic attributes of the pseudo sample.

[0086] Among them, feature attributes include discrete features and continuous features.

[0087] Step 304: If the feature attribute is a discrete feature and all the perturbation features of the pseudo sample do not exist in at least one candidate data sample, then determine the perturbation features to be updated that do not exist in at least one first candidate data sample.

[0088] Step 306 : Determine the similarity between the sample feature of at least one first candidate data sample and the disturbance feature to be updated, and obtain the candidate data sample with the highest feature similarity based on the similarity.

[0089] Specifically, if the characteristic attribute of the pseudo sample is a discrete feature, the authenticity of the feature of the pseudo sample is tested. If all the discrete features of the pseudo sample cannot find the same features in the candidate data sample, the number of discrete features that do not exist in the candidate data sample is determined, and the KNN algorithm is used to perform similarity judgment on all the discrete features of each pseudo sample and the corresponding candidate data sample to obtain the candidate data sample with the highest feature similarity, and the features that do not exist in the discrete perturbation features are replaced with the most similar features.

[0090] In the above method for determining the candidate data sample with the highest feature similarity, for the pseudo sample with discrete features, the most similar feature is selected for each discrete feature that does not exist in the candidate data sample to obtain the candidate data sample with the highest similarity, thereby ensuring the authenticity of the pseudo sample.

[0091] The pseudo sample can be generated by inputting the first attack data sample into a trained disturbance generation network. In one embodiment, Figure 4 As shown, a training method for a perturbation generation network is provided, comprising the following steps:

[0092] Step 402: Obtain a second attacked data sample and at least one second candidate data sample of the attacked data sample for training the perturbation generating network.

[0093] The perturbation generation network consists of a ModuleDict embedding layer, an encoder, a decoder, and a reconstruction network. The ModuleDict embedding layer is a model container, consisting of embedding layers for all discrete data. Each embedding input is the total number of values ​​in each dimension, and the output dimension is 8. The encoder structure includes: the first layer is a linear output layer, consisting of a linear layer and an activation function. The input is the total number of features, 86, the output dimension is 43, and the activation function is Tanh(). The second layer is a linear output layer, consisting of a linear layer and an activation function. The input is the feature dimension 43, the output dimension is 21, and the activation function is Tanh(). The third layer is a linear layer, consisting of an input feature dimension 21, and an output dimension 10. The decoder structure includes: the first layer is a linear output layer, consisting of a linear layer and an activation function. The input is the feature dimension 10, the output dimension is 21, and the activation function is Tanh(). The second layer is a linear output layer, consisting of a linear layer and an activation function. The input is the feature dimension 21, the output dimension is 43, and the activation function is Tanh(). The third layer is a linear layer with an input feature dimension of 43 and an output dimension of 86.

[0094] In one embodiment, a training sample set is obtained; the training sample set is divided according to the attributes of each sample in the training sample set to obtain an attacked data sample set and a candidate data sample set; and at least one second candidate data sample similar to each second attacked data sample set in the attacked data sample set is determined from the candidate data sample set based on a recommendation algorithm. The obtained raw data is divided to obtain a training sample set and a test set, and the number of attacked data samples in the test set must meet a set number.

[0095] In step 404, the second attacked data sample is input into the perturbation generation network, and the perturbation vector of the second attacked data sample is output after passing through the embedding layer, encoder, and decoder of the perturbation generation network.

[0096] Step 406: Input the perturbation vector into the network layer of the replacement model to obtain a prediction result.

[0097] Among them, the alternative model is determined according to the target black box model. The determination method of the alternative model is as follows: Figure 5 As shown, the following steps are included:

[0098] Step 502: Obtain the original data set and the constructed alternative model.

[0099] The surrogate model consists of an embedding layer and a network layer. The FM layer of the network layer defines three matrices: a global bias, a first-order weight matrix, and a second-order cross matrix. The global bias is added to the product of the input features and the first-order weight matrix to obtain the first-order cross matrix. The FM algorithm then calculates the second-order cross matrix using the first-order cross matrix. The sum of the first-order and second-order cross matrices is returned. The DNN layer in the network layer consists of a linear layer and a ReLU function, followed by a sequence of layers. Finally, the nn.Dropout function is used to prevent overfitting.

[0100] In step 504 , the original data set is input into the surrogate model and the target black-box model respectively to obtain a first output feature of the surrogate model and a second output feature of the target black-box model.

[0101] The target black-box model and the surrogate model share network parameters. The target black-box model consists of a ModuleDict embedding layer, a Product layer, a DNN layer, and a linear output layer. The ModuleDict embedding layer is a model container, consisting of embedding layers for all discrete data. Each embedding input is the total number of values ​​in each dimension, and the output dimension is 10. The linear output layer is composed of a linear layer and a sigmoid activation function, with the input dimension of the linear layer being all feature dimensions and the output dimension being 1.

[0102] The Product layer includes a linear part and a nonlinear part. The linear part of the Product layer is the inner product calculation of the features after the embedding layer, and the nonlinear part of the Product layer is the outer product of the features after the two embedding layers. The [embed_dim, embed_dim] matrix is ​​then summed at the corresponding positions to obtain a final [embed_dim, embed_dim] matrix. Here, embed_dim is 10. The DNN layer consists of a linear layer followed by a ReLU function, and the nn.Dropout function is used to prevent overfitting. The linear output layer consists of a linear layer followed by a sigmoid activation function. The input dimension of the linear layer is the total feature dimension, and the output dimension is 1.

[0103] Step 506 : Determine the output loss between the surrogate model and the target black-box model based on the first output feature and the second output feature.

[0104] Step 508: Determine the classification loss of the alternative model based on the first output feature.

[0105] In step 510 , a total loss function of the substitute model is determined based on the classification loss and the output loss, thereby obtaining a substitute model having the same function as the target black box model.

[0106] For example, from the original dataset , , The feature of the sample is obtained, each feature consists of n features (Feature), the feature is passed into the black-box model (Black-box Model) and the substitute model (Substitute Model), and the output features B (u i )、S(u i ), according to the output features, the cross entropy is used to obtain the distance loss:

[0107]

[0108] Among them, B(u i ) represents the feature vector of the i-th user feature obtained by the black-box model, S(u i ) represents the feature vector of the i-th user feature obtained by the substitute model.

[0109] According to the output feature S(u i ) The classification loss is calculated using cross entropy.

[0110] Among them, the classification loss is:

[0111]

[0112] Determine the total loss function of the alternative model based on the classification loss and distance loss:

[0113]

[0114] In the above embodiment, by training the constructed substitute model and the target black box model, a substitute model with the same function as the target black box model is obtained. The black box model can be attacked based on the trained substitute model, thereby reducing the difficulty of the attack while ensuring the same function.

[0115] Step 408 : Compare the prediction result with the corresponding candidate sub-data sample to obtain the comparison loss.

[0116] Specifically, in order to ensure that the perturbation vector prediction result is close to the attack target (Click=1) (that is, it can also be understood as the corresponding candidate sub-data sample), and the perturbation vector distribution is as close as possible to the candidate sub-data sample. The perturbation vector passes through the remaining network layers of the replacement model to obtain the final prediction result, and the final prediction result is compared with the corresponding candidate sub-data sample to obtain the contrast loss. For example, according to the value of the label Click, the data is divided into the sub-dataset (Attacked Subdataset) of the second attacked data sample, which is recorded as , Click is 1 and is divided into the second candidate data sample sub-dataset (Selected Subdataset), recorded as ,in, For each attacked sample, the trained Deep FM is used to calculate its similar subset and select the top-k samples for each attacked sample. The calculation formula is:

[0117]

[0118] in, , , for each attacked sample data, there will be a sub-data, that is, including multiple second candidate data samples, so the sub-datasets of all attacked samples are:

[0119] .

[0120] The obtained attacked sample is passed into the perturbation generation network (Generation Network) and passes through the Embedding Layer, Encoder, and Decoder to obtain the perturbation vector of the attack sample. That is, any sample u is selected from the attacked sample. p , , after the embedding layer becomes Vector (in order to facilitate the reconstruction of the perturbation vector later, the size of the output vector of each feature of the embedding layer is the same as the feature size of the sample), and then the embedding layer output is used as the input of the encoder to obtain the intermediate vector. The decoder generates the corresponding Perturbation vector. In order to ensure that the prediction result of the perturbation vector is close to the attack target (Click=1), the perturbation vector Distribute as close to sub as possible sim Subdataset. After replacing the remaining network layers of the model ( ) to obtain the final prediction result, and compare the prediction result with the attack target (click=1). At this time, the cross entropy function is used to obtain the contrast loss:

[0121]

[0122] in, Corresponding sub-dataset sub sim , the corresponding output vectors (v'1,...,v'(k)) are obtained through the embedding layer and the self-attention layer, and the size is ) ).

[0123] Step 410: Train the encoder and decoder based on the contrastive loss to obtain a trained perturbation generation network.

[0124] Specifically, the contrast loss function is calculated according to each training, and the encoder and decoder are trained so that the contrast loss converges to a set value to obtain a trained perturbation generation network.

[0125] The encoder and decoder are trained based on the contrastive loss to obtain a trained perturbation generation network. In order to further make the perturbation vector distribution output by the perturbation generation network close to the intermediate vector of the output of the replacement model, the average loss between the perturbation vector and the intermediate vector is calculated.

[0126] In one embodiment, based on the contrastive loss, at least one second candidate data sample is input into the replacement model to output an intermediate vector feature; similarity is solved for the perturbation vector and the intermediate vector feature to obtain an average loss; the encoder and decoder are trained based on the average loss until the average loss converges to a preset value, thereby obtaining a trained perturbation generation network.

[0127] Specifically, at least one second candidate data sample is input into the substitute model, and after the substitute model passes through the embedding layer and the neural layer, an intermediate vector feature is obtained. Similarity is solved between the perturbation vector and the intermediate vector feature to obtain an average loss; the encoder and decoder are trained based on the average loss until the average loss converges to a preset value, thereby obtaining a trained perturbation generation network. For example, the second candidate data sample is input into the substitute model, and the intermediate vector feature is output. , to ensure that the perturbation vector output by the perturbation generation network is The distribution is close to these output vectors The distribution of , calculate the similarity loss:

[0128]

[0129] Among them, q={1,..., k}, so the system in each batch, the final comparison loss is:

[0130]

[0131] In the above embodiment, the encoder and decoder of the disturbance generation network are trained based on the substitution model. The intermediate disturbance vectors generated by the encoder and decoder are used as the input data of the substitution model neural network. The encoder and decoder can be trained using a normal gradient update method to obtain a trained disturbance generation network. The disturbance vectors output by the disturbance generation network are distributed near the candidate data samples, thereby ensuring the authenticity of the output data of the disturbance generation network.

[0132] In another embodiment, Figure 6 As shown in the figure, a method for processing adversarial attack data based on a black box model is provided. Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0133] Step 602: Determine an alternative model that has the same function as the target black box model.

[0134] Step 604 : Acquire the attacked data sample set and the candidate data sample set, and determine at least one candidate data sample from the candidate data sample set that is similar to each attacked data sample set in the attacked data sample set.

[0135] The attacked data sample set includes a first attacked data sample and a second attacked data sample. The candidate data sample set includes a first candidate data sample and a second candidate data sample. The attacked data sample can be data with a click rate of 0 (click=0), and the candidate data sample can be data with a click rate of 1 (click=1).

[0136] In step 606, the second attacked data sample is input into the perturbation generation network, and the perturbation vector of the second attacked data sample is output after passing through the embedding layer, encoder, and decoder of the perturbation generation network.

[0137] Specifically, the second attacked data sample with click=0 is input into the perturbation generation network. After passing through the embedding layer, encoder and decoder of the perturbation generation network, the perturbation vector of the second attacked data sample is output. The perturbation vector is called the perturbation sample vector, or pseudo sample.

[0138] Step 608: Input the perturbation vector into the network layer of the replacement model to obtain a prediction result.

[0139] Step 610 : Compare the prediction result with the corresponding second candidate data sample to obtain the comparison loss.

[0140] Step 612: Train the encoder and decoder based on the contrastive loss to obtain a trained perturbation generation network.

[0141] In step 614, the first attacked data sample is input into the trained perturbation generation network for perturbation processing to obtain a corresponding pseudo sample.

[0142] In step 616 , the pseudo sample is input into the construction network, and features of the pseudo sample are compared with at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity.

[0143] The constructed network includes an encoder and a decoder. The first layer of the encoder is a linear output layer, consisting of a linear layer and an activation function. The input feature dimension is 72, the output dimension is 144, and the activation function is Tanh(). The second layer is a linear output layer, consisting of a linear layer and an activation function. The input feature dimension is 144, the output dimension is 288, and the activation function is Tanh(). The third layer is a linear layer, with an input feature dimension of 288 and an output dimension of 1000.

[0144] The first layer of the decoder is a linear output layer, consisting of a linear layer and an activation function. The input feature dimension is 1000, the output dimension is 288, and the activation function is Tanh(). The second layer is a linear output layer, consisting of a linear layer and an activation function. The input feature dimension is 288, the output dimension is 144, and the activation function is Tanh(). The third layer is a linear layer, with an input feature dimension of 144 and an output dimension of 72.

[0145] Specifically, the first attacked data sample is input into a trained perturbation generation network for perturbation processing. After passing through the embedding layer, encoder, and decoder, an intermediate perturbation vector is output. At this point, the intermediate perturbation vector needs to be matched with the features, so a reconstruction layer is required to reconstruct it to obtain a pseudo sample. After obtaining the pseudo sample, the authenticity of the pseudo sample needs to be ensured. Therefore, the continuity features are maintained unchanged. The most similar features are selected from the corresponding first candidate data sample for each discrete feature in the attacked data sample that does not exist in the original dataset. The most similar features are used to replace the non-existent features in the discrete perturbation features to obtain a constructed pseudo sample. The constructed pseudo sample is then used to attack the substitution model. If the attack is successful, the constructed pseudo sample is retained. If not, the step is repeated.

[0146] Step 618: Update the perturbation feature of the pseudo sample according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample.

[0147] Step 620: Use the constructed pseudo sample to attack the target black box system.

[0148] It's understandable that network parameters are shared between the black-box model and the substitute model, the substitute model and the perturbation generation network, and the perturbation generation network and the construction network. That is, the weights of all embedding layers are shared with the weights of the reconstruction network. Reconstructing the network is an inverse operation of the embedding layer. This multi-weight sharing concept simplifies the model training process.

[0149] In one embodiment, Figure 7Figure 2 shows a block diagram of the processing architecture for adversarial attack data based on a black-box model. The process involves determining an alternative model with the same functionality as the target black-box model, obtaining a dataset, partitioning the dataset into a set of attacked data samples and a set of candidate data samples, and then determining from the candidate data sets at least one candidate data sample that is similar to each attacked data sample in the attacked data set. The perturbation generation network and the embedding layer of the alternative model share network parameters.

[0150] A second attacked data sample is randomly selected from the attacked data sample set and input into the embedding layer, encoder and decoder of the perturbation generation network, and a perturbation vector Vp is output. Multiple second candidate data samples similar to the second attacked data sample are respectively input into the embedding layer of the replacement model, and multiple intermediate vector features are output. The similarity loss of the multiple intermediate vector features and the perturbation vector is calculated, and the average loss is calculated. When the average loss converges to the set value, the training of the perturbation generation network is completed.

[0151] Randomly select a first attacked data sample from the attacked data sample set and input it into the perturbation generation network for perturbation processing to obtain the corresponding pseudo sample. Input the pseudo sample into the construction network and perform feature comparison between the pseudo sample and at least one first candidate data sample to obtain the candidate data sample with the highest feature similarity. Figure 8 As shown, while keeping the continuity features in the pseudo sample unchanged, the most similar feature is selected from the corresponding first candidate data sample for each discrete feature in the attacked data sample that does not exist in the original data set, and the most similar feature is used to replace the feature that does not exist in the discrete perturbation feature to obtain a constructed pseudo sample.

[0152] In the above-described embodiment, the encoder and decoder of the perturbation generation network are trained based on the substitution model. Using the intermediate perturbation vectors generated by the encoder and decoder as input data for the substitution model neural network, the encoder and decoder can be trained using a normal gradient update method to obtain a trained perturbation generation network. By determining at least one first candidate data sample corresponding to a first attacked data sample, the first attacked data sample is perturbed by the perturbation generation network to obtain a corresponding pseudo sample. By selecting the candidate data sample with the highest similarity to the pseudo sample from the at least one first candidate data sample, the perturbation features of the pseudo sample are updated, thereby improving the authenticity of the constructed pseudo sample.

[0153] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0154] Based on the same inventive concept, embodiments of the present application also provide a device for processing data against an attack based on a black box model, which is used to implement the aforementioned method for processing data against an attack based on a black box model. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the device for processing data against an attack based on a black box model provided below can be found in the limitations of the method for processing data against an attack based on a black box model, and will not be repeated here.

[0155] In one embodiment, Figure 9 As shown, a device for processing attack data based on a black box model is provided, comprising: a data determination module 902, a disturbance processing module 904, a feature comparison module 906, and a construction module 908, wherein:

[0156] The data determination module 902 is configured to determine a first attacked data sample and at least one first candidate data sample of the attacked sample data.

[0157] The disturbance processing module 904 is configured to perform disturbance processing on the first attacked data sample to obtain a corresponding pseudo sample.

[0158] The feature comparison module 906 is configured to perform feature comparison between the pseudo sample and at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity.

[0159] The construction module 908 is used to update the perturbation features of the pseudo sample according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample.

[0160] In the above embodiment, by determining at least one first candidate data sample corresponding to the first attacked data sample, the first attacked data sample is perturbed to obtain a corresponding pseudo sample; by selecting the candidate data sample with the highest similarity to the pseudo sample from the at least one first candidate data sample, the perturbation feature of the pseudo sample is updated, thereby improving the authenticity of the generated pseudo sample.

[0161] In another embodiment, a device for processing attack data based on a black box model is provided. In addition to a data determination module 902, a disturbance processing module 904, a feature comparison module 906, and a construction module 908, the device further includes an attack module, a network training module, and a data partitioning module, wherein:

[0162] The feature comparison module 906 is used to obtain the feature attributes of the pseudo sample; if the feature attributes are discrete features and all the perturbation features of the pseudo sample do not exist in at least one candidate data sample, then determine the perturbation features to be updated that do not exist in at least one first candidate data sample; determine the similarity between the sample features of at least one first candidate data sample and the perturbation features to be updated, and obtain the candidate data sample with the highest feature similarity based on the similarity.

[0163] The attack module is used to attack the target black box model based on the constructed pseudo sample to obtain the attack result; and determine the reliability of the constructed pseudo sample based on the attack result.

[0164] The disturbance processing module 904 is used to input the first attack data sample into the trained disturbance generation network and output the corresponding pseudo sample.

[0165] a network training module, configured to obtain a second attacked data sample and at least one second candidate data sample of the attacked data sample for training the disturbance generating network;

[0166] Input the second attacked data sample into the perturbation generation network, and output the perturbation vector of the second attacked data sample after passing through the embedding layer, encoder, and decoder of the perturbation generation network;

[0167] Input the perturbation vector into the network layer of the alternative model to obtain the prediction result;

[0168] Compare the prediction results with the corresponding candidate sub-data samples to obtain the contrast loss;

[0169] The encoder and decoder are trained based on the contrastive loss to obtain a trained perturbation generation network.

[0170] The network training module is further configured to input at least one second candidate data sample into the substitution model based on the contrastive loss and output an intermediate vector feature;

[0171] Perform similarity calculation on the perturbation vector and the intermediate vector features to obtain the average loss;

[0172] The encoder and decoder are trained based on the average loss until the average loss converges to a preset value, and a trained perturbation generation network is obtained.

[0173] Model replacement module, used to obtain the original dataset and the constructed alternative model;

[0174] Input the original data set into the surrogate model and the target black-box model respectively, and obtain the first output feature of the surrogate model and the second output feature of the target black-box model;

[0175] determining an output loss between the surrogate model and the target black-box model based on the first output feature and the second output feature;

[0176] determining a classification loss for the surrogate model based on the first output feature;

[0177] According to the classification loss and output loss, the total loss function of the substitute model is determined to obtain a substitute model with the same function as the target black box model.

[0178] The data partitioning module is used to obtain a training sample set; the training sample set is partitioned according to the attributes of each sample in the training sample set to obtain an attacked data sample set and a candidate data sample set;

[0179] Based on the recommendation algorithm, at least one second candidate data sample similar to each second attacked data sample set in the attacked data sample set is determined from the candidate data sample set.

[0180] Each module in the aforementioned black-box model-based attack data processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0181] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown. The computer device includes a processor, memory, a communication interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal via wired or wireless communication. The wireless communication can be achieved via Wi-Fi, a mobile cellular network, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a method for processing attack data based on a black-box model. The display screen of the computer device can be a liquid crystal display or an electronic ink display. The input device of the computer device can be a touch layer covering the display screen, keys, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse.

[0182] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0183] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0184] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0185] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0186] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0187] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0188] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0189] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for processing adversarial attack data based on a black box model, characterized in that: The method comprises: determining a first attacked data sample and at least one first candidate data sample of the first attacked data sample; Performing perturbation processing on the first attacked data sample to obtain a corresponding pseudo sample; Performing feature comparison between the pseudo sample and the at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity; The perturbation feature of the pseudo sample is updated according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample; The step of performing feature comparison between the pseudo sample and the at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity includes: obtaining feature attributes of the pseudo sample; If the feature attribute is a discrete feature, and all the perturbation features of the pseudo sample do not exist in the at least one candidate data sample, determining the perturbation features to be updated that do not exist in the at least one first candidate data sample; Determine the similarity between the sample feature of the at least one first candidate data sample and the disturbance feature to be updated, and obtain the candidate data sample with the highest feature similarity based on the similarity; The perturbation processing is performed on the first attacked data sample to obtain a corresponding pseudo sample, including: inputting the first attacked data sample into a trained perturbation generation network, and outputting a corresponding pseudo sample; The training of the disturbance generation network includes: Obtaining a second attacked data sample and at least one second candidate data sample of the second attacked data sample for training the perturbation generation network; inputting the second attacked data sample into the perturbation generation network, and outputting a perturbation vector of the second attacked data sample after passing through an embedding layer, an encoder, and a decoder of the perturbation generation network; Inputting the disturbance vector into the network layer of the substitution model to obtain a prediction result; Comparing the prediction result with the corresponding candidate sub-data sample to obtain the contrast loss; The encoder and decoder are trained based on the contrastive loss to obtain a trained perturbation generation network.

2. The method according to claim 1, characterized in that The method further comprises: Attacking the target black box model according to the constructed pseudo sample to obtain an attack result; The reliability of the constructed pseudo sample is determined according to the attack result.

3. The method according to claim 1, characterized in that The encoder and decoder are trained based on the contrastive loss to obtain a trained disturbance generation network, comprising: Based on the contrast loss, inputting the at least one second candidate data sample into a substitution model and outputting an intermediate vector feature; Performing similarity calculation on the perturbation vector and the intermediate vector features to obtain an average loss; The encoder and decoder are trained based on the average loss until the average loss converges to a preset value, thereby obtaining a trained perturbation generation network.

4. The method according to claim 1, wherein The surrogate model is determined based on the target black box model. The surrogate model is determined by: Obtain the original dataset and the constructed alternative model; Inputting the original data set into the surrogate model and the target black-box model respectively to obtain a first output feature of the surrogate model and a second output feature of the target black-box model; determining an output loss between the surrogate model and the target black-box model based on the first output feature and the second output feature; determining a classification loss of the surrogate model based on the first output feature; According to the classification loss and the output loss, a total loss function of the replacement model is determined to obtain a replacement model with the same function as the target black box model.

5. The method according to claim 1, wherein The obtaining of a second attacked data sample and at least one second candidate data sample for training the disturbance generating network comprises: Obtain a training sample set; Dividing the training sample set according to the attributes of each sample in the training sample set to obtain an attacked data sample set and a candidate data sample set; Based on a recommendation algorithm, at least one second candidate data sample similar to each second attacked data sample set in the attacked data sample set is determined from the candidate data sample set.

6. A device for processing attack data based on a black box model, characterized in that: The device comprises: a data determination module, configured to determine a first attacked data sample and at least one first candidate data sample of the first attacked data sample; a disturbance processing module, configured to perform disturbance processing on the first attacked data sample to obtain a corresponding pseudo sample; and further configured to input the first attacked data sample into a trained disturbance generation network to output a corresponding pseudo sample; a network training module, configured to obtain a second attacked data sample and at least one second candidate data sample for training the disturbance generating network; Inputting the second attacked data sample into the perturbation generation network, and outputting a perturbation vector of the second attacked data sample after passing through the embedding layer, encoder, and decoder of the perturbation generation network; Inputting the disturbance vector into the network layer of the substitution model to obtain a prediction result; Comparing the prediction result with the corresponding candidate sub-data sample to obtain the contrast loss; Training the encoder and decoder based on the contrastive loss to obtain a trained perturbation generation network; a feature comparison module, configured to perform feature comparison on the pseudo sample and the at least one first candidate data sample to obtain a candidate data sample with the highest feature similarity; and further configured to obtain a feature attribute of the pseudo sample; if the feature attribute is a discrete feature and all perturbation features of the pseudo sample do not exist in the at least one candidate data sample, determine a perturbation feature to be updated that does not exist in the at least one first candidate data sample; determine a similarity between a sample feature of the at least one first candidate data sample and the perturbation feature to be updated, and obtain a candidate data sample with the highest feature similarity based on the similarity; The construction module is used to update the perturbation feature of the pseudo sample according to the candidate data sample with the highest feature similarity to obtain a constructed pseudo sample.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images