A Cross-Domain Black-Box Model Reverse Attack Method

Abstract

The present invention discloses a cross - domain black - box model reverse attack method, which includes the following steps: obtaining the black - box access right of the target model to be attacked; obtaining the privacy image prediction vector and making a data set; obtaining auxiliary images and making a data set; establishing an adversarial domain alignment reverse attack model; using the established auxiliary image data set and the privacy image deep feature data set to train the adversarial domain alignment reverse attack model, updating the weight parameters in the network until the loss function of the network converges, and obtaining the trained adversarial domain alignment reverse attack model; inputting the collected privacy image prediction vector into the trained adversarial domain alignment reverse attack model to obtain the restored privacy training image. The present invention realizes high - performance black - box cross - domain model reverse attack in a scenario with domain differences, and can solve problems such as the need for white - box access rights in current cross - domain model reverse attacks and poor attack effects of ordinary model reverse attacks in scenarios with domain differences.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence security, and particularly relates to a cross-domain black-box model reverse attack method. Background Art

[0002] With the wide application of deep learning, the security problems existing in it have increasingly attracted people's attention. Model Inversion Attack (MIA) aims to recover the training data used by querying a given deep neural network model. Currently, model inversion attacks can be divided into two types according to the technical routes they adopt: search-based reverse methods and training-based reverse methods. The search-based reverse method obtains the private image by searching the original image space of the input or its corresponding latent space. The training-based reverse method uses auxiliary data to train a reverse model, and directly restores the depth features or prediction vectors corresponding to the private image into the private image using the reverse model.

[0003] According to the access rights that the attacker can obtain, model inversion attacks can be further divided into black-box model reverse attacks and white-box model reverse attacks. Under black-box access rights, the attacker can only obtain the query right of the target model. Specifically, the attacker can obtain the prediction vector of the private image and query the prediction vector of the given input image. Under white-box access rights, the attacker can further obtain the network structure, weight parameters, depth features of the intermediate layer output, etc. of the target model.

[0004] The literature Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, CCS’15, page 1322–1333, New York, NY, USA, Oct 2015. Association for Computing Machinery. first proposed the search-based inversion method. The authors recovered distinguishable face images from a simple fully-connected neural network by searching the original image space under black-box access. Subsequently, the literature Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song. The secret revealer: Generative model-inversion attacks against deep neural networks. In 2020 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 250–258, Jun 2020. and the literature Si Chen, Mostafa Kahla, Ruoxi Jia, and Guo-Jun Qi. Knowledge-enriched distributional model inversion attacks. In 2021 IEEE / CVF International Conference on Computer Vision (ICCV), page 16158–16167, 2021. etc. proposed to incorporate generative adversarial networks. By searching the latent space of the generative adversarial network, the search space of the search-based inversion method is reduced and the output images are made more natural.The literature "Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang. Neural network inversion in adversarial setting via background knowledge alignment. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS’19, page 225–240, New York, NY, USA, Nov 2019. Association for Computing Machinery." proposed a training-based inversion method. The authors successfully recovered the private training set from the prediction vectors under the black-box access permission by constructing an inversion model.

[0005] Although existing model inversion attack methods have shown good recovery effects, the vast majority of methods assume that the auxiliary dataset used by the attacker has the same distribution as the private dataset used to train the target network. Therefore, existing methods perform well when the auxiliary dataset and the private dataset have the same distribution (same domain), but perform very poorly or even cannot recover the images when the auxiliary dataset and the private dataset have different distributions (cross-domain), and are even less able to be achieved under the more restrictive black-box access permission. Summary of the Invention

[0006] The present invention precisely aims at the problems that existing model inversion attacks have poor effects when the auxiliary dataset and the private dataset are cross-domain and cannot achieve cross-domain black-box model inversion attacks, etc. It provides a cross-domain black-box model inversion attack method. First, obtain the black-box access permission of the target model to be attacked; obtain the prediction vectors of private images and make a dataset; obtain auxiliary images and make a dataset; establish an adversarial domain alignment inversion attack model; use the established auxiliary image dataset and the deep feature dataset of private images to train the adversarial domain alignment inversion attack model, update the weight parameters in the network until the loss function of the network converges, and obtain the trained adversarial domain alignment inversion attack model; input the collected prediction vectors of private images into the trained adversarial domain alignment inversion attack model to obtain the recovered private training images. The method of the present invention achieves high-performance cross-domain model inversion attacks in scenarios where the auxiliary dataset and the private dataset have domain differences and the attacker only has black-box access permission.

[0007] To achieve the above object, the technical solution adopted by the present invention is: A cross-domain black-box model inversion attack method, including the following steps:

[0008] S1. Obtain the black-box access right to the target network to be attacked: The target network to be attacked has been trained with private images and has an image classification function. The private images are kept confidential. The black-box access right is limited to obtaining the classification category information of the target network and querying the classification results of the network for the input images.

[0009] S2. Obtain the private image prediction vectors and make a data set: The private image prediction vectors are the classification probability results of the target network for the private images.

[0010] S3. Obtain the auxiliary images and make a data set: The auxiliary image data set and the private image data set used for training the target network have the same categories, but there are domain differences between them.

[0011] S4. Establish an adversarial domain alignment reverse attack model: The adversarial domain alignment reverse attack model includes an auxiliary classification module, two non-linear amplification layers, an image reconstruction module, and a domain discrimination module. The auxiliary classification module generates prediction vectors similar to the private image prediction vectors from the auxiliary image data set, thereby eliminating the domain differences between the auxiliary images and the private images in the prediction vector space. The two non-linear amplification layers are respectively located behind the auxiliary classification module and behind the target network, and can improve the diversity of the images of the same category in the reverse recovered images and eliminate the negative impact of overfitting of the auxiliary classification model and the target model on the reverse. The image reconstruction module is located behind the non-linear amplification layer of the auxiliary classification module and is used to recover the images. The domain discrimination module is also located behind the non-linear amplification layer of the auxiliary classification module and is used to discriminate the domain to which the deep features belong.

[0012] S5. Train the adversarial domain alignment reverse attack model: Use the established private image prediction vector data set and auxiliary image data set to train the adversarial domain alignment reverse attack model, and update the weight parameters in the network until the loss function of the network converges to obtain the trained adversarial domain alignment reverse attack model.

[0013] S6. Result output: Input the deep features of the collected private images into the trained adversarial domain alignment reverse attack model to obtain the recovered private images.

[0014] As a further technical solution of the present invention, the network structure of the target network to be attacked in step S1 and the network structure form of the auxiliary classification module in step S4 are not limited. It can be composed of one or more convolutional layers, batch normalization layers, pooling layers, activation layers, fully connected layers, and Dropout layers, or can be composed only of fully connected layers. The pooling layer can be any one of max pooling and average pooling. The activation function forms used in the activation layer include but are not limited to ReLU, tanh, etc.

[0015] Further, the non-linear amplification layer in step S4 includes a non-linear amplification function with variable parameters, and the definition of the non-linear amplification function is as follows:

[0016] Definition 1: Let f: [0, 1] → [0, 1] be a twice differentiable function. If f satisfies the following conditions, it is called a non-linear amplification function:

[0017] (1). x ≤ f(x)

[0018] (2). f(0) = 0, f(1) = 1

[0019] (3). If x < y, then f(x) < f(y), that is, f(x) is monotonically increasing

[0020] (4). f(x) is a convex-down function.

[0021] Further, training the adversarial domain alignment reverse attack model in step S5 includes four steps: (1) pre-training the auxiliary classification module, (2) training the non-linear amplification layer corresponding to the auxiliary classification module, (3) training the non-linear amplification layer corresponding to the target network, and (4) training the image reconstruction module and the domain discrimination module and fine-tuning the auxiliary classification module.

[0022] Further, the error function used in pre-training the auxiliary classification module in step S5 is:

[0023]

[0024] where x aux represents the auxiliary image sample, y aux represents the label corresponding to the auxiliary image sample, A represents the weight parameter of the auxiliary classification module, and CrossEntropy is the cross-entropy function.

[0025] Further, the error function used in training the non-linear amplification layer corresponding to the auxiliary classification module in step S5 is:

[0026]

[0027] where α aux is the variable parameter of the non-linear amplification function, Amp is the non-linear amplification function, p aux = one_hot(y aux ), y aux = A(x aux ) is the output prediction vector of the network, and CategoricalDiversity is defined as:

[0028]

[0029] where p = one_hot(y), m = arg max y, N is the number of categories, and σ is the variance.

[0030] Further, the error function used in the non - linear amplification layer corresponding to the training target network in step S5 is:

[0031]

[0032] where α priv is the variable parameter of the non - linear amplification function, p priv = one_hot(y priv ), y priv is the predicted vector corresponding to the private image, and the definition of CategoricalDiversity is the same as that used in the non - linear amplification layer corresponding to the training auxiliary classification module.

[0033] Further, the total error function used in step S5 for training the image reconstruction module, domain discriminant module, and fine - tuning the auxiliary classification module is:

[0034]

[0035] where D represents the weight parameter of the domain discriminant module, A represents the weight parameter of the feature alignment module, R represents the weight parameter of the image reconstruction module, E R is the image reconstruction error, and E D is the domain error.

[0036] Further, the image reconstruction error E R in step S5 is specifically:

[0037]

[0038] where x aux represents the auxiliary image samples, p aux is the distribution satisfied by the auxiliary data set, F represents the feature extractor of the target network to be attacked, LOSS is the full - reference image quality evaluation index, and the full - reference image quality evaluation index includes, but is not limited to, mean squared error (MSE), peak signal - to - noise ratio (PSNR), or structural similarity (SSIM).

[0039] Further, the domain error E D in step S5 is specifically:

[0040]

[0041] where y priv represents the predicted vector of the private image, p yprivThe distribution satisfied by the privacy image prediction vector.

[0042] Compared with the prior art, the present invention adopts the above technical solutions and has the following technical effects:

[0043] (1) The present invention proposes an adversarial domain alignment reverse attack model. Compared with the existing in-domain model reverse method based on training and the cross-domain model reverse method based on training, the present invention adds an auxiliary classification network and a non-linear amplification layer on this basis. The non-linear amplification layer proposed by the present invention can enhance the diversity of images of the same category in the restored images.

[0044] (2) The adversarial domain alignment reverse attack model proposed by the present invention can use adversarial learning to train the domain discriminant module and fine-tune the auxiliary classification module while learning to reconstruct the image, so that the auxiliary classification module can output a prediction vector similar to the target network, and then eliminate the domain difference between images of different domains in the prediction vector space, thereby realizing the cross-domain model reverse attack.

[0045] (3) The network structure of the auxiliary classification network in the adversarial domain alignment reverse attack model proposed by the present invention is independent of the target network structure, and uses a non-linear amplification function to eliminate the negative impact of model overfitting on reverse, and has stronger practicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a schematic structural diagram of the target network to be attacked by the method of the present invention;

[0047] Figure 2 is a schematic diagram of the auxiliary dataset images in step S3 of Embodiment 1 of the present invention;

[0048] Figure 3 is a schematic structural diagram of the adversarial domain alignment reverse attack model in step S4 of the present invention;

[0049] Figure 4 is a schematic structural diagram of an auxiliary classification module with three convolutional groups and two linear layers in the adversarial domain alignment reverse attack model in step S4 of the present invention;

[0050] Figure 5 is a schematic diagram of another auxiliary classification module structure with five convolutional groups and two linear layers in the adversarial domain alignment reverse attack model in step S4 of the present invention;

[0051] Figure 6 is a schematic structural diagram of an auxiliary classification module with two convolutional groups and three linear layers in the adversarial domain alignment reverse attack model in step S4 of the present invention;

[0052] Figure 7 is a schematic structural diagram of the domain discriminant module in the adversarial domain alignment reverse attack model in step S4 of the present invention;

[0053] Figure 8 It is a schematic structural diagram of the image reconstruction module in the adversarial domain alignment reverse attack model of step S4 of the present invention;

[0054] Figure 9 It is a training flow chart of the adversarial domain alignment reverse attack model in step S5 of the present invention;

[0055] Figure 10 It is the image of the privacy training data set in Embodiment 1 of the present invention;

[0056] Figure 11 It is the image obtained by using the adversarial domain alignment reverse attack model to perform model reverse attack recovery, where the auxiliary classification module is the network in Figure 4 ;

[0057] Figure 12 It is the image obtained by using the adversarial domain alignment reverse attack model to perform model reverse attack recovery, where the auxiliary classification module is the network in Figure 5 ;

[0058] Figure 13 It is the image obtained by using the adversarial domain alignment reverse attack model to perform model reverse attack recovery, where the auxiliary classification module is the network in Figure 6 ;

[0059] Figure 14 It is the image obtained by using the existing model reverse attack recovery. Specific implementation manners

[0060] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.

[0061] A cross-domain black-box model reverse attack method includes the following steps:

[0062] S1: Obtain the black-box access right to the target network to be attacked: The target network to be attacked has been trained with privacy images and has an image classification function, and the privacy images are kept confidential. The black-box access right is limited to obtaining the classification category information of the target network and querying the classification result of the network for the input image;

[0063] The network structure form of the target network to be attacked is not limited. It can be composed of one or more convolutional layers, batch normalization layers, pooling layers, activation layers, fully connected layers, and Dropout layers, or it can be composed only of fully connected layers. Among them, the pooling layer is any one of the maximum pooling and average pooling, and the activation function forms used in the activation layer include but are not limited to ReLU, tanh, etc. In this embodiment, the structure of the target network model to be attacked is specifically as shown in Figure 1 and consists of the following two parts:

[0064] (1) A feature extractor consisting of 3 convolutional groups is used to extract the deep features of an input image with a size of 32*32. Each convolutional group sequentially includes a convolutional layer with a size of 3*3, a batch normalization layer, a max pooling layer, and a ReLU activation layer. The number of channels of the three convolutional modules is 128, 256, and 512 in sequence.

[0065] (2) A classifier consisting of two fully connected layers is used to classify the extracted deep features. Among them, the input feature dimension is 8192, the hidden layer dimension is 50, and the output dimension is 10.

[0066] S2: Obtain the privacy image prediction vector and make a dataset: The privacy image prediction vector is the classification probability result of the target network for the privacy image.

[0067] S3: Obtain the auxiliary image and make a dataset: The auxiliary image dataset and the privacy image dataset used to train the target network have the same categories, but there are domain differences between them.

[0068] The auxiliary image dataset and the privacy image dataset contain samples and their corresponding labels. The label refers to the category of each sample. The domain difference can be one or more of the differences brought by the device for collecting images, the environmental illumination when collecting images, and the texture difference of the object surface in the image. The auxiliary image samples used in this embodiment are as Figure 2 shown, Figure 2 The samples in are white handwritten characters with a black background, with a relatively large font size and blurred outlines.

[0069] S4: Establish an adversarial domain alignment reverse attack model: As Figure 3 shown, the adversarial domain alignment reverse attack model includes an auxiliary classification module, two non-linear amplification layers, an image reconstruction module, and a domain discrimination module.

[0070] The auxiliary classification module generates a prediction vector similar to the privacy image prediction vector from the auxiliary image dataset, thereby eliminating the domain difference between the auxiliary image and the privacy image in the prediction vector space. The network structure form of the auxiliary classification module is not limited. It can be composed of one or more convolutional layers, batch normalization layers, pooling layers, activation layers, fully connected layers, and Dropout layers, or it can be composed only of fully connected layers. Among them, the pooling layer is any one of max pooling and average pooling. The activation function forms used in the activation layer include but are not limited to ReLU, tanh, etc. Specifically, as shown in 4- Figure 6 shown, Figure 4 The auxiliary classification module shown is composed of the following two parts:

[0071] (1) A feature extractor consisting of 3 convolutional groups, which is used to extract the deep features of an input image with a size of 32 * 32. Each convolutional group sequentially includes a convolutional layer with a size of 3 * 3, a batch normalization layer, a max pooling layer, and a ReLU activation layer. The number of channels of the three convolutional modules is 128, 256, and 512 in sequence.

[0072] (2) A classifier consisting of two fully connected layers, which is used to classify the extracted deep features. Among them, the input feature dimension is 8192, the hidden layer dimension is 50, and the output dimension is 10.

[0073] Figure 5 The auxiliary classification module shown consists of the following two parts:

[0074] (1) A feature extractor consisting of 5 convolutional groups, which is used to extract the deep features of an input image with a size of 32 * 32. Among the first three convolutional groups, each group sequentially includes a convolutional layer with a size of 3 * 3, a batch normalization layer, a max pooling layer, and a ReLU activation layer. The number of channels of the three convolutional modules is 128, 256, and 512 in sequence. In the latter two convolutional groups, the first convolutional group sequentially includes a convolutional layer with a size of 3 * 3, the number of channels is 1024, a batch normalization layer, and a ReLU activation layer. The second convolutional group sequentially includes a convolutional layer with a size of 2 * 2, the number of channels is 2048, a batch normalization layer, and a ReLU activation layer.

[0075] (2) A classifier consisting of two fully connected layers, which is used to classify the extracted deep features. Among them, the input feature dimension is 2048, the hidden layer dimension is 250, and the output dimension is 10.

[0076] Figure 6 The auxiliary classification module shown consists of the following two parts:

[0077] (1) A feature extractor consisting of 2 convolutional groups, which is used to extract the deep features of an input image with a size of 32 * 32. Each convolutional group sequentially includes a convolutional layer with a size of 5 * 5, a ReLU activation layer, and a max pooling layer. The number of channels of the three convolutional modules is 6, 16 in sequence.

[0078] (2) A classifier consisting of three fully connected layers, which is used to classify the extracted deep features. Among them, the input feature dimension is 400, the hidden layer dimensions are 120 and 84 in sequence, and the output dimension is 10.

[0079] Two non-linear amplification layers are respectively located behind the auxiliary classification module and behind the target network, which can improve the diversity of images of the same category in the reverse recovery image and eliminate the negative impact of model overfitting on the reverse. The non-linear amplification layer contains a non-linear amplification function with variable parameters, and the parameters can adjust the non-linearity degree of the non-linear amplification function. The definition of the non-linear amplification function is as follows:

[0080] Definition 1: Let f:[0,1]→[0,1] be a twice differentiable function. If f satisfies the following conditions, it is called a non-linear amplification function:

[0081] (1). x ≤ f(x)

[0082] (2). f(0) = 0, f(1) = 1

[0083] (3). If x < y, then f(x) < f(y), that is, f(x) is monotonically increasing

[0084] (4). f(x) is a convex function

[0085] The non-linear amplification function used in the adversarial domain alignment reverse attack model in this embodiment is y = x α , where α ∈ (0,1) is an adjustable parameter.

[0086] The image reconstruction module is located behind the non-linear amplification layer of the auxiliary classification module and is used to restore the image. The image reconstruction module includes multiple deconvolution layers, batch normalization layers, activation layers and a sigmoid output layer. The forms of activation functions used in the activation layers include but are not limited to ReLU, tanh, etc. The structure of the image reconstruction module of the adversarial domain alignment reverse attack model in this embodiment is as Figure 8 shown, which consists of four deconvolution groups and a sigmoid layer. Among them, the first three deconvolution groups each contain a 4*4 deconvolution layer, a batch normalization layer and a ReLU activation layer in sequence. The fourth deconvolution group only contains a 4*4 deconvolution layer. The number of channels of the deconvolution groups is 512, 256, 128 in sequence.

[0087] The domain discrimination module is also located behind the non-linear amplification layer of the auxiliary classification module and is used to discriminate the domain to which the depth features belong. The domain discrimination module includes a gradient reversal layer, multiple fully connected layers and activation layers. The forms of activation functions used in the activation layers include but are not limited to ReLU, tanh, etc. The gradient reversal layer can set the output gradient value to the opposite of the input gradient value during model training. The structure of the domain discrimination module of the adversarial domain alignment reverse attack model in this embodiment is as Figure 7As shown, it consists of a gradient reversal layer, three fully connected layers, and a Sigmoid layer. There is a ReLU activation layer and a Dropout layer between every two of the three fully connected layers. The input feature dimension of the feature alignment domain discriminant module is 50, the output feature dimension is 1, and the intermediate hidden layer dimensions are all 1024.

[0088] S5: Train the adversarial domain alignment reverse attack model: Use the established privacy image deep feature dataset and the auxiliary image dataset to train the adversarial domain alignment reverse attack model, and update the weight parameters in the network until the loss function of the network converges to obtain the trained adversarial domain alignment reverse attack model. Training the adversarial domain alignment reverse attack model includes four steps: (1) Pre-train the auxiliary classification module, (2) Train the corresponding non-linear amplification layer of the auxiliary classification module, (3) Train the corresponding non-linear amplification layer of the target network, (4) Train the image reconstruction module and the domain discriminant module and fine-tune the auxiliary classification module.

[0089] The error function used for pre-training the auxiliary classification module is:

[0090]

[0091] where x aux represents the auxiliary image sample, y aux represents the label corresponding to the auxiliary image sample, A represents the weight parameter of the auxiliary classification module, and CrossEntropy is the cross-entropy function.

[0092] The error function used for training the corresponding non-linear amplification layer of the auxiliary classification module is:

[0093]

[0094] where α aux is the variable parameter of the non-linear amplification function, Amp is the non-linear amplification function, p aux = one_hot(y aux ), y aux = A(x aux ) is the output prediction vector of the network, and CategoricalDiversity is defined as:

[0095]

[0096] where p = one_hot(y), m = arg max y, N is the number of classes, and σ is the variance.

[0097] The error function used for training the corresponding non-linear amplification layer of the target network is:

[0098]

[0099] Among them, α priv is the variable parameter of the non-linear amplification function, p priv = one_hot(y priv ), where y priv is the prediction vector corresponding to the privacy image. The definition of CategoricalDiversity is the same as that used in the non-linear amplification layer corresponding to the training auxiliary classification module.

[0100] The error function used to train the image reconstruction module, the domain discrimination module, and fine-tune the auxiliary classification module is:

[0101]

[0102] Among them, D represents the weight parameter of the domain discrimination module, A represents the weight parameter of the feature alignment module, R represents the weight parameter of the image reconstruction module, and E R is the image reconstruction error, and E D is the domain error.

[0103] The image reconstruction error E R Specifically:

[0104]

[0105] Among them, x aux represents the auxiliary image samples, p aux is the distribution satisfied by the auxiliary data set, F represents the feature extractor of the target network to be attacked, LOSS is the full-reference image quality evaluation index, and the full-reference image quality evaluation index includes but is not limited to mean square error (MSE), peak signal-to-noise ratio (PSNR), or structural similarity (SSIM).

[0106] The domain error E D Specifically:

[0107]

[0108] Among them, y priv represents the privacy image prediction vector, and p ypriv is the distribution satisfied by the privacy image prediction vector.

[0109] Referring to Figure 9 , the specific steps to train the adversarial domain alignment reverse attack model are:

[0110] S51: Use image enhancement techniques to expand the established auxiliary image data set. The image enhancement techniques include geometric transformation of the image, illumination transformation of the image, or changing the image style using an artificial neural network, and generate any one or several of the new images;

[0111] S52: Train the auxiliary classification network using the augmented auxiliary image dataset in step S51 to obtain a pre-trained auxiliary classification network;

[0112] S53: Input the augmented auxiliary image dataset in step S51 into the auxiliary classification network to obtain corresponding prediction vectors, and use the prediction vectors generated by the auxiliary images to train the corresponding non-linear amplification layer of the auxiliary classification network;

[0113] S54: Use the prediction vectors corresponding to the privacy images to train the corresponding non-linear amplification layer of the target network;

[0114] S55: Input the prediction vectors corresponding to the amplified auxiliary images and the prediction vectors corresponding to the privacy images into the domain discrimination module together, and input the prediction vectors corresponding to the amplified auxiliary images into the image reconstruction module

[0115] S54: The domain discrimination module calculates the domain error E based on the input prediction vectors corresponding to the amplified auxiliary images and the prediction vectors corresponding to the privacy images D

[0116] S55: The image reconstruction module calculates the image reconstruction error E based on the input prediction vectors corresponding to the auxiliary images and in combination with the auxiliary images R

[0117] In this embodiment, the structural similarity (SSIM) is used as the LOSS in the image reconstruction error E R in the LOSS.

[0118] S56: The network completes training by minimizing the total error function.

[0119] S6: Result output: Amplify the prediction vectors of the collected privacy images using the non-linear amplification layer of the trained target network, and input them into the image reconstruction module to obtain the restored privacy images.

[0120] Figure 10 The samples in are white handwritten digits on a black background, with clear font outlines and relatively centered. Compared with the Figure 2 privacy images shown, Figure 10 the auxiliary image samples shown have a large domain difference. In this embodiment, the prediction vectors obtained by the target network to be attacked according to the Figure 10 privacy images shown are input into the adversarial domain alignment reverse attack model with three different auxiliary classification networks, and the reverse results are as shown in Figure 11 , Figure 12 , Figure 13 shown. As can be seen from the results in Figures 11 - 13 , the restored images are all clearly visible and each sample is different.Figure 2 The semantic information of the private image is well restored, and the restored samples retain Figure 10 the texture features of the auxiliary image. Compared with Figure 14 the images restored by the reverse attack of the existing models in [ ], the adversarial domain alignment reverse attack model better realizes the cross-domain model reverse attack.

[0121] Combined with the above method process, the core idea of the adversarial domain alignment reverse attack model involved in the present invention is to align the deep features of the private image and the deep features of the auxiliary image through adversarial learning, so as to eliminate the domain difference between the two. In addition, the model completes the reverse task while aligning the features, achieving the purpose of cross-domain model reverse. Once trained, the adversarial domain alignment reverse attack model involved in the present invention can restore the private image with a different distribution from the auxiliary image data from the deep features of the private image. Generally speaking, the present invention realizes high-performance cross-domain model reverse attack in the scenario with domain differences, and can solve the problems that the current model reverse attack is only applicable to the same distribution of auxiliary data and private data and cannot achieve cross-domain attack.

[0122] It should be noted that the above content only illustrates the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. For those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and retouches can be made, and these improvements and retouches all fall within the protection scope of the claims of the present invention.

Claims

1. A cross-domain black-box model reverse attack method, characterized in that : It includes the following steps: S1. Obtain the black-box access right to the target network to be attacked: The target network to be attacked has been trained with privacy images and has an image classification function, and the privacy images are kept confidential; the black-box access right is limited to obtaining the classification category information of the target network and the classification results of the query network for the input images; S2. Obtain the privacy image prediction vectors and make a data set: The privacy image prediction vectors are the classification probability results of the target network for the privacy images; S3. Obtain the auxiliary images and make a data set: The auxiliary image data set and the privacy image data set used for training the target network have the same categories, but there are domain differences between them; S4. Establish an adversarial domain alignment reverse attack model: The adversarial domain alignment reverse attack model includes an auxiliary classification module, two non-linear amplification layers, an image reconstruction module, and a domain discrimination module. The auxiliary classification module generates prediction vectors similar to the privacy image prediction vectors from the auxiliary image data set, thereby eliminating the domain differences between the auxiliary images and the privacy images in the space of the prediction vectors; The two non-linear amplification layers are respectively located behind the auxiliary classification module and behind the target network, and can enhance the diversity of images of the same category in the reverse-recovered images and eliminate the negative impacts of overfitting of the auxiliary classification model and the target model on the reverse; The image reconstruction module is located behind the non-linear amplification layer of the auxiliary classification module and is used to recover images; The domain discrimination module is located behind the non-linear amplification layer of the auxiliary classification module and is used to discriminate the domain to which the deep features belong; S5. Train the adversarial domain alignment reverse attack model: Use the privacy image prediction vector data set established in step S2 and the auxiliary image data set obtained in step S3 to train the adversarial domain alignment reverse attack model in step S4, and update the weight parameters in the network until the loss function of the network converges to obtain the trained adversarial domain alignment reverse attack model; S6. Result output: Input the deep features of the collected privacy images into the trained adversarial domain alignment reverse attack model to obtain the recovered privacy images.

2. The cross-domain black-box model reverse attack method according to claim 1, wherein : The network structure of the target network to be attacked in step S1 and the network structure form of the auxiliary classification module in step S4 are not limited, and are composed of one or more convolutional layers, batch normalization layers, pooling layers, activation layers, fully connected layers, and Dropout layers, or only composed of fully connected layers; the pooling layer is any one of max pooling and average pooling; the activation function forms used in the activation layer include ReLU and tanh.

3. The cross-domain black-box model reverse attack method according to claim 2, wherein : The non-linear amplification layer in step S4 includes a non-linear amplification function with variable parameters, and is specifically defined as: Definition 1: Let f: [0, 1] → [0, 1] be a second-order differentiable function. If f satisfies the following conditions, it is called a non-linear amplification function: (1). x ≤ f(x) (2). f(0) = 0, f(1) = 1 (3). If x < y, then f(x) < f(y), that is, f(x) is monotonically increasing (4). f(x) is a convex function.

4. A cross-domain black-box model reverse attack method according to claim 3, characterized in that : The training of the adversarial domain alignment reverse attack model in step S5 includes four steps: (1) Pre-training the auxiliary classification module; (2) Training the corresponding non-linear amplification layer of the auxiliary classification module; (3) Training the corresponding non-linear amplification layer of the target network; (4) Training the image reconstruction module and the domain discriminator module and fine-tuning the auxiliary classification module.

5. A cross-domain black-box model reverse attack method according to claim 4, characterized in that : The error function used in pre-training the auxiliary classification module in step S5 is: Among them, x aux represents the auxiliary digital image sample, y aux represents the label corresponding to the auxiliary digital image sample, A represents the weight parameter of the auxiliary classification module, and CrossEntropy is the cross-entropy function.

6. A cross-domain black-box model reverse attack method according to claim 4, characterized in that : The error function used in training the corresponding non-linear amplification layer of the auxiliary classification module in step S5 is: Among them, α aux is a variable parameter of the non-linear amplification function, Amp is the non-linear amplification function, p aux = one_hot(y aux ), y aux = A(x aux ) is the output prediction vector of the network, and CategoricalDiversity is defined as: Where p = one_hot(y), m = arg max y, N is the number of classes, and σ is the variance.

7. A cross-domain black-box model reverse attack method according to claim 4, characterized in that : The error function used in training the corresponding non-linear amplification layer of the target network in step S5 is: Among them, α priv is a variable parameter of the non-linear amplification function, p priv = one_hot(y priv ), where y priv is the prediction vector corresponding to the privacy image, and the definition of CategoricalDiversity is the same as that used in the non-linear amplification layer corresponding to the training auxiliary classification module.

8. A cross-domain black-box model reverse attack method according to claim 4, characterized in that : The error function used in training the image reconstruction module and the domain discriminator module and fine-tuning the auxiliary classification module in step S5 is: Among them, D represents the weight parameter of the domain discrimination module, A represents the weight parameter of the feature alignment module, R represents the weight parameter of the image reconstruction module, and E R is the image reconstruction error, and E D is the domain error.

9. The cross-domain black-box model reverse attack method according to claim 8, characterized in that : The image reconstruction error E in the step S5 R Specifically where x aux represents the auxiliary data image sample, p aux is the distribution satisfied by the auxiliary data set, F represents the feature extractor of the target network to be attacked, LOSS is the full-reference image quality evaluation metric, and the full-reference image quality evaluation metric includes mean squared error MSE, peak signal-to-noise ratio PSNR, or structural similarity SSIM.

10. A cross-domain black-box model reverse attack method according to claim 8 or 9, characterized in that : The domain error E in the step S5 D Specifically: Among them, y priv represents the privacy image prediction vector, and p ypriv is the distribution satisfied by the privacy image prediction vector.

Images