Image self-supervised training method based on adversarial attack

Abstract

The invention discloses an image self-supervised training method based on adversarial attacks, and belongs to the field of computer vision. According to the method and the device, the problems of poor learning ability of a semantic information mode and poor generalization ability of the MIM due to the fact that key semantic information of a traditional MIM task is lost are solved. The method comprises the following steps: firstly, performing mask processing on a non-semantic information part in an image, especially in an area with the most significant change before and after attack, and performing random mask processing on other parts; through the mask mode, a model reconstruction task aiming at the semantic information part can be generated on the basis of keeping the semantic information to the greatest extent, so that the understanding ability of the model on the semantic information is enhanced. In the subsequent downstream task fine tuning stage, through confrontation training of pixel-level PGD attacks, the perception ability of the model to semantic information is deepened, and the generalization ability, the anti-interference robustness and the semantic information extraction ability of the model are significantly improved. The method can be applied to image self-supervision training.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to an image self-supervision training method based on adversarial attacks. Background Art

[0002] In order to improve the performance of the model in a wide range of downstream tasks and reduce the overhead of repeated model training, the Masked Image Modeling (MIM) framework has received widespread attention. Inspired by the great success of the BERT method based on the Masked Language Model (MLM) framework in natural language understanding, many works in recent years have applied similar frameworks to the image field to form the MIM framework. By fine-tuning to downstream tasks, the pre-trained model based on the MIM framework has achieved state-of-the-art performance in image detection, semantic segmentation, and image classification tasks. MIM draws on the core idea of ​​MLM: randomly masking some areas in the image, and reconstructing the masked parts from the unmasked parts to promote the model's understanding of the image semantic information and learn more transferable features. The high-level features learned by this type of method through MIM pre-training can be widely transferred to fields such as medicine, e-commerce, and commodities, thereby improving the performance of the model in various field-specific tasks (such as label classification tasks), and has a wide range of application value.

[0003] However, in traditional MIM tasks, the masked parts of the image are selected through random uniform sampling. This method will lead to the loss of key semantic information, thereby limiting MIM's learning of semantic information patterns and its generalization ability. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the key semantic information of the traditional MIM task is missing, which leads to poor learning ability of the semantic information pattern and poor generalization ability of MIM, and proposes an image self-supervised training method based on adversarial attack.

[0005] The technical solution adopted by the present invention to solve the above technical problems is: a method for image self-supervision training based on adversarial attack, the method specifically comprising the following steps:

[0006] Step 1: Process the acquired source domain image dataset based on the embedding-level PGD adversarial attack to obtain the processed image;

[0007] Step 2: Use the processed image to pre-train the mask image model until the loss converges and then stop the pre-training to obtain a pre-trained mask image model;

[0008] Step 3: Fine-tune the parameters of the pre-trained mask image model based on the pixel-level PGD adversarial attack and the labeled target domain image dataset to complete the training process.

[0009] Furthermore, the mask image model is a mask autoencoder.

[0010] Furthermore, the specific process of step one is:

[0011] Step 1: For any image X in the acquired source domain image dataset, divide the image X into N′ non-overlapping image patches, N′=H×W / P 2 , P represents the size of each image patch, H and W represent the height and width of image X respectively;

[0012] The i-th image patch obtained after division is denoted as X i , i=1,2,3;

[0013] Step 1 and 2: Initialize the number of iterations l = 0 and initialize the adversarial perturbation

[0014] Step 13: uniformly randomly mask all image patches in image X, with a mask ratio of γ, i.e., mask γ·N′ image patches;

[0015] Step 14: All image patches of image X are used as input to the masked autoencoder, and in the masked autoencoder, there is a unique corresponding embedding layer for each image patch in image X;

[0016] If the image patch X i For an unmasked patch, give the image patch X i The corresponding embedding layer adds an embedding-level PGD adversarial attack, that is, to the image patch X i The corresponding embedding layer adds adversarial perturbations If the image patch X i For mask patch, then give image patch X i The embedding-level PGD adversarial attack added by the corresponding embedding layer is 0;

[0017] And the image patch X i The corresponding embedding layer output is recorded as

[0018] Step 15: Determine whether the stop condition of the iteration is met;

[0019] If satisfied, then get the image patch X i The corresponding final embedding layer f embed,i , then execute step 16;

[0020] If not satisfied, set l=l+1 and return to step 1-3;

[0021] Step 16: Pass all image patches in image X through the corresponding final embedding layer respectively, according to and f embed,i (X i ) After selecting some mask patches from the image X, randomly select some image patches from the remaining image patches of the image X for masking to obtain a processed image corresponding to the image X;

[0022] Step 17: Perform steps 11 to 16 on each image in the source domain image dataset to obtain a processed image dataset corresponding to the source domain image dataset.

[0023] Furthermore, the adversarial perturbation for:

[0024]

[0025] Among them, α is the update step size, is the adversarial perturbation after the lth iteration update, is the image patch X i The corresponding non-zero adversarial perturbation added to the last embedding layer, is the mean square error reconstruction loss of the lth iteration, represents the mean square error reconstruction loss The gradient of , sign(·) represents the sign function, and M is the set of all mask image patches selected at the lth iteration.

[0026] Furthermore, the stopping condition of the iteration is to satisfy the following method (1) or method (2);

[0027] Method (1), reaching the set maximum number of iterations;

[0028] Method (2), any image slice satisfies is greater than the threshold ∈; where, yes The p-norm of .

[0029] Furthermore, in step 16, all image patches in image X are respectively passed through the corresponding final embedding layer, according to and f embed,i (X i ) After selecting some mask patches from image X, randomly select some image patches from the remaining image patches of image X for masking to obtain the processed image corresponding to image X; the specific process is:

[0030] Step 1: Calculate the corresponding Among them, |·| means calculating the Euclidean distance, Represents X i After the output of the initial embedding layer, f embed,i (X i ) indicates X i Output after the final embedding layer;

[0031] Step 2: According to Select a fixed ratio of image patches from all image patches of image X;

[0032] Then randomly select a fixed ratio of image patches from the remaining image patches;

[0033] Step 3: Perform a mask operation on the image patch selected in step 2 in image X to obtain a masked image, that is, obtain a processed image corresponding to image X.

[0034] Furthermore, the specific process of step 2 is as follows:

[0035] For each image patch of image X, Sort in descending order and select the α in front attack The image patch corresponding to N′ values, where α attack is the proportion of the mask part selected based on the embedding attack;

[0036] Then randomly select (γ-α) from the remaining image patches attack )·N′ image patches, γ is the ratio of all masked parts.

[0037] Furthermore, the α attack The value of is 0.4.

[0038] Furthermore, the loss in step 2 is the sum of the mean square error reconstruction losses of each image in the training set. The mean square error reconstruction loss of image X is specifically:

[0039]

[0040] in, is the mean squared error reconstruction loss of image X, M is the set of mask image patches on image X, and X k is the kth mask image patch in the set M, Yes X k The reconstruction result of , ‖·‖ represents the 2-norm.

[0041] Furthermore, the parameters of the pre-trained mask image model are fine-tuned based on the pixel-level PGD adversarial attack and the target domain image dataset, specifically:

[0042]

[0043] Among them, x j is the jth image in the target domain image dataset of the downstream task, δ j It is for x j Added pixel-level PGD adversarial attack, f θ (x j +δ j ) is x j +δ j The output after the mask image model, θ is the fine-tuned mask image model parameter, is the cross entropy loss function, N is the total number of images contained in the target domain image dataset of the downstream task, and y j is x j .

[0044] The beneficial effects of the present invention are:

[0045] The present invention first masks the non-semantic information part of the image, especially the area with the most significant changes before and after the attack, and randomly masks the rest. Through this masking method, a model reconstruction task for the semantic information part can be generated on the basis of retaining the semantic information to the greatest extent, thereby enhancing the model's ability to understand semantic information. In the subsequent downstream task fine-tuning stage, adversarial training of pixel-level PGD attacks is used to further deepen the model's perception of semantic information and improve its robustness. The image self-supervised training method based on adversarial attacks of the present invention significantly improves the model's generalization ability, anti-interference robustness and semantic information extraction ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Diagram of the framework for self-supervised pre-training of masked autoencoders combined with embedding-level adversarial attack masking strategy;

[0047] Figure 2 Visual analysis of adversarial examples generated for embedding-level adversarial attacks and pixel-level adversarial attacks;

[0048] Figure 3 This is the algorithm flow chart of the mask strategy for embedding-level adversarial attack;

[0049] In the figure, Π ∈ Represents the limit ‖δ‖ p ≤∈;

[0050] Figure 4 This is a diagram of the pixel-level adversarial fine-tuning training framework after the MIM self-supervised pre-training phase is completed;

[0051] Figure 5 This is the image attention area analysis diagram of the obtained model after training is completed. DETAILED DESCRIPTION

[0052] Specific implementation method 1: This implementation method is a method for self-supervised training of images based on adversarial attack, and the method specifically includes the following steps:

[0053] Step 1: Process the acquired source domain image dataset based on the embedding-level PGD (Projected Gradient Descent) counterattack to obtain the processed image;

[0054] Step 2: Use the processed image to pre-train the mask image model (here the initial mask image model is trained, not the mask image model after the embedding layer adds perturbations) until the loss converges and stop the pre-training to obtain the pre-trained mask image model;

[0055] Step 3: Fine-tune the parameters of the pre-trained mask image model based on the pixel-level PGD adversarial attack and the labeled target domain image dataset to complete the training process.

[0056] The present invention first adopts the masked autoencoder under the MIM framework as a self-supervised training method, which can promote the model to fully understand the image. Secondly, based on the projected gradient descent method (Projected Gradient Descent, PGD), the embedding level is attacked adversarially, and the key areas in the image are highlighted by the adversarial attack method. In order to improve the generalization ability, robustness against interference, and semantic information extraction ability of the training model, a masked autoencoder and an embedded-level PGD adversarial attack are combined to propose an adversarial attack masking method, which masks the key semantic information area by masking the part with the largest difference before and after the attack, and combines random masks to construct MIM tasks of key areas and random area masks. Finally, pixel-level PGD adversarial training is used to fine-tune downstream tasks. The adversarial attack self-supervised training strategy proposed in the present invention can enhance the model's perception of semantic information and high-frequency features, and improve the generalization and robustness of the model.

[0057] Specific implementation method 2: Figure 1 As shown, the difference between this embodiment and the first embodiment is that the mask image model is a masked autoencoder (MAE).

[0058] The other steps and parameters are the same as those in the first embodiment.

[0059] The mask image model in the present invention may be a mask autoencoder, but is not limited to a mask autoencoder. Figure 1 As shown, the masked autoencoder includes an embedding layer, an encoder, and a decoder.

[0060] The masked autoencoder can reconstruct the masked image patches based on the unmasked image patches in the image. The reconstruction process requires the model to understand the underlying semantics and model the relationship between the unmasked image patches and the masked image patches, thereby facilitating the learning of meaningful representations. Given an image Among them, C, H and W represent the number of image channels, image height and image width respectively. The image X′ is divided into N = H × W / P 2 non-overlapping patches, each of size P. In this way, the image X′ is converted into a series of image patches X′={x′ 1 ,...,x′ N}, Then, the image patch index M is selected according to certain rules, and the image patches of image X′ are divided into a mask patch set X′ m ={x′ k |k∈M} and the set of visible patches During the training process, the encoder is v As input, to obtain the potential representation H v Next, the decoder attempts to embed the interpolated [mask] token into the latent representation H v In the sequence, get the image at the position corresponding to the interpolated [mask] token and output the reconstructed image patch.

[0061] Specific implementation method three: Figure 3 As shown, this embodiment is different from the specific embodiment 1 or 2 in that the specific process of step 1 is:

[0062] Step 1: For any image X in the acquired source domain image dataset, divide the image X into N′ non-overlapping image patches, N′=H×W / P 2 , P represents the size of each image patch, H and W represent the height and width of image X respectively;

[0063] The i-th image patch obtained after division is denoted as X i , i=1,2,3;

[0064] Step 1 and 2: Initialize the number of iterations l = 0 and initialize the adversarial perturbation

[0065] Step 13: uniformly randomly mask all image patches in image X, with a mask ratio of γ, i.e., mask γ·N′ image patches;

[0066] Step 14: All image patches of image X are used as input to the masked autoencoder, and in the masked autoencoder, for each image patch in image X, there is a unique corresponding embedding layer (i.e., the image patch and the embedding layer are in a one-to-one correspondence);

[0067] If the image patch X i For an unmasked patch, give the image patch X i The corresponding embedding layer adds an embedding-level PGD adversarial attack, that is, to the image patch X i The corresponding embedding layer adds adversarial perturbations If the image patch X i For mask patch, then give image patch X i The embedding-level PGD adversarial attack added by the corresponding embedding layer is 0;

[0068] And the image patch X i The corresponding embedding layer output is recorded as Right now Yes X i The corresponding embedding layer after adding adversarial perturbations for the l-1th time;

[0069] Step 15: Determine whether the stop condition of the iteration is met;

[0070] If satisfied, then get the image patch X i The corresponding final embedding layer f embed,i , then execute step 16;

[0071] If not satisfied, set l=l+1 and return to step 1-3;

[0072] Step 16: Pass all image patches in image X through the corresponding final embedding layer respectively, according to and f embed,i (X i ) After selecting some mask patches from the image X, randomly select some image patches from the remaining image patches of the image X for masking to obtain a processed image corresponding to the image X;

[0073] Step 17: Perform steps 11 to 16 on each image in the source domain image dataset (it should be noted that each image is processed based on the original model) to obtain a processed image dataset corresponding to the source domain image dataset.

[0074] The other steps and parameters are the same as those in the first or second embodiment.

[0075] Specific implementation method 4: This implementation method is different from any one of the specific implementation methods 1 to 3 in that the adversarial disturbance for:

[0076]

[0077] Among them, α is the update step size, is the adversarial perturbation after the lth iteration update, is the image patch X i The corresponding non-zero adversarial perturbation added to the last embedding layer, is the mean square error reconstruction loss of the lth iteration, represents the mean square error reconstruction loss The gradient of , sign(·) represents the sign function, and M is the set of all mask image patches selected at the lth iteration.

[0078] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0079] Because for any embedding layer, the embedding layer is not updated in every iteration. Therefore, it is necessary to calculate the adversarial perturbation to be added in the current iteration based on the non-zero adversarial perturbation added to the current embedding layer last time.

[0080] Specific implementation method 5: This implementation method is different from any one of specific implementation methods 1 to 4 in that the stopping condition of the iteration is to satisfy the following method (1) or method (2);

[0081] Method (1), reaching the set maximum number of iterations;

[0082] Method (2), any image slice satisfies is greater than the threshold ∈; where, yes The p-norm of , the threshold ∈ can be set according to the actual situation.

[0083] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0084] If only method (1) is satisfied when the iteration stops, the embedding layer obtained in the last iteration is taken as the final embedding layer; if only method (2) is satisfied when the iteration stops, the embedding layer obtained in the second-to-last iteration is taken as the final embedding layer; if both method (1) and method (2) are satisfied when the iteration stops, the embedding layer obtained in the second-to-last iteration is taken as the final embedding layer.

[0085] Specific implementation method 6: This implementation method is different from the specific implementation methods 1 to 5 in that in step 16, all image patches in the image X are respectively passed through the corresponding final embedding layer, according to and f embed,i (X i) After selecting some mask patches from image X, randomly select some image patches from the remaining image patches of image X for masking to obtain the processed image corresponding to image X; the specific process is:

[0086] Step 1: Calculate the corresponding Among them, |·| means calculating the Euclidean distance, Represents X i After the output of the initial embedding layer, f embed,i (X i ) indicates X i Output after the final embedding layer;

[0087] Step 2: According to Select a fixed ratio of image patches from all image patches of image X to obtain an image patch set M attack ;

[0088] Then randomly select a fixed ratio of image patches from the remaining image patches to obtain the image patch set M random ;

[0089] Step 3: Perform a mask operation on the image patch selected in step 2 in image X to obtain a masked image, that is, obtain a processed image corresponding to image X.

[0090] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0091] Specific implementation method 7: This implementation method is different from the specific implementation methods 1 to 6 in that the specific process of step 2 is as follows:

[0092] For each image patch of image X, Sort in descending order and select the α in front attack The image patch corresponding to N′ values, where α attack is the proportion of the mask part selected based on the embedding attack;

[0093] Then randomly select (γ-α) from the remaining image patches attack )·N′ image patches, γ is the ratio of all masked parts.

[0094] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0095] Compared with the naive randomly sampled mask patch index set M, the present invention adopts embedding-level PGD as a guide to design an adversarial attack mask method. One part of the mask patch is the non-semantic information part with the largest change before and after the attack, and the other part is randomly masked. The randomness is maintained on the basis of fully retaining the semantic information part, and an image reconstruction task focusing on the semantic information part is generated, which promotes the model's ability to understand semantic information.

[0096] Specific implementation eight: This implementation differs from any one of the specific implementations one to seven in that the α attack The value of is 0.4.

[0097] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0098] Specific implementation method 9: This implementation method is different from any one of specific implementation methods 1 to 8 in that the loss in step 2 is the sum of the mean square error reconstruction losses of each image in the training set, and the mean square error reconstruction loss of image X is specifically:

[0099]

[0100] in, is the mean squared error reconstruction loss of image X, M is the set of mask image patches on image X, and X k is the kth mask image patch in the set M, Yes X k The reconstruction result of , ‖·‖ represents the 2-norm.

[0101] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0102] Specific implementation method ten: This implementation method is different from any one of specific implementation methods one to nine in that the parameters of the pre-trained mask image model are fine-tuned based on the pixel-level PGD adversarial attack and the target domain image dataset, specifically:

[0103]

[0104] Among them, x j is the jth image in the target domain image dataset of the downstream task, δ j It is for x j Added pixel-level PGD adversarial attack, f θ (x j +δ j ) is x j +δ j The output after the mask image model, θ is the fine-tuned mask image model parameter, is the cross entropy loss function, N is the total number of images contained in the target domain image dataset of the downstream task, and y j is x j .

[0105] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0106] like Figure 4 As shown in Figure 1, after completing the self-supervised pre-training, the weights of the encoder after model training are obtained as the skeleton, and combined with the classification head of the downstream task, the encoder parameters are fine-tuned according to the needs of the downstream task. By directly incorporating adversarial examples into the training, the model is further promoted to learn more generalized features, while improving the model's robustness against interference. Figure 4 For downstream classification tasks, during the fine-tuning process, the downstream target domain image after adding pixel-level PGD adversarial attack is directly used as the input of the skeleton part. The output of the skeleton part is then passed through the classification head. The cross entropy loss is calculated based on the output of the classification head and the label of the target domain image, and then the model parameters are fine-tuned based on the cross entropy loss.

[0107] initialization Iteratively calculate the pixel-level PGD adversarial attack δ j , the specific calculation method is:

[0108]

[0109] in, is the cross entropy loss until The iteration is stopped when , and the result obtained from the second-to-last iteration is used as the final pixel-level PGD adversarial attack δ j .

[0110] The effectiveness of the method of the present invention is illustrated by experimental results below. The anti-attack parameter settings adopted by the present invention are shown in Table 1.

[0111] Table 1

[0112]

[0113]

[0114] Depend on Figure 2It can be seen that the adversarial samples generated by the embedding-level attack are more concentrated, and the adversarial samples generated are within the distribution. The adversarial samples of the pixel-level attack are more divergent, and some are even far away from the original samples. The training parameters used in the self-supervised pre-training stage of the mask image model and the downstream task adversarial fine-tuning stage of the present invention are shown in Table 2. The mask rate γ of the mask image model is set to 0.75, and the hyperparameter β of the AdamW optimizer is set to 0.9 and 0.95. The learning rate during training is dynamically adjusted according to the preheating and cosine annealing strategies. The inter-layer learning rate decays to 0.65 in the fine-tuning stage.

[0115] Table 2

[0116]

[0117] The training results of the present invention on the CIFAR10 dataset (including image samples of bulk commodity shipping carriers such as ships, freight trucks, and containers) and the Imagenette dataset (including image samples of crops such as corn and soybeans, as well as cash crops) are shown in Table 3.

[0118] Table 3

[0119]

[0120] As can be seen from Table 3, the present invention can not only significantly improve the natural accuracy of the model on clean samples after training, but also improve the robustness of the model, greatly improving the PGD adversarial accuracy and automatic attack (Autoattack, AA) accuracy. Figure 5 As shown. It can be seen that the model trained by the present invention focuses more on the semantic information part related to the category and its surrounding parts, without letting the model's focus area be too scattered in the entire image. Most of the background and the parts of objects not related to the category are ignored. This shows that the model fully understands the critical semantic information in the image and eliminates interference, thereby improving the generalization performance and enhancing the robustness.

[0121] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for self-supervised image training based on adversarial attack, characterized in that: The method specifically comprises the following steps: Step 1: Process the acquired source domain image dataset based on the embedding-level PGD adversarial attack to obtain the processed image; Step 2: Use the processed image to pre-train the mask image model until the loss converges and then stop the pre-training to obtain a pre-trained mask image model; Step 3: Fine-tune the parameters of the pre-trained mask image model based on the pixel-level PGD adversarial attack and the labeled target domain image dataset to complete the training process.

2. The image self-supervised training method based on adversarial attack according to claim 1, characterized in that: The mask image model is a mask autoencoder.

3. The image self-supervised training method based on adversarial attack according to claim 2, characterized in that: The specific process of step one is: Step 1: For any image X in the acquired source domain image dataset, divide the image X into N′ non-overlapping image patches, N′=H×W / P 2 , P represents the size of each image patch, H and W represent the height and width of image X respectively; The i-th image patch obtained after division is denoted as X i , i=1,2,3; Step 1 and 2: Initialize the number of iterations l = 0 and initialize the adversarial perturbation Step 13: uniformly randomly mask all image patches in image X, with a mask ratio of γ, i.e., mask γ·N′ image patches; Step 14: All image patches of image X are used as input to the masked autoencoder, and in the masked autoencoder, there is a unique corresponding embedding layer for each image patch in image X; If the image patch X i For an unmasked patch, give the image patch X i The corresponding embedding layer adds an embedding-level PGD adversarial attack, that is, to the image patch X i The corresponding embedding layer adds adversarial perturbations If the image patch X i is a mask patch, then the image patch X i The embedding-level PGD adversarial attack added by the corresponding embedding layer is 0; And the image patch X i The corresponding embedding layer output is recorded as Step 15: Determine whether the stop condition of the iteration is met; If satisfied, then get the image patch X i The corresponding final embedding layer f embed,i , then execute step 16; If not satisfied, set l=l+1 and return to step 1-3; Step 16: Pass all image patches in image X through the corresponding final embedding layer respectively, according to and f embed,i (X i ) After selecting some mask patches from the image X, randomly select some image patches from the remaining image patches of the image X for masking to obtain a processed image corresponding to the image X; Step 17: Perform steps 11 to 16 on each image in the source domain image dataset to obtain a processed image dataset corresponding to the source domain image dataset.

4. The image self-supervised training method based on adversarial attack according to claim 3, characterized in that: The adversarial perturbation for: Among them, α is the update step size, is the adversarial perturbation after the lth iteration update, is the image patch X i The corresponding non-zero adversarial perturbation added to the last embedding layer, is the mean square error reconstruction loss of the lth iteration, represents the mean square error reconstruction loss The gradient of , sign(·) represents the sign function, and M is the set of all mask image patches selected at the lth iteration.

5. The image self-supervised training method based on adversarial attack according to claim 4, characterized in that: The stopping condition of the iteration is to satisfy the following method (1) or method (2); Method (1), reaching the set maximum number of iterations; Method (2), any image slice satisfies is greater than the threshold ∈; where, yes The p-norm of .

6. The image self-supervised training method based on adversarial attack according to claim 5, characterized in that: In step 16, all image patches in image X are respectively passed through the corresponding final embedding layer, according to and f embed,i (X i ) After selecting some mask patches from the image X, randomly select some image patches from the remaining image patches of the image X for masking to obtain a processed image corresponding to the image X; The specific process is: Step 1: Calculate the corresponding Among them, |·| means calculating the Euclidean distance, Represents X i After the output of the initial embedding layer, f embed,i (X i ) indicates X i Output after the final embedding layer; Step 2: According to Select a fixed ratio of image patches from all image patches of image X; Then randomly select a fixed ratio of image patches from the remaining image patches; Step 3: Perform a mask operation on the image patch selected in step 2 in image X to obtain a masked image, that is, obtain a processed image corresponding to image X.

7. The image self-supervised training method based on adversarial attack according to claim 6, characterized in that: The specific process of step 2 is: For each image patch of image X, Sort in descending order and select the α in front attack The image patch corresponding to N′ values, where α attack is the proportion of the mask part selected based on the embedding attack; Then randomly select (γ-α) from the remaining image patches attack )·N′ image patches, γ is the ratio of all masked parts.

8. The image self-supervised training method based on adversarial attack according to claim 7, characterized in that: The α attack The value of is 0.

4.

9. The image self-supervised training method based on adversarial attack according to claim 8, characterized in that: The loss in step 2 is the sum of the mean square error reconstruction losses of each image in the training set. The mean square error reconstruction loss of image X is specifically: in, is the mean squared error reconstruction loss of image X, M is the set of mask image patches on image X, and X k is the kth mask image patch in the set M, Yes X k The reconstruction result of , ‖·‖ represents the 2-norm.

10. The image self-supervised training method based on adversarial attack according to claim 9, characterized in that: The parameters of the pre-trained mask image model are fine-tuned based on the pixel-level PGD adversarial attack and the target domain image dataset, specifically: Among them, x j is the jth image in the target domain image dataset of the downstream task, δ j It is for x j Added pixel-level PGD adversarial attack, f θ (x j +δ j ) is x j +δ j The output after the mask image model, θ is the fine-tuned mask image model parameter, is the cross entropy loss function, N is the total number of images contained in the target domain image dataset of the downstream task, and y j is x j .

Images