Self-supervised adversarial training method and device

Abstract

The invention provides a self-supervised adversarial training method, which comprises the following steps: constructing a first-stage initial model, obtaining a first-stage training sample, and carrying out self-supervised training on the first-stage initial model to obtain a clean model; constructing a second-stage initial model, obtaining a second-stage training sample, generating an enhanced sample of the second-stage training sample, and taking an output feature of the clean model to the enhanced sample as a pseudo-supervision signal; generating an adversarial sample of the enhanced sample, and performing self-supervised adversarial training on the second-stage initial model by using the pseudo-supervised signal, the enhanced sample and the adversarial sample to obtain a robust model; and carrying out an image classification task by using the robust model. The invention further provides a self-supervised adversarial training device, a computer readable storage medium and electronic equipment. The training method provided by the invention can be seamlessly combined with different self-supervised learning frameworks, the confrontation and clean sample classification accuracy is improved, and the performance gap with supervised confrontation learning is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of machine learning and computer vision, and in particular relates to a self-supervised adversarial training method and device based on diverse enhanced query and self-supervised dual disturbance. Background Art

[0002] In recent years, in the field of adversarial learning, self-supervised adversarial training has attracted more and more researchers' attention due to its label-free characteristics and practicality. In the field of self-supervision, the case discrimination task, as a new best-effect proxy task, provides a natural entry point for adversarial training. Previous work in this field includes proposing the use of adversarial training based on self-supervised contrast loss to improve robustness, with slight differences in the form of loss design. The assumption of this type of method is that if the surrounding sample feature space is smooth enough, the features after adversarial perturbation will be consistent with the clean features. For example, some work pointed out that contrastive learning is naturally sensitive to adversarial noise, and proposed a simple method to remove erroneous negative sample pairs, which does not involve changes to adversarial training. There is also work that proposes to introduce causal inference into adversarial contrastive learning and use adversarial invariance constraints to improve it. Despite the good performance achieved, the above methods are still confined to the framework of contrastive learning.

[0003] After that, some works began to explore robustness improvement in a wider range of self-supervised frameworks. Some works formalized self-supervised adversarial training into a two-stage framework. In the first stage, the self-supervised model is trained, and in the second stage, the model features are used to guide adversarial training. This type of work provides a strong baseline method for this method. Some works proposed a targeted adversarial perturbation generation method to improve the robustness of non-contrastive methods. However, this method cannot be improved on self-supervised methods based on contrastive learning. Some works pointed out that strong data augmentation methods are important for building self-supervised learning, but are harmful to self-supervised robustness. Although the above methods have achieved certain robust performance, there is a certain gap with supervised adversarial learning. At the same time, the above methods have not yet established an understanding of the adversarial learning process similar to that in supervised adversarial learning. Summary of the invention

[0004] In view of the above problems, the present invention proposes a self-supervised adversarial training method, including: constructing a first-stage initial model, obtaining first-stage training samples, performing self-supervised training on the first-stage initial model, and obtaining a clean model; constructing a second-stage initial model, obtaining second-stage training samples, generating enhanced samples of the second-stage training samples, and using the output features of the enhanced samples by the clean model as pseudo-supervisory signals; generating adversarial samples of the enhanced samples, and performing self-supervised adversarial training on the second-stage initial model with the pseudo-supervisory signal, the enhanced samples and the adversarial samples to obtain a robust model; and performing image classification tasks with the robust model.

[0005] Furthermore, the loss function l2 of the self-supervised adversarial training is:

[0006] l2=l clean +α*l adv

[0007]

[0008] Among them, x is the second stage training sample, To enhance the sample, for The corresponding adversarial example, l clean To use the enhanced sample The loss function for this self-supervised adversarial training, l adv To use adversarial examples The loss function for self-supervised adversarial training, f1 is the clean model, To use the enhanced sample The second stage initial model is trained and a specific normalization layer is selected, To use adversarial examples The second-stage initial model is trained and a specific normalization layer is selected. δ is the adversarial perturbation, α is the hyperparameter, and sim(,) represents the cosine similarity between features.

[0009] Furthermore, the self-supervised model parameter adversarial perturbation is introduced in the self-supervised adversarial training:

[0010]

[0011] Then the loss function of the self-supervised adversarial training is swap-diverse-aug :

[0012] l swap-diverse-aug = l clean +α*l adv

[0013]

[0014] Among them, θ2-perturb is the model parameter of the second-stage initial model after the self-supervised model parameter counter-perturbation is introduced, μ(·) is the perturbation range of the model parameter, θ2 is the model parameter of the second-stage initial model when the self-supervised model parameter counter-perturbation is not introduced, l clean To introduce the self-supervised model parameters to counter the disturbance, use the enhanced samples The loss function for this self-supervised adversarial training, l adv To introduce the adversarial perturbation of the self-supervised model parameters, use adversarial samples The loss function for self-supervised adversarial training, f1 is the clean model, To introduce the self-supervised model parameters to counter the disturbance, use the enhanced samples The second stage initial model is trained and a specific normalization layer is selected, To introduce the adversarial perturbation of the self-supervised model parameters, use adversarial samples The second-stage initial model is trained and a specific normalization layer is selected. α is a hyperparameter and sim(,) represents the cosine similarity between features.

[0015] Preferably, in the self-supervised adversarial training, the self-supervised model parameter adversarial perturbation is introduced in the last one-third of the training process.

[0016] The present invention also proposes a self-supervised adversarial training device, comprising: a first-stage training module, used to construct a first-stage initial model, obtain first-stage training samples, perform self-supervised training on the first-stage initial model, and obtain a clean model; a second-stage training module, used to construct a second-stage initial model, obtain second-stage training samples, generate enhanced samples of the second-stage training samples, and use the output features of the enhanced samples by the clean model as pseudo-supervisory signals; generate adversarial samples of the enhanced samples, and perform self-supervised adversarial training on the second-stage initial model with the pseudo-supervisory signals, the enhanced samples and the adversarial samples to obtain a robust model; and a task execution module, used to perform image classification tasks with the robust model.

[0017] Furthermore, in the second stage training module, the loss function l2 of the self-supervised adversarial training is:

[0018] l2=l clean +α*l adv

[0019]

[0020] Among them, x is the second stage training sample, To enhance the sample, for The corresponding adversarial example, l clean To use the enhanced sample The loss function for this self-supervised adversarial training, l adv To use adversarial examples The loss function for self-supervised adversarial training, f1 is the clean model, To use the enhanced sample The second stage initial model is trained and a specific normalization layer is selected, To use adversarial examples The second-stage initial model is trained and a specific normalization layer is selected. δ is the adversarial perturbation, α is the hyperparameter, and sim(,) represents the cosine similarity between features.

[0021] Furthermore, in the second stage training module, the self-supervised model parameter adversarial perturbation is introduced in the self-supervised adversarial training:

[0022]

[0023] Then the loss function of the self-supervised adversarial training is swap-diverse-aug :

[0024] l swap-diverse-aug = l clean +α*l adv

[0025]

[0026] Among them, θ 2-perturb is the model parameter of the second-stage initial model after the self-supervised model parameter counter-perturbation is introduced, μ(·) is the perturbation range of the model parameter, θ2 is the model parameter of the second-stage initial model when the self-supervised model parameter counter-perturbation is not introduced, l clean To introduce the self-supervised model parameters to counter the disturbance, use the enhanced samples The loss function for this self-supervised adversarial training, l adv To introduce the adversarial perturbation of the self-supervised model parameters, use adversarial samples The loss function for self-supervised adversarial training, f1 is the clean model, To introduce the self-supervised model parameters to counter the disturbance, use the enhanced samples The second stage initial model is trained and a specific normalization layer is selected, To introduce the adversarial perturbation of the self-supervised model parameters, use adversarial samples The second-stage initial model is trained and a specific normalization layer is selected. α is a hyperparameter and sim(,) represents the cosine similarity between features.

[0027] Preferably, in the second stage training module, the self-supervised model parameter counter-perturbation is introduced in the last one-third of the training process.

[0028] The present invention also proposes an electronic device, comprising the self-supervised adversarial training device as described above.

[0029] The present invention also proposes a computer-readable storage medium storing computer-executable instructions, characterized in that when the computer-executable instructions are executed, the self-supervised adversarial training method as described above is implemented.

[0030] The present invention proposes a self-supervised adversarial training method based on diverse enhanced query and self-supervised dual perturbation, which improves the robust generalization performance and clean sample accuracy by increasing data complexity and model regularization. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the self-supervised adversarial training method of the present invention.

[0032] Figure 2 It is a flow chart of the self-supervised adversarial training method of the present invention.

[0033] Figure 3 Schematic diagram of the self-supervised adversarial training device of the present invention.

[0034] Figure 4 It is a schematic diagram of an electronic device of the present invention.

[0035] Figure 5 It is a schematic diagram of the hardware structure of an electronic device of the present invention.

[0036] Wherein, the accompanying drawings are marked as follows:

[0037] 100: Electronic device 10: Self-supervised adversarial training device

[0038] 11: First phase training module 12: Second phase training module

[0039] 13: Task execution module

[0040] S1, S2, S3: Steps DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation method described herein is only used to explain the present invention and is not used to limit the present invention.

[0042] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0043] In response to the problems existing in the prior art, the inventor proposed a self-supervised adversarial training method based on diverse enhanced queries and self-supervised dual perturbations by studying the self-supervised adversarial learning process. Specifically, a diverse strong data enhancement method is used to query the model trained on clean samples. By designing different feature batch normalization techniques, the diverse strong data enhancement that does not rely on labels can improve the self-supervised adversarial performance. At the same time, the perturbation of the self-supervised model parameters is introduced in the training process to further avoid the decline in robust generalization error and clean sample accuracy.

[0044] Self-supervised adversarial training is a learning paradigm that learns robust features without using labels. The existing solutions have narrowed the performance gap between self-supervised adversarial training and supervised adversarial training, but rarely explored the connection between the two and established a general form of self-supervised adversarial training. After analysis, the current methods still have the problem of large robust generalization gap and decreased accuracy of clean samples.

[0045] The present invention proposes a self-supervised adversarial training method based on diverse enhanced query and self-supervised dual perturbation, which alleviates the above problems by increasing data complexity and model regularization. Specifically, this method first challenges the conclusion in previous work that complex data enhancement is harmful to the robustness of self-supervised training, and proposes to use diverse enhanced data as query samples to guide adversarial training. At the same time, the method proposes a self-supervised dual perturbation, which improves the robustness of downstream classification tasks. The method of the present invention can be seamlessly combined with different self-supervised learning frameworks, improve the classification accuracy of adversarial and clean samples, and reduce the performance gap with supervised adversarial learning.

[0046] Starting from the framework of the self-supervised method based on instance discrimination, and in combination with the accompanying drawings, the self-supervised adversarial training method based on diverse enhanced query and self-supervised dual perturbation proposed in the present invention is described in detail.

[0047] As a popular pre-training proxy task for self-supervised learning, case discrimination tasks are divided into contrastive learning and non-contrastive learning. Contrastive learning learns feature space by pulling away negative sample pairs from different sources and pulling closer positive sample pairs from the same source. Non-contrastive learning only constructs loss functions by pulling closer sample pairs from the same source and other model components. Self-supervised adversarial training tasks require that this pre-trained feature is not easily disturbed by adversarial noise when used for downstream tasks.

[0048] The present invention uses a general two-stage framework to perform self-supervised adversarial training. Figure 1 As shown, in the first stage, the first stage model f1 is first constructed, and the first stage training samples (clean samples) are obtained. The framework uses normal samples to perform self-supervised training on the first stage model f1. The loss function l1 of the first stage is:

[0049] l1=l ssl

[0050] Among them, l ssl Represents the loss function of the self-supervised pre-training method, as defined by the respective supervised training method.

[0051] In the second stage, firstly, the second stage model f2 is constructed, and the second stage training samples are obtained, wherein the structure of f2 is consistent with that of f1. In some embodiments of the present invention, f1 can be used as the initialization model of the second stage; the features obtained by f1 from the input second stage training samples are used as pseudo supervision signals for adversarial training of the second stage model f2. The loss function l2 of the second stage is used to generate adversarial samples x adv :

[0052] l2=sim(f1(x),f2(x adv ))+α*sim(f2(x),f2(x adv ))

[0053]

[0054] sim(,) represents the cosine similarity between features, and α represents the value used to balance the recognition performance of the second-stage training sample x and the adversarial sample x adv is a hyperparameter for the identification performance and δ represents the added adversarial perturbation.

[0055] Previous work proposed that the enhancement strength of augmented data is detrimental to the adversarial nature of self-supervised models. This method uses TrivialAugment as a data augmentation method that is independent of the dataset and does not search for augmentation strategies to construct diverse data and draw different conclusions. Among them, TrivialAugment is a data augmentation strategy that randomly selects a data augmentation operation for each image and randomly determines its enhancement magnitude, and then enhances the image. On this basis, in order to improve the generalization of different data augmentation strengths while maintaining the performance on the test data distribution, this method simultaneously inputs samples of ordinary data augmentation (horizontal flip, random resized crop) and strong data augmentation into the first-stage model and the second-stage robust model to make the features of the two consistent. In order to make the robust model adapt to different data distributions, this method proposes a "(strong / weak) augmentation-(yes / no) adversarial" paired batch normalization strategy, setting a specific batch normalization layer for each input distribution of the robust model. After the training is completed, only the "weak augmentation-adversarial" batch normalization layer is retained. Therefore, the first-stage training method remains unchanged, and the second-stage training loss function l2 is as follows:

[0056] l2=l clean +α*l adv

[0057]

[0058] Among them, x is the second stage training sample, To enhance samples for diversity, for The corresponding adversarial example, l clean To use the enhanced sample The loss function for this self-supervised adversarial training, l adv To use adversarial examples The loss function for self-supervised adversarial training, To use the enhanced sample The second stage initial model is trained and a specific normalization layer is selected, To use adversarial examples The second-stage initial model model is trained and a specific normalization layer is selected. δ is the adversarial perturbation, α is the hyperparameter, and sim(,) represents the cosine similarity between features.

[0059] In order to further alleviate the training-test robust generalization gap, this method also introduces model parameter perturbations into self-supervised pre-training, and regularizes the smoothness of the constraint feature similarity loss in a self-supervised manner. Specifically, the self-supervised weight adversarial perturbation perturbs the model parameters in the direction of increasing the self-supervised similarity loss, thereby increasing the smoothness of the similarity loss landscape with respect to the model weights. The self-supervised weight adversarial perturbation can be formalized as:

[0060]

[0061] θ 2-perturb are the model parameters of the initial model in the second stage after the self-supervised weight adversarial perturbation is introduced, θ2 is the model parameters of the initial model in the second stage when the self-supervised weight adversarial perturbation is not introduced, and μ(·) is the perturbation range of the model parameters.

[0062] Since additional adversarial components will increase the difficulty of self-supervised tasks, the present invention introduces model parameter perturbations into the second-stage adversarial training process to stabilize the features of adversarial samples and clean samples. Model parameter perturbations can be introduced at any time during the second-stage adversarial training process, such as during the entire process or half of the process. According to the inventor's research and experiments, the best effect can be achieved by introducing model parameter perturbations in the last third of the adversarial training process.

[0063] Therefore, integrating the above diverse data enhancement queries and self-supervised model perturbation methods, the overall training objective function of the model is:

[0064] l swap-diverse-aug = l clean +α*l adv

[0065] in:

[0066]

[0067] In the above formula, l clean To introduce self-supervised model parameters to counteract disturbances, use enhanced samples The loss function for self-supervised adversarial training, l adv To introduce adversarial perturbations to the parameters of the self-supervised model, adversarial samples are used The loss function for self-supervised adversarial training, f1 is the clean model, To introduce self-supervised model parameters to counteract disturbances, use enhanced samples The second stage initial model is trained and a specific normalization layer is selected, To introduce adversarial perturbations to the parameters of the self-supervised model, adversarial samples are used The second stage initial model is trained and a specific normalization layer is selected.

[0068] In general, this method addresses the problem of insufficient robust generalization of existing self-supervised adversarial training and the reduced accuracy of clean samples. The self-supervised model trained normally in the first stage is used as a guide for the robust model in the second stage, and diverse data enhancement is used for query to retain generalization performance. At the same time, after the training is stable, self-supervised adversarial perturbations of model parameters are introduced. After the training is completed, only the model corresponding to the "weak enhancement-adversarial" batch normalization layer is retained. It is a robust pre-trained model obtained through self-supervised adversarial training and tested on downstream image classification tasks.

[0069] Figure 2 : is a flow chart of the self-supervised adversarial training method of the present invention. Figure 2 As shown, in the first embodiment of the present invention, a self-supervised adversarial training method based on diverse enhanced query and self-supervised dual perturbation is proposed, which specifically includes:

[0070] Step S1: construct the first-stage initial model, obtain the first-stage training samples, perform self-supervised training on the first-stage initial model, and obtain a clean model;

[0071] In the first stage, normal samples are used to perform self-supervisory training on the first stage model f1, and the loss function l1 of the first stage is:

[0072] l1=l ssl

[0073] Step S2: construct the second-stage initial model f2, obtain the second-stage training sample x, and generate the enhanced sample of the second-stage training sample x Enhance samples with clean models The output feature of is a pseudo supervisory signal; the adversarial sample of the enhanced sample is generated The second-stage initial model is trained in self-supervisory adversarial manner using pseudo-supervisory signals, enhanced samples, and adversarial samples to obtain a robust model.

[0074] The loss function l2 of the second stage self-supervised adversarial training is:

[0075] l2=l clean +α*l adv

[0076]

[0077] In order to further alleviate the training-test robust generalization gap, this method also introduces model parameter perturbations into self-supervised pre-training, and regularizes the smoothness of the constraint feature similarity loss in a self-supervised manner. Specifically, the self-supervised weight adversarial perturbation perturbs the model parameters in the direction of increasing the self-supervised similarity loss, thereby increasing the smoothness of the similarity loss landscape with respect to the model weights. The self-supervised weight adversarial perturbation can be formalized as:

[0078]

[0079] Since additional adversarial components will increase the difficulty of the self-supervised task, the present invention introduces model parameter perturbations into the last third of the second-stage adversarial training process to stabilize the features of adversarial samples and clean samples in the early stage of the second-stage self-supervised adversarial training.

[0080] Therefore, integrating the above diverse data enhancement queries and self-supervised model perturbation methods, the overall training objective function of the model is:

[0081] l swap-diverse-aug = l clean +α*l adv

[0082] in:

[0083]

[0084]

[0085] Step S3: Perform image classification tasks using the trained robust model.

[0086] It should be noted that, in various embodiments of the present invention, the sequence numbers of the above steps do not mean the order of execution. The order of execution of the steps should be determined by their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0087] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0088] Figure 3 Schematic diagram of the self-supervised adversarial training device of the present invention. Figure 3 As shown, in a second embodiment of the present invention, a self-supervised adversarial training device 10 is proposed, comprising:

[0089] The first stage training module 11 is used to construct the first stage initial model, obtain the first stage training samples, perform self-supervised training on the first stage initial model, and obtain a clean model;

[0090] The second-stage training module 12 is used to construct the second-stage initial model, obtain the second-stage training samples, generate enhanced samples of the second-stage training samples, use the output features of the clean model on the enhanced samples as pseudo-supervisory signals; generate adversarial samples of the enhanced samples, and perform self-supervised adversarial training on the second-stage initial model with the pseudo-supervisory signals, the enhanced samples and the adversarial samples to obtain a robust model;

[0091] The task execution module 13 is used to perform image classification tasks using the trained robust model.

[0092] In the third embodiment of the present invention, a computer-readable storage medium is proposed. The self-supervised adversarial training device of the present invention, if its function is implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a computer-readable storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. Therefore, in the third embodiment of the present invention, a computer-readable storage medium is provided for storing a computer program for executing a self-supervised adversarial training method. It should be understood that the computer-readable storage medium in the embodiment of the present invention can be a volatile memory and / or a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM) that is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0093] Figure 4 Schematic diagram of an electronic device of the present invention. Figure 4As shown, in the fourth embodiment of the present invention, an electronic device 100 is proposed, including the self-supervised adversarial training device as described above. A person of ordinary skill in the art can understand that all or part of the steps in the above method can be completed by instructing related hardware (such as a processor, FPGA, ASIC, etc.) through a program. All or part of the steps of the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module in the above embodiment can be implemented in the form of hardware, such as implementing its corresponding functions through an integrated circuit, or in the form of a software functional module, such as implementing its corresponding functions through a processor executing a program / instruction stored in a memory. The embodiments of the present invention are not limited to any particular form of combination of hardware and software.

[0094] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0095] The electronic device of the present invention may be any device with data processing capability, and the device with data processing capability may be a device or apparatus such as a computer. The device embodiment may be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the processor of any device with data processing capability reads the corresponding computer program instructions in the non-volatile memory into the memory and runs the instructions. Figure 5 This is a schematic diagram of the hardware structure of an electronic device of the present invention. Figure 5 As shown in the figure, from the hardware level, it is a hardware structure diagram of any device with data processing capability where the self-supervised adversarial training device of the present invention is located, except Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0096] When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0097] The following are the details and results of the experiments conducted on the self-supervised adversarial training method based on diverse enhanced queries and self-supervised dual perturbations of the present invention.

[0098] Experimental conditions:

[0099] The inventors conducted experiments on the pytorch platform. All models were trained using two RTX3090 GPUs. The self-supervised model was first trained with normal clean samples for 1000 rounds to obtain the first-stage model, and then it was trained for 100 rounds of adversarial training to obtain the second-stage model. The adversarial training process used a 5-step projected gradient descent with a range of 8 / 255 to optimize the adversarial samples. After 60 rounds, model parameter perturbations were added, and the parameter perturbation range was limited to 0.002. AutoAttack and PGD attack methods were used for testing, and the number of attack steps was 20.

[0100] Experimental results:

[0101] The inventors used the pre-trained model obtained by this method to conduct image classification task experiments on the CIFAR10 and CIFAR100 datasets, and used three different self-supervised frameworks (SimCLR, BYOL and SimSiam) to verify the effectiveness and versatility of this method. Table 1 shows the results of image classification on the CIFAR10 dataset using ResNet34 as the SimCLR encoder. Table 2 shows the results of image classification on the CIFAR100 dataset using this method. Table 3 shows the results of image classification on CIFAR10 using the SimSiam and BYOL self-supervised frameworks. Table 4 shows the image classification results of transfer learning from CIFAR100 to CIFAR10. The values ​​in the table are the classification accuracy (%) of the samples in this column. The values ​​in the following tables are all the results of simple linear fine-tuning of the pre-trained models.

[0102] method Clean PGD AutoAttack DynACL+AIR 79.79 51.07 47.61 TARO 84.23 53.36 45.68 DecoupledACL 82.46 56.86 47.99 Ours 84.57 58.57 49.22

[0103] Table 1

[0104] method Clean PGD AutoAttack DynACL+AIR 47.02 23.91 20.66 TARO 51.28 29.46 21.14 DecoupledACL 51.44 30.68 21.31 Ours 53.54 33.09 23.42

[0105] Table 2

[0106] method Clean PGD AutoAttack BYOL TARO 86.84 52.01 44.76 BYOL DecoupledACL 83.15 55.22 47.67 BYOL Ours 85.91 56.53 49.07 SimSiam TARO 81.71 52.61 44.46 SimSiam DecoupledACL 78.40 57.17 47.20 SimSiam Ours 80.42 58.53 47.69

[0107] Table 3

[0108] method Clean PGD TARO 57.13 23.99 DecoupledACL 55.03 25.78 Ours 57.66 26.83

[0109] Table 4

[0110] The present invention proposes a self-supervised adversarial training method based on diverse enhanced query and self-supervised dual perturbation, which improves robust generalization performance and clean sample accuracy by increasing data complexity and model regularization. The present invention includes the following features:

[0111] (1) We propose to use diverse and strongly data-augmented samples to query the normally trained model, and design an auxiliary “enhancement-adversarial” paired batch normalization technique to improve performance.

[0112] (2) Introducing model parameter perturbations into self-supervised pre-training. Introducing self-supervised double perturbations in the later stages of adversarial training can improve the robustness and generalization of the model and avoid interfering with the learning of clean features.

[0113] The experimental results demonstrate the superior performance of the self-supervised adversarial training method based on diverse enhanced queries and self-supervised dual perturbations proposed in the present invention, and verify the effectiveness and versatility of the method.

[0114] The above implementation modes are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the present invention. The patent protection scope of the present invention should be defined by the claims.

Claims

1. A self-supervised adversarial training method, characterized in that: include: Construct the first-stage initial model, obtain the first-stage training samples, perform self-supervised training on the first-stage initial model, and obtain a clean model; Constructing a second-stage initial model, obtaining second-stage training samples, generating enhanced samples of the second-stage training samples, and using the output features of the clean model for the enhanced samples as pseudo supervisory signals; Generate an adversarial sample of the enhanced sample, and perform self-supervised adversarial training on the second-stage initial model using the pseudo supervisory signal, the enhanced sample, and the adversarial sample to obtain a robust model; Use this robust model to perform image classification tasks.

2. The self-supervised adversarial training method according to claim 1, characterized in that: The loss function l2 of this self-supervised adversarial training is: l2＝l clean +α*l adv Among them, x is the second stage training sample, To enhance the sample, for The corresponding adversarial example, l clean To use the enhanced sample The loss function for this self-supervised adversarial training, l adv To use adversarial examples The loss function for self-supervised adversarial training, f1 is the clean model, To use the enhanced sample The second stage initial model is trained and a specific normalization layer is selected, To use adversarial examples The second-stage initial model is trained and a specific normalization layer is selected. δ is the adversarial perturbation, α is the hyperparameter, and sim(,) represents the cosine similarity between features.

3. The self-supervised adversarial training method according to claim 1, characterized in that: In this self-supervised adversarial training, the self-supervised model parameter adversarial perturbation is introduced: Then the loss function of the self-supervised adversarial training is swap-diverse-aug : l swap-diverse-aug ＝l clean +α*l adv Among them, θ 2-perturb is the model parameter of the second-stage initial model after the self-supervised model parameter counter-perturbation is introduced, μ(·) is the perturbation range of the model parameter, θ2 is the model parameter of the second-stage initial model when the self-supervised model parameter counter-perturbation is not introduced, l clean To introduce the self-supervised model parameters to counter the disturbance, use the enhanced samples The loss function for this self-supervised adversarial training, l adv To introduce the adversarial perturbation of the self-supervised model parameters, use adversarial samples The loss function for self-supervised adversarial training, f1 is the clean model, To introduce the self-supervised model parameters to counter the disturbance, use the enhanced samples The second stage initial model is trained and a specific normalization layer is selected, To introduce the adversarial perturbation of the self-supervised model parameters, use adversarial samples The second-stage initial model is trained and a specific normalization layer is selected. α is a hyperparameter and sim(,) represents the cosine similarity between features.

4. The self-supervised adversarial training method according to claim 3, characterized in that: In the self-supervised adversarial training, adversarial perturbations of the self-supervised model parameters are introduced in the last one-third of the training process.

5. A self-supervised adversarial training device, characterized in that: include: The first-stage training module is used to build the first-stage initial model, obtain the first-stage training samples, perform self-supervised training on the first-stage initial model, and obtain a clean model; The second-stage training module is used to construct a second-stage initial model, obtain second-stage training samples, generate enhanced samples of the second-stage training samples, and use the output features of the clean model on the enhanced samples as pseudo supervisory signals; Generate an adversarial sample of the enhanced sample, and perform self-supervised adversarial training on the second-stage initial model using the pseudo supervisory signal, the enhanced sample, and the adversarial sample to obtain a robust model; The task execution module is used to perform image classification tasks using the robust model.

6. The self-supervised adversarial training apparatus according to claim 5, characterized in that: In the second stage training module, the loss function l2 of the self-supervised adversarial training is: l2＝l clean +α*l adv Among them, x is the second stage training sample, To enhance the sample, for The corresponding adversarial example, l clean To use the enhanced sample The loss function for this self-supervised adversarial training, l adv To use adversarial examples The loss function for self-supervised adversarial training, f1 is the clean model, To use the enhanced sample The second stage initial model is trained and a specific normalization layer is selected, To use adversarial examples The second-stage initial model is trained and a specific normalization layer is selected. δ is the adversarial perturbation, α is the hyperparameter, and sim(,) represents the cosine similarity between features.

7. The self-supervised adversarial training apparatus according to claim 5, characterized in that: In the second stage training module, the self-supervised model parameter adversarial perturbation is introduced in the self-supervised adversarial training: Then the loss function of the self-supervised adversarial training is swap-diverse-aug : l swap-diverse-aug ＝l clean +α*l adv Among them, θ 2-perturb is the model parameter of the second-stage initial model after the self-supervised model parameter counter-perturbation is introduced, μ(·) is the perturbation range of the model parameter, θ2 is the model parameter of the second-stage initial model when the self-supervised model parameter counter-perturbation is not introduced, l clean To introduce the self-supervised model parameters to counter the disturbance, use the enhanced samples The loss function for this self-supervised adversarial training, l adv To introduce the adversarial perturbation of the self-supervised model parameters, use adversarial samples The loss function for self-supervised adversarial training, f1 is the clean model, To introduce the self-supervised model parameters to counter the disturbance, use the enhanced samples The second stage initial model is trained and a specific normalization layer is selected, To introduce the adversarial perturbation of the self-supervised model parameters, use adversarial samples The second-stage initial model is trained and a specific normalization layer is selected. α is a hyperparameter and sim(,) represents the cosine similarity between features.

8. The self-supervised adversarial training apparatus according to claim 7, characterized in that: In the second stage training module, the self-supervised model parameter adversarial perturbation is introduced in the last one-third of the training process.

9. An electronic device comprising the self-supervised adversarial training device as claimed in any one of claims 5 to 8.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed, the self-supervised adversarial training method according to any one of claims 1 to 4 is implemented.

Images