Robust unsupervised domain adaptive image classification method and device based on adversarial distillation

Abstract

The present invention discloses a robust unsupervised domain adaptive image classification method and device based on adversarial distillation, the method comprising the following steps: obtaining an unsupervised target domain natural sample set; constructing a robust unsupervised domain adaptive image classification framework, including a non-robust target domain teacher model and a robust target domain student model; using a pre-trained non-robust source domain model to initialize the parameters of the non-robust target domain teacher model, and performing end-to-end iterative training on an unsupervised target domain natural sample set; constructing a robust target domain student model, performing adversarial distillation training on an unsupervised target domain natural sample set, and outputting image classification results. This method combines knowledge distillation with adversarial training, and in the case of complete lack of source domain data, only a non-robust source domain model is used to obtain a robust model on the target domain, while maintaining the classification performance of the target domain natural samples, the classification performance of the target domain adversarial samples and the robustness of the model are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer image classification, and specifically relates to a robust unsupervised domain adaptive image classification method and device based on adversarial distillation. Background Art

[0002] Unsupervised domain adaptation learning can transfer knowledge from a labeled source domain to an unlabeled target domain, and promote model transfer in scenarios where labels are scarce or annotations are cumbersome. However, due to data privacy and security issues, source domain data may not be accessible during the domain adaptation stage, and it is computationally intractable to use target domain data together with large-scale source data to train models on the target domain. Therefore, unsupervised domain adaptation learning with no source domain data has emerged, such as a model for unsupervised domain adaptation learning with no source domain data (SHOT model). Despite significant progress in existing research, most existing unsupervised domain adaptation or unsupervised domain adaptation with no source domain data methods ignore the robustness of deep learning models, which are sensitive to imperceptible perturbations in input images and are vulnerable to adversarial examples; in particular, since they are optimistically trained without precise supervision of the target domain, unsupervised domain adaptation models (with no source domain data) may be more sensitive to such perturbations, exacerbating the model's vulnerability and posing a huge threat to security-sensitive applications.

[0003] In existing research, on the one hand, robust unsupervised domain adaptation models are trained by transferring robustness from robust source domain models or robust pre-trained models to improve robustness; however, in many practical applications, it is impractical to assume the availability of robust source domain models or robust pre-training, so it is difficult to directly apply them to classification tasks. On the other hand, adversarial training is another way to improve model robustness, but adversarial training can lead to serious overfitting, which greatly affects the effectiveness of classification results. Summary of the invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a robust unsupervised domain adaptive image classification method and device based on adversarial distillation. The method combines knowledge distillation and adversarial training. When the source domain data is completely missing, only a non-robust source domain model is used to obtain a robust model on the target domain. While maintaining the classification performance of natural samples in the target domain, the classification performance of adversarial samples in the target domain and the robustness of the model are effectively improved.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] On the one hand, the present invention provides a robust unsupervised domain adaptive image classification method based on adversarial distillation, characterized in that it includes the following steps:

[0007] Obtain a natural sample set of unsupervised target domain;

[0008] Constructing a robust unsupervised domain adaptive image classification framework; the robust unsupervised domain adaptive image classification framework includes a non-robust target domain teacher model and a robust target domain student model;

[0009] Use the pre-trained non-robust source domain model to initialize the parameters of the non-robust target domain teacher model, perform end-to-end iterative training on the unsupervised target domain natural sample set, and obtain the trained non-robust target domain teacher model;

[0010] A robust target domain student model is constructed based on the trained non-robust target domain teacher model, and adversarial distillation training is performed on an unsupervised target domain natural sample set to obtain a trained robust target domain student model and output image classification results.

[0011] As a preferred technical solution, the objective function of the robust unsupervised domain adaptive image classification framework is derived based on the interval difference divergence under the condition of passive domain data, specifically:

[0012] According to the interval learning theory, for any score function f, it satisfies:

[0013]

[0014] in is an ideal interval loss, is the score function f in the target adversarial domain The classification error based on 0-1 loss, is the score function f in the source domain The classification error is spaced by a constant ρ. is the source domain spaced by a constant ρ and target domain The interval difference divergence, is the target domain separated by a constant ρ and target adversarial domain The interval difference divergence of

[0015] Let in the target adversarial domain Classification error based on 0-1 loss To achieve the minimum optimal score function f, according to the right side of formula (1), we can get:

[0016]

[0017] Under the condition that the source domain data is completely missing, it can be seen that the score function f is The classification error is spaced by a constant ρ is a constant, so according to formula (2), the objective function of the robust unsupervised domain adaptive image classification framework is derived as follows:

[0018]

[0019] in, is the objective function of the non-robust target domain teacher model, is the objective function of the student model in the robust target domain.

[0020] As a preferred technical solution, the trained non-robust target domain teacher model is specifically:

[0021] An unsupervised domain adaptation learning model that does not use source domain data is used to perform standard unsupervised domain adaptation learning to obtain a non-robust target domain teacher model;

[0022] Initialize the parameters of the non-robust target domain teacher model using the parameters of the pre-trained non-robust source domain model;

[0023] The unsupervised target domain natural sample set is input into the non-robust target domain teacher model for end-to-end iterative training to obtain a trained non-robust target domain teacher model.

[0024] As a preferred technical solution, the trained robust target domain student model is specifically:

[0025] The robust target domain student model is constructed using the same structure as the non-robust target domain teacher model;

[0026] According to the parameter information of the robust target domain student model, a corresponding adversarial sample is generated for each natural sample in the unsupervised target domain natural sample set;

[0027] Adversarial distillation training is performed. During each iterative training process, the parameters of the non-robust target domain teacher model are fixed, and the robust target domain student model is trained end-to-end to obtain the trained robust target domain student model and output the image classification result.

[0028] As a preferred technical solution, adversarial distillation training is performed based on the adversarial training method TRADES, which is aimed at balancing robustness and accuracy.

[0029] The generation formula of the adversarial sample is:

[0030]

[0031] The adversarial distillation loss function is established based on the output of the non-robust target domain teacher model and the output of the robust target domain student model, and is expressed as:

[0032]

[0033] where φ represents the robust target domain student model, φ T represents the non-robust target domain teacher model, x is a natural sample in the unsupervised target domain natural sample set, x' is the adversarial sample generated corresponding to x, is the KL divergence loss function, β is a constant coefficient, p is the p-norm, and ∈ is a constant range.

[0034] As a preferred technical solution, the adversarial training method PGD of projected gradient descent is used for adversarial distillation training;

[0035] The generation formula of the adversarial sample is:

[0036]

[0037] The adversarial distillation loss function is established based on the output of the non-robust target domain teacher model and the output of the robust target domain student model, and is expressed as:

[0038]

[0039] where φ represents the robust target domain student model, φ T represents the non-robust target domain teacher model, x is a natural sample in the unsupervised target domain natural sample set, x' is the adversarial sample generated corresponding to x, is the KL divergence loss function, β is a constant coefficient, p is the p-norm, and ∈ is a constant range.

[0040] On the other hand, the present invention also provides a robust unsupervised domain adaptive image classification system based on adversarial distillation, characterized in that it is applied to the above-mentioned robust unsupervised domain adaptive image classification method based on adversarial distillation, including a data acquisition module, a classification framework construction module, a teacher model training module and a student model training module;

[0041] The data acquisition module is used to obtain an unsupervised target domain natural sample set;

[0042] The classification framework construction module is used to construct a robust unsupervised domain adaptive image classification framework; the robust unsupervised domain adaptive image classification framework includes a non-robust target domain teacher model and a robust target domain student model;

[0043] The teacher model training module is used to initialize the parameters of the non-robust target domain teacher model using the pre-trained non-robust source domain model, and perform end-to-end iterative training on an unsupervised target domain natural sample set to obtain a trained non-robust target domain teacher model;

[0044] The student model training module is used to construct a robust target domain student model based on a trained non-robust target domain teacher model, perform adversarial distillation training on an unsupervised target domain natural sample set, obtain a trained robust target domain student model and output an image classification result.

[0045] As a preferred technical solution, the teacher model training module is specifically:

[0046] An unsupervised domain adaptation learning model that does not use source domain data is used to perform standard unsupervised domain adaptation learning to obtain a non-robust target domain teacher model;

[0047] Initialize the parameters of the non-robust target domain teacher model using the parameters of the pre-trained non-robust source domain model;

[0048] The unsupervised target domain natural sample set is input into the non-robust target domain teacher model for end-to-end iterative training to obtain a trained non-robust target domain teacher model.

[0049] As a preferred technical solution, the student model training module is specifically:

[0050] The robust target domain student model is constructed using the same structure as the non-robust target domain teacher model;

[0051] According to the parameter information of the robust target domain student model, a corresponding adversarial sample is generated for each natural sample in the unsupervised target domain natural sample set;

[0052] Adversarial distillation training is performed. During each iterative training process, the parameters of the non-robust target domain teacher model are fixed, and the robust target domain student model is trained end-to-end to obtain the trained robust target domain student model and output the image classification result.

[0053] On the other hand, the present invention provides a computer-readable storage medium storing a program, characterized in that when the program is executed by a processor, the above-mentioned robust unsupervised domain adaptive image classification method based on adversarial distillation is implemented.

[0054] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0055] 1. Compared with the general unsupervised domain adaptive image classification method, this method uses a non-robust source domain model to obtain a robust model on the target domain when the source domain data is completely missing. The model is trained by the knowledge distillation + adversarial learning method, which can significantly improve the robustness of the model.

[0056] 2. This method does not require the use of source domain data or pre-trained robust models. Only using non-robust source domain models can still obtain robust models on the target domain, making many source domain models suitable for robust unsupervised domain transfer adaptation applications in the real world, and has wide applicability.

[0057] 3. This method uses two unsupervised domain adaptive adversarial distillation methods for training, which alleviates the problem of overfitting, can improve the model's category discrimination ability and image classification performance, and has a higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 Flow chart of a robust unsupervised domain adaptive image classification method based on adversarial distillation in an embodiment of the present invention;

[0060] Figure 2 4 is a structural diagram of a robust unsupervised domain adaptive image classification method based on adversarial distillation in an embodiment of the present invention;

[0061] Figure 3 4 is a structural diagram of a robust unsupervised domain adaptive image classification system based on adversarial distillation in an embodiment of the present invention;

[0062] Figure 4 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0064] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0065] Knowledge distillation is a common method for model compression. Different from pruning and quantization in model compression, knowledge distillation is to build a lightweight small model and use the supervisory information of a large model with better performance to train the small model in order to achieve better performance and accuracy. The large model is called the teacher model, and the small model is called the student model. The supervisory information output from the teacher model is called knowledge, and the process of the student model learning to transfer the supervisory information from the teacher model is called distillation. In the field of transfer learning, knowledge distillation is a common transfer learning method.

[0066] Therefore, the present invention combines knowledge distillation and adversarial training, and uses the adversarial distillation method to train a robust target domain student model. In the case of a complete lack of source domain data, a robust model on the target domain can be obtained using only a non-robust source domain model, making many source domain models suitable for robust unsupervised domain transfer adaptation applications in the real world.

[0067] like Figure 1 , Figure 2 As shown, the robust unsupervised domain adaptive image classification method based on adversarial distillation in this embodiment includes the following steps:

[0068] S1. Obtain a natural sample set of unsupervised target domain;

[0069] S2. Construct a robust unsupervised domain adaptive image classification framework, including a non-robust target domain teacher model and a robust target domain student model;

[0070] After the robust unsupervised domain adaptive image classification framework is constructed, the interval learning theory is used to estimate the upper bound of the learning error of the model in the target domain adversarial training process under the condition of no source domain data, and a more reasonable and effective learning target is heuristically designed under the condition of no source domain data; the objective function of the robust unsupervised adaptive image classification framework in this embodiment is derived based on the interval difference divergence under the condition of no source domain data, specifically:

[0071] According to the interval learning theory, for any score function f, it satisfies:

[0072]

[0073] in is an ideal interval loss, is the score function f in the target adversarial domain The classification error based on 0-1 loss, is the score function f in the source domain The classification error is spaced by a constant ρ. is the source domain spaced by a constant ρ and target domain The interval difference divergence, is the target domain separated by a constant ρ and target adversarial domain The interval difference divergence of

[0074] It is known that there is the following lemma, that is, for any score function f, it satisfies

[0075]

[0076] and Satisfies the triangle inequality Here are the reasons:

[0077]

[0078] Then the theorem in formula (1) is proved;

[0079] Since the ultimate goal of the present invention is to obtain a robust target domain student model with good image discrimination ability through adversarial distillation training, it is necessary to set Classification error based on 0-1 loss To achieve the minimum optimal score function f, according to the right side of formula (1), we can get:

[0080]

[0081] Under the condition that the source domain data is completely missing, it can be seen that the score function f is The classification error is spaced by a constant ρ is a constant, so according to formula (2), the objective function of the robust unsupervised domain adaptive image classification framework is derived as follows:

[0082]

[0083] in, is the objective function of the non-robust target domain teacher model, is the objective function of the student model in the robust target domain.

[0084] S3, use the pre-trained non-robust source domain model to initialize the parameters of the non-robust target domain teacher model, perform end-to-end iterative training on the unsupervised target domain natural sample set, and obtain the trained non-robust target domain teacher model;

[0085] The learning of the robust unsupervised domain adaptive image classification framework of the present invention is divided into two parts, such as Figure 2 As shown, the first part is transfer learning based on unsupervised domain adaptation, and the second part is adversarial distillation learning based on unsupervised domain adaptation;

[0086] Step S3 is the first part of the robust unsupervised domain adaptive image classification framework in this embodiment, wherein the trained non-robust target domain teacher model is based on the learning objective function of the non-robust target domain teacher model. The training is based on the transfer learning method of unsupervised domain adaptation, which is used to improve the category discrimination of the target domain teacher model for the target domain natural samples under the premise of complete lack of source domain data. Specifically:

[0087] S301. Due to data privacy and security issues, the source domain data cannot be accessed during the domain adaptation stage. Therefore, any existing unsupervised domain adaptation learning model that does not use source domain data is used to perform standard unsupervised domain adaptation learning to obtain a non-robust target domain teacher model.

[0088] S302, initializing the parameters of the non-robust target domain teacher model using the parameters of the pre-trained non-robust source domain model;

[0089] S303, input the unsupervised target domain natural sample set into the non-robust target domain teacher model for end-to-end iterative training, and the training loss function is expressed as:

[0090] S4. Construct a robust target domain student model based on the trained non-robust target domain teacher model, perform adversarial distillation training on the unsupervised target domain natural sample set, obtain the trained robust target domain student model and output the image classification result.

[0091] Step S4 is the second part of the robust unsupervised domain adaptive image classification framework in this embodiment, wherein the trained robust target domain student model is based on the learning objective function of the robust target domain student model The adversarial distillation learning method based on unsupervised domain adaptation is used to improve the class discrimination of the robust target domain student model for adversarial samples in the target domain. Specifically:

[0092] S401, constructing a robust target domain student model using the same structure (including backbone network and classifier) ​​as the non-robust target domain teacher model;

[0093] S402, generating a corresponding adversarial sample for each natural sample in the unsupervised target domain natural sample set according to the parameter information of the robust target domain student model;

[0094] S403, conduct adversarial distillation training. During each iterative training process, the parameters of the non-robust target domain teacher model are fixed, and the robust target domain student model is trained end-to-end to obtain a trained robust target domain student model and output the image classification result.

[0095] In this embodiment, there are two methods for unsupervised domain adaptive adversarial distillation learning, namely:

[0096] 1. Based on the adversarial training method TRADES for the trade-off between robustness and accuracy, adversarial distillation training is performed, specifically:

[0097] The robust target domain student model is constructed using the same structure (including backbone network and classifier) ​​as the non-robust target domain teacher model;

[0098] According to the parameter information of the robust target domain student model, a corresponding adversarial sample is generated for each natural sample in the unsupervised target domain natural sample set. The generation formula is:

[0099]

[0100] Conduct adversarial distillation training. During each iterative training process, the parameters of the non-robust target domain teacher model are fixed, and the robust target domain student model is trained end-to-end to obtain a trained robust target domain student model and output the image classification result.

[0101] The adversarial distillation loss function is established based on the output of the non-robust target domain teacher model and the output of the robust target domain student model, which is expressed as:

[0102]

[0103] where φ represents the robust target domain student model, φ T represents the non-robust target domain teacher model, x is a natural sample in the unsupervised target domain natural sample set, x' is the adversarial sample generated corresponding to x, is the KL divergence loss function, β is a constant coefficient, p is the p-norm, and ∈ is a constant range.

[0104] 2. Use the adversarial training method PGD of projected gradient descent for adversarial distillation training, specifically:

[0105] The robust target domain student model is constructed using the same structure (including backbone network and classifier) ​​as the non-robust target domain teacher model;

[0106] According to the parameter information of the robust target domain student model, a corresponding adversarial sample is generated for each natural sample in the unsupervised target domain natural sample set. The generation formula is:

[0107]

[0108] Conduct adversarial distillation training. During each iterative training process, the parameters of the non-robust target domain teacher model are fixed, and the robust target domain student model is trained end-to-end to obtain a trained robust target domain student model and output the image classification result.

[0109] The adversarial distillation loss function is established based on the output of the non-robust target domain teacher model and the output of the robust target domain student model, which is expressed as:

[0110]

[0111] where φ represents the robust target domain student model, φ T represents the non-robust target domain teacher model, x is a natural sample in the unsupervised target domain natural sample set, x' is the adversarial sample generated corresponding to x, is the KL divergence loss function, β is a constant coefficient, p is the p-norm, and ∈ is a constant range.

[0112] The present invention is based on the unsupervised domain adaptive non-robust target domain teacher model learning of source domain data. In the case of complete lack of source domain data, the parameters of the non-robust target domain teacher model are initialized using a pre-trained non-robust source domain model. Then, the target domain teacher model is iteratively trained through an unsupervised domain adaptive algorithm to improve the category discrimination of natural samples in the target domain. Then, unsupervised adversarial distillation learning is performed on the target domain, the parameters of the non-robust target domain teacher model are fixed, and a robust target domain student model is iteratively trained using unsupervised target domain data. In each round of training iteration, a corresponding adversarial sample is generated for each natural sample in the target domain according to the model parameter information of the student model. Then, a distillation loss function is established based on the output of the teacher model for the natural samples of the target domain and the output of the student model for the corresponding adversarial samples. The parameters of the student model are updated accordingly to improve the category discrimination and image classification performance of the student model for the adversarial samples in the target domain.

[0113] In this embodiment, the experimental details of the robust unsupervised domain adaptive image classification method based on adversarial distillation are as follows:

[0114] 1. The present invention has been effectively verified by conducting a large number of experiments on four natural sample datasets, namely: Office-31 dataset, Office-Home dataset, PACS dataset and VisDA-C dataset;

[0115] 2. On the Office-31, Office-home and PACS datasets, both the non-robust target domain teacher model and the robust target domain student model use ResNet-50 as the backbone network, while on the VisDA-C dataset, both the non-robust target domain teacher model and the robust target domain student model use ResNet-101 as the backbone network; the classifiers are all single fully connected layers;

[0116] 3. Use the stochastic gradient descent method, with an initial learning rate of 0.1 and gradually decrease it using the cosine annealing method; the number of training iterations on the VisDA-C dataset is 50, and the number of training iterations on the other three datasets is 100; the range of adversarial samples ∈ is 8 / 255, and the norm p is ∞; β is set to 1 in the TRADES method;

[0117] 4. The experimental results of the present invention are shown in the following table, where SHOT is the benchmark model, TRADES-AD and PGD-AD are two adversarial distillation training methods of the present invention; classification accuracy refers to the classification accuracy of the target domain model for the target domain natural test samples; robustness refers to the classification accuracy of the target domain model for the target domain adversarial test samples.

[0118]

[0119] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0120] Based on the same idea as the robust unsupervised domain adaptive image classification method based on adversarial distillation in the above-mentioned embodiment, the present invention also provides a robust unsupervised domain adaptive image classification system based on adversarial distillation, which can be used to execute the above-mentioned robust unsupervised domain adaptive image classification method based on adversarial distillation. For ease of explanation, the structural schematic diagram of the embodiment of the robust unsupervised domain adaptive image classification system based on adversarial distillation only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0121] like Figure 3 As shown, another embodiment of the present invention provides a robust unsupervised domain adaptive image classification system 100 based on adversarial distillation, including a data acquisition module 101, a classification framework construction module 102, a teacher model training module 103 and a student model training module 104;

[0122] The data acquisition module 101 is used to obtain an unsupervised target domain natural sample set;

[0123] The classification framework construction module 102 is used to construct a robust unsupervised domain adaptive image classification framework, including a non-robust target domain teacher model and a robust target domain student model;

[0124] The teacher model training module 103 is used to initialize the parameters of the non-robust target domain teacher model using the pre-trained non-robust source domain model, and perform end-to-end iterative training on an unsupervised target domain natural sample set to obtain a trained non-robust target domain teacher model;

[0125] The student model training module 104 is used to construct a robust target domain student model based on the trained non-robust target domain teacher model, perform adversarial distillation training on an unsupervised target domain natural sample set, obtain a trained robust target domain student model and output image classification results.

[0126] Specifically, the teacher model training module 103 is specifically:

[0127] An unsupervised domain adaptation learning model that does not use source domain data is used to perform standard unsupervised domain adaptation learning to obtain a non-robust target domain teacher model;

[0128] Initialize the parameters of the non-robust target domain teacher model using the parameters of the pre-trained non-robust source domain model;

[0129] The unsupervised target domain natural sample set is input into the non-robust target domain teacher model for end-to-end iterative training to obtain a trained non-robust target domain teacher model.

[0130] Specifically, the student model training module 104 is specifically as follows:

[0131] The robust target domain student model is constructed using the same structure as the non-robust target domain teacher model;

[0132] According to the parameter information of the robust target domain student model, a corresponding adversarial sample is generated for each natural sample in the unsupervised target domain natural sample set;

[0133] Adversarial distillation training is performed. During each iterative training process, the parameters of the non-robust target domain teacher model are fixed, and the robust target domain student model is trained end-to-end to obtain the trained robust target domain student model and output the image classification result.

[0134] It should be noted that the robust unsupervised domain adaptive image classification system based on adversarial distillation of the present invention corresponds one-to-one to the robust unsupervised domain adaptive image classification method based on adversarial distillation of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the robust unsupervised domain adaptive image classification method based on adversarial distillation are applicable to the embodiment of the robust unsupervised domain adaptive image classification system based on adversarial distillation. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here. This is hereby declared.

[0135] In addition, in the implementation of the robust unsupervised domain adaptive image classification system based on adversarial distillation in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the robust unsupervised domain adaptive image classification system based on adversarial distillation is divided into different program modules to complete all or part of the functions described above.

[0136] like Figure 4 As shown, in one embodiment, a computer-readable storage medium 200 is provided, in which a program is stored in a memory 201. When the program is executed by a processor 202, the robust unsupervised domain adaptive image classification method based on adversarial distillation is implemented, specifically:

[0137] Obtain a natural sample set of unsupervised target domain;

[0138] Constructing a robust unsupervised domain adaptive image classification framework; the robust unsupervised domain adaptive image classification model includes a non-robust target domain teacher model and a robust target domain student model;

[0139] Use the pre-trained non-robust source domain model to initialize the parameters of the non-robust target domain teacher model, perform end-to-end iterative training on the unsupervised target domain natural sample set, and obtain the trained non-robust target domain teacher model;

[0140] A robust target domain student model is constructed based on the trained non-robust target domain teacher model, and adversarial distillation training is performed on an unsupervised target domain natural sample set to obtain a trained robust target domain student model and output image classification results.

[0141] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0142] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A robust unsupervised domain adaptive image classification method based on adversarial distillation, characterized in that: The steps include: Obtain a natural sample set of unsupervised target domain; Constructing a robust unsupervised domain adaptive image classification framework; the robust unsupervised domain adaptive image classification framework includes a non-robust target domain teacher model and a robust target domain student model; The objective function of the robust unsupervised domain adaptive image classification framework is derived based on the interval difference divergence under the condition of passive domain data, specifically: According to the interval learning theory, for any score function f, it satisfies: in is an ideal interval loss, is the score function f in the target adversarial domain The classification error based on 0-1 loss, is the score function f in the source domain The classification error is spaced by a constant ρ. is the source domain spaced by a constant ρ and target domain The interval difference divergence, is the target domain separated by a constant ρ and target adversarial domain The interval difference divergence of Let in the target adversarial domain Classification error based on 0-1 loss To achieve the minimum optimal score function f, according to the right side of formula (1), we can get: Under the condition that the source domain data is completely missing, it can be seen that the score function f is The classification error is spaced by a constant ρ is a constant, so according to formula (2), the objective function of the robust unsupervised domain adaptive image classification framework is derived as follows: in, is the objective function of the non-robust target domain teacher model, is the objective function of the student model in the robust target domain; Use the pre-trained non-robust source domain model to initialize the parameters of the non-robust target domain teacher model, perform end-to-end iterative training on the unsupervised target domain natural sample set, and obtain the trained non-robust target domain teacher model; The trained non-robust target domain teacher model is specifically: An unsupervised domain adaptation learning model that does not use source domain data is used to perform standard unsupervised domain adaptation learning to obtain a non-robust target domain teacher model; Initialize the parameters of the non-robust target domain teacher model using the parameters of the pre-trained non-robust source domain model; Input the unsupervised target domain natural sample set into the non-robust target domain teacher model for end-to-end iterative training to obtain a trained non-robust target domain teacher model; A robust target domain student model is constructed based on the trained non-robust target domain teacher model, and adversarial distillation training is performed on the unsupervised target domain natural sample set to obtain a trained robust target domain student model and output the image classification result; The trained robust target domain student model is specifically: The robust target domain student model is constructed using the same structure as the non-robust target domain teacher model; According to the parameter information of the robust target domain student model, a corresponding adversarial sample is generated for each natural sample in the unsupervised target domain natural sample set; Adversarial distillation training is performed. During each iterative training process, the parameters of the non-robust target domain teacher model are fixed, and the robust target domain student model is trained end-to-end to obtain the trained robust target domain student model and output the image classification result.

2. The robust unsupervised domain adaptive image classification method based on adversarial distillation according to claim 1, characterized in that: Adversarial distillation training is performed based on the adversarial training method TRADES, which is aimed at balancing robustness and accuracy; The generation formula of the adversarial sample is: The adversarial distillation loss function is established based on the output of the non-robust target domain teacher model and the output of the robust target domain student model, and is expressed as: where φ represents the robust target domain student model, φ T represents the non-robust target domain teacher model, x is a natural sample in the unsupervised target domain natural sample set, x' is the adversarial sample generated corresponding to x, is the KL divergence loss function, β is a constant coefficient, p is the p-norm, and ϵ is a constant range.

3. The robust unsupervised domain adaptive image classification method based on adversarial distillation according to claim 2, characterized in that: Use the projected gradient descent adversarial training method PGD for adversarial distillation training; The generation formula of the adversarial sample is: The adversarial distillation loss function is established based on the output of the non-robust target domain teacher model and the output of the robust target domain student model, and is expressed as: where φ represents the robust target domain student model, φ T represents the non-robust target domain teacher model, x is a natural sample in the unsupervised target domain natural sample set, x' is the adversarial sample generated corresponding to x, is the KL divergence loss function, β is a constant coefficient, p is the p-norm, and ϵ is a constant range.

4. A robust unsupervised domain adaptive image classification system based on adversarial distillation, characterized in that: A robust unsupervised domain adaptive image classification method based on adversarial distillation applied to any one of claims 1-3, comprising a data acquisition module, a classification framework construction module, a teacher model training module and a student model training module; The data acquisition module is used to obtain an unsupervised target domain natural sample set; The classification framework construction module is used to construct a robust unsupervised domain adaptive image classification framework; the robust unsupervised domain adaptive image classification framework includes a non-robust target domain teacher model and a robust target domain student model; The objective function of the robust unsupervised domain adaptive image classification framework is derived based on the interval difference divergence under the condition of passive domain data, specifically: According to the interval learning theory, for any score function f, it satisfies: in is an ideal interval loss, is the score function f in the target adversarial domain The classification error based on 0-1 loss, is the score function f in the source domain The classification error is spaced by a constant ρ. is the source domain spaced by a constant ρ and target domain The interval difference divergence, is the target domain separated by a constant ρ and target adversarial domain The interval difference divergence of Let in the target adversarial domain Classification error based on 0-1 loss To achieve the minimum optimal score function f, according to the right side of formula (1), we can get: Under the condition that the source domain data is completely missing, it can be seen that the score function f is The classification error is spaced by a constant ρ is a constant, so according to formula (2), the objective function of the robust unsupervised domain adaptive image classification framework is derived as follows: in, is the objective function of the non-robust target domain teacher model, is the objective function of the student model in the robust target domain; The teacher model training module is used to initialize the parameters of the non-robust target domain teacher model using the pre-trained non-robust source domain model, and perform end-to-end iterative training on an unsupervised target domain natural sample set to obtain a trained non-robust target domain teacher model; The trained non-robust target domain teacher model is specifically: An unsupervised domain adaptation learning model that does not use source domain data is used to perform standard unsupervised domain adaptation learning to obtain a non-robust target domain teacher model; Initialize the parameters of the non-robust target domain teacher model using the parameters of the pre-trained non-robust source domain model; Input the unsupervised target domain natural sample set into the non-robust target domain teacher model for end-to-end iterative training to obtain a trained non-robust target domain teacher model; The student model training module is used to construct a robust target domain student model based on the trained non-robust target domain teacher model, perform adversarial distillation training on an unsupervised target domain natural sample set, obtain a trained robust target domain student model and output an image classification result; The trained robust target domain student model is specifically: The robust target domain student model is constructed using the same structure as the non-robust target domain teacher model; According to the parameter information of the robust target domain student model, a corresponding adversarial sample is generated for each natural sample in the unsupervised target domain natural sample set; Adversarial distillation training is performed. During each iterative training process, the parameters of the non-robust target domain teacher model are fixed, and the robust target domain student model is trained end-to-end to obtain the trained robust target domain student model and output the image classification result.

5. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the robust unsupervised domain adaptive image classification method based on adversarial distillation according to any one of claims 1 to 3 is implemented.

Images