Deep Hash model adversarial training method and system based on representation distillation

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to a deep hash model adversarial training method and system based on representation distillation. The method comprises the following steps: 1) using a clean sample to train a deep hash model having the same network structure as a student model as a teacher model; 2) iteratively updating an adversarial sample by taking maximization of a distance with semantic representation output by the teacher model as a loss function; 3) giving a generated adversarial sample and an original clean sample, and realizing semantic representation distillation from the teacher model to the student model by using a hash code alignment loss function and a feature alignment loss function so as to guide the adversarial training process of the student model; and 4) alternately carrying out the step 2) and the step 3) until the maximum iteration round is achieved. According to the method, discriminative and generalization semantic information learned from a clean sample by a teacher model can be fully stored, so that the problem of poor model robustness generalization caused by the fact that the semantic information cannot be fully learned by an existing method is solved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a deep hash model adversarial training method and system based on representation distillation. Background Art

[0002] With the rapid development of artificial intelligence and big data, Internet visual data has exploded, giving rise to the application demand of large-scale visual retrieval. How to efficiently store and analyze huge visual data and provide customized query services for users has become a highly concerned issue in academia and industry. Deep hashing technology uses deep neural networks to learn compact hash codes to manage and analyze visual big data. It has the advantages of small storage space and fast retrieval speed. It has gradually become a key technology for solving large-scale visual retrieval problems and has been widely used in various fields. However, in recent years, researchers have shown that deep hashing models inevitably inherit the fragility of deep neural networks and are very vulnerable to adversarial samples. By adding adversarial perturbations that are difficult for the human eye to detect to the query data or database data, adversarial samples can interfere with the deep hashing model and return erroneous and irrelevant retrieval results. This phenomenon directly threatens the robustness and reliability of image retrieval systems based on deep hashing, and brings great security challenges to its practical application. Therefore, studying adversarial defense methods for deep hashing models and directly improving the robustness of deep hashing models, that is, enhancing the ability of deep hashing models to resist adversarial attacks, has important theoretical and practical significance for various practical applications based on deep hashing models.

[0003] The current adversarial defense methods are mainly divided into three categories, including active defense based on adversarial training and adversarial purification and passive defense methods based on adversarial sample detection. Among them, adversarial training refers to a method in which adversarial samples are added to the model training in the form of data enhancement during the model training process, thereby directly improving the robustness of the model against adversarial samples. Adversarial training is usually modeled as a min-max optimization problem. Among them, the max part is an internal maximization optimization problem, which generates strong adversarial samples for model training by maximizing the prediction error loss, and the min part is an external minimization optimization problem, which trains the model parameters by using clean samples and adversarial samples generated by the max part to directly reduce the model's misjudgment of adversarial samples, thereby maintaining accurate predictions. Adversarial training is regarded as the most effective method to improve the adversarial robustness of deep neural networks in adversarial defense methods. However, the current adversarial training methods for deep hashing models face the serious problem of poor robust generalization, that is, the deep hashing model has poor robustness performance on unseen test data. Therefore, studying how to effectively improve the robust generalization of deep hashing models is an urgent problem to be solved.

[0004] Compared with adversarial attack methods, there are currently fewer explorations of adversarial training methods for deep hashing models, mainly two works, namely ATRDH and CgAT methods. Among them, the ATRDH method solves the internal maximization problem. First, the deep hashing model is fixed to train the prototype network, and then the prototype network is used to generate the target category anchor point, and then the targeted adversarial samples are generated under its supervision; then in the external minimization problem, the deep hashing model is trained by combining the original deep hashing loss function and narrowing (pushing away) the distance between the adversarial samples of the same (different) category and the clean samples. However, the random sampling of the target category anchor point by this method cannot guarantee the quality of the generated adversarial samples. To solve this problem, the CgAT method proposes to generate non-targeted adversarial samples with the optimization goal of being as far away from the own category anchor point as possible, and then trains a robust deep hashing model by combining the original deep hashing model training loss and narrowing the Hamming distance between the adversarial sample and the own category anchor point. However, the above methods do not take into account various model regularization methods, which limits the robust generalization of the model. In addition, existing methods dynamically narrow (pull apart) the Hamming distance between adversarial samples of the same (different) category and clean samples during the training process. Since adversarial samples greatly destroy the discriminative semantics of clean samples, in order to narrow the distance between adversarial samples and clean samples, the model is prone to discard some discriminative and generalizable semantics during the adversarial training process, thereby limiting the robust generalization of the model. Summary of the invention

[0005] The purpose of the present invention is to propose an adversarial training method based on representation distillation, that is, introducing semantic representation distillation from a teacher model to a student model during the adversarial training process, thereby regularizing the student model during the adversarial training process to improve the robust generalization of the student model.

[0006] The basic process of the present invention is: first, a clean deep hash model is trained, that is, a deep hash model that is only trained on clean samples as a teacher model. The teacher model can learn rich semantic information from clean samples to distinguish the similarity and dissimilarity of samples; then the hash code output by the teacher model and the semantic representation of fine-grained features are used to guide the adversarial training process of the student model. The ultimate goal of the present invention is to improve the robustness of the deep hash model on unseen data, that is, the correct prediction and retrieval of adversarial samples as query data.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A deep hashing model adversarial training method based on representation distillation includes the following steps:

[0009] 1) Use clean samples to train a deep hashing model with the same network structure as the student model as the teacher model;

[0010] 2) Iteratively update the adversarial sample by maximizing the distance to the semantic representation output by the teacher model as the loss function;

[0011] 3) Given the generated adversarial samples and the original clean samples, the hash code alignment loss function and feature alignment loss function are used to realize the semantic representation distillation from the teacher model to the student model to guide the adversarial training process of the student model;

[0012] 4) Step 2) and step 3) are performed alternately until the maximum number of iterations is reached.

[0013] Furthermore, the training process of the teacher model includes: selecting the same network structure as the student model for the teacher model, and then using clean samples as training data sets to train the teacher model; the teacher model adopts a deep hash model and uses a loss function to preserve the similarity and dissimilarity between samples into a binary hash code; the teacher model first uses a deep neural network to extract high-dimensional image features, and then reduces the dimension of the high-dimensional features and then quantizes them to obtain a binary hash code, and uses quantization loss to ensure that the information loss in the process of quantization into a discrete binary hash code is as small as possible.

[0014] Furthermore, the hash code alignment loss function is used to shorten the Hamming distance between the hash codes output by the student model and the teacher model.

[0015] Furthermore, the feature alignment loss function better enhances the ability of the student model to preserve the discriminative and generalizable semantic information of the teacher model by bringing the intermediate layer feature representations of the student model and the teacher model closer.

[0016] Furthermore, the adversarial sample is generated by iteratively updating the output hash code of the teacher model as far away from the hash code as possible in the Hamming space.

[0017] A deep hash-based image retrieval method comprises the following steps:

[0018] Get pictures as training data sets;

[0019] Using the training data set, the teacher model and the student model are trained using the above method;

[0020] The trained student model is used to hash the images in the image database, and image retrieval is performed based on the hash codes.

[0021] A deep hashing model adversarial training system based on representation distillation, comprising:

[0022] The teacher model pre-training module is used to use clean samples to train a deep hashing model with the same network structure as the student model as the teacher model;

[0023] An adversarial sample generation module is used to iteratively update adversarial samples by maximizing the distance to the semantic representation output by the teacher model as the loss function;

[0024] The adversarial training module is used to implement semantic representation distillation from the teacher model to the student model given the generated adversarial samples and the original clean samples using the hash code alignment loss function and the feature alignment loss function to guide the adversarial training process of the student model until the maximum number of iterations is reached.

[0025] The beneficial effects of the present invention are as follows:

[0026] Compared with the existing methods, the present invention can significantly improve the robust generalization performance of the deep hash model while ensuring that the accuracy remains unchanged. Specifically, inspired by the fact that knowledge distillation can be regarded as a model regularization, thereby improving the standard generalization of the model, the present invention introduces the idea of ​​knowledge distillation in the adversarial training method of the deep hash model for the first time, thereby improving the adversarial robust generalization of the model. The present invention adopts a deep hash model with the same network structure as the student model and only performs standard training on clean samples as the teacher model, without the need to train a robust large model as the teacher model, so the training cost of the teacher model is relatively low. The present invention proposes hash code alignment loss and intermediate layer feature alignment loss to distill the two semantic representations, so as to fully preserve the discriminative and generalizable semantic information learned by the teacher model from the clean samples, thereby solving the problem that the existing methods cannot fully learn these semantic information, resulting in poor robust generalization of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is the adversarial training process of the CRDAT method.

[0028] Figure 2 It is the robustness performance of model training and testing during the training process of different methods.

[0029] Figure 3 It is the impact of distillation of different layer features on robustness.

[0030] Figure 4 This is the plane visualization result of the loss function. DETAILED DESCRIPTION

[0031] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below through specific embodiments and drawings.

[0032] like Figure 1As shown, the basic process of the adversarial training method based on representation distillation (Clean model Representation Distillation based Adversarial Training, CRDAT) proposed in the present invention is: first train a clean deep hashing model as the teacher model, then fix the parameters of the teacher model unchanged, and use the output of the teacher model to guide the adversarial training of the student model. The steps of the adversarial training method proposed in the present invention include: 1) using the standard deep hashing method on clean samples to train a teacher model with the same network structure as the student model; 2) in order to generate stronger adversarial samples to promote the adversarial training of the model, the present invention uses the distance between the semantic representation and the output of the teacher model as the loss function to iteratively update the adversarial samples; 3) given the adversarial samples generated in the second step and the original clean samples, the proposed hash code alignment loss and feature alignment loss are used to realize the semantic representation distillation from the teacher model to the student model, guiding the adversarial training process of the student model; 4) the second step and the third step are performed alternately until the maximum number of iterations is reached.

[0033] 1. Background knowledge:

[0034] Given a clean sample training dataset It contains N training data, x i represents the i-th training data, y i =[y i1 ,y i2 , ..., y iC ]∈{0,1} C Represents the corresponding label, which contains C categories in total. Represents a trained deep hashing model, given a query image x i , the corresponding hash code b i It can be directly obtained according to formula (1):

[0035]

[0036] stb i ∈{-1, 1} K , (1)

[0037] where f θ (x i ) is the output of the last hash layer of the deep hash model, θ represents the parameters of the deep hash model, and K represents the hash code length.

[0038] Teacher model pre-training:

[0039] In order to improve the student model to be trained The robust generalization of the teacher model Select the same network structure as the student model, where g ψ (x i ) represents the teacher model’s response to input x i The prediction results are then used as the training data set to train the teacher model. The deep hash model uses a loss function to save the similarity and dissimilarity between samples into a binary hash code. Specifically, a deep neural network is first used to extract high-dimensional image features, and then the high-dimensional features are reduced in dimension and quantized to obtain a binary hash code. In order to ensure that the information loss in the process of quantization into discrete binary hash codes is as small as possible, the quantization loss is usually also an important part of the deep hash model training loss function. For example: The loss function of the classic deep pairwise supervised hashing (DPSH) method based on pairwise supervision information is shown in formula (2):

[0040]

[0041] Among them, ψ represents the parameters of the teacher model, s ij Represents sample x i and x j The semantic similarity of the two, 1 represents similarity, 0 represents dissimilarity, h i =tanh(g ψ (xi)), b i =sign(h i ), And α is a hyperparameter.

[0042] (II) External minimization-model training:

[0043] Using the output of the teacher model and the generated adversarial samples and their corresponding clean samples, the student model can be trained to improve the robustness of the model against adversarial samples. Specifically, it is expected that the student model can have the same semantic representation and retrieval results for both clean samples and adversarial samples, and can fully preserve the rich semantic information of discriminating similarity and dissimilarity of samples learned by the teacher model from clean samples without discarding it during adversarial training. Since the deep hash model uses the output hash code to retrieve semantically similar samples in the database, the present invention first proposes a hash code alignment loss to shorten the Hamming distance between the hash codes output by the student model and the teacher model. The hash code alignment loss function (Hashcode Alignment Loss, HAL) is shown in formula (3).

[0044]

[0045] Among them, x′ is the adversarial example corresponding to x, represents the result of calculating the hash code alignment loss for the clean sample x, represents the result of calculating the hash code alignment loss for the adversarial sample x′, λ represents the hyperparameter that weighs the two losses, and d H (·,·) represents the Hamming distance between the outputs of the teacher model and the student model.

[0046] In order to provide a more fine-grained feature semantic representation as guidance, the present invention proposes a feature alignment loss function, which can better enhance the student model's ability to preserve the discriminative and generalizable semantic information of the teacher model by bringing the intermediate layer feature representations of the student model and the teacher model closer. The feature alignment loss function (Feature Alignment Loss, FAL) is shown in formula (4).

[0047]

[0048] in, represents the output feature of the i-th layer of the student model for x′, Represents the output features of the i-th layer of the teacher model for x.

[0049] Specifically, in order to select the optimal deep neural network output layer for guidance, the present invention conducts the following analysis. First, the shallow layer of the deep neural network learns some information such as color, texture, edge, etc. from the input data distribution, while the high layer will learn some more abstract semantic information related to the task. Then, since the clean sample and the adversarial sample are in different manifold distributions, the shallow information learned by the clean model and the robust model trained using them will be quite different. However, since the training tasks of the clean model and the robust model are the same, that is, the same deep hash loss function is used for training, the high-level semantic representations required for judgment by the two models will have greater similarity. Therefore, the present invention believes that aligning the shallow outputs of the teacher model and the student model will destroy the adversarial training results of the model, making it impossible to learn robust data features from the adversarial samples; on the contrary, aligning the high-level outputs of the teacher model and the student model will fully preserve the discriminative and generalizable semantic information of the teacher model, thereby improving the robust performance of the student model. Therefore, the present invention selects the output of the last fully connected layer of the deep hash model to calculate the feature alignment loss function to provide more refined guidance information.

[0050] In summary, the overall loss function of the external minimization part of the present invention is as shown in formula (5), where μ represents a hyperparameter.

[0051]

[0052] (III) Internal maximization-adversarial sample generation:

[0053] Since adversarial training is defined as a min-max training method, the stronger the adversarial sample is, the more robust the model using it for adversarial training will be. In order to generate stronger adversarial samples, the present invention proposes to use the output hash code of the teacher model as a guide and iteratively update the hash code as far away from it as possible in the Hamming space to generate stronger adversarial samples. The specific internal maximization loss function for generating adversarial samples is shown in formula (6):

[0054]

[0055] Where x′ is the adversarial sample corresponding to x, and ∈ represents the maximum perturbation amount. The purpose of this loss function is to destroy the original discriminative semantic information contained in the adversarial sample as much as possible, so that the model makes a misjudgment and retrieves as many samples of the wrong category as possible. In order to further improve the quality of generating adversarial samples, the present invention adopts the PGD algorithm to update the adversarial sample by iteratively calculating the gradient of the adversarial sample multiple times. The specific calculation process of the tth time is shown in formula (7):

[0056]

[0057] Among them, x′ t represents the adversarial sample generated by the tth update, η is the update step size of PGD, represents the gradient of HAL loss with respect to x′, and x′0 represents the initial point of adversarial sample update, that is, the clean sample x.

[0058] The key points of the present invention are:

[0059] 1. The present invention proposes a new adversarial training method for deep hash models based on representation distillation. It combines the idea of ​​knowledge distillation with the adversarial training method for deep hash models for the first time, and significantly improves the robust generalization of the model by regularizing the adversarial training process of the model.

[0060] 2. This paper proposes a new representation distillation method, namely the proposed hash code alignment loss and feature alignment loss. By distilling the semantic representation of the teacher model into the student model to guide the adversarial training process of the student model, the student model can fully preserve the discriminative semantic information of the teacher model during the adversarial training process.

[0061] 3. The method proposed in the present invention not only provides a new adversarial training method for deep hashing models, but also a large number of experiments have proved that the method of the present invention significantly improves the robustness and generalization of the model compared with the existing methods.

[0062] Specific application scenarios of the present invention:

[0063] The deep hash model adversarial training method based on representation distillation of the present invention can be used in fields such as image retrieval, and can improve the robustness and reliability of the deep hashing-based image retrieval system, and enhance the ability of the deep hashing model of the image retrieval system to resist adversarial attacks.

[0064] For example, one embodiment of the present invention provides an image retrieval method based on deep hashing, the steps of which include:

[0065] 1) Obtain pictures as training data sets;

[0066] 2) Using the training data set, the teacher model and the student model are trained using the deep hash model adversarial training method based on representation distillation of the present invention;

[0067] 3) Using the trained student model to perform image retrieval: Specifically, using the trained student model to perform hash coding on the images in the image database, and performing image retrieval based on the hash coding.

[0068] Effects of the present invention:

[0069] Dataset: The present invention selects three commonly used datasets in the field of deep hashing to verify the effectiveness of the method of the present invention, namely FLICKR-25K, NUS-WIDE and MS-COCO datasets. FLICKR-25K contains a total of 25,000 images belonging to 38 categories. 1,700 images are randomly sampled from all the images as query datasets, and the other data constitute the database. In addition, 5,000 images are randomly sampled from the database as training datasets to train clean teacher models and robust student models. The NUS-WIDE dataset contains 269,648 images belonging to 81 categories, and 195,834 images from the 21 most commonly used categories are selected for training and testing. Specifically, 100 images are selected for each of the 21 categories as query datasets, and the other data constitute the database. In addition, 500 images are selected from each category in the database data to form a training dataset. The MS-COCO dataset contains a total of 82,783 training data and 40,504 verification data, and each image is labeled with 80 categories. The training data and validation data consist of a total of 122,218 images, from which 5,000 images are randomly selected as query data, and the rest of the data constitute the database. In addition, 10,000 images are randomly selected from the database to form the training data set.

[0070] Target model: The present invention adopts the AlexNet network as the network structure of the teacher and student deep hashing models by default, and uses the DPSH deep hashing method to train a clean teacher model.

[0071] Evaluation indicators: Targeted mean Average Precision (t-mAP) and mean Average Precision (mAP) are used as evaluation indicators to evaluate the ability of the trained robust model to resist untargeted adversarial attacks and targeted adversarial attacks respectively. t-mAP directly reflects the ability of adversarial samples with targeted adversarial perturbations to retrieve target category data.

[0072] Overall experimental results: In order to verify the improvement effect of the present invention on the robust generalization of deep hash models under various experimental settings, the present invention compares all the current adversarial training methods for deep hash models, namely ATRDH and CgAT methods, and uses the current strong non-targeted adversarial attack methods, such as HAG, SDHA and CgAA methods, and targeted adversarial attack methods, such as P2P, DHTA and THA methods, to evaluate the ability of robust deep hash models to resist these adversarial attacks. The robustness performance of the (student) deep hash model trained on three datasets, FLIKCR-25K, NUS-WIDE and MS-COCO, in the face of non-targeted adversarial attacks is shown in Table 1. Among them, the teacher deep hash model is trained using the DPSH method, and the network structures of the teacher model and the student model are both AlexNet, with hash code lengths of 16 bits, 32 bits and 64 bits, respectively. According to the experimental results, the following conclusions can be drawn: (1) Since the standard-trained deep hash model is not trained with adversarial samples during the training phase, it is very vulnerable to adversarial sample attacks and has the worst robustness performance on the three non-targeted adversarial attack methods; (2) Compared with the other two adversarial attack methods, namely ATRDH and CgAT methods, the CRDAT method of the present invention significantly improves the ability of the robust deep hash model to resist non-targeted adversarial attacks. For example, when facing the CgAA adversarial attack method, the method of the present invention improves the retrieval accuracy by 20%, 26% and 21% on the deep hash model with 32 bits trained on the FLICKR-25K, NUS-WIDE and MS-COCO datasets, respectively. This shows that the method of the present invention can significantly improve the robustness of the deep hash model. The robustness of the above-mentioned robust deep hash model against targeted adversarial attacks is shown in Table 2. According to the experimental results, the following conclusions can be drawn: the robust deep hash model trained by the method of the present invention can also resist targeted adversarial attacks well, and compared with other methods, the robustness of the deep hash model against targeted adversarial attack methods is significantly improved. In addition, the training and testing robustness performance of the adversarial training model during the training process is plotted, and the results are shown as follows: Figure 2As can be seen from the figure, the gap between the training and test robustness of the method of the present invention is the smallest, which further illustrates that the method of the present invention can effectively improve the robust generalization of the adversarial training model.

[0073] Ablation experiment: In order to verify the effect of different components of the CRDAT method of the present invention on improving the robustness of the deep hashing model, a series of ablation analysis experiments were conducted. Specifically, in this part of the analysis experiment, a clean deep hashing model with AlexNet as the network structure and 32 bits trained on the FLICKR-25K dataset using the DPSH deep hashing method was used as the teacher model.

[0074] Effectiveness of different parts: In order to verify the effectiveness of different components of the method, an experiment was conducted to examine the changes in model robustness after removing and replacing different components of the method of the present invention. The experimental results are shown in Table 3, from which the following conclusions can be drawn: 1) First, after replacing the internal maximization adversarial sample generation method of the present invention with the internal maximization adversarial sample generation method of the CgAT method, the robustness of the student model trained using the adversarial sample has decreased to a certain extent, indicating that the internal maximization optimization method of the method of the present invention can generate stronger adversarial samples, thereby promoting the improvement of the robustness of the adversarial training model. 2) Then, by replacing the external minimization optimization method of the present invention, that is, the adversarial training method based on representation distillation, with the adversarial training method of ATRDH, the robustness of the student model showed a significant decrease, which fully demonstrated the effectiveness of the adversarial training method based on representation distillation of the present invention, which significantly improved the robust generalization of the model by regularizing the training process of the model. 3) Finally, by removing the hash code alignment loss and feature alignment loss of the present invention respectively, the robustness of the student model has decreased significantly, which fully demonstrated the effectiveness and necessity of these two loss functions.

[0075] The impact of different layer features on model robustness: In order to test the impact of different layer feature alignment losses on the robustness of the student model, the following analysis experiments were conducted. The experimental results are as follows: Figure 3 As shown. C1 represents the first convolutional layer of the AlexNet network, A represents the average pooling layer, and L1 to L3 represent the first to third fully connected layers, respectively. The following conclusions can be drawn from the experimental results: when the shallow features of the clean model are used for distillation, the robustness of the student model is significantly reduced, while when the high-level features are used for distillation, the robustness of the student model is improved. This experimental result shows that the shallow features of the clean model and the robust model are inconsistent. When the distance between the shallow features is forcibly pulled in, it will affect the learning of the model's robustness, and the behavior closer to the clean model, the high-level features of the clean model and the robust model are consistent. When the distance between the high-level features is pulled in, the shallow layer of the model can fully learn the robust features and maintain the discriminability of the high-level features.

[0076] Loss plane visualization: In order to intuitively demonstrate the improvement effect of the adversarial training method of the present invention on the robustness of the deep hash model, the loss function planes of the models trained by different adversarial training methods are visualized, where the smoother the loss function plane, the better the model robustness. Figure 4 As shown, the method of the present invention can train a model with a smoother loss function plane, which verifies the robustness improvement effect of the method of the present invention from another perspective.

[0077] Table 1. Robustness of adversarial training methods against untargeted attacks on different datasets

[0078]

[0079] Table 2. Robustness of adversarial training methods against untargeted attacks on different datasets

[0080]

[0081] Table 3 Ablation experiments on effectiveness of different parts

[0082]

[0083] Another embodiment of the present invention provides a deep hashing model adversarial training system based on representation distillation, comprising:

[0084] The teacher model pre-training module is used to use clean samples to train a deep hashing model with the same network structure as the student model as the teacher model;

[0085] An adversarial sample generation module is used to iteratively update adversarial samples by maximizing the distance to the semantic representation output by the teacher model as the loss function;

[0086] The adversarial training module is used to implement semantic representation distillation from the teacher model to the student model given the generated adversarial samples and the original clean samples using the hash code alignment loss function and the feature alignment loss function to guide the adversarial training process of the student model until the maximum number of iterations is reached.

[0087] The specific implementation process of each module refers to the above description of the method of the present invention.

[0088] Another embodiment of the present invention provides a computer device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0089] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0090] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. A deep hashing model adversarial training method based on representation distillation, characterized in that: The following steps are involved: 1) Use clean samples to train a deep hashing model with the same network structure as the student model as the teacher model; 2) Iteratively update the adversarial sample by maximizing the distance to the semantic representation output by the teacher model as the loss function; 3) Given the generated adversarial samples and the original clean samples, the hash code alignment loss function and feature alignment loss function are used to realize the semantic representation distillation from the teacher model to the student model to guide the adversarial training process of the student model; 4) Step 2) and step 3) are performed alternately until the maximum number of iterations is reached.

2. The method according to claim 1, characterized in that The training process of the teacher model includes: selecting the same network structure as the student model for the teacher model, and then using clean samples as training data sets to train the teacher model; the teacher model adopts a deep hash model and uses a loss function to save the similarity and dissimilarity between samples into binary hash codes; the teacher model first uses a deep neural network to extract high-dimensional image features, then reduces the dimension of the high-dimensional features and then quantizes them to obtain binary hash codes, and uses quantization loss to ensure that the information loss in the process of quantization into discrete binary hash codes is as small as possible.

3. The method according to claim 1, characterized in that The hash code alignment loss function is used to shorten the Hamming distance between the hash codes output by the student model and the teacher model. The calculation formula of the hash code alignment loss function is as follows: Among them, x′ is the adversarial sample corresponding to the clean sample x, represents the result of calculating the hash code alignment loss for the clean sample x, represents the result of calculating the hash code alignment loss for the adversarial sample x′, λ is a hyperparameter, d H represents the Hamming distance between the output of the teacher model and the student model, S(x) represents the student model, T(x) represents the teacher model, and f θ (x i ) is the output of the last hash layer of the student model, and K represents the hash code length.

4. The method according to claim 3, characterized in that The feature alignment loss function is to better enhance the ability of the student model to preserve the discriminative and generalizable semantic information of the teacher model by bringing the intermediate layer feature representations of the student model and the teacher model closer. The calculation formula of the feature alignment loss function is as follows: in, represents the output features of the i-th layer of the student model for the input x′, Represents the output features of the i-th layer of the teacher model for the input x.

5. The method according to claim 4, characterized in that The output of the last fully connected layer of the deep hashing model is used to calculate the feature alignment loss function to provide more detailed guidance information.

6. The method according to claim 3, characterized in that Taking the output hash code of the teacher model as a guide, the adversarial sample is generated by iteratively updating as far away from the hash code as possible in the Hamming space, where the loss function for generating adversarial samples is: Where x′ is the adversarial sample corresponding to x, ∈ represents the maximum perturbation amount, and the purpose of this loss function is to destroy the original discriminative semantic information contained in the adversarial sample as much as possible, so as to make the model make misjudgments and retrieve as many samples of the wrong category as possible; then the PGD algorithm is used to update the adversarial sample by calculating the gradient of the adversarial sample through multiple iterations.

7. A deep hash-based image retrieval method, characterized in that: The following steps are involved: Get pictures as training data sets; Using a training data set, training a teacher model and a student model using the method described in any one of claims 1 to 6; The trained student model is used to hash the images in the image database, and image retrieval is performed based on the hash codes.

8. A deep hashing model adversarial training system based on representation distillation, characterized in that: include: The teacher model pre-training module is used to use clean samples to train a deep hashing model with the same network structure as the student model as the teacher model; An adversarial sample generation module is used to iteratively update adversarial samples by maximizing the distance to the semantic representation output by the teacher model as the loss function; The adversarial training module is used to implement semantic representation distillation from the teacher model to the student model given the generated adversarial samples and the original clean samples using the hash code alignment loss function and the feature alignment loss function to guide the adversarial training process of the student model until the maximum number of iterations is reached.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images