A dynamically scalable persistent adversarial defense method
By constructing a dynamic and scalable continuous adversarial defense architecture and a pseudo-task knowledge replay strategy with feature space enhancement, the problems of low training capability and low accuracy of existing models in the face of diverse attacks are solved, and dynamic adaptation to adversarial attacks and efficient training are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NORTHWESTERN POLYTECHNICAL UNIV
- Filing Date
- 2025-01-25
- Publication Date
- 2026-05-15
AI Technical Summary
Existing continuous adversarial defense models share a large number of parameters when facing different adversarial attack methods. The large amount of training data stored leads to low model training capability and difficulty in coping with diverse attack strategies, resulting in low accuracy of training results.
Construct a dynamic and scalable continuous adversarial defense architecture, including multiple defense experts and routers, dynamic routing and classifiers for defense experts, and dynamically adjust model parameters to cope with new types of attacks through a pseudo-task knowledge replay strategy enhanced by feature space.
This approach enables the model to dynamically adapt to different adversarial attacks, mitigates the suboptimal performance issues caused by shared parameters, avoids high memory consumption and privacy protection problems, and improves the accuracy of training results.
Smart Images
Figure CN120032206B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and more specifically, to a dynamic and scalable persistent adversarial defense method. Background Technology
[0002] Deep neural networks are vulnerable to adversarial attacks. Attackers generate adversarial examples by adding carefully designed, imperceptible perturbations to the input image, deceiving the model and causing it to produce incorrect outputs. Adversarial examples pose a significant security threat to systems based on deep learning models. For example, attackers design attack algorithms to create adversarial glasses to mislead facial recognition systems, causing misidentification and enabling unauthorized access or identity impersonation, threatening public safety and personal privacy. To enhance the robustness of deep learning models against adversarial examples, researchers have proposed various defense mechanisms, including defensive distillation and adversarial example detection, to ensure the reliability of models in practical applications. Training, as an effective defense strategy, incorporates adversarial examples into the model training process, enabling the model to learn adversarial features and thus improve its robustness. Current adversarial training methods typically rely on a limited set of adversarial examples, and after training, the model parameters are fixed, lacking the ability to adapt and expand in the face of new adversarial examples. However, in real-world scenarios, as research on model security continues to advance, adversarial attacks are constantly updated and iterated, with increasing attack intensity, making it difficult for existing defense models to cope with ever-changing attack threats. Therefore, researchers have begun to study continuous adversarial defense strategies, which involve continuously adjusting model parameters based on new adversarial attacks to ensure the model's continued robustness.
[0003] To improve the model's sustained robustness against iterative adversarial attacks, existing persistent adversarial defense methods typically employ the classic data replay-based continuous learning process and regularize parameters shared across all tasks. However, the difference between adversarial examples and clean examples lies in the fact that adding perturbations makes the examples more complex, thus making sustained adversarial training difficult. Current persistent adversarial defense methods suffer from the following drawbacks:
[0004] 1. The existing persistent adversarial defense model, which shares all task parameters, is ill-equipped to cope with continuously generated adversarial attacks. Adversarial examples are challenging samples with high specificity and aggressiveness, located at or across the model's classification boundary. The continuous development of various attack methods further increases the difficulty of achieving high-quality feature extraction using a unified parameter-sharing network framework, which usually leads to suboptimal problems for persistent adversarial defense models with respect to different adversarial attack methods, and thus cannot maintain optimal performance.
[0005] 2. Existing persistent adversarial defense models based on data replay are ill-equipped to handle diverse attack strategies. Given the wide variety of attack types, targets, and intensities, replay-based methods require a large amount of memory, significantly increasing the storage burden on the model. Furthermore, replaying only a small number of samples makes it difficult to maintain high-quality persistent defense. In addition, this approach may raise privacy concerns, especially in data-sensitive application scenarios. Summary of the Invention
[0006] This invention provides a dynamically scalable persistent adversarial defense method to at least address the technical problems of existing persistent adversarial defense models having a high degree of parameter sharing among different adversarial attack methods, low model training capability due to the large amount of training data stored, and low accuracy of training results.
[0007] According to one aspect of the present invention, a dynamically scalable persistent adversarial defense method is provided. The method may include: acquiring training set images, wherein each image in the training set consists of an initial image and perturbation information of the attack type corresponding to the initial image; constructing a target classifier, wherein the target classifier includes an image encoder, a text encoder, and a similarity calculation module; determining a cue for each type of attack image based on each type of attack image in the training set images; inputting each type of attack image and its corresponding cue into the target classifier to obtain a successfully trained target classifier; constructing a target convolutional neural network, wherein the target convolutional neural network includes layer normalization, multi-head attention, feature fusion layer, defense expert dynamic routing, multilayer perceptron, layer standardization, and addition operations, and the defense expert dynamic routing includes several... The system consists of one router and several defense experts. Each router carries a tag, and each type of attack perturbation information corresponds to a router of that type. The training set images are input into the target convolutional neural network to obtain a successfully trained target convolutional neural network and each router determines the output proportion of each defense expert for each type of attack image. The system then acquires a test image, which consists of a target image and unknown perturbation information of the target image. The test image is input into a successfully trained target classifier to obtain the attack type corresponding to the unknown perturbation information of the test image. Finally, the test image is input into the successfully trained target convolutional neural network to obtain the target result of the attack type of the test image.
[0008] Optionally, determining the prompt for each type of attack image based on the training set images includes: defining the attack type of each type of attack image in the training set images as initial definition information, defining the content of each type of attack image other than the initial definition information as a context vector; and obtaining the prompt for each type of attack image based on the initial definition information and the context vector.
[0009] Optionally, the step of inputting each type of attack image and its corresponding prompt into the target classifier to obtain a successfully trained target classifier includes: inputting the first type of attack image into an image encoder to obtain the features of the first type of attack image, and calculating the feature center and covariance matrix of the first type of attack image; inputting the prompt of the first type of attack image into a text encoder to obtain the features corresponding to the prompt of the first type of attack image; calculating the similarity between the features of the first type of attack image and the features corresponding to the prompt of the first type of attack image to obtain the attack type corresponding to the perturbation information of the first type of attack image; when the second type of attack image is about to be input into the image encoder, performing Gaussian sampling on the feature center of the first type of attack image to obtain the sampled features; fine-tuning the target classifier trained on the first type of attack image based on the sampled features to obtain the fine-tuned target classifier; inputting the second type of attack image into the fine-tuned target classifier, iterating in a loop to obtain a successfully trained target classifier.
[0010] Optionally, in the process of inputting the training set images into the target convolutional neural network to obtain a successfully trained target convolutional neural network, the method further includes: during the training process, when the target convolutional neural network encounters a new attack type image, replacing the defense expert parameters of the new attack type image after training based on the stored defense expert parameters of the old attack type image and the defense expert parameters of the new attack type image after training.
[0011] Optionally, replacing the defense expert parameters of the newly trained attack type image based on the defense expert parameters of the stored old attack type image and the defense expert parameters of the newly trained attack type image includes: linearly fusing the defense expert parameters of the stored old attack type image and the defense expert parameters of the newly trained attack type image to obtain the target defense expert parameters; and replacing the defense expert parameters of the newly trained attack type image with the target defense expert parameters.
[0012] Optionally, the step of inputting the image to be tested into a successfully trained target classifier to obtain the attack type corresponding to the unknown type of perturbation information of the image to be tested includes: inputting the target image into an image encoder to obtain the features of the target image; inputting the prompts of each type of attack image into a text encoder to obtain the features corresponding to the prompts of each type of attack image; and calculating the features of the target attack and the features corresponding to each prompt through a similarity calculation module to obtain the attack type corresponding to the perturbation information of the target image.
[0013] Optionally, the step of inputting the image to be tested into a successfully trained target convolutional neural network to obtain the target result of the attack type of the image to be tested includes: determining the target router in the dynamic routing of defense experts based on the attack type corresponding to the perturbation information of the target image; extracting features from the image to be tested through layer normalization, multi-head attention, and feature fusion to obtain feature labels; inputting the feature labels into each defense expert in the dynamic routing of defense experts to obtain a first result of the attack type of the target expert corresponding to the target router for the image to be tested; inputting the feature labels into a multilayer perceptron and a layer standard in sequence to obtain a second result of the attack type of the image to be tested; and obtaining the target result of the attack type of the image to be tested based on the first result and the second result.
[0014] Optionally, obtaining the target result of the attack type of the image to be tested based on the first result and the second result includes: determining the sum of the first result and the second result as the target result of the attack type of the image to be tested.
[0015] The beneficial effects of this invention are:
[0016] (1) The present invention constructs a dynamic and scalable continuous adversarial defense architecture, which includes a dynamic routing of defense experts and a classifier of sample types composed of multiple defense experts and a set of routers, thereby mitigating the problem of suboptimal model performance caused by shared parameters.
[0017] (2) This invention proposes a pseudo-task knowledge replay strategy based on feature space enhancement, which avoids the high memory consumption and privacy protection problems caused by storing a large amount of historical training data. Attached Figure Description
[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:
[0019] Figure 1 This is a flowchart of a dynamic and scalable persistent adversarial defense method according to an embodiment of the present invention;
[0020] Figure 2 This is a schematic diagram of a target classifier according to an embodiment of the present invention;
[0021] Figure 3 This is a schematic diagram of a target convolutional neural network according to an embodiment of the present invention;
[0022] Figure 4 This is a schematic diagram of pseudo-task knowledge replay with feature space enhancement according to an embodiment of the present invention. Detailed Implementation
[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0025] Example 1
[0026] According to an embodiment of the present invention, a dynamic and scalable persistent adversarial defense method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system containing at least one set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0027] Figure 1 This is a flowchart of a dynamically scalable persistent adversarial defense method according to an embodiment of the present invention, such as... Figure 1 As shown, the method may include the following steps:
[0028] Step S101: Obtain training set images, wherein each image in the training set consists of an initial image and perturbation information of the attack type corresponding to the initial image.
[0029] In the technical solution provided by step S101 of the present invention, training set images are obtained, and the training set images include several groups of images with different attack types.
[0030] Step S102: Construct a target classifier, which includes an image encoder, a text encoder, and a similarity calculation module.
[0031] In the technical solution provided by step S102 of the present invention, Figure 2This is a schematic diagram of a target classifier according to an embodiment of the present invention, such as... Figure 2 As shown, the target classifier includes an image encoder, a text encoder, and a similarity calculation module.
[0032] Step S103: Based on each type of attack image in the training set images, determine the hints for each type of attack image.
[0033] In the technical solution provided by step S103 of the present invention, a prompt for each type of attack image is obtained based on each type of attack image in the training set images.
[0034] Step S104: Input each type of attack image and its corresponding prompt into the target classifier to obtain a successfully trained target classifier.
[0035] In the technical solution provided by step S104 of the present invention, each type of attack image and its corresponding prompt are used to train the target classifier to obtain a successfully trained target classifier.
[0036] Step S105: Construct the target convolutional neural network, which includes layer normalization, multi-head attention, feature fusion layer, defense expert dynamic routing, multilayer perceptron, layer standardization and addition operation. The defense expert dynamic routing includes several routers and several defense experts. Each router carries a label, and the perturbation information of each attack type corresponds to a router of a certain type.
[0037] In the technical solution provided by step S105 of the present invention, Figure 3 This is a schematic diagram of a target convolutional neural network according to an embodiment of the present invention, such as... Figure 3 As shown, the target convolutional neural network includes layer normalization, multi-head attention, feature fusion layer, defense expert dynamic routing, multilayer perceptron, layer standardization, and addition operation. The defense expert dynamic routing includes several routers and several defense experts. Each router carries a label, and the perturbation information of each attack type corresponds to a router of a certain type. Each defense expert is a LoRA. LoRA can decouple the original heavy and frozen parameters to a low-rank trainable space, improve training speed, and reduce training burden.
[0038] Step S106: Input the training set images into the target convolutional neural network to obtain the successfully trained target convolutional neural network and the output proportion of each defense expert for each type of attack image determined by each router.
[0039] In the technical solution provided by step S106 of the present invention, the training set images are used to train the target convolutional neural network to obtain a successfully trained target convolutional neural network and each router determines the output ratio of each defense expert for each type of attack image.
[0040] Step S107: Obtain the image to be tested; wherein the image to be tested consists of the target image and unknown type of perturbation information of the target image.
[0041] In the technical solution provided by step S107 of the present invention, a test image is obtained; wherein, the test image consists of a target image and unknown type of perturbation information of the target image, and the target image is an image without noise.
[0042] Step S108: Input the image to be tested into the successfully trained target classifier to obtain the attack type corresponding to the unknown type of perturbation information of the image to be tested.
[0043] In the technical solution provided by step S108 of the present invention, the successfully trained target classifier processes the image to be tested to obtain the attack type corresponding to the unknown type of perturbation information of the image to be tested.
[0044] Step S109: Input the image to be tested into the successfully trained target convolutional neural network to obtain the target result of the attack type of the image to be tested.
[0045] In the technical solution provided by step S109 of the present invention, the successfully trained target convolutional neural network processes the image to be tested to obtain the target result of the attack type of the image to be tested, and the target result can be a percentage.
[0046] The method described in this embodiment will be further described below.
[0047] As an optional embodiment, step S103, determining the prompt for each type of attack image based on the training set images, includes: defining the attack type of each type of attack image in the training set images as initial definition information, defining the content of each type of attack image other than the initial definition information as a context vector; and obtaining the prompt for each type of attack image based on the initial definition information and the context vector.
[0048] In this embodiment, the attack type of each type of attack image in the training set is defined as initial definition information. The content of each type of attack image, excluding the initial definition information, is defined as a context vector. Therefore, the cue for each type of attack image can be represented as:
[0049]
[0050] in, For each type of attack image, a hint is provided, where t represents the attack type. The context consists of a series vectors Indicates (where) These vectors reside in the text embedding space instead of the original text, using continuous parameters, which offers greater flexibility compared to the discrete parameters of the text formula. Each vector has the same dimension as the word embedding. The final input to the text encoder is a concatenation of the context vector and the class word embeddings. The cue for each attack image class can also be represented as:
[0051]
[0052] In addition, a variant of context parameterization called type-specific context is employed, which assigns a separate context vector to each attack type, allowing different attack types to have different customized contexts. This helps to distinguish attack types, and the loss for updating context parameters uses cross-entropy loss.
[0053] As an optional embodiment, step S104, which involves inputting each type of attack image and its corresponding prompt into the target classifier to obtain a successfully trained target classifier, includes: inputting the first type of attack image into an image encoder to obtain the features of the first type of attack image, and calculating the feature center and covariance matrix of the first type of attack image; inputting the prompt of the first type of attack image into a text encoder to obtain the features corresponding to the prompt of the first type of attack image; calculating the similarity between the features of the first type of attack image and the features corresponding to the prompt of the first type of attack image to obtain the attack type corresponding to the perturbation information of the first type of attack image; when the second attack image is about to be input into the image encoder, performing Gaussian sampling on the feature center of the first type of attack image to obtain the sampled features; fine-tuning the target classifier trained on the first type of attack image based on the sampled features to obtain the fine-tuned target classifier; inputting the second type of attack image into the fine-tuned target classifier, iterating in a loop to obtain a successfully trained target classifier.
[0054] In this embodiment, such as Figure 2 As shown, the first type of attack image is input into the image encoder to obtain the features of the first type of attack image, and the feature center and covariance matrix of the first type of attack image are calculated; the prompt of the first type of attack image is input into the text encoder to obtain the features corresponding to the prompt of the first type of attack image; by calculating the similarity between the features of the first type of attack image and the features corresponding to the prompt of the first type of attack image, the attack type corresponding to the perturbation information of the first type of attack image is obtained. Figure 4 This is a schematic diagram of pseudo-task knowledge replay with feature space enhancement according to an embodiment of the present invention, as shown below. Figure 4 As shown, when the second attack image is about to be input into the image encoder, Gaussian sampling is performed on the feature centers of the first type of attack image to obtain the sampled features. Figure 4The target classifier trained on the first type of attack image is fine-tuned based on the sampled features (past features in the image). The second type of attack image is then input into the fine-tuned target classifier, and the process is iterated until a successfully trained target classifier is obtained.
[0055] The feature centers of the first type of attack image To represent the average feature of the first type of attack image samples, calculate the feature centers and the features of samples with the same attack type in each mini-batch of the first type of attack images. The variances between the variances are aggregated stepwise using moving averages to form a covariance matrix, which is expressed as:
[0056]
[0057] in, Let be the covariance matrix of the image subjected to the first type of attack.
[0058] When the second type of attack image is about to be input into the image encoder, Gaussian sampling is performed on the feature centers of the first type of attack image to obtain the sampled features. The past sample features are the sample features of the first type of attack image. Therefore, the expressions for the sampled features and covariance matrix of the first type of attack image are:
[0059]
[0060]
[0061] in, The features of the sampled image from the first type of attack. The sampled covariance matrix of the image from the first type of attack. Covariance matrix The Cholesky decomposition matrix, From the standard normal distribution Random vectors drawn from the sample. Covariance matrix The transpose of the Cholesky decomposition matrix. Feature center of the first type of attack image When training for new types of attacks, past features are randomly sampled from the stored mean and covariance matrices of past types to generate a feature subset; this process enriches the diversity of features and prevents the classifier of the sample type from overfitting to the current attack.
[0062] As an optional embodiment, in step S106, during the process of inputting the training set images into the target convolutional neural network to obtain a successfully trained target convolutional neural network, the method further includes: during the training process, when the target convolutional neural network encounters a new attack type image, replacing the defense expert parameters of the new attack type image after training based on the stored defense expert parameters of the old attack type image and the defense expert parameters of the new attack type image after training.
[0063] In this embodiment, when the target convolutional neural network encounters a new attack type image, it calculates the defense expert parameters of the stored old attack type image and the defense expert parameters of the trained new attack type image, and then replaces the trained defense expert parameters of the new attack type image with the calculated values.
[0064] As an optional embodiment, replacing the defense expert parameters of the newly trained attack type image based on the defense expert parameters of the stored old attack type image and the defense expert parameters of the newly trained attack type image includes: linearly fusing the defense expert parameters of the stored old attack type image and the defense expert parameters of the newly trained attack type image to obtain the target defense expert parameters; and replacing the defense expert parameters of the newly trained attack type image with the target defense expert parameters.
[0065] In this embodiment, the defense expert parameters of the stored old attack type images and the defense expert parameters of the trained new attack type images are linearly fused to obtain the expression for the target defense expert parameters as follows:
[0066]
[0067] For target defense expert parameters, Defense expert parameters for images of new attack types after training. Defense expert parameters for stored images of older attack types. These are the weights used during the fusion process.
[0068] Using target defense expert parameters Defense expert parameters for replacing the new attack type image after training .
[0069] As an optional embodiment, step S108, which involves inputting the image to be tested into a successfully trained target classifier to obtain the attack type corresponding to the unknown type of perturbation information of the image to be tested, includes: inputting the target image into an image encoder to obtain the features of the target image; inputting the prompts for each type of attack image into a text encoder to obtain the features corresponding to the prompts for each type of attack image; and calculating the features of the target attack and the features corresponding to each prompt through a similarity calculation module to obtain the attack type corresponding to the perturbation information of the target image.
[0070] In this embodiment, such as Figure 2 As shown, the target image is input into the image encoder to obtain the features of the target image; the prompts for each type of attack image are input into the text encoder to obtain the features corresponding to the prompts for each type of attack image; the similarity calculation module calculates the features of the target attack and the features corresponding to each prompt to obtain several values; the several values are sorted, and the type of the prompt corresponding to the maximum sorted value is determined as the attack type corresponding to the perturbation information of the target image.
[0071] As an optional embodiment, step S109, which involves inputting the image to be tested into a successfully trained target convolutional neural network to obtain the target result of the attack type of the image to be tested, includes: determining the target router in the dynamic routing of defense experts based on the attack type corresponding to the perturbation information of the target image; extracting features from the image to be tested through layer normalization, multi-head attention, and feature fusion to obtain feature labels; inputting the feature labels into each defense expert in the dynamic routing of defense experts to obtain a first result of the attack type of the target expert corresponding to the target router for the image to be tested; inputting the feature labels into a multilayer perceptron and a layer standard in sequence to obtain a second result of the attack type of the image to be tested; and obtaining the target result of the attack type of the image to be tested based on the first result and the second result.
[0072] In this embodiment, based on the attack type corresponding to the perturbation information of the target image, a target router in the dynamic routing of defense experts with the same attack type as the perturbation information of the target image is selected. Feature extraction is performed on the image to be tested through layer normalization, multi-head attention, and feature fusion to obtain feature labels. The feature labels are input into each defense expert in the dynamic routing of defense experts to obtain the weight of each defense expert for the attack type of the image to be tested.
[0073] The feature labels are input into each defense expert in the dynamic routing of defense experts, and the expression for the weight of each defense expert for the attack type of the image to be tested is as follows:
[0074]
[0075] in, This represents the percentage of attack types output by the target router to each defense expert for the image under test. The label for each router, ,function Will Projected onto a one-dimensional vector, this represents the percentage of each defense expert's contribution to the first outcome. The choice of function has the greatest impact on the correct result. One expert, setting the contributions of the remaining experts as... Finally, application The function normalizes these weights.
[0076] Based on the weight of each defense expert's attack type for the image under test, the first result expression for the attack type of the target expert for the image under test corresponding to the target router is as follows:
[0077]
[0078] in, The first result for the attack type of the target expert corresponding to the target router on the image to be tested. For feature labeling, i The output of each defense expert selected for the router.
[0079] As an optional embodiment, obtaining the target result of the attack type of the image to be tested based on the first result and the second result includes: determining the sum of the first result and the second result as the target result of the attack type of the image to be tested.
[0080] In this embodiment, the first result and the second result are added together to obtain the target result of the attack type of the image to be tested.
[0081] In this embodiment of the invention, a training set of images is acquired, wherein each image in the training set consists of an initial image and perturbation information of the attack type corresponding to the initial image; a target classifier is constructed, wherein the target classifier includes an image encoder, a text encoder, and a similarity calculation module; based on each type of attack image in the training set images, a prompt for each type of attack image is determined; each type of attack image and its corresponding prompt are input into the target classifier to obtain a successfully trained target classifier; a target convolutional neural network is constructed, wherein the target convolutional neural network includes layer normalization, multi-head attention, feature fusion layer, defense expert dynamic routing, multilayer perceptron, layer standardization, and addition operations; the defense expert dynamic routing includes several routers and several defense experts, each router carries a label, and the perturbation information of each attack type corresponds to a router of that type; the training set images are input into the target convolutional neural network to obtain a successfully trained target convolutional neural network. The output proportion of each defense expert for each type of attack image is determined through the network and each router; the test image is acquired, which consists of the target image and unknown type perturbation information of the target image; the test image is input into a successfully trained target classifier to obtain the attack type corresponding to the unknown type perturbation information of the test image; the test image is input into a successfully trained target convolutional neural network to obtain the target result of the attack type of the test image. This solves the technical problems of existing continuous adversarial defense models having a large degree of parameter sharing for different adversarial attack methods and low model training capability due to the large amount of training data stored, resulting in low accuracy of training results. It achieves the construction of a dynamic and scalable continuous adversarial defense architecture, reduces the sharing of parameters, and proposes a pseudo-task knowledge replay strategy based on feature space enhancement, which avoids the low model training capability due to the storage of a large amount of historical training data and improves the technical effect of training result accuracy.
[0082] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0083] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0084] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.
[0085] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0086] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a first processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0087] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A dynamically scalable persistent adversarial defense method, characterized in that, include: Obtain training set images, wherein each image in the training set consists of an initial image and perturbation information of the attack type corresponding to the initial image; Construct a target classifier, which includes an image encoder, a text encoder, and a similarity calculation module; Based on the training set images, determine the cue for each type of attack image; Each type of attack image and its corresponding prompt are input into the target classifier to obtain a successfully trained target classifier. The step of inputting each type of attack image and its corresponding prompt into the target classifier to obtain a successfully trained target classifier includes: The first type of attack image is input into the image encoder to obtain the features of the first type of attack image, and the feature center and covariance matrix of the first type of attack image are calculated. The prompts of the first type of attack image are input into the text encoder to obtain the features corresponding to the prompts of the first type of attack image; By calculating the similarity between the features of the first type of attack image and the features corresponding to the prompts of the first type of attack image, the attack type corresponding to the perturbation information of the first type of attack image can be obtained. When the second attack image is about to be input into the image encoder, Gaussian sampling is performed on the feature center of the first type of attack image to obtain the sampled features. The target classifier trained on the first type of attack image is fine-tuned based on the sampled features to obtain the fine-tuned target classifier. The second type of attack image is input into the fine-tuned target classifier, and the process is iterated until a successfully trained target classifier is obtained. Construct a target convolutional neural network, which includes layer normalization, multi-head attention, feature fusion layer, defense expert dynamic routing, multilayer perceptron, layer standardization and addition operation. The defense expert dynamic routing includes several routers and several defense experts. Each router carries a label, and the perturbation information of each attack type corresponds to a router of a certain type. The training set images are input into the target convolutional neural network to obtain the successfully trained target convolutional neural network and each router determines the output proportion of each defense expert for each type of attack image; In the process of inputting the training set images into the target convolutional neural network to obtain a successfully trained target convolutional neural network, the method further includes: During training, when the target convolutional neural network encounters a new attack type image, it replaces the defense expert parameters of the new attack type image with those of the old attack type image based on the stored defense expert parameters of the old attack type image and the defense expert parameters of the new attack type image after training. The replacement of the defense expert parameters for the newly trained attack type images with the defense expert parameters based on the stored old attack type images and the trained new attack type images includes: The defense expert parameters of the stored old attack type images and the defense expert parameters of the trained new attack type images are linearly fused to obtain the target defense expert parameters. Replace the defense expert parameters of the newly trained attack type image with the target defense expert parameters; Acquire the image to be tested; wherein the image to be tested consists of the target image and unknown type of perturbation information of the target image; The image to be tested is input into the successfully trained target classifier to obtain the attack type corresponding to the unknown type of perturbation information of the image to be tested; The image to be tested is input into the successfully trained target convolutional neural network to obtain the target result of the attack type of the image to be tested.
2. The method according to claim 1, characterized in that, The method for determining a cue for each type of attack image based on the training set images includes: The attack type of each attack image in the training set is defined as the initial definition information, and the content of each attack image other than the initial definition information is defined as the context vector. Based on the initial definition information and context vector of each type of attack image, a hint is obtained for each type of attack image.
3. The method according to claim 1, characterized in that, The step of inputting the image to be tested into a successfully trained target classifier to obtain the attack type corresponding to the unknown type of perturbation information of the image to be tested includes: The target image is input into the image encoder to obtain the features of the target image; The prompts for each type of attack image are input into the text encoder to obtain the features corresponding to the prompts for each type of attack image; The similarity calculation module calculates the characteristics of the target attack and the characteristics corresponding to each prompt to obtain the attack type corresponding to the perturbation information of the target image.
4. The method according to claim 3, characterized in that, The step of inputting the image to be tested into a successfully trained target convolutional neural network to obtain the target result of the attack type of the image to be tested includes: Based on the attack type corresponding to the perturbation information of the target image, the target router in the dynamic routing of the defense expert is determined. Feature labels are obtained by extracting features from the test image through layer normalization, multi-head attention, and feature fusion. The feature tags are input into each defense expert in the dynamic routing of defense experts to obtain the first result of the attack type of the target experts for the image to be tested, corresponding to the target router; The feature labels are sequentially input into the multilayer perceptron and the layer standard to obtain the second result of the attack type of the image under test; Based on the first and second results, the target result of the attack type of the image to be tested is obtained.
5. The method according to claim 4, characterized in that, The target result of the attack type of the image to be tested, based on the first result and the second result, includes: The sum of the first and second results is determined as the target result of the attack type of the image to be tested.