Adversarial training method for deep convolutional neural network, image recognition method and device

Abstract

The present application provides an adversarial training method, image recognition method and device for a deep convolutional neural network, the method comprising: extracting the initial features of the adversarial attack sample corresponding to the target image in the adversarial branch during the adversarial training of the deep convolutional neural network; performing feature decoupling on the initial features of the adversarial attack sample to obtain adversarial type features and interference features corresponding to the initial features, and performing feature extraction on the adversarial type features to obtain a first robust representation for identifying the target image type. The present application takes into account the dynamic nature of the adversarial attack samples participating in the training, and utilizes feature decoupling to separate the interference features in the adversarial attack samples, which can ensure the adversarial training quality of the deep convolutional neural network, and can enhance the adversarial training effect for the deep convolutional neural network, improve the robustness of the deep convolutional neural network, and thus effectively improve the effectiveness and reliability of the recognition results of the deep convolutional neural network.

Description

Technical Field

[0001] The present application relates to the field of adversarial training technology, and in particular to an adversarial training method, image recognition method, and device for deep convolutional neural networks. Background Art

[0002] The fundamental working principle of a deep convolutional neural network is to continuously map the input image using the neural network learned within the model, ultimately extracting the features required for the corresponding recognition task, identifying these features, and implementing the application function. Adversarial attack techniques essentially aim to disrupt this feature extraction process, misleading the model into extracting incorrect recognition features. Therefore, extracting robust recognition features is crucial for adversarial defense techniques.

[0003] Existing adversarial training techniques improve the robustness of model recognition by introducing adversarial attack samples into normal neural network training. However, this approach, which simply introduces adversarial attack samples into the training process, can cause the trained model to overfit to the adversarial attack patterns used in training, reducing its robustness against untrained and unknown attack patterns, leading to poor generalization of the trained model. Furthermore, existing training techniques ignore the dynamic nature of the adversarial attack samples used in training and fail to address the problem of poor training quality caused by excessive inclusion of features from other categories in the samples in the later stages of training. Summary of the Invention

[0004] In view of this, embodiments of the present application provide an adversarial training method, an image recognition method, and an apparatus for deep convolutional neural networks to eliminate or improve one or more defects in the prior art.

[0005] One aspect of the present application provides an adversarial training method for a deep convolutional neural network, comprising:

[0006] In the adversarial branch of the adversarial training process of the deep convolutional neural network, the initial features of the adversarial attack sample corresponding to the target image are extracted;

[0007] Performing feature decoupling on the initial features of the adversarial attack sample to obtain adversarial type features and interference features corresponding to the initial features, and performing feature extraction on the adversarial type features to obtain a first robust representation for identifying the target image type.

[0008] In some embodiments of the present application, further comprising:

[0009] Inputting the interference features into a pattern-level classifier and training corresponding dynamic adversarial labels;

[0010] Wherein, if the proportion of the interference feature in the initial feature is greater than the adversarial type feature, the dynamic adversarial label of the adversarial attack sample is: interference category;

[0011] If the proportion of the interference feature in the initial feature is less than or equal to the adversarial type feature, the dynamic adversarial label of the adversarial attack sample is: the original image category of the target image;

[0012] And, inputting the first robust representation into an image-level classifier and training a corresponding label, wherein the label is the original image category of the target image.

[0013] In some embodiments of the present application, the feature decoupling of the initial features of the adversarial attack sample to obtain the adversarial type features and interference features corresponding to the initial features includes:

[0014] Using a preset feature decoupler to perform feature decoupling on the initial features of the adversarial attack sample;

[0015] The feature distribution distance between the adversarial type feature and the interference feature is determined based on the KL divergence loss method, and the feature distribution distance is maximized.

[0016] In some embodiments of the present application, extracting the adversarial type features to obtain a first robust representation for identifying the target image type further includes:

[0017] Performing MLP mapping on the adversarial type feature to remove noise components in the adversarial type feature to obtain a corresponding first denoising feature;

[0018] Performing MLP feature prediction on the first denoising features to extract a first robust representation for identifying the target image type.

[0019] In some embodiments of the present application, further comprising:

[0020] Extracting regular type features of regular samples corresponding to target images in a natural branch of the adversarial training process of deep convolutional neural networks;

[0021] Feature extraction is performed on conventional type features of the conventional samples to obtain a second robust representation, and alignment processing of activated neurons is performed based on the second robust representation and the first robust representation.

[0022] In some embodiments of the present application, extracting the regular type features of the regular samples to obtain a second robust representation, and aligning the activated neurons based on the second robust representation and the first robust representation, includes:

[0023] Performing MLP mapping on the regular type features of the regular samples to remove noise components in the regular type features to obtain corresponding second denoised features, and performing MLP feature prediction on the second denoised features to extract a second robust representation;

[0024] The first denoised feature is aligned with the second robust representation, and the second denoised feature is aligned with the first robust representation to align neurons activated in the deep convolutional neural network by the regular sample and the adversarial attack sample, respectively.

[0025] Another aspect of the present application provides an image type recognition method, comprising:

[0026] receiving image data to be identified and preprocessing the image data;

[0027] Inputting the preprocessed image data into a deep convolutional neural network so that the deep convolutional neural network outputs a type recognition result corresponding to the image data, wherein the deep convolutional neural network is pre-trained based on an adversarial training method of the deep convolutional neural network;

[0028] The type recognition result is output.

[0029] Another aspect of the present application provides an adversarial training apparatus for a deep convolutional neural network, comprising:

[0030] The initial feature extraction module is used to extract the initial features of the adversarial attack sample corresponding to the target image in the adversarial branch of the adversarial training process of the deep convolutional neural network;

[0031] A feature decoupling module is used to perform feature decoupling on the initial features of the adversarial attack sample to obtain adversarial type features and interference features corresponding to the initial features, and to perform feature extraction on the adversarial type features to obtain a first robust representation for identifying the target image type.

[0032] Another aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements the adversarial training method of the deep convolutional neural network, or implements the image type recognition method.

[0033] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the adversarial training method of the deep convolutional neural network, or implements the image type recognition method.

[0034] The adversarial training method for a deep convolutional neural network provided in the present application extracts the initial features of the adversarial attack sample corresponding to the target image in the adversarial branch of the adversarial training process of the deep convolutional neural network; by performing feature decoupling on the initial features of the adversarial attack sample, the adversarial type features and interference features corresponding to the initial features are obtained; considering the dynamic nature of the adversarial attack samples participating in the training, the feature decoupling technology is used to separate the interference features in the adversarial attack samples, which can avoid overfitting the adversarial attack mode participating in the training. Even if too many other interference features are mixed into the samples in the later stage of training, the adversarial training quality of the deep convolutional neural network can be guaranteed, the adversarial training effect of the deep convolutional neural network can be enhanced, and the robustness of the deep convolutional neural network can be further improved; and the deep convolutional neural network obtained based on the adversarial training method can be applied to many artificial intelligence applications based on deep convolutional neural network recognition, such as autonomous driving, smart home, and monitoring and security systems, which can effectively improve the effectiveness and reliability of the recognition results of the deep convolutional neural network.

[0035] Additional advantages, purposes, and features of the present application will be described in part in the following description and will become apparent to those skilled in the art upon study of the following or may be learned from practice of the present application. The purposes and other advantages of the present application may be achieved and obtained by the structures specifically pointed out in the specification and drawings.

[0036] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description, and the above and other purposes that can be achieved by the present application will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are intended to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. The components in the drawings are not drawn to scale, but are only for the purpose of illustrating the principles of the present application. In order to facilitate the illustration and description of some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, they may become larger than other components in the exemplary device actually manufactured according to the present application. In the drawings:

[0038] Figure 1 Schematic diagram of the phenomenon of mixing in features of other categories in adversarial training.

[0039] Figure 2 Schematic diagram of feature decoupling state in feature space.

[0040] Figure 3 Schematic diagram of the overall process of the adversarial training method of the deep convolutional neural network in one embodiment of the present application.

[0041] Figure 4 The technical roadmap for feature decoupling-based adversarial training provided for this application.

[0042] Figure 5 Schematic diagram of a specific process of the adversarial training method for a deep convolutional neural network in one embodiment of the present application.

[0043] Figure 6 Schematic diagram of the structure of the adversarial training device for deep convolutional neural networks in another embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail in conjunction with the embodiments and drawings. Here, the illustrative embodiments of this application and their descriptions are used to explain this application, but are not intended to limit this application.

[0045] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, the accompanying drawings only show structures and / or processing steps that are closely related to the scheme according to the present application, while other details that are not closely related to the present application are omitted.

[0046] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0047] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0048] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0049] Artificial intelligence algorithms, with performance comparable to or even exceeding human recognition capabilities, have gradually taken a key position in fields such as autonomous driving, smart homes, and surveillance and security. Deep convolutional neural networks are particularly prominent. Deep convolutional neural networks use intelligent convolution operations to perform multi-stage feature extraction on target images, gradually mapping the extracted features from low-dimensional texture, shape, color, and other features to high-dimensional features with complex semantics, enabling intelligent recognition and classification tasks in fields such as autonomous driving, smart homes, and surveillance and security. However, in recent years, adversarial attack techniques targeting deep learning models have emerged. By exploiting vulnerabilities in neural networks during inference and applying subtle perturbations to images that are invisible to the human eye, they aim to deceive deep neural network models, resulting in a sharp decline in their security and a serious threat to the deployment and popularization of artificial intelligence applications. Therefore, there is an urgent need to design adversarial defense technologies that correspond to adversarial attack techniques to improve model robustness.

[0050] The current mainstream adversarial defense method is adversarial training. This technique, which incorporates adversarial attack samples into model training, simulates the interference of adversarial attacks in advance, and enhances the robustness of the model. In recent years, researchers have gradually begun to explore the fundamental reasons why adversarial attacks can disrupt neural networks, for example, by analyzing the robustness of the model from the perspective of the frequency of input samples.

[0051] In recent years, some research has delved into the mechanisms of adversarial training and the generation of adversarial attack samples, using techniques such as neural network visualization and image Fourier transforms to investigate the interpretability of these mechanisms. This research has demonstrated that the dynamic nature of adversarial attack samples during training can cause the model to mistake features from other categories for the labeled categories of the training samples. Despite this, existing technologies offer solutions to this dynamic nature.

[0052] In other words, existing adversarial training methods for deep convolutional neural networks all have problems such as low generalization of training models and gradual decline in training quality.

[0053] Based on this, the present application uses feature decoupling technology to decompose the features of the adversarial attack samples, further explores its perturbation mechanism, and enhances the adversarial training effect. In adversarial training, the present application found that the feature composition of the adversarial attack samples participating in the training is dynamic, and will be mixed with interference features of other categories, affecting the training effect. Therefore, the present application uses feature decoupling and dynamic labeling technology to separate the interference features, enhance the adversarial training effect, and further achieve the improvement of the robustness of adversarial training. The present application can be applied to many artificial intelligence applications based on deep convolutional neural network recognition, such as autonomous driving, smart home, and monitoring security systems.

[0054] In one or more embodiments of the present application, CL (Contrastive Learning) refers to contrastive learning; KLDivergence (Kullback-Leibler Divergence) refers to KL divergence, also known as relative entropy; PGD (Project Gradient Descent) refers to gradient projection descent technology; Epoch refers to training round; TSNE (t-distributed stochastic neighbor embedding) refers to stochastic neighbor embedding based on t-distribution.

[0055] First, the basic principles of the adversarial training method for deep convolutional neural networks provided in this application are as follows:

[0056] See also Figure 1 First, we use PGD attack to generate training samples for adversarial training. We can see that as the number of training epochs increases, the training samples gradually change from the original clear "4" and "5" to training samples with other digital categories. Figure 1 The "5" in the first row of images gradually takes on the characteristics of "3". Figure 1 The "4" in the second row of images gradually takes on the features of a "9." Therefore, it can be clearly stated that the feature composition of the adversarial attack samples during the adversarial training process is dynamic, and such samples mixed with features from other categories will cause a decline in model training quality and affect the final model's recognition accuracy.

[0057] Therefore, this application needs to separate the mixed features during the training process to ensure the training effect. Figure 2 First, we use the TSNE data dimensionality reduction technique to perform high-dimensional feature mapping on the 5,000 sample data. We can observe that the initial features of natural samples and adversarial attack samples have different distributions in the feature space. The initial features of adversarial attack samples have many overlapping areas, which are misleading and misclassified samples. Then, we separate the mixed features of natural samples and adversarial attack samples, and the results are as follows: Figure 2 As shown in the second row of the figure, the mixed features and the initial features have significant overlapping areas, which are interference features, while the remaining robust representations are better divided into multiple clustered areas according to categories, making them easier to classify. Therefore, referring to this process, the basic principle of the adversarial training method for deep convolutional neural networks provided in this application is to use the designed interference feature classification subtask to divide the initial features into two parts: mixed features and robust representations. The mixed features serve the interference feature classification subtask, and the robust representations are used for normal recognition and classification tasks.

[0058] Based on this, the present invention provides a method for adversarial training of deep convolutional neural networks. Figure 3 The adversarial training method of the deep convolutional neural network specifically includes the following contents:

[0059] Step 100: In the adversarial branch of the adversarial training process of the deep convolutional neural network, extract the initial features of the adversarial attack sample corresponding to the target image.

[0060] Understandably, see Figure 4 The entire training framework for adversarial training of deep convolutional neural networks is divided into two branches: the adversarial branch and the natural branch. Both branches take the same target image as input, for example, the original image x. Using adversarial attack and traditional image processing techniques, the original image x is converted into adversarial attack samples x1 and regular samples x2, respectively, corresponding to the target image. The backbone network of the deep convolutional neural network then extracts initial features from the adversarial attack samples x1 and regular samples x2, respectively, yielding the initial features of the adversarial samples and the regular type features z2 of the regular samples.

[0061] Step 200: Decouple the initial features of the adversarial attack sample to obtain adversarial type features and interference features corresponding to the initial features, and extract the adversarial type features to obtain a first robust representation for identifying the target image type.

[0062] In one or more embodiments of the present application, the adversarial type features may also be referred to as residual features, such as Figure 4 z1 in ; Interference features can also be called mixed-in features of other categories or mixed-in features of other categories, such as Figure 4 r1 shown in .

[0063] From the above description, it can be seen that the adversarial training method of the deep convolutional neural network provided in the embodiment of the present application decouples the initial features of the adversarial attack samples to obtain the adversarial type features and interference features corresponding to the initial features. Taking into account the dynamic nature of the adversarial attack samples participating in the training, the feature decoupling technology is used to separate the interference features in the adversarial attack samples, which can avoid overfitting the adversarial attack patterns participating in the training. Even if too many other interference features are mixed into the samples in the later stage of training, the training quality can be effectively guaranteed, and the adversarial training effect for the deep convolutional neural network can be effectively enhanced, further improving the robustness of the adversarial training. The deep convolutional neural network obtained based on this adversarial training method can be applied to many artificial intelligence applications based on deep convolutional neural network recognition, such as autonomous driving, smart homes, and monitoring and security systems, which can effectively improve the effectiveness and reliability of the recognition results of deep convolutional neural networks.

[0064] In order to further characterize the changing rules of the characteristics of the adversarial attack samples, in the adversarial training method of a deep convolutional neural network provided in the embodiment of the present application, see Figure 5 The adversarial training method of the deep convolutional neural network also specifically includes the following contents:

[0065] Step 300: Input the interference features into a pattern-level classifier and train the corresponding dynamic adversarial labels; wherein, if the proportion of the interference features in the initial features is greater than the adversarial type features, then the dynamic adversarial label of the adversarial attack sample is: interference category; if the proportion of the interference features in the initial features is less than or equal to the adversarial type features, then the dynamic adversarial label of the adversarial attack sample is: the original image category of the target image;

[0066] Step 400: Input the first robust representation into an image-level classifier and train a corresponding label, wherein the label is the original image category of the target image.

[0067] Specifically, after the feature extraction of the adversarial branch is completed, the mixed-in features r1 of other categories will be used for classification by the pattern-level classifier. The training label is the dynamic adversarial label, and its label value is the image category recognized by the current model. When the proportion of mixed-in features of other categories is greater than the features of the original category, the dynamic adversarial label is the mixed-in other categories, otherwise it is the original true label. The robust representation q1 is used for the correct recognition and classification of the model, and the training label is the true label.

[0068] Based on this, in order to address the problem in the existing technology that it is impossible to accurately characterize the dynamics of the feature structure of the training adversarial attack samples, as the adversarial training evolves, the features of the training adversarial attack samples will gradually change from imperceptible attack features to robust shape features, color features, texture features, etc. Although the general change pattern can be known by observing the appearance of the sample, it is very difficult to quantitatively characterize these features due to their abstract nature. Therefore, in response to this problem, this application uses the classification labels output by the training model as a quantitative reference and designs a dynamic adversarial labeling technology to characterize the change pattern of attack sample features.

[0069] In order to further enhance the robustness of the neural network model representation capability, in the adversarial training method of a deep convolutional neural network provided in the embodiment of the present application, see Figure 5 , step 200 in the adversarial training method of the deep convolutional neural network specifically includes the following content:

[0070] Step 210: Decouple the initial features of the adversarial attack sample using a preset feature decoupler; wherein, the feature distribution distance between the adversarial type feature and the interference feature is determined based on the KL divergence loss method, and the feature distribution distance is maximized.

[0071] Specifically, when training the feature decoupler f, this application uses KL divergence loss to calculate the feature distribution distance between the part of r1 mixed with other category features and the remaining features z1, and enhances the feature decoupling effect by maximizing this distance.

[0072] Based on this, the existing technology is unable to decouple the features of the training samples and filter out the mixed features of other categories. Due to the dynamic nature of the feature structure of the training adversarial attack samples, the specific proportion of other categories of features mixed in the samples is difficult to measure accurately. In addition, this high-dimensional feature is extremely abstract and cannot be characterized by specific parameters, which makes feature decoupling difficult. In response to this problem, the present application uses self-supervised learning technology to design an adversarial attack feature decoupling subtask to enhance the robustness of the representation ability of the neural network model.

[0073] In order to further improve the robustness of the neural network model representation capability, in the adversarial training method of a deep convolutional neural network provided in the embodiment of the present application, see Figure 5 , step 200 in the adversarial training method for a deep convolutional neural network further specifically includes the following content:

[0074] Step 220: Perform MLP mapping on the adversarial type feature to remove noise components in the adversarial type feature to obtain a corresponding first denoising feature.

[0075] Step 230: Perform MLP feature prediction on the first denoising feature to extract a first robust representation for identifying the target image type.

[0076] Specifically, in the adversarial branch, this application uses the feature decoupler f to divide the initial features of the adversarial attack sample into the part r1 mixed with other category features and the remaining features z1. Then, this application uses the MLP feature mapping module p to remove the noise components in z1, and finally uses the MLP feature prediction module to extract the robust representation q1 for recognition and classification.

[0077] In order to further improve the robustness of the neural network model representation capability, in the adversarial training method of a deep convolutional neural network provided in the embodiment of the present application, see Figure 5 The adversarial training method for a deep convolutional neural network further includes step 500, which can be performed simultaneously or asynchronously with step 100 and specifically includes the following content:

[0078] Step 500: In a natural branch of the adversarial training process of a deep convolutional neural network, extract regular type features of regular samples corresponding to the target image.

[0079] Step 600: extracting features of the regular type of the regular sample to obtain a second robust representation, and aligning the activated neurons based on the second robust representation and the first robust representation.

[0080] It can be understood that for natural branches, this application uses the feature extraction module other than the feature decoupler f to obtain the final robust representation q2, and the features of this branch do not participate in the training of the two classifiers, but only participate in the feature alignment optimization task of the two branches.

[0081] In order to further enhance the model's adversarial defense capability, in an adversarial training method for a deep convolutional neural network provided in an embodiment of the present application, step 600 of the adversarial training method for a deep convolutional neural network specifically includes the following:

[0082] Step 610: performing MLP mapping on the regular type features of the regular samples to remove noise components in the regular type features to obtain corresponding second denoised features, and performing MLP feature prediction on the second denoised features to extract a second robust representation;

[0083] Step 620: Align the first denoising feature with the second robust representation, and align the second denoising feature with the first robust representation to align neurons activated in the deep convolutional neural network by the regular sample and the adversarial attack sample, respectively.

[0084] See also Figure 4 As shown in the dotted box, after the two branch feature extractions are completed, this application uses asynchronous feature alignment to align q1 and k2 as well as q2 and k1 respectively, and aligns the neurons activated by conventional samples in the neural network with the neurons activated by robust features, thereby enhancing the robustness of the neural network model's representation capabilities and improving the model's adversarial defense capabilities.

[0085] With respect to the embodiment of the adversarial training method for the above-mentioned deep convolutional neural network, the present application further provides an online application process of the deep convolutional neural network. The image recognition method corresponding to the online application process specifically includes the following contents:

[0086] Step 010: Receive image data to be identified and pre-process the image data.

[0087] Step 020: Input the preprocessed image data into a deep convolutional neural network so that the deep convolutional neural network outputs a type recognition result corresponding to the image data, wherein the deep convolutional neural network is pre-trained based on any embodiment of the aforementioned adversarial training method of the deep convolutional neural network.

[0088] Step 030: Output the type recognition result.

[0089] From the above description, it can be seen that the image recognition method provided in the embodiment of the present application uses a deep convolutional neural network obtained by the adversarial training method based on the aforementioned deep convolutional neural network, which can be applied to many artificial intelligence applications based on deep convolutional neural network recognition, such as autonomous driving, smart home, and monitoring and security systems, and can effectively improve the effectiveness and reliability of the recognition results of the deep convolutional neural network.

[0090] From the software level, this application also provides a deep convolutional neural network adversarial training device for executing all or part of the deep convolutional neural network adversarial training method, see Figure 6 The adversarial training device for a deep convolutional neural network specifically includes the following contents:

[0091] The initial feature extraction module 10 is used to extract the initial features of the adversarial attack sample corresponding to the target image in the adversarial branch of the adversarial training process of the deep convolutional neural network;

[0092] The feature decoupling module 20 is used to perform feature decoupling on the initial features of the adversarial attack sample to obtain adversarial type features and interference features corresponding to the initial features, and perform feature extraction on the adversarial type features to obtain a first robust representation for identifying the target image type.

[0093] The embodiment of the adversarial training device for deep convolutional neural networks provided in this application can be specifically used to execute the processing flow of the embodiment of the adversarial training method for deep convolutional neural networks in the above-mentioned embodiment. Its functions are not described in detail here, and reference can be made to the detailed description of the embodiment of the adversarial training method for deep convolutional neural networks mentioned above.

[0094] The adversarial training device for deep convolutional neural networks performs adversarial training of deep convolutional neural networks in a portion that can be executed in a server. In another practical application scenario, all operations can also be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application is not limited to this. If all operations are completed in the client device, the client device may further include a processor for specific processing of the adversarial training of the deep convolutional neural network.

[0095] The client device may include a communication module (i.e., a communication unit) that can establish a communication connection with a remote server to implement data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0096] The server and the client device may communicate using any suitable network protocol, including network protocols that have not yet been developed as of the filing date of this application. Examples of such network protocols include TCP / IP, UDP / IP, HTTP, and HTTPS. Furthermore, examples of such network protocols include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer) protocols, which are used on top of the aforementioned protocols.

[0097] From the above description, it can be seen that the adversarial training device for the deep convolutional neural network provided in the embodiment of the present application decouples the initial features of the adversarial attack samples to obtain the adversarial type features and interference features corresponding to the initial features. Taking into account the dynamic nature of the adversarial attack samples participating in the training, the feature decoupling technology is used to separate the interference features in the adversarial attack samples, which can avoid overfitting the adversarial attack patterns participating in the training. Even if too many other interference features are mixed into the samples in the later stage of training, the adversarial training quality of the deep convolutional neural network can be guaranteed, the adversarial training effect for the deep convolutional neural network can be enhanced, and the robustness of the deep convolutional neural network can be further improved. The deep convolutional neural network obtained based on this adversarial training method can be applied to many artificial intelligence applications based on deep convolutional neural network recognition, such as autonomous driving, smart home, and monitoring and security systems, which can effectively improve the effectiveness and reliability of the recognition results of the deep convolutional neural network.

[0098] To further illustrate this solution, this application also provides a specific application example of an adversarial training method for a deep convolutional neural network, which is used to solve the following problems:

[0099] 1. How to accurately characterize the dynamics of the feature structure of the training adversarial attack samples. With the evolution of adversarial training, the features of the training adversarial attack samples will gradually change from imperceptible attack features to robust shape features, color features, texture features, etc. By observing the appearance of the samples, we can know the general change pattern, but it is very difficult to quantitatively characterize these features due to their abstract nature. To address this problem, the present invention uses the classification labels output by the training model as a quantitative reference and designs a dynamic adversarial labeling technology to characterize the change pattern of attack sample features.

[0100] 2. How to decouple the features of the training samples and filter out the mixed features of other categories. Due to the dynamic nature of the feature structure of the training adversarial attack samples, the specific proportion of other categories of features mixed in the samples is difficult to measure accurately. In addition, this high-dimensional feature is extremely abstract and cannot be characterized by specific parameters, which makes feature decoupling difficult. To address this problem, this application uses self-supervised learning technology to design adversarial attack feature decoupling subtasks, and designs a subsequent series of feature alignment methods to enhance the robustness of the neural network model's representation capabilities and improve the model's adversarial defense capabilities.

[0101] The application example of this application proposes an adversarial training technology based on feature decoupling. By modeling the dynamic composition of features of the adversarial attack samples in the adversarial training process, it is found that their features will gradually change from imperceptible attack features to robust shape features, color features, texture features, etc., and a dynamic adversarial labeling technology is designed to characterize its change law, and this change law is used to separate and decouple the robust features. With the robust features as cognitive anchors, the neural network aligns the erroneous features of the interference samples with them to achieve efficient robust enhancement. Then, a cognitive alignment method based on adversarial attack feature decoupling is designed, and the semantic feature structure of the adversarial attack samples is modeled and analyzed to identify the robust features and attack features therein. Then, the self-supervised learning technology is used to design the adversarial attack feature decoupling subtask, and the feature alignment technology is used to align the neurons activated by conventional samples in the neural network with the neurons activated by the robust features, so as to enhance the robustness of the neural network model representation ability and improve the model's adversarial defense capability.

[0102] See also Figure 4 The application example of this application shows the adversarial training technology based on feature decoupling. First, the application example of this application introduces the overall process of the adversarial training technology based on feature decoupling. Figure 4 As shown in Figure 2, the entire training framework is divided into two branches: the adversarial branch and the natural branch. Both branches take the same original image x as input and transform it into x1 and x2 through adversarial attacks and traditional image processing techniques. The backbone network then performs initial feature extraction on each of the two images.

[0103] In the adversarial branch, the application example of this application uses the feature decoupler f to divide the initial features of the adversarial attack sample into the part r1 mixed with other category features and the remaining features z1. Then, the application example of this application uses the MLP feature mapping module p to remove the noise components in z1, and finally uses the MLP feature prediction module to extract the robust representation q1 for recognition and classification.

[0104] After the feature extraction of the adversarial branch is completed, the mixed-in features r1 of other categories will be used for classification by the pattern-level classifier. The training label is the dynamic adversarial label, and its label value is the image category recognized by the current model. When the proportion of mixed-in features of other categories is greater than the features of the original category, the dynamic adversarial label is the mixed-in other category, otherwise it is the original true label. The robust representation q1 is used for the correct recognition and classification of the model, and the training label is the true label.

[0105] In addition, when training the feature decoupler f, the application example of this application uses KL divergence loss to calculate the feature distribution distance between the part of r1 mixed with other category features and the remaining features z1, and enhances the feature decoupling effect by maximizing this distance.

[0106] For natural branches, the application example of this application uses the feature extraction module other than the feature decoupler f to obtain the final robust representation q2, and the features of this branch do not participate in the training of the two classifiers, but only participate in the feature alignment optimization task of the two branches.

[0107] like Figure 4 As shown in the dotted box position in the figure, after the two branch features are extracted, the application example of this application uses asynchronous feature alignment to align q1 and k2 and q2 and k1 respectively, and aligns the neurons activated by conventional samples in the neural network with the neurons activated by robust features, thereby enhancing the robustness of the neural network model's representation ability and improving the model's defense capability.

[0108] The optimization loss functions involved in the entire feature decoupling-based adversarial training technology include the classification loss of the two classifiers, KL divergence loss, and contrast loss. First, the classification loss of the two classifiers is introduced:

[0109]

[0110] Where T is the matrix transpose operation, y adv ,c p , r1 are Figure 4 The dynamic adversarial labels, pattern-level classifiers, and other category features are mixed in. Substituting the elements in the formula with the parameters required by the image-level classifier, we can get its loss function:

[0111]

[0112] where y t ,c m ,q1 are Figure 4 The true label, image-level classifier, and robust representation in . The KL divergence loss can be expressed as:

[0113]

[0114] Using Cosine similarity metric In this application example, the contrast loss can be expressed as:

[0115]

[0116] By optimizing the above loss function in adversarial training, the application example of this application can use these auxiliary tasks to enhance the robustness of the neural network model's representation ability and improve the model's adversarial defense capabilities.

[0117] In summary, the application example of this application proposes an adversarial training technology based on feature decoupling, which models the dynamic change law of the feature composition of the adversarial attack samples in the adversarial training process, and uses this change law to separate and decouple the robust features. With the robust features as the cognitive anchor, the neural network aligns the erroneous features of the interference samples with them to achieve efficient robust enhancement. This method is simple to deploy, can be applied to any basic deep neural network, and has high generalization. The application example of this application can be applied in many artificial intelligence applications based on deep neural network recognition, such as autonomous driving, smart home, and monitoring security systems.

[0118] The key technical points of this application example are:

[0119] 1. A feature decoupling-based adversarial training technique is proposed. By modeling the dynamic change pattern of the feature composition of the adversarial attack samples during the adversarial training process, this change pattern is used to separate and decouple the robust features. Using the robust features as cognitive anchors, the neural network aligns the erroneous features of the interference samples with them, achieving efficient robust enhancement.

[0120] 2. This application demonstrates the dynamic nature of the feature structure of adversarial attack samples during training. As adversarial training progresses, the features of these samples gradually transform from subtle attack features to robust shape, color, and texture features. Based on this, a dynamic adversarial labeling algorithm based on the inference results of the participating models is proposed.

[0121] 3. The application examples of this application are highly generalizable and easy to deploy. They can be applied to any basic deep neural network and can be used in many applications based on deep neural network recognition, such as autonomous driving, smart home, and monitoring security systems.

[0122] The embodiment of the present application also provides a computer device (ie, an electronic device), which may include a processor, a memory, a receiver and a transmitter, wherein the processor is used to execute the adversarial training method or image recognition method of the deep convolutional neural network mentioned in the above embodiment, wherein the processor and the memory may be connected by a bus or other means, taking a bus connection as an example. The receiver may be connected to the processor and the memory by wire or wirelessly. The computer device is communicatively connected to the adversarial training device of the deep convolutional neural network to receive real-time motion data from the sensors in the wireless multimedia sensor network and to receive the original video sequence from the video acquisition device.

[0123] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0124] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the adversarial training method for a deep convolutional neural network or the image recognition method in the embodiments of the present application. The processor executes the non-transitory software programs, instructions, and modules stored in the memory to perform various functional applications and data processing of the processor, that is, to implement the adversarial training method for a deep convolutional neural network or the image recognition method in the above-mentioned method embodiments.

[0125] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0126] The one or more modules are stored in the memory, and when executed by the processor, perform the adversarial training method of the deep convolutional neural network in the embodiment.

[0127] In some embodiments of the present application, the user equipment may include a processor, a memory and a transceiver unit, and the transceiver unit may include a receiver and a transmitter. The processor, memory, receiver and transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0128] As an implementation method, the functions of the receiver and transmitter in this application can be considered to be implemented through a transceiver circuit or a dedicated transceiver chip, and the processor can be considered to be implemented through a dedicated processing chip, a processing circuit or a general-purpose chip.

[0129] As another implementation method, it is possible to use a general-purpose computer to implement the server provided in the embodiments of the present application. That is, the program code for implementing the functions of the processor, receiver, and transmitter is stored in a memory, and the general-purpose processor implements the functions of the processor, receiver, and transmitter by executing the code in the memory.

[0130] The embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the adversarial training method of the aforementioned deep convolutional neural network or the image recognition method. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the art.

[0131] It should be understood by those skilled in the art that the various exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0132] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0133] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0134] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Those skilled in the art will appreciate that various modifications and variations of the present embodiment are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for adversarial training of a deep convolutional neural network, characterized in that: include: In the adversarial branch of the adversarial training process of the deep convolutional neural network, the initial features of the adversarial attack samples corresponding to the target image are extracted; Performing feature decoupling on the initial features of the adversarial attack sample to obtain adversarial type features and interference features corresponding to the initial features, and performing MLP mapping on the adversarial type features to remove noise components in the adversarial type features to obtain corresponding first denoising features; performing MLP feature prediction on the first denoising features to extract a first robust representation for identifying the target image type; Inputting the interference features into a pattern-level classifier and training corresponding dynamic adversarial labels; Inputting the first robust representation into an image-level classifier and training a corresponding label, wherein the label is the original image category of the target image; Extracting regular type features of regular samples corresponding to target images in a natural branch of the adversarial training process of deep convolutional neural networks; Performing MLP mapping on the conventional type features of the conventional samples to remove noise components in the conventional type features to obtain corresponding second denoising features, and performing MLP feature prediction on the second denoising features to extract a second robust representation; The first denoised feature is aligned with the second robust representation, and the second denoised feature is aligned with the first robust representation to align neurons activated in the deep convolutional neural network by the regular sample and the adversarial attack sample, respectively.

2. The adversarial training method for a deep convolutional neural network according to claim 1, characterized in that: If the proportion of the interference feature in the initial feature is greater than the adversarial type feature, the dynamic adversarial label of the adversarial attack sample is: interference category; If the proportion of the interference features in the initial features is less than or equal to the adversarial type features, the dynamic adversarial label of the adversarial attack sample is: the original image category of the target image.

3. The adversarial training method for a deep convolutional neural network according to claim 1, characterized in that: The step of performing feature decoupling on the initial features of the adversarial attack sample to obtain adversarial type features and interference features corresponding to the initial features respectively includes: Using a preset feature decoupler to perform feature decoupling on the initial features of the adversarial attack sample; Wherein, the feature distribution distance between the adversarial type feature and the interference feature is determined based on the KL divergence loss method, and the feature distribution distance is maximized.

4. A method for identifying image types, characterized in that: include: Receiving image data to be identified and preprocessing the image data; Inputting the preprocessed image data into a deep convolutional neural network so that the deep convolutional neural network outputs a type recognition result corresponding to the image data, wherein the deep convolutional neural network is pre-trained based on the adversarial training method of the deep convolutional neural network according to any one of claims 1 to 3; The type identification result is output.

5. A deep convolutional neural network adversarial training device, characterized in that: include: An initial feature extraction module is used to extract the initial features of the adversarial attack sample corresponding to the target image in the adversarial branch of the adversarial training process of the deep convolutional neural network; A feature decoupling module is used to perform feature decoupling on the initial features of the adversarial attack sample to obtain adversarial type features and interference features corresponding to the initial features, and perform MLP mapping on the adversarial type features to remove noise components in the adversarial type features to obtain corresponding first denoising features; perform MLP feature prediction on the first denoising features to extract a first robust representation for identifying the target image type; The adversarial training device of the deep convolutional neural network is also used to perform the following: Inputting the interference features into a pattern-level classifier and training corresponding dynamic adversarial labels; Inputting the first robust representation into an image-level classifier and training a corresponding label, wherein the label is the original image category of the target image; Extracting regular type features of regular samples corresponding to target images in a natural branch of the adversarial training process of deep convolutional neural networks; Performing MLP mapping on the conventional type features of the conventional samples to remove noise components in the conventional type features to obtain corresponding second denoising features, and performing MLP feature prediction on the second denoising features to extract a second robust representation; The first denoised feature is aligned with the second robust representation, and the second denoised feature is aligned with the first robust representation to align neurons activated in the deep convolutional neural network by the regular sample and the adversarial attack sample, respectively.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the adversarial training method of the deep convolutional neural network as described in any one of claims 1 to 3, or implements the image type recognition method as described in claim 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the adversarial training method of the deep convolutional neural network as described in any one of claims 1 to 3, or implements the image type recognition method as described in claim 4.

Images