Model adversarial training method, device, storage medium and electronic device

By generating first-order and second-order adversarial samples, using perturbation information and phase information, the convolutional neural network model is subject to adversarial training, which solves the problem of the model's ability to classify clean samples after adversarial training, and achieves robustness and The balance of generalization.

CN118608899BActive Publication Date: 2024-11-05ZHEJIANG SCI RES INST OF TRANSPORT +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411036226.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-05
Estimated Expiration
2044-07-31

AI Technical Summary

Technical Problem

After adversarial training, while improving robust accuracy, the convolutional neural network model will reduce the classification ability of clean samples, making it difficult to balance the robustness and generalization of the model.

Method used

By generating first-order adversarial samples and second-order adversarial samples, using perturbation information and phase information of clean samples, the convolutional neural network model is trained to ensure that the model retains the classification ability of clean samples while improving robust accuracy while retaining the ability to classify clean samples. .

Benefits of technology

While improving the robust accuracy of the model, it effectively protects the classification ability of clean samples and avoids the reduction of generalization caused by adversarial training.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118608899B_ABST
    Figure CN118608899B_ABST
Patent Text Reader

Abstract

The present invention proposes a model adversarial training method, device, storage medium and electronic device, including: generating first-order adversarial samples based on disturbance information and clean samples, wherein the clean samples are label image samples without disturbance information added; generating second-order adversarial samples based on the amplitude information of the first-order adversarial samples and the phase information of the clean samples; performing adversarial training on the convolutional neural network model based on the second-order adversarial samples, and the convolutional neural network model after adversarial training is used to classify images. Because the second-order adversarial samples carry the phase information of the clean samples and the amplitude information of the first-order adversarial samples, the convolutional neural network model is adversarially trained based on the second-order adversarial samples, and the classification ability of the convolutional neural network model for clean samples is guaranteed while improving the robust accuracy of the convolutional neural network model through adversarial training.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data recognition technology, and in particular to a model adversarial training method, device, storage medium and electronic device. Background Art

[0002] With the continuous development of deep learning, convolutional neural network models (CNN) have been widely used in image classification tasks, but even the most advanced neural networks are vulnerable to adversarial samples. In practical applications, adversarial samples can even harm the recognition of traffic signs and deceive face recognition systems in autonomous driving. In order to make the convolutional neural network model robust against adversarial attacks, it needs to be trained adversarially. In the field of image classification, adversarial training uses adversarial samples to train the model, forcing the model to adapt to the adversarial samples and classify them correctly, thereby achieving a defensive effect.

[0003] Adversarial training is one of the most powerful ways to enhance the robustness of models, but there is a problem. Although the model after adversarial training can greatly improve the robustness accuracy, it will reduce the generalization, that is, it will affect the classification ability of ordinary clean samples. Because the samples used in adversarial training are adversarial samples, the model learns too many features of adversarial samples and ignores some essential features of clean samples, thereby reducing the classification ability of clean samples.

[0004] How to ensure that adversarial training improves the robustness and accuracy of the model while ensuring the model's ability to classify clean samples has become a difficult problem that plagues technical personnel in this field. Summary of the invention

[0005] The object of the present invention is to provide a model adversarial training method, device, storage medium and electronic device to at least partially improve the above-mentioned problems.

[0006] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0007] In a first aspect, an embodiment of the present invention provides a model adversarial training method, the method comprising:

[0008] Generate a first-order adversarial sample based on the disturbance information and the clean sample, wherein the clean sample is a labeled image sample without adding disturbance information;

[0009] Generate a second-order adversarial sample based on the amplitude information of the first-order adversarial sample and the phase information of the clean sample;

[0010] The convolutional neural network model is adversarially trained based on the second-order adversarial samples, and the convolutional neural network model after the adversarial training is used to classify images.

[0011] In a second aspect, an embodiment of the present invention provides an image classification method, the method comprising:

[0012] Get the image to be classified;

[0013] The image to be classified is input into a target network model to obtain category information of the image to be classified; wherein the target network model is a convolutional neural network model completed by the model adversarial training method described in the first aspect.

[0014] In a third aspect, an embodiment of the present invention provides a model adversarial training device, the device comprising:

[0015] A first processing unit, configured to generate a first-order adversarial sample based on disturbance information and a clean sample, wherein the clean sample is a labeled image sample to which no disturbance information is added;

[0016] The first processing unit is further used to generate a second-order adversarial sample based on the amplitude information of the first-order adversarial sample and the phase information of the clean sample;

[0017] The second processing unit is used to perform adversarial training on the convolutional neural network model based on the second-order adversarial samples, and the convolutional neural network model after the adversarial training is used to classify images.

[0018] In a fourth aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon, and the computer program implements the above method when executed by a processor.

[0019] In a fifth aspect, an embodiment of the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store one or more programs; when the one or more programs are executed by the processor, the above method is implemented.

[0020] Compared with the prior art, the model adversarial training method, device, storage medium and electronic device provided by the embodiments of the present invention include: generating first-order adversarial samples based on disturbance information and clean samples, wherein the clean samples are label image samples without disturbance information added; generating second-order adversarial samples based on the amplitude information of the first-order adversarial samples and the phase information of the clean samples; adversarially training the convolutional neural network model based on the second-order adversarial samples, and the convolutional neural network model after adversarial training is used to classify images. Because the second-order adversarial samples carry the phase information of the clean samples and the amplitude information of the first-order adversarial samples, adversarially training the convolutional neural network model based on the second-order adversarial samples ensures the classification ability of the convolutional neural network model for clean samples while improving the robust accuracy of the convolutional neural network model through adversarial training.

[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic diagram of the structure of an electronic device provided by the present invention;

[0024] Figure 2 This is one of the flow charts of the model adversarial training method provided by the present invention;

[0025] Figure 3 The second flowchart of the model adversarial training method provided by the present invention;

[0026] Figure 4 The third flowchart of the model adversarial training method provided by the present invention;

[0027] Figure 5 A schematic diagram of the process flow of the image classification method provided by the present invention;

[0028] Figure 6 A unit schematic diagram of a model adversarial training device provided by the present invention.

[0029] In the figure: 10 - processor; 11 - memory; 12 - bus; 13 - communication interface; 301 - first processing unit; 302 - second processing unit. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0031] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0033] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0034] In the description of the present invention, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, or are directions or positional relationships in which the product of the invention is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as a limitation on the present invention.

[0035] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "disposed" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0036] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0037] In order to better play the role of defense methods, researchers began to explore the essential characteristics of adversarial samples from the perspective of signal processing in the frequency domain, so as to improve the effectiveness of defense methods. Researchers studied adversarial samples from the frequency domain perspective and found that the convolutional neural network model (CNN) mainly uses high-frequency components that cannot be perceived by humans for image classification, while the low-frequency components of the image are more related to semantic information. In addition, it was found that the convolutional neural network model (CNN) mainly relies on the amplitude spectrum of the image for prediction, while humans rely more on the phase spectrum to identify objects. For example, the phase spectrum of the pistol image and the amplitude spectrum of the puzzle image are reconstructed into a new image. The human eye can still recognize the pistol in it, but the convolutional neural network model (CNN) will only classify it as a puzzle when recognizing a new image.

[0038] Adversarial training has the following problems: although the model after adversarial training can greatly improve the robust accuracy, it will cause a decrease in generalization, that is, it will affect the classification ability of ordinary clean samples. Since the samples used in adversarial training are adversarial samples, the model has learned too many features of adversarial samples and ignored some essential features of clean samples, thereby reducing the classification ability of clean samples. In order to improve the classification ability of the model for clean samples after adversarial training, an embodiment of the present invention proposes a model adversarial training method, which uses the essential characteristics of adversarial samples to guide the training of a convolutional neural network model, and ensures the classification ability of the convolutional neural network model for clean samples while improving the robust accuracy of the convolutional neural network model through adversarial training.

[0039] The embodiment of the present invention provides an electronic device, which may be a computer device, a mobile phone device, a server device, etc. Figure 1 , a schematic diagram of the structure of an electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected via the bus 12, and the processor 10 is used to execute an executable module stored in the memory 11, such as a computer program.

[0040] The processor 10 may be an integrated circuit chip having signal processing capabilities. In the implementation process, each step of the model adversarial training method and the image classification method may be completed by an integrated logic circuit of hardware or software instructions in the processor 10. The above-mentioned processor 10 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components.

[0041] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0042] The bus 12 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Figure 1 Although only one bidirectional arrow is used in the diagram, it does not mean that there is only one bus 12 or only one type of bus 12 .

[0043] The memory 11 is used to store programs, such as programs corresponding to the model adversarial training device and the image classification device. The model adversarial training device and the image classification device include at least one software function module that can be stored in the memory 11 in the form of software or firmware or solidified in the operating system (OS) of the electronic device. After receiving the execution instruction, the processor 10 executes the program to implement the model adversarial training method and the image classification method.

[0044] Possibly, the electronic device provided in the embodiment of the present invention further includes a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus. The electronic device can receive image data transmitted by other terminals via the communication interface 13, including clean samples for training and their corresponding category label data.

[0045] It should be understood that Figure 1 The structure shown is only a schematic diagram of a portion of the electronic device. The electronic device may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0046] The model adversarial training method provided by the embodiment of the present invention can be applied to, but not limited to, Figure 1 For detailed procedures, please refer to the electronic equipment shown in Figure 2 , the model adversarial training method includes: S102, S103 and S104, which are specifically described as follows.

[0047] S102, generating a first-order adversarial sample based on the perturbation information and the clean sample.

[0048] Among them, clean samples are labeled image samples without added perturbation information.

[0049] Optionally, the formula for the first-order adversarial sample is:

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056] in, Indicates Clean samples The corresponding first-order adversarial example, Indicates A clean sample, represents disturbance information, Indicates the sensitivity of high and low frequencies. represents the perturbation step size, represents the symbolic function, represents discrete cosine transform, represents the inverse discrete cosine transform, Represents the loss function For variables Find the gradient, represents the cross entropy loss function, Indicates that the parameter is Classification model For Clean samples The classification results, Indicates The labels corresponding to the clean samples are represents the mask matrix used to select low-frequency information, represents the mask matrix used to select high-frequency information, represents the first in the mask matrix used to select low-frequency information Line The element value of the column, represents the number of the mask matrix used to select high-frequency information. Line Column element value, low frequency band Indicates that the Euclidean distance to the center of the matrix is ​​less than area, high frequency band Indicates that the Euclidean distance to the center of the matrix is ​​greater than or equal to area, r is the preset distance threshold.

[0057] Considering that adversarial samples have specific properties in the frequency domain, discrete cosine transform is used to transform the image from pixel space to frequency space. The problem of single location introduces high and low frequency sensitivity parameters , which is used to control the distribution of adversarial perturbations in the frequency space. Since different types of images have different sensitivities to low-frequency and high-frequency components in the frequency, the high and low frequency sensitivity parameters are used. The weights of the adversarial perturbations on the low-frequency and high-frequency components can be controlled.

[0058] when When , it means that only disturbance information is added to the high-frequency component; when , it means that disturbance information is only added to the low-frequency component. The larger the value, the more the adversarial disturbance is concentrated in the low-frequency area. Considering that low-frequency information has a strong correlation with the label, in the subsequent adversarial training, it is hoped that even if there is adversarial perturbation on the low-frequency component of the image, the convolutional neural network model can still classify it correctly, that is, improve the adversarial robustness of the model.

[0059] In an optional implementation, the high and low frequency sensitivity parameters It can be set to 0.6 to add more perturbation information to the low-frequency components, forcing the low-frequency components infected by the perturbation to still be strongly correlated with the correct label, thereby achieving an adversarial effect.

[0060] S103, generating a second-order adversarial sample based on the amplitude information of the first-order adversarial sample and the phase information of the clean sample.

[0061] The convolutional neural network model is constructed by imitating the structure of the human brain. The human brain mainly uses the phase information of the image to distinguish objects, and the transformation of the amplitude spectrum does not affect the classification of objects by the human eye. Therefore, a good convolutional neural network model should be similar to the human brain and pay more attention to the phase information of the image. Compared with the adversarial training method in which the model overfits the adversarial samples and forgets some features of the clean samples at the expense of clean accuracy, the present invention introduces the phase information of the clean samples in the adversarial training, and at the same time, in order to ensure that the model is robust to the adversarial samples, the amplitude information of the first-order adversarial samples is used to form the second-order adversarial samples.

[0062] S104, performing adversarial training on the convolutional neural network model based on the second-order adversarial samples.

[0063] Among them, the convolutional neural network model after adversarial training is used to classify images.

[0064] Because the second-order adversarial samples carry the phase information of clean samples and the amplitude information of the first-order adversarial samples, adversarial training is performed on the convolutional neural network model based on the second-order adversarial samples. While the robust accuracy of the convolutional neural network model is improved through adversarial training, the classification ability of the convolutional neural network model for clean samples is guaranteed.

[0065] In summary, the present invention provides a model adversarial training method, including: generating first-order adversarial samples based on disturbance information and clean samples, wherein the clean samples are label image samples without added disturbance information; generating second-order adversarial samples based on the amplitude information of the first-order adversarial samples and the phase information of the clean samples; performing adversarial training on the convolutional neural network model based on the second-order adversarial samples, and the convolutional neural network model after the adversarial training is used to classify images. Because the second-order adversarial samples carry the phase information of the clean samples and the amplitude information of the first-order adversarial samples, the convolutional neural network model is adversarially trained based on the second-order adversarial samples, and the classification ability of the convolutional neural network model for clean samples is guaranteed while improving the robust accuracy of the convolutional neural network model through adversarial training.

[0066] exist Figure 2 On the basis of how to ensure the accuracy of the convolutional neural network model, the embodiment of the present invention also provides an optional implementation method, please refer to Figure 3 , after performing adversarial training on the convolutional neural network model based on the second-order adversarial sample in S104, the model adversarial training method also includes: S105 and S106, which are specifically described as follows.

[0067] S105, determining whether the convolutional neural network model after adversarial training has converged. If so, the process ends; if not, the process proceeds to S106.

[0068] S106, updating the disturbance information based on the loss function of the convolutional neural network model.

[0069] As mentioned above, the perturbation information Loss function with convolutional neural network model For variables The result of finding the gradient When the loss function of the convolutional neural network model changes, the perturbation information needs to be updated.

[0070] In the first iteration, we can select Determine the random value as the initial disturbance information , to generate corresponding adversarial samples.

[0071] After executing S106 and the disturbance information is updated, S102 is repeatedly executed to generate first-order adversarial samples based on the disturbance information and the clean samples until the convolutional neural network model converges or the number of iterative training exceeds a preset value.

[0072] It should be noted that when the execution result of S105 is that the convolutional neural network model has not converged, it can be determined whether the current number of iterations exceeds the preset value. If not, S106 is executed, otherwise, the process ends.

[0073] It should be noted that the high and low frequency sensitivity parameters The value of directly affects the effect of the model adversarial training provided by the present invention. In order to balance improving the robustness accuracy of the convolutional neural network model and ensuring the classification ability of the convolutional neural network model for clean samples, how to set the high and low frequency sensitivity parameters The present invention also provides an optional implementation method, please refer to Figure 4 , before generating the first-order adversarial sample based on the disturbance information and the clean sample in S102, the model adversarial training method also includes: S101, which is specifically described as follows.

[0074] S101, obtaining the high and low frequency sensitivity of the clean sample.

[0075] Optionally, the high and low frequency sensitivity of the clean sample can be determined based on the characteristic information of the clean sample, including whether the image has a large block of uniform color or brightness and whether the image contains fine structures and textures.

[0076] Alternatively, the feature information includes a first proportion of images with large blocks of uniform color or brightness in the clean sample and a second proportion of images containing fine structures and textures in the clean sample. The first proportion and the second proportion are combined with corresponding weights to determine the high and low frequency sensitivity.

[0077] Among them, images with large blocks of uniform color or brightness are usually more sensitive to low-frequency noise, because these images are mainly composed of color and brightness changes in low-frequency components, and noise will directly affect their overall color or brightness. Images containing a large number of subtle structures and textures are more susceptible to high-frequency noise, because these details are often reflected in high-frequency components, such as natural scene images with rich textures such as leaves, grass, and feathers.

[0078] exist Figure 2 Based on the content in S103, the embodiment of the present invention further provides an optional implementation, please refer to the following. S103, the step of generating a second-order adversarial sample based on the amplitude information of the first-order adversarial sample and the phase information of the clean sample, includes: S103A, specifically described as follows.

[0079] S103A, performing an inverse discrete Fourier transform based on the amplitude information of the first-order adversarial sample and the phase information of the clean sample to obtain a second-order adversarial sample.

[0080] Optionally, the formula for the second-order adversarial sample is:

[0081]

[0082] in, represents the second-order adversarial sample corresponding to the i-th clean sample, represents the inverse discrete Fourier transform, is the amplitude spectrum, It represents the amplitude information of the first-order adversarial sample corresponding to the i-th clean sample, is the phase spectrum, represents the phase information of the i-th clean sample, Represents the element-wise multiplication of two matrices.

[0083] The algorithm of the frequency domain-based model adversarial training provided by the present invention is as follows:

[0084] Input: Input image ;Label ;Model ; Disturbance size ; Step size ; High and low frequency sensitivity ;Multi-step iterative attack times ; Learning rate ;

[0085] Output: Robust model

[0086] 1: Repeat

[0087] 2: Initialization ,

[0088] 3:for do

[0089] 4:

[0090] 5:

[0091] 6:

[0092] 7:

[0093] 8:

[0094] 9:

[0095] 10:

[0096] 11:

[0097] 12: end for

[0098] 13:

[0099] 14: Until training converges.

[0100] in, .

[0101] The present invention also provides an image classification method, which can be applied to but not limited to Figure 1 It should be noted that the electronic device corresponding to the image classification method and the electronic device corresponding to the model adversarial training method may be different. Figure 5 , the image classification method provided by the embodiment of the present invention includes: S201 and S202, which are specifically described as follows.

[0102] S201, obtaining an image to be classified.

[0103] S202, inputting the image to be classified into the target network model to obtain category information of the image to be classified.

[0104] Among them, the target network model is a convolutional neural network model completed based on the above-mentioned model adversarial training method.

[0105] It should be noted that the category information referred to in the present invention may be category information of objects included in the image, such as leaves, kittens, puppies, cars, traffic lights, airplanes, and the like.

[0106] See also Figure 6 , Figure 6 A model adversarial training device and an image classification device are provided in an embodiment of the present invention. Optionally, the model adversarial training method and the image classification device are applied to the electronic device described above.

[0107] The model adversarial training device includes: a first processing unit 301 and a second processing unit 302.

[0108] A first processing unit, configured to generate a first-order adversarial sample based on the disturbance information and the clean sample, wherein the clean sample is a labeled image sample to which no disturbance information is added;

[0109] The first processing unit is also used to generate a second-order adversarial sample based on the amplitude information of the first-order adversarial sample and the phase information of the clean sample;

[0110] The second processing unit is used to perform adversarial training on the convolutional neural network model based on the second-order adversarial sample, and the convolutional neural network model after the adversarial training is used to classify the image.

[0111] It should be noted that the model adversarial training device provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effect. For the sake of brief description, for parts not mentioned in this embodiment, please refer to the corresponding content in the above embodiment.

[0112] The embodiment of the present invention further provides a storage medium, which stores computer instructions and programs, which, when read and run, execute the model adversarial training method and image classification method of the above embodiment. The storage medium may include memory, flash memory, registers, or a combination thereof.

[0113] An electronic device is provided below, which may be a computer device, a mobile phone device, a server device, etc. Figure 1 As shown, the above-mentioned model adversarial training method and image classification method can be implemented; specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 can be a CPU. The memory 11 is used to store one or more programs. When the one or more programs are executed by the processor 10, the model adversarial training method and image classification method of the above-mentioned embodiment are executed.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0115] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A model adversarial training method, characterized in that: The method comprises: Generate a first-order adversarial sample based on the disturbance information and the clean sample, wherein the clean sample is a labeled image sample without adding disturbance information; Generate a second-order adversarial sample based on the amplitude information of the first-order adversarial sample and the phase information of the clean sample; Performing adversarial training on the convolutional neural network model based on the second-order adversarial sample, and using the convolutional neural network model after the adversarial training to classify images; The formula for the first-order adversarial sample is: in, Indicates Clean samples The corresponding first-order adversarial example, Indicates A clean sample, represents disturbance information, Indicates the sensitivity of high and low frequencies. represents the perturbation step size, represents the symbolic function, represents discrete cosine transform, represents the inverse discrete cosine transform, Represents the loss function For variables Find the gradient, represents the cross entropy loss function, Indicates that the parameter is Classification model For Clean samples The classification results, Indicates The labels corresponding to the clean samples are represents the mask matrix used to select low-frequency information, represents the mask matrix used to select high-frequency information, represents the first in the mask matrix used to select low-frequency information Line The element value of the column, represents the number of the mask matrix used to select high-frequency information. Line Column element value, low frequency band Indicates that the Euclidean distance to the center of the matrix is ​​less than area, high frequency band Indicates that the Euclidean distance to the center of the matrix is ​​greater than or equal to area; Before generating the first-order adversarial sample based on the disturbance information and the clean sample, the model adversarial training method further includes: Determine the high and low frequency sensitivity of the clean sample according to the first proportion value and the second proportion value combined with corresponding weights; The first proportion is a proportion of images with large blocks of uniform color or brightness in the clean sample, and the second proportion is a second proportion of images containing fine structures and textures in the clean sample.

2. The model adversarial training method according to claim 1, characterized in that: After performing adversarial training on the convolutional neural network model based on the second-order adversarial sample, the method further includes: Determine whether the convolutional neural network model converges after adversarial training is completed; If not converged, updating the disturbance information based on the loss function of the convolutional neural network model; After the disturbance information is updated, first-order adversarial samples are repeatedly generated based on the disturbance information and clean samples until the convolutional neural network model converges or the number of iterative training exceeds a preset value.

3. The model adversarial training method according to claim 1, characterized in that: The step of generating a second-order adversarial sample based on the amplitude information of the first-order adversarial sample and the phase information of the clean sample comprises: Based on the amplitude information of the first-order adversarial sample and the phase information of the clean sample, an inverse discrete Fourier transform is performed to obtain the second-order adversarial sample.

4. The model adversarial training method according to claim 3, characterized in that: The formula for the second-order adversarial sample is: in, represents the second-order adversarial sample corresponding to the i-th clean sample, represents the inverse discrete Fourier transform, is the amplitude spectrum, represents the amplitude information of the first-order adversarial sample corresponding to the i-th clean sample, is the phase spectrum, represents the phase information of the i-th clean sample, Represents the element-wise multiplication of two matrices.

5. An image classification method, characterized in that: The method comprises: Get the image to be classified; The image to be classified is input into a target network model to obtain category information of the image to be classified; wherein the target network model is a convolutional neural network model completed by the model adversarial training method according to any one of claims 1 to 4.

6. A model adversarial training device, characterized in that: The device comprises: A first processing unit, configured to generate a first-order adversarial sample based on disturbance information and a clean sample, wherein the clean sample is a labeled image sample to which no disturbance information is added; The first processing unit is further used to generate a second-order adversarial sample based on the amplitude information of the first-order adversarial sample and the phase information of the clean sample; A second processing unit, used for performing adversarial training on the convolutional neural network model based on the second-order adversarial sample, and the convolutional neural network model after the adversarial training is used to classify the image; The formula for the first-order adversarial sample is: in, Indicates Clean samples The corresponding first-order adversarial example, Indicates A clean sample, represents disturbance information, Indicates the sensitivity of high and low frequencies. represents the perturbation step size, represents the symbolic function, represents discrete cosine transform, represents the inverse discrete cosine transform, Represents the loss function For variables Find the gradient, represents the cross entropy loss function, Indicates that the parameter is Classification model For Clean samples The classification results, Indicates The labels corresponding to the clean samples are represents the mask matrix used to select low-frequency information, represents the mask matrix used to select high-frequency information, represents the first in the mask matrix used to select low-frequency information Line The element value of the column, represents the number of the mask matrix used to select high-frequency information. Line Column element value, low frequency band Indicates that the Euclidean distance to the center of the matrix is ​​less than area, high frequency band Indicates that the Euclidean distance to the center of the matrix is ​​greater than or equal to area; Before generating the first-order adversarial sample based on the disturbance information and the clean sample, the model adversarial training device is further used to determine the high and low frequency sensitivity of the clean sample according to the first proportion value and the second proportion value combined with the corresponding weights; The first proportion is a proportion of images with large blocks of uniform color or brightness in the clean sample, and the second proportion is a second proportion of images containing fine structures and textures in the clean sample.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store one or more programs; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 5 is implemented.