Generative adversarial sample generation method based on high-frequency feature guidance

Abstract

The invention provides a generative adversarial sample generation method based on high-frequency feature guidance. The method comprises the following steps: acquiring an original image sample and a corresponding real category index, processing the original image sample, and extracting a high-frequency feature component of the original image sample; splicing the original image sample and the corresponding high-frequency feature component in a channel dimension to obtain multi-channel fusion data; inputting the multi-channel fusion data into a generative adversarial network model, wherein the model comprises a generator and a discriminator; the generator is used for generating a confrontation sample; the discriminator is used for generating category output values of the original image sample and the adversarial sample, mapping the difference between the category output values of the original image sample and the adversarial sample into category probability distribution, and obtaining a relative cross entropy loss function based on the category probability distribution and a real category index of the original image sample; and by maximizing the relative cross entropy loss function, network parameters of the generator are updated by utilizing back propagation, so that the optimized generator is obtained.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of artificial intelligence security, in particular to a generative adversarial sample generation method, system and device based on high-frequency feature guidance and a storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, artificial intelligence technology represented by deep learning has made remarkable achievements in image recognition, speech recognition, natural language processing and other tasks. Although deep learning technology has made breakthrough progress in image recognition and other tasks, the inherent endogenous vulnerability of deep learning models has brought serious security risks to image recognition tasks. Researchers have found that adding some tiny and visually imperceptible perturbations to the input image of a deep learning model can cause the model to produce incorrect recognition results, which is called adversarial attack, and the new image data derived from the perturbation is called adversarial sample.

[0003] However, existing methods are mainly divided into two categories: white-box attack and black-box attack. In white-box attack, the attacker knows all the details of the target model, including model structure, parameters and other information. Black-box attack cannot obtain the internal structure and parameter information of the target model, which leads to significant technical bottlenecks in actual application. The specific performance is that the cross-model migration ability of the adversarial sample is weak, and the attack efficiency is low. Studies have shown that deep learning models have significant sensitivity to high-frequency features. Traditional methods are mostly based on pixel-level perturbation optimization and cannot fully exploit the influence of image high-frequency components (such as edges and textures) on model decision-making. Existing methods do not construct an effective mechanism to guide high-frequency feature learning, resulting in a lack of pertinence in the frequency domain distribution of the generated perturbation, and the attack efficiency is limited. For example, iterative methods based on gradient optimization (such as fast gradient descent (FGSM) and gradient projection (PGD)) only focus on global error and do not establish the relevance of high-frequency features and perturbation generation, making it difficult to generate adversarial samples with strong penetration.

[0004] Therefore, how to study an adversarial sample generation method to solve the problems of insufficient utilization of high-frequency features and weak generalization of loss function in existing adversarial attack methods. SUMMARY

[0005] The embodiments of the application provide a generative adversarial sample generation method, system, device and storage medium based on high-frequency feature guidance to solve the problems of insufficient utilization of high-frequency features and weak generalization of loss function in existing adversarial attack methods.

[0006] To achieve the above purpose, the technical scheme adopted by the application is as follows:

[0007] In a first aspect, the application provides a method for generating generative adversarial samples based on high-frequency feature guidance, which comprises the following steps: S1, obtaining an original image sample and a true class index of the original image sample, performing high-pass filtering on the original image sample, and extracting a high-frequency feature component of the original image sample;

[0008] S2, splicing the original image sample and the high-frequency feature component of the original image sample in the channel dimension to obtain multi-channel fusion data;

[0009] S3, inputting the multi-channel fusion data into a generative adversarial network model, wherein the generative adversarial network model comprises a generator and a discriminator;

[0010] The generator is configured to generate an adversarial sample with high-frequency sensitive characteristics, wherein the perturbation amplitude of the adversarial sample is limited within a preset threshold;

[0011] The discriminator is configured to generate a class output value of the original image sample and the adversarial sample, map the difference between the class output values of the original image sample and the adversarial sample into a class probability distribution by using a Softmax function, and obtain a relative cross-entropy loss function based on the class probability distribution and the true class index of the original image sample;

[0012] S4, updating the network parameters of the generator by maximizing the relative cross-entropy loss function through back propagation;

[0013] S5, repeating steps S3-S4 until a preset number of iterations is reached, and obtaining an optimized generator.

[0014] In a second aspect, the application provides a system for generating generative adversarial samples based on high-frequency feature guidance, which comprises:

[0015] An acquisition module and a processing module are configured to obtain an original image sample and a true class index of the original image sample, perform high-pass filtering on the original image sample, and extract a high-frequency feature component of the original image sample;

[0016] A splicing module is configured to splice the original image sample and the high-frequency feature component of the original image sample in the channel dimension to obtain multi-channel fusion data;

[0017] A generative adversarial network model module is configured to input the multi-channel fusion data into a generative adversarial network model, wherein the generative adversarial network model comprises a generator and a discriminator;

[0018] The generator is configured to generate an adversarial sample with high-frequency sensitive characteristics, wherein the perturbation amplitude of the adversarial sample is limited within a preset threshold;

[0019] The discriminator is configured to generate a category output value of the original image sample and the adversarial sample, and map a difference between the category output value of the original image sample and the adversarial sample into a category probability distribution by using a Softmax function, and obtain a relative cross-entropy loss function based on the category probability distribution and a true category index of the original image sample.

[0020] The optimization module is configured to update the network parameter of the generator by maximizing the relative cross-entropy loss function by using back propagation.

[0021] In a third aspect, a device for generating a generative adversarial sample based on high-frequency feature guidance is provided. The device includes modules configured to perform the method of the first aspect.

[0022] In a possible design, the device for generating a generative adversarial sample based on high-frequency feature guidance of the third aspect can further include a transceiver. The transceiver can be a transceiver circuit or an interface circuit. The transceiver can be configured to enable the device for generating a generative adversarial sample based on high-frequency feature guidance of the third aspect to communicate with other devices.

[0023] In a possible design, the device for generating a generative adversarial sample based on high-frequency feature guidance of the third aspect can further include a memory. The memory can be integrated with the processor, or can be separately arranged. The memory can be configured to store instructions related to the method of the first aspect.

[0024] In a fourth aspect, a device for generating a generative adversarial sample based on high-frequency feature guidance is provided. The device includes a processor and a memory. The processor is coupled to the memory. The processor is configured to execute instructions stored in the memory, so that the device for generating a generative adversarial sample based on high-frequency feature guidance performs the method of the first aspect.

[0025] In a possible design, the device for generating a generative adversarial sample based on high-frequency feature guidance of the fourth aspect can further include a transceiver. The transceiver can be a transceiver circuit or an interface circuit. The transceiver can be configured to enable the device for generating a generative adversarial sample based on high-frequency feature guidance of the fourth aspect to communicate with other devices.

[0026] In a fifth aspect, a device for generating a generative adversarial sample based on high-frequency feature guidance is provided. The device includes a processor and a memory. The memory is configured to store instructions. When the processor executes the instructions, the device for generating a generative adversarial sample based on high-frequency feature guidance performs the method of the first aspect.

[0027] In a possible design, the high-frequency feature guided generative adversarial sample generation apparatus of the fifth aspect can further include a transceiver. The transceiver can be a transceiver circuit or an interface circuit. The transceiver can be configured to enable the high-frequency feature guided generative adversarial sample generation apparatus of the fifth aspect to communicate with other apparatuses.

[0028] In a sixth aspect, a computer-readable storage medium is provided, which includes a computer program or instructions stored therein, and when the computer program or instructions are executed, the high-frequency feature guided generative adversarial sample generation method of the first aspect is performed.

[0029] In the embodiments of the present application, the high-frequency feature components (such as edges and textures) of an image are extracted by high-pass filtering, and are spliced with original image samples in a channel dimension to form multi-channel fusion data, so as to realize dynamic association of high-frequency features and spatial semantics, and guide the generator to focus on the model sensitive area. Meanwhile, based on a relative cross loss function, the difference between the output values of the adversarial samples and the original image samples on the discriminator is calculated, the difference is mapped to a probability distribution by using a Sofmax function, the generator is optimized to maximize the distance between the probability distribution and the true label distribution, and the generator is forced to learn the essential feature offset rule rather than the local decision boundary, so as to alleviate the overfitting problem caused by the excessive dependence of the traditional cross entropy loss on the training data.

[0030] Other features and advantages of the present application will be described in detail in the following detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0032] Figure 1 A flowchart of the high-frequency feature guided generative adversarial sample generation method provided by the embodiments of the present application is shown in the figure.

[0033] Figure 2 A schematic diagram of the generator generating unconstrained perturbation adversarial samples and constrained perturbation adversarial samples using different classification discriminators is shown in the figure.

[0034] Figure 3 A structure diagram of the high-frequency feature guided generative adversarial sample generation apparatus provided by the embodiments of the present application is shown in the figure. Figure 1 ;

[0035] Figure 4Structure diagram of the device for generating generative adversarial samples based on high-frequency feature guidance provided by the embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person skilled in the art without creative work fall within the scope of protection of the present application. Meanwhile, in the description of the embodiments of the present application, the terms “first”, “second”, and the like are only used for differentiation and description, and cannot be understood as indicating or implying relative importance. Therefore, the features with “first” and “second” can explicitly or implicitly include one or more features. In the description of the embodiments of the present application, the meaning of “multiple” is two or more, unless otherwise specifically limited.

[0037] Figure 1 Flowchart of the method for generating generative adversarial samples based on high-frequency feature guidance provided by the embodiment of the present application.

[0038] The flow of the method for generating generative adversarial samples based on high-frequency feature guidance is as follows:

[0039] In step S1, an original image sample and a true class index of the original image sample are obtained, and high-pass filtering is performed on the original image sample to extract a high-frequency feature component of the original image sample.

[0040] The true class index of the original image sample can be understood as the position index of the actual class corresponding to the image in the class set, that is, there is a class set C = {c1, c2, c3,..., cn}, and the original image sample belongs to the second class c2 in the class set. Therefore, the true class index of the original image sample is {0, 1, 0,..., 0}. Specific applications are described below, and will not be described here.

[0041] The high-pass filtering performed on the original image sample to extract the high-frequency feature component of the original image sample can be understood as performing spatial domain convolution operation on the original image sample using a preset high-pass convolution kernel to extract the high-frequency feature component of the original image sample. The extraction expression is:

[0042] H(x, y) = I(x, y) * K hp

[0043] Wherein, H(x, y) represents the high-frequency feature component of the original image sample, I(x, y) represents the original image sample, (x, y) represents the pixel position in the original image sample, and Khp represents a preset high-pass convolution kernel, and * represents a convolution operation.

[0044] Exemplarily, a spatial domain high-pass filtering process is performed on an input original image sample I(x, y) e R H×W×3 (Height H, width W, RGB three channels) to obtain a high-frequency feature component H(x, y) e R hp of the original image. Specifically, a preset Laplacian convolution kernel K H×W×3 is adopted to perform a two-dimensional convolution operation on the original image to extract the high-frequency feature component H(x, y) e R

[0045] wherein the Laplacian convolution kernel is specifically expressed as:

[0046] The convolution operation is specifically expressed as: wherein (x, y) represents a pixel position, and c e {R, G, B} represents a channel index.

[0047] Through the operation, the rapidly changing regions (such as edges, textures and high-frequency details) in the image are significantly enhanced, and the low-frequency background information is suppressed.

[0048] Step S2, the original image sample and the high-frequency feature component of the original image sample are spliced in the channel dimension to obtain multi-channel fusion data.

[0049] That is, the RGB three-channel data of the original image sample and the three-channel data of the high-frequency feature component of the original image sample are spliced in the channel dimension to obtain multi-channel fusion data, wherein the RGB three-channel data of the original image sample contains color information of the original image sample, and the three-channel data of the high-frequency feature component of the original image sample contains detail features of the original image sample.

[0050] Exemplarily, the 3-channel data of the original image I(x, y) and the 3-channel data of the high-frequency component H(x, y) are spliced in the channel dimension to form a 6-channel fusion input matrix, and its expression is X fuse = Concat(I, H) e R H×W×6 .

[0051] It can be understood that, in the above step S2, the high-frequency component in the image data is extracted by the Laplacian convolution kernel, which can accurately capture the local detail features (such as object edges and texture mutation regions) relied on model decision, and the multi-channel fusion mechanism enables the network to adaptively learn the relevance of the high-frequency features and the original image sample, thereby guiding the subsequent generator to focus on the model sensitive area and enhancing the targeted attack ability and cross-model migration of the adversarial sample.

[0052] It should be further pointed out that, after step S2 and before step S3, the method further comprises:

[0053] The multi-channel fusion data is subjected to data enhancement processing in the spatial domain to obtain processed multi-channel fusion data. Correspondingly, in step S3, the multi-channel fusion data is input into the generative adversarial network model, which is replaced by inputting the processed multi-channel fusion data into the generative adversarial network model. The data enhancement processing includes size normalization, center cropping, random horizontal flipping, and random rotation. For example, the size normalization operation linearly interpolates the image size to 225x225, the center cropping operation cuts a 224x224 pixel region from the center of the scaled image, the random horizontal flipping operation horizontally mirrors the image with a probability of p = 0.5, the random rotation operation randomly rotates the image by an angle in the range of [-15°, +15°], the enhanced image is converted into a floating-point tensor, and is normalized to the range of [0, 1].

[0054] In the following steps, only the input of the multi-channel fusion data into the generative adversarial network model is described. The specific processing method of inputting the processed multi-channel fusion data into the generative adversarial network model is consistent and can be understood by reference, and will not be described again.

[0055] In step S3, the multi-channel fusion data is input into the generative adversarial network model, wherein the generative adversarial network model includes a generator and a discriminator.

[0056] The generator is used to generate an adversarial sample with high-frequency sensitive characteristics, wherein the perturbation amplitude of the adversarial sample is limited within a preset threshold.

[0057] It can be understood that, within the preset threshold, the adversarial sample is generated by the generator, and the expression of the adversarial sample with high-frequency sensitive characteristics is:

[0058]

[0059] wherein x' represents the adversarial sample, x represents the original image sample, represents the processing process of the generator on the original image sample, ∈ is the preset threshold, and clip() represents a clipping function for constraining the pixel value of the adversarial sample in the range of 0-1. By constraining the pixel value in the range of 0-1 through the clipping function, the normality of the image pixel value can be guaranteed.

[0060] In addition, the preset threshold is used to control the perturbation amplitude of the generated adversarial sample, which is specifically set according to the actual situation and is not limited herein.

[0061] In addition, the generator adopts a residual structure design and is composed of serial residual modules. Each residual module includes a 1x1 convolutional layer, a 3x3 convolutional layer and a skip connection branch. The input multi-channel fusion data is subjected to channel dimension compression and expansion through the 1x1 convolutional layer, spatial features are extracted through the 3x3 convolutional layer, the multi-channel fusion data is added to the convolutional output through the introduction of the skip connection branch to form a residual connection, which can effectively alleviate the problems such as gradient disappearance in the training process, and also facilitates the dynamic updating of the weight parameters of the generator through back propagation in the subsequent steps.

[0062] The discriminator is used to generate the category output values of the original image samples and the adversarial samples, and a Softmax function is used to map the difference between the category output values of the original image samples and the adversarial samples into a category probability distribution, and based on the category probability distribution and the true category index of the original image samples, a relative cross-entropy loss function is obtained.

[0063] Specifically, it can be understood that:

[0064] The discriminator generates the category output values of the original image samples and the adversarial samples, and calculates the difference value between the category output values of the original image samples and the adversarial samples, and the difference value expression is:

[0065] ΔD ψ =D ψ (x′)-D ψ (x),

[0066] Wherein, ΔD ψ represents the difference value between the category output values of the original image samples and the adversarial samples, respectively represent the category output value of the adversarial sample and the category output value of the original image sample.

[0067] For example, assuming that the original image sample is a picture of a cat, the category output value of the discriminator for the original image sample is [0.7, 0.3], which means that there is a 70% probability of being a cat and a 30% probability of being a dog. For the adversarial sample, the category output value of the discriminator is [0.3, 0.7], and at this time ΔD ψ is [-0.4, 0.4].

[0068] The difference between the category output values of the original image samples and the adversarial samples is mapped into a category probability distribution by using a Softmax function, and the category probability distribution expression is:

[0069]

[0070] Wherein, k represents the target category index, n represents the total number of categories, h represents the index traversing n categories in the summation symbol, σ() represents the Softmax function, and σ(ΔD ψ , k) represents the relative probability of the kth target category index.

[0071] For example, in this application, the total number of categories is 2, when k = 1, that is, the summary of cats is calculated; when k = 2, that is, the summary of dogs is calculated.

[0072] Based on the category probability distribution and the true category index of the original image sample, a relative cross-entropy loss function is obtained, and the expression of the relative cross-entropy loss function is:

[0073]

[0074] Wherein, k represents the target category index, n represents the total number of categories, Q(k) takes the value of 1 or 0, when the kth target category index is consistent with the true category index of the original image sample, Q(k) takes the value of 1, otherwise Q(k) takes the value of 0, σ(ΔD ψ (k) represents the relative probability of the kth target category index.

[0075] In this application, L rce It can also be represented by the function CrossEntropy(), but the actual meaning is the same, and specific reference can be made to the prior art for understanding, which will not be repeated here.

[0076] In addition, the discriminator parameters are fixed and do not participate in gradient update, and are only used as a classification discriminator.

[0077] Step S4, by maximizing the relative cross-entropy loss function, the network parameters of the generator are updated by using back propagation.

[0078] That is, by maximizing the relative cross-entropy loss function, the network parameters of the generator are updated by using back propagation, and the expression is:

[0079]

[0080] Wherein, The function represents L rce The mathematical calculation process of the weight parameters of the generator when the maximum value is obtained, θ * represents L rce The weight parameters of the generator when the maximum value is obtained.

[0081] Step S5, repeat steps S3-S4 until a preset iteration number is reached, and obtain an optimized generator.

[0082] The above-mentioned preset iteration number can be determined according to the actual situation, and is not limited.

[0083] It should be noted that the above discriminators can be Inception-v3, VGG19 and ResNet18 respectively. When the discriminators are processing, the images need to be preprocessed. The subsequent parameter update of the generator may be slightly different when different classification discriminators are used, and the specific selection is based on the actual situation. For example Figure 2 As shown in FIG. 8, the subsequent generator generates unconstrained perturbation adversarial samples and constrained perturbation adversarial samples.

[0084] For example, the present application obtains a total of 1000 categories and 100,000 images. In the generator optimization stage, the Adam optimizer is used, the initial step size is set to 0.0002, the exponential decay rate of the first order moment and the second order moment is set to 0.5 and 0.999 respectively, and the iteration round is 50 times.

[0085] The present application trains the generator on Inception-v3, ResNet18 and VGG19 as the proxy model of the discriminator, and generates adversarial samples on the trained generator for any image. The generated adversarial samples are subjected to attack experiments on Inception-v3 (denoted as Incv3), VGG19, ResNet18, ResNet152, Densenet201 (denoted as Dense201), Squeezenet (denoted as Sqz) and other pre-trained models.

[0086] Table 1 shows the attack success rate (%) of the adversarial samples generated by the generator trained by different proxy models, and Table 2 shows the fooling success rate (%) of the adversarial samples generated by the generator trained by different proxy models.

[0087]

[0088] Table 1

[0089]

[0090] Table 2

[0091] Experiments show that the cross-model migration ability of the adversarial samples generated by Incv3 and ResNet18 as proxy models is generally better than that of VGG19 as proxy model, and the subsequent selection can be based on the actual situation.

[0092] In summary, in the embodiments of the present application, the high-frequency feature components (such as edges and textures) of the image are extracted by high-pass filtering, and are spliced with the original image samples in the channel dimension to form multi-channel fusion data, so as to realize dynamic association of high-frequency features and spatial semantics, and guide the generator to focus on the model sensitive area. Meanwhile, based on the relative cross-entropy loss function, the difference between the output values of the adversarial samples and the original image samples on the discriminator is calculated, the difference is mapped to a probability distribution by using the Softmax function, the generator is optimized to maximize the distance between the probability distribution and the true label distribution, and the generator is forced to learn the essential feature offset rule rather than the local decision boundary, so as to alleviate the overfitting problem caused by the excessive dependence of the traditional cross-entropy loss on the training data.

[0093] The above Figures 1-2 The generation method of the adversarial samples guided by the high-frequency features provided in the embodiments of the present application is described in detail, and the generation system of the adversarial samples guided by the high-frequency features provided in the embodiments of the present application is described in detail as follows.

[0094] The system specifically includes: an acquisition and processing module, a splicing module, a generative adversarial network model module, and an optimization module, and specifically as follows.

[0095] The acquisition and processing module is used to acquire the original image samples and the true class indexes of the original image samples, perform high-pass filtering on the original image samples, and extract the high-frequency feature components of the original image samples.

[0096] The splicing module is used to splice the original image samples and the high-frequency feature components of the original image samples in the channel dimension to obtain multi-channel fusion data.

[0097] The generative adversarial network model module is used to input the multi-channel fusion data into the generative adversarial network model, wherein the generative adversarial network model includes a generator and a discriminator.

[0098] The generator is used to generate adversarial samples with high-frequency sensitive characteristics, wherein the perturbation amplitude of the adversarial samples is limited within a preset threshold.

[0099] The discriminator is used to generate the class output values of the original image samples and the adversarial samples, and map the difference between the class output values of the original image samples and the adversarial samples to a class probability distribution by using the Softmax function, and obtain a relative cross-entropy loss function based on the class probability distribution and the true class indexes of the original image samples.

[0100] The optimization module is used to update the network parameters of the generator by maximizing the relative cross-entropy loss function.

[0101] In addition, for the system implementation, as it is basically similar to the method implementation, the description is relatively simple, and the relevant parts are described in the method implementation. Moreover, it should be noted that in each module of the system of the present application, the components are logically divided according to the functions to be implemented, but the present application is not limited thereto, and each component can be re-divided or combined as needed.

[0102] The above describes the method and system for generating generative adversarial samples based on high-frequency feature guidance provided by the embodiments of the present application. The following describes the device for generating generative adversarial samples based on high-frequency feature guidance provided by the embodiments of the present application. Figures 3-4 The device for generating generative adversarial samples based on high-frequency feature guidance provided by the embodiments of the present application is described in detail.

[0103] Figure 3 The device for generating generative adversarial samples based on high-frequency feature guidance provided by the embodiments of the present application is described in detail. Figure 1 . As shown in Figure 3 , the device for generating generative adversarial samples based on high-frequency feature guidance 300 includes a transceiving module 301 and a processing module 302. For ease of description, Figure 3 only the main components of the device for generating generative adversarial samples based on high-frequency feature guidance are shown.

[0104] The transceiving module 301 is configured to perform the transceiving functions of the method for generating generative adversarial samples based on high-frequency feature guidance, and the processing module 302 is configured to perform other functions of the method for generating generative adversarial samples based on high-frequency feature guidance other than the transceiving functions.

[0105] Optionally, the transceiving module 301 can include a sending module (not shown in Figure 3 ) and a receiving module (not shown in Figure 3 ). The sending module is configured to implement the sending functions of the device for generating generative adversarial samples based on high-frequency feature guidance 300, and the receiving module is configured to implement the receiving functions of the device for generating generative adversarial samples based on high-frequency feature guidance 300.

[0106] Optionally, the device for generating generative adversarial samples based on high-frequency feature guidance 300 can further include a storage module (not shown in Figure 3 ), which stores programs or instructions. When the processing module 302 executes the programs or instructions, the device for generating generative adversarial samples based on high-frequency feature guidance 300 can perform the method for generating generative adversarial samples based on high-frequency feature guidance in the embodiments of the present application.

[0107] The following describes the device for generating generative adversarial samples based on high-frequency feature guidance provided by the embodiments of the present application. Figure 4Each component of the generative adversarial example generation device 400 guided by high-frequency features will be described in detail:

[0108] The processor 401 is the control center of the generative adversarial sample generation device 400 guided by high-frequency features. It can be a single processor or a collective term for multiple processing elements. For example, the processor 401 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0109] Optionally, the processor 401 can execute various functions of the high-frequency feature-guided generative adversarial sample generation device 400 by running or executing software programs stored in the memory 402 and calling data stored in the memory 402, such as executing the high-frequency feature-guided generative adversarial sample generation method in the embodiments of this application.

[0110] In a specific implementation, as one example, processor 401 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0111] In a specific implementation, as one example, the generative adversarial example generation device 400 based on high-frequency feature guidance may also include multiple processors, for example... Figure 4 The processors 401 and 404 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). The memory 402 is used to store the software program executing the scheme of this application, and its execution is controlled by the processor 401. Specific implementation methods can be found in the above method embodiments, and will not be repeated here.

[0112] Optionally, the memory 402 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 402 may be integrated with the processor 401 or may exist independently, and may be connected via the interface circuit of the generative adversarial sample generation device 400 guided by high-frequency features. Figure 4 (Not shown in the image) is coupled to processor 401, and this embodiment of the application does not specifically limit this.

[0113] Transceiver 403 is used for communication with other communication devices. For example, if the generative adversarial example generation device 400 guided by high-frequency features is the first device, transceiver 403 can be used to communicate with a second device or a third device.

[0114] Alternatively, transceiver 403 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0115] Optionally, the transceiver 403 can be integrated with the processor 401 or exist independently, and can be connected to the interface circuit of the generative adversarial example generation device 400 guided by high-frequency features. Figure 4 (Not shown in the image) is coupled to processor 401, and this embodiment of the application does not specifically limit this.

[0116] Understandable, Figure 4 Figure 4 The structure of the high-frequency feature-guided generative adversarial sample generation device 400 shown does not constitute a limitation on the high-frequency feature-guided generative adversarial sample generation device. Actual high-frequency feature-guided generative adversarial sample generation devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0117] In addition, the technical effects of the high-frequency feature guided generative adversarial sample generation apparatus 400 can refer to the technical effects of the methods described in the above method embodiments, which will not be repeated here.

[0118] It should be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0119] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRAM).

[0120] The above-described embodiments can be implemented in whole or in part by software, hardware (such as a circuit), firmware, or any combination thereof. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid state disk.

Claims

1. A method for generating generative adversarial samples based on high-frequency feature guidance, characterized in that, The method includes: S1. Obtain the original image sample and the true category index of the original image sample, perform high-pass filtering on the original image sample, and extract the high-frequency feature components of the original image sample; S2. The original image sample and the high-frequency feature components of the original image sample are concatenated along the channel dimension to obtain multi-channel fused data; S3. Input the multi-channel fused data into the generative adversarial network model, wherein the generative adversarial network model includes a generator and a discriminator; The generator is used to generate adversarial samples with high-frequency sensitive characteristics, wherein the perturbation amplitude of the adversarial samples is limited to a preset threshold. The discriminator is used to generate class output values ​​for the original image sample and the adversarial sample, and uses the Sofimax function to map the difference between the class output values ​​of the original image sample and the adversarial sample into a class probability distribution. Based on the class probability distribution and the true class index of the original image sample, the relative cross-entropy loss function is obtained. S4. Update the network parameters of the generator by maximizing the relative cross-entropy loss function using backpropagation; S5. Repeat steps S3-S4 until the preset number of iterations is reached to obtain the optimized generator.

2. The high-frequency feature guidance-based generative adversarial sample generation method according to claim 1, characterized in that, The step of performing high-pass filtering on the original image samples to extract the high-frequency feature components of the original image samples includes: The original image samples are subjected to spatial domain convolution operations using a preset high-pass convolution kernel to extract high-frequency feature components. The extraction expression is as follows: H(x, y) = I(x, y) * K hp Wherein, the H(x, y) represents a high frequency feature component of the original image sample, the I(x, y) represents the original image sample, the (x, y) represents a pixel position in the original image sample, and the K hp represents a preset high-pass convolution kernel, and the * represents a convolution operation.

3. The high-frequency feature guidance-based generative adversarial sample generation method according to claim 1, characterized in that, The step of concatenating the original image samples and their high-frequency feature components along the channel dimension to obtain multi-channel fused data includes: The RGB three-channel data of the original image sample and the three-channel data of the high-frequency feature components of the original image sample are concatenated along the channel dimension to obtain multi-channel fused data. The RGB three-channel data of the original image sample contains the color information of the original image sample, and the three-channel data of the high-frequency feature components of the original image sample contains the detailed features of the original image sample.

4. The high-frequency feature guidance-based generative adversarial sample generation method according to claim 1, characterized in that, After step S2 and before step S3, the method further includes: The multi-channel fused data is subjected to spatial domain data augmentation processing to obtain the processed multi-channel fused data. Correspondingly, step S3, which inputs the multi-channel fused data into the generative adversarial network model, is replaced by inputting the processed multi-channel fused data into the generative adversarial network model. The data augmentation process includes size normalization, center pruning, random horizontal flipping, and random rotation.

5. The high-frequency feature guidance-based generative adversarial sample generation method according to claim 1, characterized in that, The generator is used to generate adversarial samples with high-frequency sensitivity, wherein the perturbation amplitude of the adversarial samples is limited within a preset threshold, including: Within the preset threshold, adversarial examples are generated by the generator mapping. The expression for generating adversarial examples with high-frequency sensitive characteristics is: Wherein, the x' represents the adversarial sample, the x represents the original image sample, and the represents the processing process of the generator on the original image sample, the ∈ is the preset threshold, and the clip() represents a clipping function for constraining the pixel value of the adversarial sample in the range of 0-1.

6. The high-frequency feature guidance-based generative adversarial sample generation method according to claim 1, characterized in that, The discriminator is configured to generate category output values of the original image sample and the adversarial sample, and map a difference between the category output values of the original image sample and the adversarial sample into a category probability distribution by using a Softmax function, and obtain a relative cross-entropy loss function based on the category probability distribution and a true category index of the original image sample. The discriminator generates category output values of the original image sample and the adversarial sample, and calculates a difference value between the category output values of the original image sample and the adversarial sample, and the difference value is expressed as: ΔD ψ = D ψ (x') - D ψ (x), wherein the ΔD ψ represents a difference value of the category output value of the original image sample and the adversarial sample, the respectively represent the category output value of the adversarial sample and the category output value of the original image sample. The difference between the category output values of the original image sample and the adversarial sample is mapped into a category probability distribution by using a Softmax function, and the category probability distribution is expressed as: wherein the k represents a target category index, the n represents a total number of categories, the j represents an index traversing n categories in a summation symbol, the σ() represents a Softmax function, the σ(ΔD ψ , k) represents a relative probability of the kth target category index; The relative cross-entropy loss function is obtained based on the category probability distribution and a true category index of the original image sample, and the relative cross-entropy loss function is expressed as: wherein k represents a target category index, n represents a total number of categories, Q(k) takes a value of 1 or 0, Q(k) takes a value of 1 when the kth target category index is consistent with a true category index of the original image sample, otherwise Q(k) takes a value of 0, and σ(ΔD ψ (k) represents a relative probability of the kth target category index. wherein k represents a target category index, n represents a total number of categories, Q(k) takes a value of 1 or 0, Q(k) takes a value of 1 when the kth target category index is consistent with a true category index of the original image sample, otherwise Q(k) takes a value of 0, and σ(ΔD ψ (k) represents a relative probability of the kth target category index.

7. The high-frequency feature guidance-based generative adversarial sample generation method according to claim 1, characterized in that, The network parameters of the generator are updated by using back propagation by maximizing the relative cross-entropy loss function, and the network parameters of the generator are updated by using back propagation by maximizing the relative cross-entropy loss function, and the network parameters of the generator are updated by using back propagation by maximizing the relative cross-entropy loss function. The system comprises: Wherein, the The function represents the L rce The mathematical calculation process of the weight parameters of the generator when the maximum value is obtained, and the θ * The function represents the L rce The weight parameters of the generator when the maximum value is obtained.

8. A high-frequency feature guidance based generative adversarial sample generation system, characterized by, The acquisition module and the processing module are configured to acquire an original image sample and a true category index of the original image sample, perform high-pass filtering processing on the original image sample, and extract a high-frequency feature component of the original image sample; The splicing module is configured to splice the original image sample and the high-frequency feature component of the original image sample in a channel dimension to obtain multi-channel fusion data; The generative adversarial network model module is configured to input the multi-channel fusion data into a generative adversarial network model, and the generative adversarial network model comprises a generator and a discriminator; The generator is configured to generate an adversarial sample with high-frequency sensitive characteristics, and a perturbation amplitude of the adversarial sample is limited within a preset threshold; The discriminator is configured to generate category output values of the original image sample and the adversarial sample, and map a difference between the category output values of the original image sample and the adversarial sample into a category probability distribution by using a Softmax function, and obtain a relative cross-entropy loss function based on the category probability distribution and a true category index of the original image sample. The optimization module is configured to update the network parameters of the generator by using back propagation by maximizing the relative cross-entropy loss function. The apparatus comprises modules for performing the method of any one of claims 1-7.

9. An apparatus for generating generative adversarial samples based on high-frequency feature guidance, comprising: The computer-readable storage medium comprises a computer program or instructions, which, when executed, cause the method of any one of claims 1-7 to be performed.

10. A computer-readable storage medium, characterized in that, ​

Images