Semantic attack method based on style suppression and frequency domain limitation
By introducing style suppression and frequency domain restriction technologies into semantic attack methods, the problem of insufficient concealment and stability of the adversarial sample in the prior art is solved, and a higher attack success rate and concealment are achieved.
Patent Information
- Application Number
- CN202411640822.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-18
- Publication Date
- 2025-02-18
AI Technical Summary
The existing semantic attack methods fail to effectively distinguish the style characteristics and content characteristics of the image, resulting in the visually easy to detect the adversarial samples, affecting concealment and stability.
The semantic attack method based on style suppression and frequency domain restriction is adopted to style the adversarial samples through an adaptive instance normalization layer, and the frequency component decomposition of the samples is used by discrete wavelet transformation to optimize the perturbation distribution and intensity.
The high concealment and stability of the adversarial samples are achieved, which reduces the obvious traces of the adversarial samples in the high-frequency region, and reduces the impact of low-frequency perturbation components on the neural network model, thereby improving the attack success rate.
Smart Images
Figure CN119476405A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of artificial intelligence and computer vision security, and in particular to a semantic attack method based on style suppression and frequency domain limitation. Background Art
[0002] With the rapid development of deep learning technology, neural networks have been widely used in image classification, speech recognition, target detection and other fields. However, the robustness problem of neural networks has gradually been exposed, especially in adversarial sample attacks, which pose a great security risk. Adversarial sample attacks add small perturbations to the input samples to cause significant deviations in the output of the model, leading to misjudgments. Traditional adversarial attacks mostly rely on the choice of perturbation direction. The attack effect is not stable enough and is easily detected by the human eye, making it difficult to achieve the requirements of high concealment and stability.
[0003] Existing semantic attack methods often fail to distinguish between the style and content features of images, resulting in the generated adversarial samples being easily perceived visually, affecting the concealment and stability of the adversarial attack. On the other hand, the perturbations of traditional adversarial sample generation methods mostly appear in low-frequency components, which are more easily detected by the human visual system (HVS), thus limiting their effectiveness in practical applications. Therefore, there is an urgent need for an adversarial attack method that can effectively control the style and frequency components of the sample without relying on a specific perturbation direction to improve the concealment and attack stability of the adversarial sample. Summary of the invention
[0004] The present invention provides a semantic attack method based on style suppression and frequency domain limitation to solve the problem of how to conceal semantic features through style suppression and frequency domain limitation without relying on a specific disturbance direction, so as to generate adversarial samples with high concealment and strong stability, thereby improving the effectiveness of adversarial attacks in neural network models.
[0005] In order to solve the above technical problems, the present invention provides a semantic attack method based on style suppression and frequency domain limitation, comprising:
[0006] Adding slight noise to the benign sample to generate an initial adversarial sample, and calculating the semantic similarity between the initial adversarial sample and the benign sample to obtain the semantic loss;
[0007] Inputting the initial adversarial sample into an adaptive instance normalization (IN) layer, performing style conversion processing on the initial adversarial sample based on the style information of the reference sample to generate a stylized adversarial sample, and calculating the semantic similarity between the benign sample and the stylized adversarial sample to obtain a style suppression loss;
[0008] Decomposing the stylized adversarial sample by frequency components based on discrete wavelet transform (DWT) to obtain low-frequency components and high-frequency components, and calculating the frequency difference between the low-frequency components and the high-frequency components to obtain frequency loss;
[0009] The style suppression loss and the frequency loss are combined to generate an overall loss, and the perturbation distribution and intensity of the stylized adversarial sample are optimized and adjusted based on the overall loss to generate a final optimized adversarial sample.
[0010] Furthermore, in the step of generating an initial adversarial sample, the tiny noise is used to generate an initial adversarial sample having semantic features different from those of the benign sample.
[0011] Furthermore, the step of calculating the semantic similarity includes:
[0012] Inputting the initial adversarial sample and the benign sample into a neural network model to obtain corresponding feature vectors;
[0013] The semantic similarity between the initial adversarial sample and the benign sample is calculated based on the feature vector.
[0014] Furthermore, the style conversion process includes:
[0015] Acquire style parameters of the initial adversarial sample and style parameters of the reference sample;
[0016] The initial adversarial sample is subjected to style transformation based on the style parameters to generate the stylized adversarial sample.
[0017] Furthermore, the style parameters include the mean and variance of the initial adversarial sample and the reference sample.
[0018] Furthermore, the step of frequency component decomposition includes:
[0019] The stylized adversarial sample is decomposed into a low-frequency component and a high-frequency component based on discrete wavelet transform (DWT).
[0020] Furthermore, the low-frequency component contains basic information of the stylized adversarial sample, and the high-frequency component contains detail information and noise of the stylized adversarial sample.
[0021] Furthermore, the step of calculating the frequency difference includes:
[0022] The frequency loss of the stylized adversarial sample and the benign sample is calculated based on an L1 norm between the low-frequency component and the high-frequency component.
[0023] Further, the overall loss is generated based on a weighted combination of the style suppression loss and the frequency loss.
[0024] Furthermore, the optimization and adjustment step iteratively optimizes the perturbation distribution and strength of the stylized adversarial sample based on the overall loss.
[0025] The key innovative features of the present invention include:
[0026] (1) Style suppression module: The IN layer performs style conversion on adversarial samples and uses style-related features to generate adversarial samples to improve the concealment of attack samples.
[0027] (2) Frequency domain restriction module: Through DWT frequency decomposition, the high-frequency and low-frequency perturbation distribution of adversarial samples is controlled, which enhances the attack stability and concealment from the human visual system.
[0028] The following are its main beneficial effects:
[0029] The beneficial effect of the present invention is that the concealment and stability of adversarial samples are achieved by introducing style suppression and frequency domain limitation technology. Compared with the traditional attack method that relies on a specific disturbance direction, the present invention no longer needs to select a disturbance direction, but uses style suppression to suppress interference features, reducing the obvious traces of adversarial samples in high-frequency areas; at the same time, through frequency domain control, the influence of low-frequency disturbance components on the neural network model is reduced, thereby achieving adversarial attacks with higher concealment and stability. The adversarial samples generated by the present invention have a higher attack success rate, while reducing interference in areas susceptible to the human eye, making them more concealed and effective in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 A flowchart of a semantic attack method based on style suppression and frequency domain limitation provided in an embodiment of the present application. DETAILED DESCRIPTION
[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.
[0032] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0033] Reference Figure 1 , is a flow chart of a semantic attack method based on style suppression and frequency domain limitation provided by an embodiment of the present invention. The flow chart may at least include steps S100-S400:
[0034] S100: Add small noise to benign samples to generate initial adversarial samples x adv , calculate the semantic similarity between the initial adversarial sample and the benign sample, and get the semantic loss L sem ;
[0035] S200: Input the initial adversarial sample into the adaptive instance normalization (IN) layer, use the reference sample to perform style conversion processing, generate a stylized adversarial sample for the adversarial sample, calculate the semantic similarity between the benign sample and the stylized adversarial sample, and obtain the style suppression loss L res ;
[0036] S300: Decompose the frequency components of the stylized adversarial sample based on discrete wavelet transform (DWT) to obtain low-frequency components and high-frequency components, calculate the frequency difference between low-frequency and high-frequency components, and obtain the frequency loss L fre ;
[0037] S400: Combined with style suppression loss L res and frequency loss L fre Generate the overall loss L total , based on the overall loss, the adversarial sample is optimized and adjusted to generate a final optimized adversarial sample.
[0038] Step S100 at least includes steps S110-S130:
[0039] S110: Get benign sample x cln , and add a small noise δ to the benign sample to generate the initial adversarial sample x adv,0 .
[0040] Specifically, for each benign input sample x cln , add the disturbance noise δ to x cln Based on the feature representation of adv,0 =x cln+δ. The perturbation noise δ is a noise vector with a small amplitude to ensure visual invisibility to humans and the initial characteristics of the adversarial sample.
[0041] S120: Calculate the initial adversarial sample x adv,0 With benign samples x cln The semantic similarity s(x adv,0 ,x cln ), and based on this similarity, we get the semantic loss L sem,0 .
[0042] Specifically, by transforming the initial adversarial sample x adv,0 With benign samples x cln Input into the feature extraction layer of the neural network classification model to obtain its feature vector representation f(x adv,0 ) and f(x cln ). Then, the cosine similarity formula is used to measure their semantic similarity:
[0043]
[0044] where · represents the dot product of the vectors, and ||·|| represents the norm of the vectors. Thus, by calculating L sem,0 =1-s(x adv,0 ,x cln ) to get the semantic loss L sem,0 , and use this loss for subsequent adversarial sample optimization.
[0045] S130: Based on the semantic loss L sem,0 For the initial adversarial sample x adv,0 The perturbation is further adjusted to ensure that the generated adversarial samples have characteristics that are more deviated from the semantic features of benign samples.
[0046] Specifically, in the optimization process, the gradient descent method is applied to minimize the loss function L sem,0 Update the perturbation noise δ to generate a new adversarial sample x adv,1 , where the update formula is:
[0047]
[0048] Among them, α is the step size parameter, which is used to control the amplitude of the perturbation adjustment. In this process, by continuously optimizing the initial adversarial sample x adv,0 Make its semantic features gradually move away from benign samples x cln The features of the updated adversarial sample x are obtained adv,1 .
[0049] Step S200 at least includes steps S210-S230:
[0050] S210: The initial adversarial sample x adv Input to the adaptive instance normalization (IN) layer to obtain the style parameters of the sample.
[0051] Specifically, the initial adversarial sample x adv Input to the IN layer and calculate the mean μ of its feature map adv and variance σ adv By adv The feature map of is processed by adaptive instance normalization to obtain its corresponding style parameter μ adv and σ adv , thus providing a parameter basis for subsequent style transfer operations.
[0052] S220: Based on the style parameters and the style information of the reference sample, perform style conversion processing on the initial adversarial sample to generate a stylized adversarial sample.
[0053] Specifically, the reference sample x is introduced sty Calculate the style information of the reference sample x sty The mean μ sty and variance σ sty Then, the initial adversarial sample x is calculated by the following formula adv Perform style conversion to obtain a stylized adversarial sample x adv,sty :
[0054]
[0055] Among them, x adv,sty The generation is based on x adv With x sty The mean and variance of are interpolated to ensure the smoothness and accuracy of the style transfer effect.
[0056] S230: Calculate the stylized adversarial sample x adv,sty With benign samples x cln The semantic similarity of the style suppression loss L is obtained. res .
[0057] Specifically, using the stylized adversarial sample x adv,sty With benign samples x cln The eigenvector of f(x adv,sty ) and f(x cln ), calculate the semantic similarity s(x adv,sty ,x cln ):
[0058]
[0059] And calculate the style suppression loss L based on semantic similarity res =1-s(x adv,sty ,x cln ).
[0060] Step S300 at least includes steps S310-S330:
[0061] S310: Based on discrete wavelet transform (DWT), the stylized adversarial sample x adv,sty Decompose the frequency components to obtain low-frequency components and high-frequency components.
[0062] Specifically, the stylized adversarial sample x adv,sty Input to the discrete wavelet transform module, the image sample is decomposed into four frequency components by DWT: low frequency component x ll , and three high-frequency components x lh 、x hl 、x hh Among them, x ll represents the main information of the image, containing fewer details, while x lh 、x hl and x hh It contains high-frequency details and noise information in the image.
[0063] S320: Calculate the low-frequency component x ll The frequency difference between the high frequency component and the frequency loss L is obtained. fre .
[0064] Specifically, calculate the low-frequency component x of the stylized adversarial sample ll and high frequency components (x lh 、x hl 、x hh ) to limit the disturbance amplitude between low-frequency components and high-frequency components. Frequency loss L fre is defined as follows:
[0065]
[0066] Among them, x adv,sty,ll and x cln,ll are the low-frequency components of stylized adversarial samples and benign samples, respectively, and x adv,sty,i and x cln,i Respectively represent the corresponding high frequency components, w lf and w hf are low-frequency and high-frequency weight parameters, which control the contribution of low-frequency and high-frequency losses respectively.
[0067] S330: The frequency loss L fre An optimization process is performed to enhance the perturbation concealment of the adversarial sample.
[0068] Specifically, the frequency loss L fre Added to the overall optimization objective to control the stylized adversarial example x adv,sty The frequency component of the disturbance is enhanced. During the optimization process, keep x adv,sty The high-frequency perturbation amplitude is low, while the low-frequency information is kept consistent with the benign samples as much as possible, thereby improving concealment.
[0069] Step S400 at least includes steps S410-S430:
[0070] S410: Combined style suppression loss L res and frequency loss L fre , generating the overall loss L total .
[0071] Specifically, the style suppression loss L obtained in the previous module is res With frequency loss L fre Perform weighted combination to get the overall loss L total :
[0072] L total =λ res L res +λ fre L fre
[0073] Among them, λ res and λ fre is the corresponding weight parameter, which is used to control the contribution of style suppression loss and frequency loss in the overall loss. total As the objective function for optimizing adversarial samples, it will be used for subsequent optimization adjustments.
[0074] S420: Based on the overall loss L total For the adversarial sample x adv The disturbance distribution and intensity are optimized and adjusted.
[0075] Specifically, by transforming the overall loss L total As the loss function, apply the gradient descent method to the stylized adversarial sample x adv,sty Perform iterative updates to optimize its perturbation distribution and intensity. The update formula is as follows:
[0076]
[0077] Among them, α represents the learning rate, which is used to control the update step size. Through this optimization process, the adversarial sample gradually reaches the optimal state, and its perturbation characteristics can meet the requirements of concealment and attack stability.
[0078] S430: Output the optimized and adjusted adversarial sample as the final adversarial sample.
[0079] Specifically, after multiple iterative updates, the final adversarial sample x is determined adv,final , and output it as the final optimized adversarial sample. The final adversarial sample meets the requirements of style suppression and frequency domain limitation, and has concealment and stability.
[0080] The key innovative features of the present invention include:
[0081] (1) Style suppression module: The IN layer performs style conversion on adversarial samples and uses style-related features to generate adversarial samples to improve the concealment of attack samples.
[0082] (2) Frequency domain restriction module: Through DWT frequency decomposition, the high-frequency and low-frequency perturbation distribution of adversarial samples is controlled, which enhances the attack stability and concealment from the human visual system.
[0083] The following are its main beneficial effects:
[0084] The beneficial effect of the present invention is that the concealment and stability of adversarial samples are achieved by introducing style suppression and frequency domain limitation technology. Compared with the traditional attack method that relies on a specific disturbance direction, the present invention no longer needs to select a disturbance direction, but uses style suppression to suppress interference features, reducing the obvious traces of adversarial samples in high-frequency areas; at the same time, through frequency domain control, the influence of low-frequency disturbance components on the neural network model is reduced, thereby achieving adversarial attacks with higher concealment and stability. The adversarial samples generated by the present invention have a higher attack success rate, while reducing interference in areas susceptible to the human eye, making them more concealed and effective in practical applications.
[0085] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.
Claims
1. A semantic attack method based on style suppression and frequency domain restriction, characterized in that: The steps include: Adding slight noise to the benign sample to generate an initial adversarial sample, and calculating the semantic similarity between the initial adversarial sample and the benign sample to obtain the semantic loss; Inputting the initial adversarial sample into an adaptive instance normalization (IN) layer, performing style conversion processing on the initial adversarial sample based on the style information of the reference sample to generate a stylized adversarial sample, and calculating the semantic similarity between the benign sample and the stylized adversarial sample to obtain a style suppression loss; Decomposing the stylized adversarial sample by frequency components based on discrete wavelet transform (DWT) to obtain low-frequency components and high-frequency components, and calculating the frequency difference between the low-frequency components and the high-frequency components to obtain frequency loss; The style suppression loss and the frequency loss are combined to generate an overall loss, and the perturbation distribution and intensity of the stylized adversarial sample are optimized and adjusted based on the overall loss to generate a final optimized adversarial sample.
2. The semantic attack method based on style suppression and frequency domain limitation according to claim 1, characterized in that: In the step of generating an initial adversarial sample, the tiny noise is used to generate an initial adversarial sample having a semantic feature different from that of the benign sample.
3. The semantic attack method based on style suppression and frequency domain limitation according to claim 1, characterized in that: The step of calculating semantic similarity comprises: Inputting the initial adversarial sample and the benign sample into a neural network model to obtain corresponding feature vectors; The semantic similarity between the initial adversarial sample and the benign sample is calculated based on the feature vector.
4. The semantic attack method based on style suppression and frequency domain limitation according to claim 1, characterized in that: The style conversion process includes: Acquire style parameters of the initial adversarial sample and style parameters of the reference sample; The initial adversarial sample is subjected to style transformation based on the style parameters to generate the stylized adversarial sample.
5. The semantic attack method based on style suppression and frequency domain limitation according to claim 4, characterized in that: The style parameters include means and variances of the initial adversarial samples and the reference samples.
6. The semantic attack method based on style suppression and frequency domain limitation according to claim 1, characterized in that: The step of frequency component decomposition comprises: The stylized adversarial sample is decomposed into a low-frequency component and a high-frequency component based on discrete wavelet transform (DWT).
7. The semantic attack method based on style suppression and frequency domain limitation according to claim 6, characterized in that: The low-frequency component contains basic information of the stylized adversarial sample, and the high-frequency component contains detail information and noise of the stylized adversarial sample.
8. The semantic attack method based on style suppression and frequency domain limitation according to claim 1, characterized in that: The step of calculating the frequency difference comprises: The frequency loss of the stylized adversarial sample and the benign sample is calculated based on an L1 norm between the low-frequency component and the high-frequency component.
9. The semantic attack method based on style suppression and frequency domain limitation according to claim 1, characterized in that: The overall loss is generated based on a weighted combination of the style suppression loss and the frequency loss.
10. The semantic attack method based on style suppression and frequency domain limitation according to claim 9, characterized in that: The optimization and adjustment step iteratively optimizes the perturbation distribution and strength of the stylized adversarial sample based on the overall loss.