Hidden space confrontation sample generation method and system based on multi-scale feature separation

Abstract

The invention discloses a hidden space adversarial sample generation method and system based on multi-scale feature separation, and the method comprises the steps: employing a neural network quantization training method based on straight-through estimation, and training a hierarchical vector quantization variational auto-encoder; carrying out differentiable Haar wavelet transformation on the input image by adopting a wavelet packet transformation algorithm, decomposing the input image into a low-frequency component and a high-frequency component, and realizing multi-scale feature separation; inputting the high-frequency component into a hierarchical vector quantization variational auto-encoder, and extracting and quantizing global high-frequency features and local high-frequency detail features; in the potential space, a learnable disturbance variable is introduced, a potential vector after disturbance is constructed, and the potential vector is reconstructed into an adversarial sample through a decoder; and based on a preset disturbance target, carrying out iterative optimization on the disturbance vector until a confrontation sample which satisfies an attack success condition and is optimized in visual quality is generated. According to the method, a wavelet domain variational auto-encoder and a hidden space iterative attack algorithm are fused, and an adversarial sample with high fidelity and clear interpretation is generated.

Description

Technical Field

[0001] This invention relates to the field of adversarial sample generation technology, and in particular to a latent space adversarial sample generation method and system based on multi-scale feature separation. Background Technology

[0002] In recent years, deep learning and general artificial intelligence have become major driving technologies in many key fields such as autonomous driving, smart healthcare, industrial automation, and smart security. However, with the increasing complexity of deep learning systems and the diversification of application scenarios, their security and robustness have gradually become key bottlenecks restricting practical applications. While achieving outstanding performance, deep neural networks have also exposed many security vulnerabilities, especially in adversarial attack scenarios, where their vulnerability is even more pronounced.

[0003] Traditional adversarial example generation methods primarily rely on adding subtle perturbations at the pixel level to confuse the target model. While this method can achieve its attack objectives to some extent, relying solely on local perturbations often fails to simultaneously maintain the overall realism of the image and the integrity of global features. In practical applications, if the generated adversarial examples lack visual realism, it may not only weaken the attack's effectiveness but also trigger misjudgments by security defense systems, thereby affecting the overall system's credibility. Furthermore, existing adversarial example generation techniques lack targeted perturbation strategies for key feature vectors in the low-dimensional latent space, making it difficult to achieve high-fidelity example generation and in-depth vulnerability mining within limited computational resources.

[0004] Due to the high complexity and black-box nature of deep learning models, only by systematically revealing their potential vulnerabilities can overall security be effectively improved. Research on adversarial attacks not only helps to gain a deeper understanding of the internal mechanisms and potential vulnerabilities of models, but also provides a theoretical basis for developing more robust and reliable defense strategies and intelligent systems. However, existing adversarial example generation techniques have significant technical shortcomings in maintaining visual realism, revealing global model vulnerabilities, and efficiently optimizing latent space features, severely restricting the improvement of deep learning system security. Therefore, it is urgent to explore more efficient and accurate adversarial example generation techniques to effectively improve the security of deep learning and artificial intelligence applications. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for generating adversarial examples in latent space based on multi-scale feature separation. It comprehensively adopts multi-scale feature extraction and low-dimensional latent space optimization techniques, and combines the advantages of wavelet transform in multi-frequency domain decomposition with advanced variational modeling methods. It can accurately characterize and perturb key features while maintaining high image visual fidelity. This helps to reveal the vulnerability of deep learning black-box models when faced with subtle perturbations, and also provides innovative ideas and technical support for developing more reliable and robust intelligent systems.

[0006] To achieve the above objectives, the present invention provides the following solution: A latent space adversarial sample generation method based on multi-scale feature separation includes the following steps: S1 employs a neural network quantization training method based on direct-pass estimation to train a hierarchical vector quantization variational autoencoder; S2 uses the wavelet packet transform algorithm to perform a differentiable Haar wavelet transform on the input image, decomposing the input image into one low-frequency component and three high-frequency components to achieve multi-scale feature separation; wherein, the three high-frequency components are horizontal, vertical and diagonal high-frequency components, respectively. S3. Input the high-frequency components obtained in step S2 into the trained hierarchical vector quantization variational autoencoder to extract and quantize global high-frequency features and local high-frequency detail features, and obtain the corresponding top-level latent vector and bottom-level latent vector. S4, in the latent space of the hierarchical vector quantization variational autoencoder, based on the top-level latent vector and the bottom-level latent vector, introduces learnable perturbation variables to construct perturbation latent vectors, and reconstructs them into adversarial examples through the decoder; S5 initializes the learnable perturbation variables. Starting from the original adversarial sample, it iteratively optimizes the perturbation vector based on the preset perturbation target until an adversarial sample that meets the attack success conditions and has optimized visual quality is generated.

[0007] Furthermore, the hierarchical vector quantization variational autoencoder includes: The top-level encoder takes the feature map obtained by concatenating three high-frequency components as input and uses a multi-layer convolutional network, residual module and self-attention mechanism to process and extract high-frequency information that represents the global structure of the image as global high-frequency features. After the initial feature extraction, the continuous features are discretized by a vector quantizer to form quantized features with high-level structure description ability. The bottom encoder fuses the original high-frequency components with the quantized output of the top encoder. It further extracts local high-frequency detail features through the same network structure as the top encoder, and then quantizes them through a vector quantizer for use in the subsequent decoding process. The decoder includes multi-level convolutional layers, a residual module, a self-attention module, and an upsampling module. The decoder input is equipped with a feature fusion module, which integrates the quantization features of the top-level encoder and the bottom-level encoder, reconstructs the high-frequency components through layer-by-layer decoding, and finally combines them with the low-frequency components to achieve complete image reconstruction through wavelet inverse transform.

[0008] Furthermore, in S1, training the hierarchical vector quantization variational autoencoder specifically includes: S101, during the forward propagation process, the continuous latent vector output by the hierarchical vector quantization variational autoencoder is discretized based on the nearest neighbor search method and mapped to a discrete codebook vector. S102, during the backpropagation process, bypassing the non-differentiability of discretization, a direct estimation method is used to directly pass the gradient of the decoder with respect to the discrete codebook vector to the corresponding gradient of the encoder output. S103 adopts a codebook update strategy based on the exponential moving average algorithm to dynamically maintain the usage frequency and corresponding feature statistics of each discrete codebook vector, and update the spatial position of the discrete codebook vector accordingly. S104, design a dead code detection and reset mechanism, detect discrete codebook vectors whose usage feature statistics are lower than a preset threshold, and reset them to the result of adding Gaussian noise to the high-frequency codebook vectors.

[0009] Furthermore, step S103 further includes: using the gradient information obtained in step S102 to optimize the encoder parameters through gradient descent, and updating the discrete codebook vector based on the codebook loss.

[0010] Furthermore, in step S103, a codebook update strategy based on the exponential moving average algorithm is adopted to dynamically maintain the usage frequency and corresponding feature statistics of each discrete codebook vector, and update the spatial position of the discrete codebook vector accordingly. Specifically, this includes: In each training iteration, the discrete codebook vectors are statistically analyzed. Real-time usage frequency and the sum of the latent vectors output by the corresponding encoder Where e represents the encoder level, including the top encoder (top) and the bottom encoder (bottom). The top encoder extracts global high-frequency structural features and outputs a latent vector z. top The bottom-level encoder fuses the original high-frequency components with the top-level encoding results, outputting local detail features z. bottom Therefore, z e ∈z top , z bottom , used to distinguish the features from different levels of the encoder; i represents the index of the feature vector, which is related to the codebook mapping mechanism of the vector quantizer: the continuous features output by the encoder are discretized into codewords in the codebook through nearest neighbor search, and each feature vector corresponds to a codebook index i. In the multi-stage optimization framework, i can further identify the iterative batch or spatial position of the perturbation variable. Subsequently, the feature statistics were smoothed using the exponential moving average algorithm, and the updated formula was defined as follows: , ; in, The representative only applies to all satisfied... This condition Corresponding The terms are summed, where q(·) denotes the discretization mapping operation of the vector quantizer. This represents the i-th continuous latent vector output by the encoder. This is the attenuation coefficient, used to control the proportion of historical information retained; , Let represent the values ​​of the k-th discrete codebook vector at iteration time steps t-1 and t, respectively, and generate the update state of the k-th discrete codebook vector in the model. These represent the intermediate calculation results of the k-th feature channel in the generative model at iteration time steps t and t-1, respectively, corresponding to the moving average of the discrete codebook vector; The core function of q(·) is to map continuous feature vectors to discrete codewords in the codebook through nearest neighbor search. Specifically, this can be understood as mapping continuous latent vectors output by the encoder to discrete codewords in the codebook. (A point in a higher-dimensional space); calculate With codebook E={e1,e2,...,e K The Euclidean distance of all codewords in the array is calculated, and the nearest neighbor codeword e is selected. k As a quantization result, the output index k is... =k represents z i The codeword e is mapped to the k-th codeword in the codebook k This process ensures the consistency between encoded features and codebook vectors through joint optimization of codebook loss and reconstruction loss. z i It is the i-th continuous latent vector output by the encoder, and its specific features depend on the encoder level: in a two-level encoder structure, z i It may originate from the top-level encoder (global high-frequency structure) or the bottom-level encoder (fusion of global and local details); after q(·) mapping, z i Replaced with discrete codeword e in the codebook k This enhances the expressive power of the latent space and improves the stability of generation; Based on the updated feature statistics, the discrete codebook vector is adjusted in spatial position using a weighted average: = ; in, This represents the k-th codebook vector after the (t+1)-th update step; This represents the sum of latent vectors corresponding to the k-th codebook vector after EMA smoothing; This represents the frequency of use of the k-th codebook vector after EMA smoothing; This is a numerically stable term. This formula is used for codebook updates, summing the smoothed latent vectors. Divide by smooth usage frequency The new codebook vector position is obtained. Compared with the traditional VQ-VAE direct gradient update, the EMA strategy avoids training oscillations and improves stability by gradually adjusting the codebook vector. It also enhances the adaptability of the codebook to the data distribution. Combined with dead code detection and reset mechanisms, it further solves the codebook diversity problem.

[0011] Further, in step S3, the high-frequency components obtained in step S2 are input into the trained hierarchical vector quantization variational autoencoder to extract and quantize global high-frequency features and local high-frequency detail features, thereby obtaining the corresponding top-level latent vector and bottom-level latent vector, specifically including: S301 uses the top-level encoder in the hierarchical vector quantization variational autoencoder to extract global high-frequency features from the high-frequency components of the input, and uses vector quantization to discretize the extracted global high-frequency features to obtain the top-level latent vector. S302: After concatenating the discretization result of the top-level encoder with the input high-frequency components, the input is used to the bottom-level encoder in the hierarchical vector quantization variational autoencoder to extract local high-frequency detail features. The extracted local high-frequency detail features are then quantized to obtain the bottom-level latent vector.

[0012] Furthermore, in step S4, within the latent space of the hierarchical vector quantization variational autoencoder, a learnable perturbation variable is introduced based on the top-level and bottom-level latent vectors to construct a perturbed latent vector, which is then reconstructed into an adversarial example by the decoder. Specifically, this includes: Learnable perturbation variables are introduced into the latent space respectively. and The perturbated latent vector is constructed as follows: , ; in, and These represent the top-level latent vector and the bottom-level latent vector, respectively. and These represent the top-level latent vector and the bottom-level latent vector after perturbation, respectively. The constraints on the disturbance variables are: , , , These are the preset top-level disturbance threshold and bottom-level disturbance threshold, respectively; The decoder combines the perturbed latent vector with the low-frequency component obtained in step S2, and reconstructs it into an adversarial example through wavelet packet inverse transform. .

[0013] Further, in step S5, the learnable perturbation variables are initialized. Starting from the original adversarial example, and based on a preset perturbation target, the perturbation vector is iteratively optimized until an adversarial example that satisfies the attack success conditions and has optimized visual quality is generated. Specifically, this includes: S501, using the latent vectors of the original adversarial examples. and Starting from this point, initialize the perturbation variables. ; In S502, at each iteration, the decoder reconstructs the perturbed latent vector into adversarial examples. Input the target model and calculate the classification loss. ,in, p (·) represents the probability distribution, and y represents the true category label; S503 aims to maximize the classification loss of the target model by iteratively optimizing the perturbation variables using the gradient ascent method. During the iteration process, the perturbation variables are projected to satisfy the constraints of the perturbation variables until an adversarial example that satisfies the attack success conditions and has optimized visual quality is generated.

[0014] Furthermore, in S503, projecting the perturbation variable to satisfy the constraint conditions of the perturbation variable during the iteration process specifically includes: Update the perturbation variable through backpropagation: ; in, For learning rate, For projection operations, ensure that the perturbation variables meet the constraints of the perturbation variables. This represents the gradient of the adversarial loss function with respect to the perturbation δ, combined with adversarial examples. The specific details are as follows x +δ, x This is a normal sample; If the attack is successful within a certain number of consecutive steps and the loss is not significantly improved, then the adversarial sample with the best visual quality and the most successful attack is output.

[0015] This invention also provides a latent space adversarial example generation system based on multi-scale feature separation, applied to the above-described latent space adversarial example generation method based on multi-scale feature separation, comprising: The hierarchical vector quantization variational autoencoder training module is used to train the hierarchical vector quantization variational autoencoder using a neural network quantization training method based on pass-through estimation. The multi-scale feature separation module is used to perform a differentiable Haar wavelet transform on the input image using the wavelet packet transform algorithm, decomposing the input image into one low-frequency component and three high-frequency components to achieve multi-scale feature separation; wherein, the three high-frequency components are horizontal, vertical and diagonal high-frequency components, respectively. The hierarchical encoding module is used to input the obtained high-frequency components into the trained hierarchical vector quantization variational autoencoder to extract and quantize global high-frequency features and local high-frequency detail features, and obtain the corresponding top-level latent vector and bottom-level latent vector. The feature reconstruction and preliminary sample generation module is used to introduce learnable perturbation variables into the latent space of the hierarchical vector quantization variational autoencoder based on the top-level and bottom-level latent vectors, construct perturbation latent vectors, and reconstruct them into adversarial samples by the decoder. The adversarial perturbation injection and optimization module is used to initialize learnable perturbation variables. Starting from the original adversarial sample, it iteratively optimizes the perturbation vector based on the preset perturbation target until an adversarial sample that meets the attack success conditions and has improved visual quality is generated.

[0016] As can be seen from the above technical solutions, compared with the prior art, the latent space adversarial sample generation method and system based on multi-scale feature separation provided by the present invention has the following beneficial effects: 1) This invention achieves accurate capture and perturbation of core image features through multi-scale feature extraction and low-dimensional latent space optimization, rather than simply modifying pixels directly. This not only improves the accuracy of perturbation generation, but also makes more efficient use of the image's inherent structural information, which is consistent with human visual perception. 2) This invention utilizes wavelet transform to perform multi-frequency domain decomposition on the image, allowing the perturbation to be finely adjusted across multiple scales and frequencies. This ensures the integrity and naturalness of the image's visual characteristics while generating attack samples, achieving a better balance between attack effectiveness and image quality compared to traditional methods. 3) This invention uses a variational modeling strategy to optimize the low-dimensional latent space, which can finely control the perturbation distribution. This makes the generated adversarial examples more targeted in revealing the vulnerability of deep models at specific feature levels, avoiding noise or uncontrollable risks introduced by excessive perturbation in the pixel space. 4) By combining global and local, multi-scale analysis, this invention can more comprehensively explore the weaknesses of deep neural networks, thereby improving the success rate of attacks and providing a more detailed theoretical basis for subsequent security protection strategies. Compared with the direct pixel perturbation method, it is more systematic and accurate.

[0017] In summary, this invention employs multi-scale feature extraction and low-dimensional latent space optimization techniques, combining the advantages of wavelet transform in multi-frequency domain decomposition with advanced variational modeling methods. This allows for the precise characterization and perturbation of key features while maintaining high visual fidelity in images. This strategy not only helps reveal the vulnerability of deep learning black-box models to subtle perturbations but also provides innovative ideas and technical support for developing more reliable and robust intelligent systems. The novel high-fidelity adversarial sample generation and optimization method based on the integration of wavelet domain and low-dimensional latent space can deeply explore the weaknesses of deep models and point the way to building secure and trustworthy artificial intelligence systems, thus having significant theoretical and practical implications for promoting the development of AI security technologies. Experimental results show that, compared to existing methods, the proposed solution significantly enhances the adversarial attack effect while preserving image details and overall structure. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the latent space adversarial sample generation method based on multi-scale feature separation of the present invention. Figure 2 This is a schematic diagram of the training process of the hierarchical vector quantization variational autoencoder in an embodiment of the present invention; Figure 3 This is a schematic diagram of the potential space optimization perturbation process in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The purpose of this invention is to provide a method and system for generating adversarial examples in latent space based on multi-scale feature separation. First, the input image is divided into low-frequency and high-frequency components at different scales using differentiable wavelet packet transform, allowing for separate processing of the overall structure and detailed features of the image. Then, a hierarchical vector quantization variational autoencoder is used to encode and vector quantize the high-frequency components. Furthermore, the feature representation capability of the latent space can be improved by refining the codebook update strategy and dead code reset mechanism. When generating adversarial examples, the algorithm directly applies iterative optimization perturbations to the hierarchical latent space, ensuring both attack effectiveness and image quality, generating adversarial examples that are closer to real images and possess high attack power, high fidelity, and interpretability.

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] Example 1

[0024] like Figures 1-3 As shown, the latent space adversarial sample generation method based on multi-scale feature separation provided by this invention includes the following steps: S1, Training of the hierarchical vector quantization variational autoencoder: A neural network quantization training method based on direct-pass estimation is employed to train a hierarchical vector quantization variational autoencoder. Specifically, this includes: S101, during the forward propagation process, the continuous latent vector output by the hierarchical vector quantization variational autoencoder is discretized based on the nearest neighbor search method and mapped to a discrete codebook vector. S102, during the backpropagation process, bypassing the non-differentiability of discretization, a direct estimation method is used to directly pass the gradient of the decoder with respect to the discrete codebook vector to the corresponding gradient of the encoder output. Using the gradient information obtained in step S102, the encoder parameters are optimized by gradient descent, and the discrete codebook vector is updated based on the codebook loss. S103 employs a codebook update strategy based on the Exponential Moving Average (EMA) algorithm. This strategy dynamically maintains the usage frequency and corresponding feature statistics of each discrete codebook vector, and updates the spatial position of the discrete codebook vectors accordingly. This improves training stability and prevents codebook collapse. Specifically, it includes: In each training iteration, the discrete codebook vectors are statistically analyzed. Real-time usage frequency and the sum of the latent vectors output by the corresponding encoder , where e is used to characterize the encoder level, including the top encoder top and the bottom encoder bottom, and i represents the index of the feature vector; Subsequently, the feature statistics were smoothed using the exponential moving average algorithm, and the updated formula was defined as follows: , ; in, The representative only applies to all satisfied... This condition Corresponding The terms are summed, where q(·) denotes the discretization mapping operation of the vector quantizer. Indicates the first output of the encoder i A continuous potential vector, This is the attenuation coefficient, used to control the proportion of historical information retained; , Let represent the values ​​of the k-th discrete codebook vector at iteration time steps t-1 and t, respectively, and generate the update state of the k-th discrete codebook vector in the model. These represent the intermediate calculation results of the k-th feature channel in the generative model (weighting coefficients of the self-attention mechanism or intermediate features of the residual module), respectively, and the moving average of the corresponding discrete codebook vector; Based on the updated feature statistics, the discrete codebook vector is adjusted in spatial position using a weighted average: = ; in, This represents the k-th codebook vector after the (t+1)-th update step; This represents the sum of latent vectors corresponding to the k-th codebook vector after EMA smoothing; This represents the frequency of use of the k-th codebook vector after EMA smoothing; It is a numerically stable term.

[0025] S104, Design a dead code detection and reset mechanism to detect discrete codebook vectors whose feature statistics are lower than a preset threshold, and reset them to the result of adding Gaussian noise to the codebook vectors used in high frequency; this step prevents the generation mode from becoming too uniform.

[0026] S2, Multi-scale Feature Separation: The input image is subjected to a differentiable Haar wavelet transform using the wavelet packet transform algorithm, which decomposes the input image into one low-frequency component (LL) and three high-frequency components to achieve multi-scale feature separation. The three high-frequency components are the horizontal high-frequency component (HL), the vertical high-frequency component (LH), and the diagonal high-frequency component (HH), respectively, thereby realizing feature separation of the image at multiple scales.

[0027] S3, layered encoding: The high-frequency components obtained in step S2 are input into the trained hierarchical vector quantization variational autoencoder to extract and quantize global high-frequency features and local high-frequency detail features, obtaining the corresponding top-level latent vector and bottom-level latent vector. Specifically, this includes: S301 uses the top-level encoder in the hierarchical vector quantization variational autoencoder to extract global high-frequency features from the high-frequency components of the input, and uses vector quantization to discretize the extracted global high-frequency features to obtain the top-level latent vector. S302: After concatenating the discretization result of the top-level encoder with the input high-frequency components, the input is used to the bottom-level encoder in the hierarchical vector quantization variational autoencoder to extract local high-frequency detail features. The extracted local high-frequency detail features are then quantized to obtain the bottom-level latent vector.

[0028] S4, Feature Reconstruction and Preliminary Sample Generation: In the latent space of a hierarchical vector quantization variational autoencoder, learnable perturbation variables are introduced based on the top-level and bottom-level latent vectors to construct perturbed latent vectors, which are then reconstructed into adversarial examples by a decoder. Specifically, this includes: Learnable perturbation variables are introduced into the latent space respectively. and The perturbated latent vector is constructed as follows: , ; in, and These represent the top-level latent vector and the bottom-level latent vector, respectively. and These represent the top-level latent vector and the bottom-level latent vector after perturbation, respectively. The constraints on the disturbance variables are: , , , These are the preset top-level disturbance threshold and bottom-level disturbance threshold, respectively; The decoder combines the perturbed latent vector with the low-frequency component (LL) obtained in step S2, and reconstructs it into an adversarial example through wavelet packet inverse transform. .

[0029] S5, Combat Disturbance Injection and Optimization: The learnable perturbation variables are initialized. Starting with the original adversarial example, and based on a preset perturbation target, the perturbation vector is iteratively optimized until an adversarial example that satisfies the attack success conditions and has improved visual quality is generated. Specifically, this includes: S501, using the latent vectors of the original adversarial examples. and Starting from this point, initialize the perturbation variables. ; In S502, at each iteration, the decoder reconstructs the perturbed latent vector into adversarial examples. Input the target model and calculate the classification loss. ,in, p (·) represents the probability distribution, and y represents the true category label; S503 aims to maximize the classification loss of the target model by iteratively optimizing the perturbation variables using the gradient ascent method. During the iteration process, the perturbation variables are projected to satisfy the constraints of the perturbation variables until an adversarial example that satisfies the attack success conditions and has optimized visual quality is generated.

[0030] In step S503, the projected perturbation variable during the iteration process satisfies the constraint conditions of the perturbation variable, specifically including: Update the perturbation variable through backpropagation: ; in, For learning rate, For projection operations, ensure that the perturbation variables meet the constraints of the perturbation variables. This represents the gradient of the adversarial loss function with respect to the perturbation δ, combined with adversarial examples. The specific details are as follows x +δ, x This is a normal sample; If the attack is successful within a certain number of consecutive steps and the loss is not significantly improved, then the adversarial sample with the best visual quality and the most successful attack is output.

[0031] Specifically, the latent space adversarial sample generation method based on multi-scale feature separation in Embodiment 1 of the present invention includes the following implementation steps: First, a hierarchical vector quantization variational autoencoder was trained based on the ImageNet dataset (J. Deng, W. Dong, R. Socher, L.-J. Li, KaiLi and Li Fei-Fei, "ImageNet: A large-scale hierarchical image database," 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 2009, pp. 248-255, doi: 10.1109 / CVPR.2009.5206848.) and a direct-pass estimation method, specifically as follows: Figure 2 As shown, it includes: a. Forward propagation phase: Propagating the continuous latent vectors output by the encoder. Discretization is employed, which includes replacing the vector with a discrete codebook vector based on a nearest neighbor search method. ; b. Backpropagation stage: Bypassing the non-differentiability of the discretization operation, the decoder is directly propagated with respect to the discrete codebook vector. gradient Copy and pass to encoder output corresponding gradient ; c. Parameter Update Phase: Using the gradient information obtained in step b, the encoder parameters are optimized using gradient descent, and the discrete codebook vector is updated based on the codebook loss. Update.

[0032] Specifically, this embodiment employs Vector Quantization (VQ) technology (R. Gray, "Vector quantization," in IEEE ASSP Magazine, vol. 1, no. 2, pp. 4-29, April 1984, doi: 10.1109 / MASSP.1984.1162229.), which maps vectors in a continuous domain to a finite discrete set (codebook). This technology has been widely applied in signal compression and feature coding. In this invention, the VQ technology not only significantly enhances the feature representation capability of the latent space but also helps improve the stability and diversity of the generative model in adversarial example generation tasks. The specific implementation scheme is as follows: 1. Construction of a hierarchical VQ-VAE framework: To fully leverage the generation potential of VAE and the discretization advantages of VQ, this embodiment constructs an end-to-end generation system of hierarchical VQ-VAE. Its basic modules include an encoder, a vector quantizer, and a decoder; ① Encoder: Maps the input image to the latent space to obtain a continuous latent variable representation; ② Vector quantizer: Uses the nearest neighbor search method to map the continuous latent variables to vectors in a pre-designed discrete codebook, completing the discretization of the latent space; ③ Decoder: Receives the discretized codebook vectors, reconstructs and generates an output that is close to the original image, achieving high-fidelity image reproduction.

[0033] 2. Improved quantization strategy: To improve the update efficiency of the codebook and the robustness of the latent space, this embodiment not only supports the standard VQ strategy, but also introduces improved quantization techniques, including the use of an EMA update strategy and a K-means-based initialization method, thereby further optimizing the mutual proximity effect between the codebook vector and the encoder output.

[0034] 3. Loss Function Design: To ensure collaborative optimization of the encoder, vector quantizer, and decoder during training, this embodiment designs a rigorous loss function system, mainly comprising the following two parts: Reconstruction Loss: Constrains the similarity between the decoder output and the original input to ensure the quality of the reconstructed image; Quantization Loss: Constrains the consistency between the encoder output and the codebook vector, prompting the latent variable distribution to converge towards the discrete codebook. In summary, the sum of the two loss functions constitutes the final objective function, expressed as: ,in, For encoder output, For codebook vectors, This indicates that the gradient has stopped. The commitment loss coefficient, This represents the vector after decoding the codebook vector, specifically as follows: =decode(e).

[0035] In the latent space codebook adaptive update stage, this implementation achieves adaptive control of the codebook update by dynamically maintaining the usage frequency and feature statistics of the codebook vectors. Specifically, the update process involves first statistically analyzing each discrete codebook vector in each training iteration. Real-time usage frequency and the sum of the latent vectors output by the corresponding encoder Subsequently, the EMA mechanism was used to smooth the historical statistics, and the update formula was defined as: ,

[0036] (This represents summing only over all corresponding terms that satisfy this condition), where This is the attenuation coefficient, used to control the proportion of historical information retained. Based on the updated statistics, the codebook vector adjusts its spatial position through a weighted average:

[0037] in, This represents the k-th codebook vector after the (t+1)-th update step; This represents the sum of latent vectors corresponding to the k-th codebook vector after EMA smoothing; This represents the frequency of use of the k-th codebook vector after EMA smoothing; For numerically stable terms (the value can be set to...) ).

[0038] This embodiment employs a dead code detection and reset phase, based on EMA statistics. Set threshold ,when At that time, it was determined If the code is dead, perform a reset operation. For example, randomly select a frequently used codebook vector. After applying Gaussian noise perturbation to it, the original vector is replaced. .

[0039] Optionally, this implementation uses the classic Haar wavelet as the wavelet basis, and performs multi-resolution frequency domain analysis of the two-dimensional image through the following steps, such as... Figure 2 As shown, this process is embedded into a deep neural network framework to construct a differentiable wavelet transform module with complete gradient propagation characteristics, specifically including: 1. Haar wavelet basis parameter settings: In this embodiment, the low-pass filter and high-pass filter defined by the Haar wavelet are as follows:

[0040] 2. Two-dimensional image wavelet decomposition process: Using the above filters, this embodiment realizes two-dimensional image decomposition based on multi-resolution analysis. Its core steps include: (1) Horizontal filtering and downsampling. First, the input image is subjected to convolution operation of low-pass filter (L) and high-pass filter (H) along the row direction, and downsampling is performed to initially extract horizontal frequency domain information. (2) Vertical filtering and downsampling: Subsequently, the result after the above horizontal filtering is applied to the column direction with low-pass and high-pass filters respectively, and downsampling is performed to generate four feature subbands: LL subband: double low-pass filtering result, retaining the low-frequency components and overall structural features of the image; HL subband: first horizontal high-pass and then vertical low-pass filtering, extracting horizontal edge information; LH subband: first vertical high-pass and then horizontal low-pass filtering, capturing vertical edge features; HH subband: double high-pass filtering result, reflecting diagonal high-frequency components and texture details. (3) Constructing a differentiable wavelet transform module: To achieve end-to-end gradient propagation, this embodiment embeds the wavelet decomposition process into the deep learning framework. Specifically, this includes: registering the low-pass and high-pass filter parameters as trainable buffer parameters of the network in tensor form to ensure their automatic transfer between GPU / CPU; ensuring that the gradient transferability is maintained during backpropagation throughout the wavelet transform process, thereby supporting joint optimization of model parameters. (4) Multi-channel color image processing strategy: When processing multi-channel color images, this embodiment adopts a channel-independent wavelet decomposition strategy, that is, performing a complete wavelet decomposition operation on each color channel separately. This strategy ensures the complete preservation of the frequency domain features of each channel and effectively avoids information interference that may be introduced due to cross-channel processing.

[0041] Optionally, this embodiment addresses the problem that traditional vector quantization variational autoencoder models with single-layer coding structures cannot simultaneously model the global structure of an image (such as object contours) and local details (such as texture edges). It proposes an improved layered vector quantization variational autoencoder based on a layered coding architecture to achieve multi-level feature extraction and quantization, thereby obtaining multi-granular representations of image features and significantly improving the accuracy and stability of the generated reconstruction.

[0042] This embodiment of the hierarchical vector quantization variational autoencoder specifically includes: 1. Top-level encoder design: The top-level encoder takes the feature map obtained by stitching together high-frequency components of the image (including HL, LH, HH) as input, and uses a multi-layer convolutional network, residual module, and self-attention mechanism to process it collaboratively, focusing on extracting high-frequency information that characterizes the global structure of the image. After preliminary feature extraction, the continuous features are discretized through a vector quantizer to form quantized features with high-level structural descriptive capabilities. 2. Bottom-level encoder design: The bottom-level encoder fuses the original high-frequency components with the quantized output of the top-level encoder, and further extracts local detail features through a network structure similar to the top-level encoder. This fine-grained feature is also vector quantized and used in the subsequent decoding process, providing refined feature support and ensuring that local information is fully expressed in image reconstruction.

[0043] In this embodiment, the decoder structure is as follows: The decoder adopts an inverse structure symmetrical to the encoder, and its key components include multi-level convolutional layers, a residual module, a self-attention module, and an upsampling module. The decoder input has a feature fusion module that organically integrates the quantized features of the top and bottom layers, reconstructs high-frequency components through layer-by-layer decoding, and finally combines them with the original LL low-frequency components, achieving complete image reconstruction through wavelet inverse transform. The residual module in this embodiment uses a two-layer convolutional structure (in the order of convolutional layer → normalization layer → activation function) combined with identity mapping. When the input and output channel dimensions do not match, dimensionality adaptation is achieved through 1×1 convolution. This design not only alleviates the gradient vanishing problem in deep networks to some extent but also significantly improves feature representation capabilities. The self-attention module captures long-range dependencies in the image by constructing three spatial correlation matrices: Query, Key, and Value. This module weightedly fuses the original features and attention features and introduces learnable coefficients to achieve adaptive feature enhancement, thereby effectively improving the joint modeling capability of global and local features.

[0044] Furthermore, this application proposes a vector quantizer as a core component of feature discretization. The vector quantizer employs a codebook mapping mechanism to convert continuously encoded features into discrete representations. Its workflow includes two steps: a. For each encoder output feature vector, perform nearest neighbor matching using Euclidean distance, and select the closest codeword from the codebook to replace it; b. By using the dual constraints of reconstruction loss and codebook loss, we ensure consistency between encoded features and codebook vectors.

[0045] This implementation achieves perturbation injection in a low-dimensional discrete latent space by optimizing the preset attack target loss, such as... Figure 3 As shown, this optimization process ensures both the effectiveness of the adversarial attack and the visual quality of the adversarial examples. Specifically, it includes: a. Latent Space Perturbation Optimization: The input sample is first decomposed into low-frequency (LL) and high-frequency (HL, LH, HH) components by differentiable wavelet transform. The high-frequency components are then used by a hierarchical encoder to generate quantized latent vectors at the top level (global features) and the bottom level (local details). and Subsequently, learnable perturbation variables are introduced into the latent space. and Construct the perturbed latent representation: , , , ,here , This is a preset perturbation threshold. The classification loss of the target model is maximized. The perturbation variables are iteratively optimized using the gradient ascent method. The decoder then reconstructs the perturbed latent variables into adversarial examples. .

[0046] b. Multi-stage adversarial example generation: To effectively improve attack efficiency and sample quality, the method designs an iterative optimization framework (such as...). Figure 3 As shown), it includes the following core steps: First, the perturbation variables are initialized with the latent vectors of the original samples. , Starting from this point, initialize the perturbation variables. Then, iterative optimization is performed. In each iteration, the decoder reconstructs the perturbed latent vectors into adversarial examples. Input the target model to calculate the loss. Then, the perturbation variables are updated through backpropagation. .in For learning rate, For projection operations, ensure that the disturbance conforms to the above definition. Constraints. The adversarial sample with the best visual quality and successful attack is output only if the attack succeeds within a certain number of consecutive steps without significant improvement in loss.

[0047] Example 2 This invention also provides a latent space adversarial example generation system based on multi-scale feature separation, applied to the above-described latent space adversarial example generation method based on multi-scale feature separation, comprising: The hierarchical vector quantization variational autoencoder training module is used to train the hierarchical vector quantization variational autoencoder using a neural network quantization training method based on pass-through estimation. The multi-scale feature separation module is used to perform a differentiable Haar wavelet transform on the input image using the wavelet packet transform algorithm, decomposing the input image into one low-frequency component and three high-frequency components to achieve multi-scale feature separation; wherein, the three high-frequency components are horizontal, vertical and diagonal high-frequency components, respectively. The hierarchical encoding module is used to input the obtained high-frequency components into the trained hierarchical vector quantization variational autoencoder to extract and quantize global high-frequency features and local high-frequency detail features, and obtain the corresponding top-level latent vector and bottom-level latent vector. The feature reconstruction and preliminary sample generation module is used to introduce learnable perturbation variables into the latent space of the hierarchical vector quantization variational autoencoder based on the top-level and bottom-level latent vectors, construct perturbation latent vectors, and reconstruct them into adversarial samples by the decoder. The adversarial perturbation injection and optimization module is used to initialize learnable perturbation variables. Starting from the original adversarial sample, it iteratively optimizes the perturbation vector based on the preset perturbation target until an adversarial sample that meets the attack success conditions and has improved visual quality is generated.

[0048] Furthermore, embodiments of the present invention provide a computer-readable storage medium storing executable instructions, which, when executed, cause a processor to perform the latent space adversarial sample generation method based on multi-scale feature separation as described in Embodiment 1.

[0049] Matters not covered in this invention are common knowledge.

[0050] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0051] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A latent space adversarial sample generation method based on multi-scale feature separation, characterized in that, Includes the following steps: S1 employs a neural network quantization training method based on direct-pass estimation to train a hierarchical vector quantization variational autoencoder; S2 uses the wavelet packet transform algorithm to perform a differentiable Haar wavelet transform on the input image, decomposing the input image into one low-frequency component and three high-frequency components to achieve multi-scale feature separation; wherein, the three high-frequency components are horizontal, vertical and diagonal high-frequency components, respectively. S3. Input the high-frequency components obtained in step S2 into the trained hierarchical vector quantization variational autoencoder to extract and quantize global high-frequency features and local high-frequency detail features, and obtain the corresponding top-level latent vector and bottom-level latent vector. S4, in the latent space of the hierarchical vector quantization variational autoencoder, based on the top-level latent vector and the bottom-level latent vector, introduces learnable perturbation variables to construct perturbation latent vectors, and reconstructs them into adversarial examples through the decoder; S5 initializes the learnable perturbation variables. Starting from the original adversarial sample, it iteratively optimizes the perturbation vector based on the preset perturbation target until an adversarial sample that meets the attack success conditions and has optimized visual quality is generated.

2. The latent space adversarial sample generation method based on multi-scale feature separation according to claim 1, characterized in that, The hierarchical vector quantization variational autoencoder includes: The top-level encoder takes the feature map obtained by concatenating three high-frequency components as input and uses a multi-layer convolutional network, residual module and self-attention mechanism to process and extract high-frequency information that represents the global structure of the image as global high-frequency features. After the initial feature extraction, the continuous features are discretized by a vector quantizer to form quantized features with high-level structure description ability. The bottom encoder fuses the original high-frequency components with the quantized output of the top encoder. It further extracts local high-frequency detail features through the same network structure as the top encoder, and then quantizes them through a vector quantizer for use in the subsequent decoding process. The decoder includes multi-level convolutional layers, a residual module, a self-attention module, and an upsampling module. The decoder input is equipped with a feature fusion module, which integrates the quantization features of the top-level encoder and the bottom-level encoder, reconstructs the high-frequency components through layer-by-layer decoding, and finally combines them with the low-frequency components to achieve complete image reconstruction through wavelet inverse transform.

3. The latent space adversarial sample generation method based on multi-scale feature separation according to claim 1, characterized in that, In S1, training the hierarchical vector quantization variational autoencoder specifically includes: S101, during the forward propagation process, the continuous latent vector output by the hierarchical vector quantization variational autoencoder is discretized based on the nearest neighbor search method and mapped to a discrete codebook vector. S102, during the backpropagation process, bypassing the non-differentiability of discretization, a direct estimation method is used to directly pass the gradient of the decoder with respect to the discrete codebook vector to the corresponding gradient of the encoder output. S103 adopts a codebook update strategy based on the exponential moving average algorithm to dynamically maintain the usage frequency and corresponding feature statistics of each discrete codebook vector, and update the spatial position of the discrete codebook vector accordingly. S104, design a dead code detection and reset mechanism, detect discrete codebook vectors whose usage feature statistics are lower than a preset threshold, and reset them to the result of adding Gaussian noise to the high-frequency codebook vectors.

4. The latent space adversarial sample generation method based on multi-scale feature separation according to claim 3, characterized in that, S103 further includes: using the gradient information obtained in step S102 to optimize the encoder parameters by gradient descent, and updating the discrete codebook vector based on the codebook loss.

5. The latent space adversarial sample generation method based on multi-scale feature separation according to claim 3, characterized in that, S103 employs a codebook update strategy based on the exponential moving average algorithm to dynamically maintain the usage frequency and corresponding feature statistics of each discrete codebook vector, and updates the spatial position of the discrete codebook vector accordingly. Specifically, this includes: In each training iteration, the discrete codebook vectors are statistically analyzed. Real-time usage frequency and the sum of the latent vectors output by the corresponding encoder , where e is used to characterize the encoder level, including the top encoder top and the bottom encoder bottom, and i represents the index of the feature vector; Subsequently, the feature statistics were smoothed using the exponential moving average algorithm, and the updated formula was defined as follows: , ； in, The representative only applies to all satisfied... This condition Corresponding The terms are summed, where q(·) denotes the discretization mapping operation of the vector quantizer. This represents the i-th continuous latent vector output by the encoder. This is the attenuation coefficient, used to control the proportion of historical information retained; , Let represent the values ​​of the k-th discrete codebook vector at iteration time steps t-1 and t, respectively, and generate the update state of the k-th discrete codebook vector in the model. These represent the intermediate calculation results of the k-th feature channel in the generative model at iteration time steps t and t-1, respectively, corresponding to the moving average of the discrete codebook vector; Based on the updated feature statistics, the discrete codebook vector is adjusted in spatial position using a weighted average: = ； in, This represents the k-th codebook vector after the (t+1)-th update step; This represents the sum of latent vectors corresponding to the k-th codebook vector after EMA smoothing; This represents the frequency of use of the k-th codebook vector after EMA smoothing; It is a numerically stable term.

6. The latent space adversarial sample generation method based on multi-scale feature separation according to claim 1, characterized in that, Step S3 involves inputting the high-frequency components obtained in step S2 into the trained hierarchical vector quantization variational autoencoder to extract and quantize global high-frequency features and local high-frequency detail features, thereby obtaining the corresponding top-level latent vector and bottom-level latent vector. Specifically, this includes: S301 uses the top-level encoder in the hierarchical vector quantization variational autoencoder to extract global high-frequency features from the high-frequency components of the input, and uses vector quantization to discretize the extracted global high-frequency features to obtain the top-level latent vector. S302: After concatenating the discretization result of the top-level encoder with the input high-frequency components, the input is used to the bottom-level encoder in the hierarchical vector quantization variational autoencoder to extract local high-frequency detail features. The extracted local high-frequency detail features are then quantized to obtain the bottom-level latent vector.

7. The latent space adversarial sample generation method based on multi-scale feature separation according to claim 1, characterized in that, S4, in the latent space of the hierarchical vector quantization variational autoencoder, introduces learnable perturbation variables based on the top-level and bottom-level latent vectors to construct perturbed latent vectors, which are then reconstructed into adversarial examples by the decoder. Specifically, this includes: Learnable perturbation variables are introduced into the latent space respectively. and The perturbated latent vector is constructed as follows: ， ； in, and These represent the top-level latent vector and the bottom-level latent vector, respectively. and These represent the top-level latent vector and the bottom-level latent vector after perturbation, respectively. The constraints on the disturbance variables are: , , , These are the preset top-level disturbance threshold and bottom-level disturbance threshold, respectively; The decoder combines the perturbed latent vector with the low-frequency component obtained in step S2, and reconstructs it into an adversarial example through wavelet packet inverse transform. .

8. The latent space adversarial sample generation method based on multi-scale feature separation according to claim 7, characterized in that, S5 initializes the learnable perturbation variables. Starting with the original adversarial example, and based on a preset perturbation target, iteratively optimizes the perturbation vector until an adversarial example that satisfies the attack success conditions and has improved visual quality is generated. Specifically, this includes: S501, using the latent vectors of the original adversarial examples. and Starting from this point, initialize the perturbation variables. ; In S502, at each iteration, the decoder reconstructs the perturbed latent vector into adversarial examples. Input the target model and calculate the classification loss. ,in, p (·) represents the probability distribution, and y represents the true category label; S503 aims to maximize the classification loss of the target model by iteratively optimizing the perturbation variables using the gradient ascent method. During the iteration process, the perturbation variables are projected to satisfy the constraints of the perturbation variables until an adversarial example that satisfies the attack success conditions and has optimized visual quality is generated.

9. The latent space adversarial sample generation method based on multi-scale feature separation according to claim 8, characterized in that, In step S503, the projection of the perturbation variable during the iteration process to satisfy the constraint conditions of the perturbation variable specifically includes: Update the perturbation variable through backpropagation: ； in, For learning rate, For projection operations, ensure that the perturbation variables meet the constraints of the perturbation variables. This represents the gradient of the adversarial loss function with respect to the perturbation δ, combined with adversarial examples. The specific details are as follows x +δ, x This is a normal sample; If the attack is successful within a certain number of consecutive steps and the loss is not significantly improved, then the adversarial sample with the best visual quality and the most successful attack is output.

10. A latent space adversarial example generation system based on multi-scale feature separation, applied to the latent space adversarial example generation method based on multi-scale feature separation as described in any one of claims 1-9, characterized in that, include: The hierarchical vector quantization variational autoencoder training module is used to train the hierarchical vector quantization variational autoencoder using a neural network quantization training method based on pass-through estimation. The multi-scale feature separation module is used to perform a differentiable Haar wavelet transform on the input image using the wavelet packet transform algorithm, decomposing the input image into one low-frequency component and three high-frequency components to achieve multi-scale feature separation; wherein, the three high-frequency components are horizontal, vertical and diagonal high-frequency components, respectively. The hierarchical encoding module is used to input the obtained high-frequency components into the trained hierarchical vector quantization variational autoencoder to extract and quantize global high-frequency features and local high-frequency detail features, and obtain the corresponding top-level latent vector and bottom-level latent vector. The feature reconstruction and preliminary sample generation module is used to introduce learnable perturbation variables into the latent space of the hierarchical vector quantization variational autoencoder based on the top-level and bottom-level latent vectors, construct perturbation latent vectors, and reconstruct them into adversarial samples by the decoder. The adversarial perturbation injection and optimization module is used to initialize learnable perturbation variables. Starting from the original adversarial sample, it iteratively optimizes the perturbation vector based on the preset perturbation target until an adversarial sample that meets the attack success conditions and has improved visual quality is generated.

Images