A high-robustness image recognition method based on a frequency domain cross-domain attention mechanism

Abstract

This invention relates to a highly robust image recognition method based on a frequency-domain cross-domain attention mechanism, belonging to the fields of artificial intelligence and computer vision. The method constructs a full-frequency-domain processing architecture within the network, utilizes discrete cosine transform to achieve dual embedding of space and frequency, introduces a frequency grouping strategy and a cross-domain attention mechanism, and combines adaptive frequency filtering to achieve deep spatial-frequency interaction and noise-resistant extraction of image features. This invention achieves inherent robustness without requiring expensive adversarial training, improves the diversity and anti-interference ability of feature extraction, breaks the trade-off between accuracy and robustness, maintains high recognition accuracy, and effectively replaces the base model of adversarial training methods.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and computer vision technology, and relates to an image classification method that uses frequency domain information to extract features in a deep learning model in order to improve the robustness of the model under non-adversarial training conditions. Background Technology

[0002] In recent years, deep learning technology has made groundbreaking progress in the field of computer vision. Among them, the Vision Transformer (ViT), with its powerful global feature modeling capabilities and adaptive mechanism, has gradually replaced Convolutional Neural Networks (CNNs) and become the preferred backbone network for large-scale general-purpose models (such as CLIP, DINO, SAM, etc.). These models are widely used in key areas such as autonomous driving, facial recognition, medical image analysis, and security monitoring. To promote technology sharing, many technology companies and academic institutions have chosen to open-source their model weights.

[0003] However, this openness also exposes serious security vulnerabilities. Research shows that existing ViT models suffer from a significant lack of adversarial robustness. Attackers can generate "adversarial examples" by superimposing minute perturbations imperceptible to the human eye (i.e., adversarial noise) onto the original image, easily inducing the model to make incorrect classification decisions. For example, in autonomous driving scenarios, misidentifying a "stop" sign as a "speed limit" sign could lead to serious safety accidents. Therefore, improving the security of ViT models so that they can maintain correct judgments in the face of malicious attacks has become a core issue that urgently needs to be addressed in the field of artificial intelligence security.

[0004] To defend against adversarial attacks, existing technical solutions are mainly divided into two categories: "model optimization" and "data optimization," with specific representative technologies as follows: (1) Adversarial Training (AT): This is currently recognized as the most effective defense strategy and falls under the category of "model optimization". Its basic principle is to integrate the generation process of adversarial examples into the training cycle of the model. Specifically, in each training iteration, adversarial examples are dynamically generated using attack algorithms such as PGD (Projected Gradient Descent), and these examples are mixed with the original clean examples to force the deep neural network (DNN) to learn robust features that are insensitive to perturbations. Representative algorithms include PGD-AT, TRADES (a trade-off algorithm that balances cleanliness accuracy and robustness), and UIAT (optimizing the decision boundary using inverse adversarial examples).

[0005] (2) Data Purification / Optimization: This is a defense strategy based on "data optimization," also known as input transformation defense. Its core idea is to add a preprocessing step before the image is input into the classification network, attempting to "clean" away adversarial noise in the image and restore it to a clean image. Representative methods include DiffPure and DISCO, which typically utilize diffusion models or generative adversarial networks (GANs) to reconstruct the input data. Through manifold projection or denoising operations, they eliminate potential malicious perturbations, thereby protecting the subsequent classification model from attacks.

[0006] Although the two types of existing technologies mentioned above have achieved certain results in academia, they still have serious limitations in practical industrial applications and high-performance scenarios, specifically in the following three aspects: (1) High cost and excessive accuracy degradation caused by adversarial training: ① Extremely high training overhead: Adversarial training requires generating adversarial examples through backpropagation in each iteration, which increases the training time by several times or even tens of times compared to normal training, resulting in huge computational resource consumption. ② Decreased accuracy of clean samples: Existing adversarial training methods have an inherent "robustness-accuracy trade-off". In order to gain resistance to noise, the model is often forced to ignore some high-frequency detail features that are beneficial to normal classification but vulnerable to adversarial attacks. This leads to a significant decrease in classification accuracy when the model faces "clean images" without attacks, making it difficult to meet the high-precision business requirements.

[0007] (2) Information loss and inference delay caused by data purification (efficiency and integrity issues): ① Destruction of effective information: Data purification methods (such as DiffPure) often have difficulty perfectly distinguishing between "adversarial perturbations" and "the original high-frequency texture details of the image" during the noise removal process. This results in the loss of a large amount of detailed information that is crucial to the semantic understanding of the image while cleaning the data, which in turn leads to a significant drop in the performance of the model when processing non-malicious noise images caused by compression or shooting. ② Increased inference latency: Introducing an additional data purification network means adding a complex computational step in the inference stage, which significantly increases the system's response latency, which is unacceptable for scenarios such as autonomous driving with extremely high real-time requirements.

[0008] (3) Traditional spatial domain ViT architecture lacks feature diversity (inherent vulnerability of the architecture): Regardless of whether the above-mentioned defense methods are adopted, most existing ViT basic models process images directly in the spatial domain. Studies have found that adversarial perturbations in the spatial domain often manifest as high-frequency noise, which is highly coupled with the original texture of the image and is difficult for the model to distinguish. In addition, the existing multi-head attention mechanism (MHA) design has defects, with a lack of interaction between different "heads", resulting in severe homogenization of extracted features (Lack of Feature Diversity). This means that model decisions often rely on local, single feature subsets, and once these features are targeted by attackers, the model's defenses will collapse instantly.

[0009] Therefore, there is an urgent need for an image recognition method that can overcome the problems of high training costs, decreased accuracy in clean image recognition, insufficient feature extraction, and poor network architecture adaptability of existing visual transformation models (ViT) when defending against adversarial attacks. Summary of the Invention

[0010] In view of this, the purpose of this invention is to provide a highly robust image recognition method based on a frequency-domain cross-domain attention mechanism. This method constructs a full-frequency-domain processing architecture within the network, utilizes Discrete Cosine Transform (DCT) to achieve dual embedding of spatial and frequency characteristics, introduces a frequency grouping strategy and a cross-domain attention mechanism (CDA), and combines adaptive frequency filtering to achieve deep spatial-frequency interaction and noise-resistant extraction of image features. Specifically, this invention addresses the following problems: 1) Achieving "Inherent" Robustness Without Expensive Adversarial Training: Addressing the issue that existing technologies rely on costly adversarial training to defend against attacks, this invention leverages the decoupling of signal and noise distributions in the frequency domain, directly endowing the model with defensive capabilities through network architecture design. In the frequency domain, adversarial perturbations typically manifest as high-frequency noise, separate from the semantic content of the image. This invention enables the model to naturally distinguish between semantic content and adversarial noise, significantly enhancing its defense against both white-box and black-box attacks without increasing training costs or generating adversarial examples.

[0011] 2) Significantly Enhances Feature Extraction Diversity and Anti-interference Capability: Addressing the issues of insufficient feature interaction between heads in the traditional Multi-Head Attention (MHA) mechanism of ViT, and its susceptibility to getting trapped in local features and thus becoming vulnerable to attacks, this invention proposes a frequency grouping strategy. This strategy forcibly assigns different frequency components to different attention heads, prompting the model to extract features from rich and complementary frequency bands. This mechanism effectively disperses the destructive power of adversarial noise on a single feature, ensuring that even if some frequency bands are interfered with, the model can still make correct decisions based on the features of other frequency bands.

[0012] 3) Breaking the "Accuracy-Robustness" Trade-off and Maintaining High Recognition Accuracy: Addressing the issue that existing defense methods (such as data purification) can easily lead to the accidental deletion of image details, resulting in a decrease in the recognition rate of clean images, this invention designs a cross-domain attention mechanism (CDA). This mechanism uses spatial features as a query term to retrieve key / value pairs that combine spatial and frequency information, introducing frequency domain information while preserving the integrity of spatial semantics. Combined with adaptive frequency filtering (such as retaining the first 36 key frequency components), it effectively filters out interfering noise while preserving key textures, enabling the model to achieve classification accuracy on "clean" images that is no lower than, and in some cases even better than, mainstream spatial domain models.

[0013] 4) Effective replacement of the base model of adversarial training method: At present, the most effective defense method is still adversarial training. Although the model of this invention can show certain inherent robustness and maintain high classification accuracy under non-adversarial training, in order to further improve the effect in practical use, we can use various existing adversarial training methods to conduct adversarial training on this model. This can achieve better robustness than the mainstream ViT while reducing the decline in clean sample recognition accuracy.

[0014] To achieve the above objectives, the present invention provides the following technical solution: Solution 1: A highly robust image recognition method based on a frequency-domain cross-domain attention mechanism. First, it utilizes Discrete Cosine Transform (DCT) to construct a dual embedding representation in the frequency and spatial domains. Then, it deeply fuses frequency domain information during the feature extraction stage through a cross-domain attention mechanism (CDA). Finally, it combines adversarial training to enhance the model's defense boundary. This method specifically includes the following steps: S1: Obtain the raw RGB image; S2: Constructing an image recognition network model, specifically including: Frequency domain transformation and dual embedding module: Performs frequency domain transformation and dual embedding on the original RGB image to generate two sets of feature vector sequences, namely spatial feature vector S and frequency feature vector F; Frequency Feature Adaptive Filtering and Multi-head Grouping Module: This module applies adaptive filtering and multi-head grouping to the frequency feature vector F to obtain a subset of frequency features. ,in i Representing the i One point of attention; Cross-domain attention computation module: combines spatial feature vector S and frequency feature subset By using a cross-domain attention mechanism, the context feature vector O of the frequency domain robust features is obtained through fusion. S3: Train the model using Projected Gradient Descent (PGD) or Fast Signed Gradient Method (FGSM) to obtain the robust network model weights after adversarial fine-tuning. S4: Input the image to be identified into the adversarially trained model and output the classification result and confidence score of the image.

[0015] Furthermore, in step S2, the frequency domain transformation and dual embedding module includes: Frequency domain transformation: Divide the input image x into a P×P non-overlapping grid in each color channel, perform discrete cosine transform (DCT) on each grid, and extract the frequency coefficients; Shared spatial embedding: In order to preserve the spatial location information of the image, a large sliding window is used to extract features and generate a spatial feature vector S, which mainly carries the spatial structure and low-frequency semantics of the image. Frequency embedding: P after discrete cosine transform 2 (e.g., 64) frequency coefficients are grouped according to their frequency positions to generate a frequency feature vector F, which accurately captures the high-frequency texture and detail information of the image.

[0016] Furthermore, in step S2, the frequency feature adaptive filtering and multi-head grouping module includes adaptive filtering and multi-head grouping strategies.

[0017] Furthermore, in step S2, the adaptive filtering specifically includes: inputting the frequency feature vector F into a multilayer perceptron (MLP); according to the preset hyperparameter μ, the multilayer perceptron automatically selects the top μ frequency components that contribute the most to image classification and discards the remaining high-frequency components (which usually contain adversarial perturbations invisible to the human eye).

[0018] Furthermore, in step S2, the multi-head grouping strategy specifically includes: assuming the network's multi-head attention mechanism has h heads; dividing the selected μ frequency feature vectors into h subgroups on average; each subgroup... It contains different frequency components.

[0019] Furthermore, in step S2, the cross-domain attention calculation module specifically includes: constructing query terms: generating a query matrix using the spatial location feature vector S. :

[0020] in, This represents the query vector projection matrix. It is a trainable parameter matrix, typically implemented using a linear layer in practice, with the aim of... Projected onto the query subspace for subsequent attention mechanism computation; Constructing key-value pairs: using the spatial feature vector S and the grouped frequency feature vectors Generate key-value pairs together ( The calculation formula is:

[0021]

[0022] in, The key vector (Key) computed for attention. The value vector computed for attention. , , , and They serve the same purpose: to transform trainable parameter matrices that change during training based on gradient optimization. Their function is to... and They are projected onto different subspaces for subsequent attention mechanism calculations.

[0023] Calculate attention weights:

[0024] in, The formula for calculating the attention mechanism; The scaling factor, equal in size to the feature dimension of the Q matrix, is used for scaling. The results were normalized.

[0025] Furthermore, in step S3, training the model specifically includes: adversarial example generation: based on the gradient information of the current network, using the Projected Gradient Descent (PGD) algorithm or the Fast Signed Gradient Method (FGSM), constrained perturbations are superimposed on the original input image to generate adversarial examples. Since the preceding module (CDA) already has the ability to identify frequency domain noise, the adversarial examples generated here will be more "covert" and "difficult".

[0026] Hybrid loss optimization: The original clean image and the generated adversarial example are simultaneously input into the network to calculate the hybrid loss function; this module updates the network parameters and weights through backpropagation.

[0027] Option 2: A highly robust image recognition system based on a frequency domain cross-domain attention mechanism, comprising: The frequency domain transformation and dual embedding module includes frequency domain transformation, shared spatial embedding and frequency embedding. It performs frequency domain transformation and dual embedding on the original RGB image to generate two sets of feature vector sequences, namely spatial feature vector S and frequency feature vector F. The frequency feature adaptive and filtering multi-head grouping module includes adaptive filtering and multi-head grouping strategies. It processes the frequency feature vector F through adaptive filtering and multi-head grouping to obtain a subset of frequency features. ,in i Representing the i One point of attention; The cross-domain attention processing module achieves deep interaction between spatial semantics and frequency domain details by constructing a cross-domain query mechanism. Specifically, it integrates the spatial feature vector S and the frequency feature subset. By using a cross-domain attention mechanism, the context feature vector O of the frequency domain robust features is obtained through fusion. Adversarial training enhancement module, including adversarial sample generation and hybrid loss optimization; The classification decision output module includes global feature aggregation and classification prediction; the global feature aggregation is to perform global average pooling on the output feature sequence or extract specific categories; the classification prediction is to calculate the probability distribution of each classification label through a fully connected layer.

[0028] The beneficial effects of this invention are as follows: 1) This invention effectively alleviates the contradiction between "improved robustness" and "decreased cleanliness accuracy" in adversarial defense. In existing technologies, intensive adversarial training for adversarial attacks often significantly sacrifices the model's recognition accuracy on normal images. This invention utilizes a frequency-domain cross-domain attention mechanism, enabling the model to distinguish between semantic content (mainly distributed in low and mid-frequency ranges) and adversarial noise (mainly distributed in high frequency ranges) in images. This architectural characteristic allows the model to better maintain its ability to extract key semantic features during adversarial training. Experiments show that compared to direct adversarial training on ordinary spatial domain models, this invention improves the model's resistance to attacks while maintaining classification accuracy on clean images with a 5%–10% less accuracy drop after adversarial training compared to other networks, thus balancing security and usability to a certain extent.

[0029] 2) Improved overall defense capabilities of the model across various attack scenarios. Compared to ordinary visual transformation models that are not optimized for the frequency domain, this invention constructs a more robust defense system through the combination of network architecture improvement (CDT) and adversarial training strategies. Experimental data shows that, thanks to frequency domain feature grouping and adaptive filtering strategies, the model has a natural insensitivity to some adversarial perturbations. Furthermore, after adding adversarial training, the model's attack success rate is lower than the baseline model when facing mainstream white-box attacks such as projective gradient descent (PGD) and some black-box transfer attacks, demonstrating stronger generalization defense capabilities and an average robustness improvement of 5%–8%.

[0030] 3) This invention improves upon the problem of single feature extraction in traditional multi-head attention mechanisms. Addressing the issue of low feature redundancy in traditional VisionTransformers where different attention heads tend to focus on similar features, this invention utilizes a frequency grouping strategy to guide different attention heads to focus on different frequency components. This mechanism increases the feature diversity of the model to a certain extent. When an attacker interferes with a specific feature frequency band, the model may still make the correct judgment through complementary features in other frequency bands, thus increasing the difficulty for attackers to generate effective adversarial examples.

[0031] 4) The efficiency of frequency domain information utilization in deep networks is optimized. Unlike existing technologies that simply convert images into DCT coefficients as input, this invention designs a dual embedding and cross-domain interaction structure. By preserving spatial location information and fusing it with frequency information, this invention improves the shortcomings of traditional frequency domain models in spatial structure perception. This design allows the model to utilize the noise resistance of the frequency domain while maintaining good spatial semantic understanding capabilities, providing a new approach that balances performance and efficiency for deploying visual models in security-sensitive scenarios.

[0032] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0033] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a block diagram of the highly robust image recognition system based on the frequency domain cross-domain attention mechanism proposed in this invention. Figure 2This is a flowchart of the highly robust image recognition method based on the frequency domain cross-domain attention mechanism proposed in this invention. Detailed Implementation

[0034] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0035] Please see Figure 1 This invention provides a highly robust image recognition system based on a frequency domain cross-domain attention mechanism, comprising the following five core functional modules: (1) Frequency domain transformation and dual embedding module: Function Description: This module is the system's entry point, responsible for converting the input spatial domain RGB image into a frequency domain signal and constructing feature vectors suitable for Transformer processing. Unlike traditional methods that only perform simple patch segmentation, this module executes a dual embedding strategy: Frequency domain transformation: Perform discrete cosine transform (DCT) on each channel of the image to extract frequency coefficients.

[0036] Shared spatial embedding: In order to preserve the spatial location information of the image, features are extracted using a large sliding window (preferably 16×16, covering 4 adjacent 8×8 DCT blocks) to generate a spatial feature vector S, which mainly carries the spatial structure and low-frequency semantics of the image.

[0037] Frequency embedding: The P-value after DCT transformation 2 (e.g., 64) frequency coefficients are grouped according to their frequency positions to generate a frequency feature vector F, which accurately captures the high-frequency texture and detail information of the image.

[0038] Input: Raw RGB image data (size such as H×W×3).

[0039] Output: Two sets of feature vector sequences—spatial feature vector S and frequency feature vector F.

[0040] (2) Frequency Feature Adaptive Filtering and Multi-Head Grouping Module: Function Description: This module is designed to optimize the utilization efficiency of frequency information and filter out noise.

[0041] Adaptive filtering: A multilayer perceptron (MLP) is built in as a filter. Based on the preset hyperparameter μ (preferably μ=36), it adaptively retains the top μ frequency components that are most critical to classification and filters out high-frequency interference terms that may contain noise redundancy.

[0042] Multi-head grouping strategy: To address the feature homogenization problem in traditional multi-head attention mechanisms (MHA), this module divides the filtered frequency feature vector F into multiple subgroups. Each subgroup contains different frequency components and is assigned to different "heads" in the subsequent attention mechanism, forcing the model to observe the image from different frequency band perspectives.

[0043] Input: Frequency feature vector F.

[0044] Output: Filtered and head-grouped frequency feature subsets (where i represents the i-th attention head).

[0045] (3) Cross-domain attention processing module: Functional Description: This module is the core computing unit of this invention, replacing the standard self-attention layer in the Transformer. It achieves deep interaction between spatial semantics and frequency domain details by constructing a cross-domain query mechanism. Query (Q) generation: generated solely from spatial feature vectors S, ensuring that the focus of attention is always based on the semantic structure of the image.

[0046] Key / Value (K / V) generation: Innovatively combining spatial feature vector S with grouped frequency feature vectors Concatenation is performed.

[0047] Attention calculation: Q is used to query K, which contains a mixture of spatial and frequency information, and the attention weights are calculated. This means that when the model processes a spatial location, it will simultaneously refer to its corresponding frequency domain features, thereby being able to distinguish between normal textures and adversarial noise.

[0048] Input: Spatial feature vector S and grouped frequency feature subsets .

[0049] Output: Context feature vector O that incorporates frequency domain robust features.

[0050] (4) Adversarial training enhancement module: Function Description: In order to further improve the model's ability to defend against extreme attacks on top of its inherent robustness, this module is embedded in the system as an enhancement component during training.

[0051] Adversarial example generation: Based on the gradient information of the current network, the projective gradient descent (PGD) algorithm is used. Alternatively, the Fast Gradient Sign Method (FGSM) can be used to superimpose restricted perturbations onto the original input image to generate adversarial examples. Since the preceding module (CDA) already has the ability to identify frequency domain noise, the adversarial examples generated here will be more "covert" and "difficult".

[0052] Hybrid Loss Optimization: The original clean image and the generated adversarial examples are simultaneously input into the network to calculate the hybrid loss function. This module updates the network parameters through backpropagation, forcing the model to not only adapt to frequency domain features but also form a wider "safety margin" at the decision boundary.

[0053] Input: Training set images, current network model parameters, adversarial attack hyperparameters (such as perturbation step size α, perturbation radius) ).

[0054] Output: Weights of the robust network model after adversarial fine-tuning (5) Classification decision output module: Functional description: The system's end-execution module is responsible for mapping the features, which have undergone multi-layer cross-domain attention processing and adversarial training enhancement, to specific category spaces.

[0055] Global feature aggregation: Perform global average pooling on the output feature sequence or extract specific category tokens.

[0056] Classification prediction: The probability distribution of each class label is calculated through a fully connected layer. Due to the effective processing of the front-end frequency domain module, the feature distribution output by this module is sparser in terms of inter-class distance, and has better robustness against interference.

[0057] Input: High-dimensional feature representation output by the cross-domain attention module.

[0058] Output: The classification result (category label) and confidence score of the image.

[0059] Please see Figure 2 Based on the above system architecture, the specific implementation method and process are as follows: Step 1: Construct an initial network model based on frequency domain cross-domain attention. First, a deep neural network (CDT model) for image recognition is constructed. This network does not use the traditional convolutional neural network (CNN) structure, but rather an improvement upon the Transformer architecture. The specific construction process includes: Input layer definition: Set the network to receive standard RGB image input with dimensions of H×W×3 (e.g., 224×224×3).

[0060] Core Module Stacking: The main body of the network consists of L stacked "Cross-Domain Transformer Modules (CDT Modules)". Each CDT module replaces the standard multi-head self-attention mechanism (MSA) with the cross-domain attention mechanism (CDA) unique to this invention, while retaining the feedforward neural network (FFN) and layer normalization (LayerNorm) components.

[0061] Classification Head Initialization: Initialize a fully connected classification head at the end of the network, with the output dimension corresponding to the number of categories of the task to be identified (such as 1000 categories in ImageNet or 10 categories in CIFAR).

[0062] Step 2: Image Frequency Domain Transformation and Dual Embedding Representation When the image enters the first layer of the network, unique preprocessing and embedding operations are performed to capture both spatial semantics and frequency details simultaneously: Block DCT Transform: The input image x is divided into a P×P non-overlapping grid (preferably P=8) for each color channel. A Discrete Cosine Transform (DCT) is performed on each 8×8 grid to convert the pixel-domain values ​​into 64 frequency coefficients.

[0063] Generating Spatial Feature Vectors (S): To avoid the DCT transformation destroying the spatial structure information of the image, this invention introduces shared spatial embedding. A sliding window of size 2P×2P (e.g., 16×16) is used to slide sample on the feature map after the DCT transformation with a step size P (e.g., 8). This means that each sampling window covers 4 adjacent DCT blocks. These sampled data are mapped to dimension C to form a spatial feature vector sequence S. This vector mainly encodes the macroscopic geometric structure and positional information of the image.

[0064] Generating a frequency feature vector (F): To exploit the noise resistance in the frequency domain, this invention introduces frequency embedding. The P obtained from the above DCT transformation... 2 The 64 frequency coefficients are categorized according to their positions in an 8×8 matrix. For example, the DC components at position (0,0) are grouped together, the AC components at position (0,1) are grouped together, and so on. This results in 64 independent frequency feature vectors, each representing the overall response distribution of the graph at a specific frequency. These vectors are also mapped to dimension C, forming a sequence of frequency feature vectors F.

[0065] Step 3: Adaptive filtering and multi-head grouping of frequency features To eliminate high-frequency redundancy that may contain adversarial noise and to improve feature diversity, F is processed as follows: Adaptive filtering: The frequency feature vector F is input into a multilayer perceptron (MLP).

[0066] Based on the preset hyperparameter μ (preferably μ=36 in this embodiment), the MLP automatically selects the top 36 frequency components that contribute the most to image classification and discards the remaining 28 high-frequency components (which usually contain adversarial perturbations invisible to the human eye).

[0067] Multi-head grouping: Assume the network's multi-head attention mechanism has h heads. Divide the selected μ frequency feature vectors into h subgroups on an average basis. Each subgroup... It incorporates different frequency components. For example, the first head focuses on low-frequency information, while the second head focuses on mid-frequency information. This grouping strategy forces different attention heads to focus on different frequency domain characteristics of the image.

[0068] Step 4: Perform cross-domain attention calculation: This is the most critical step in the network forward propagation. Within each CDT Block, attention is calculated according to the following logic: Constructing the query term (Query, Q): Generating the query matrix using only the spatial location feature vector S:

[0069] This ensures that the model's proactive search behavior (Query) is initiated based on the spatial semantics of the image.

[0070] Constructing key-value pairs (Key, K and Value, V): using the spatial feature vector S and the grouped frequency feature vectors. Together, key-value pairs are generated. The specific formula is:

[0071]

[0072] Here, a concat operation is used so that the key and value contain information about both the spatial location of the image (spatial) and the distribution of frequency features (frequency domain).

[0073] Attention weight calculation and output: Perform standard dot product attention operation:

[0074] Since K contains frequency information, Q will automatically avoid regions that are heavily polluted by high-frequency noise when calculating similarity (because frequency embedding has already filtered out some noise through step S3), thus achieving inherent robustness.

[0075] Step 5: Adversarial Training Enhancement Based on Gradient Projection Although the above network structure already possesses inherent robustness, to cope with extreme attacks, an adversarial training step is introduced during the training phase to generate adversarial examples: in each training iteration, for the original input image... x Generate adversarial examples using the gradient information of the current network. .

[0076] The PGD (Projected Gradient Descent) algorithm is used, and the formula is as follows:

[0077] Here, Π represents the projection operation. It is the maximum perturbation radius (e.g., 8 / 255), and α is the step size. It represents the loss function. About the input image gradient, It is the cross-entropy loss function. These are the parameters of the current neural network. It is the first t Adversarial examples of steps, It is the original, correct category label of the image.

[0078] Since the CDT model itself is sensitive to frequency domain noise, the adversarial examples generated here will be more challenging than those generated for ordinary CNNs, thus enabling the training of stronger models.

[0079] Hybrid loss function optimization: This involves optimizing the original clean image. x and generated adversarial examples Simultaneously input the network.

[0080] Calculate the total loss:

[0081] in, It is a coefficient, usually set to 0.5. It is the loss between the clean sample identification result and the correct label. This is the loss between the adversarial example identification result and the correct label. The network weights are updated by minimizing this loss through backpropagation. This step forces the network not only to identify clean frequency domain features, but also to learn to retain the ability to lock onto key frequency components (those preserved in step 3) even when the feature space is strongly distorted (adversarial attack).

[0082] Step 6: Reasoning and Classification Output: After performing steps 1 to 4, the CDT network is constructed. Then, step 5 is performed using the CDT network for adversarial training, which enables the input image to still be correctly recognized after perturbation is added.

[0083] (1) Input the image to be recognized into the adversarially trained CDT network; (2) The network finally outputs a confidence vector for classification; (3) Select the category with the highest confidence as the final identification result.

[0084] Thanks to structural optimization and adversarial training, the results maintain extremely high accuracy even when images are maliciously attacked (such as by adding noise or texture interference).

[0085] Verification Example: To verify the effectiveness of the high-robustness image recognition method based on frequency domain cross-domain attention mechanism proposed in this invention in real industrial scenarios, this section will describe its implementation through specific application scenarios.

[0086] This embodiment aims to demonstrate how the system can efficiently uncover security vulnerabilities in cloud models through a proactive local screening mechanism, under limited query budget and cloud defense (WAF) conditions.

[0087] 1) Experimental scenario and environment setup: (1) Application Background and Specific Technical Issues: In autonomous driving scenarios, traffic sign images captured by vehicle cameras are highly susceptible to interference from physical adversarial attacks. For example, malicious attackers may affix specially designed black and white stickers to "STOP" signs, or image sensors may be affected by electromagnetic noise from rain or snow. Traditional spatial domain convolutional neural networks (CNNs) are prone to misidentifying these interfered "STOP" signs as "speed limit 45", causing vehicles to rush through intersections without slowing down, resulting in serious safety accidents.

[0088] (2) System Deployment and Configuration: Hardware Environment: In-vehicle edge computing platform (such as NVIDIA Orin or Horizon Robotics Journey series chips), equipped with a forward-looking wide-angle camera. Software Model: Deploy the CDT (Cross Domain Transformer) model enhanced by adversarial training as described in this invention. Input Data: 224×224 resolution RGB traffic sign images acquired in real time by the camera.

[0089] 2) Specific implementation steps The processing flow in this embodiment includes two stages: "offline training" and "online inference". Phase 1: Offline adversarial training (model preparation) (1) Collect a large-scale dataset containing various traffic signs (stop, speed limit, yield, etc.).

[0090] (2) Construct the CDT network architecture of the present invention and set the frequency filtering parameter μ=36.

[0091] (3) Use the PGD (projected gradient descent) algorithm to generate adversarial samples that simulate physical sticker attacks.

[0092] (4) Mix the clean samples and adversarial samples, input them into the network for training, and obtain a robust model weight file.

[0093] Phase Two: Online Real-Time Inference (While Vehicle is in Operation) (1) Image acquisition: When the vehicle arrives at the intersection, the camera captures an image of a "stop" sign with malicious noise disturbance.

[0094] (2) Frequency domain transformation and preprocessing: The vehicle chip performs 8×8 block discrete cosine transform (DCT) on the image. At this time, the high-frequency interference signal introduced by the malicious sticker is separated into the lower right corner of the DCT matrix.

[0095] (3) Adaptive filtering: The system’s MLP filter automatically identifies and suppresses the high-frequency noise components generated by the sticker texture, while preserving the low-frequency structural features of the sign body.

[0096] (4) Cross-domain attention recognition: The model uses spatial features (the octagonal outline of the sign) as the query to query frequency domain features (the frequency distribution of red background and white text). Since the model has been trained adversarially, the attention mechanism automatically ignores the abnormal frequency response of the sticker area and accurately focuses on the semantic features of the word "STOP".

[0097] (5) Decision output: The confidence level of the classification head output "stop sign" is 60%. Based on this, the vehicle control system performs braking operation and successfully avoids the attack risk.

[0098] 3) Comparison of experimental results (verification of beneficial effects) Table 1 shows the classification accuracy of various transformer networks under the AutoAttack attack framework.

[0099] Table 1. Classification accuracy of various network structures under the AutoAttack attack framework

[0100] Experimental data show that the CDT network of this invention exhibits superior defensive performance in both adversarial training scenarios.

[0101] (1) Inherent robustness comparison (non-adversarial training state) As shown in the row marked "Adversarial Training is ×" in Table 1, traditional spatial domain models (DeiT, PVT, Swin, BiFormer) all exhibit a 0% robust accuracy when facing adversarial attacks. This means they are completely ineffective against malicious attacks without any defensive measures. In contrast, the CDT model of this invention achieves a robust accuracy of 8% without any adversarial training, breaking the "zero defense" dilemma of spatial domain models and outperforming the existing frequency domain model JPEG's 5%. This strongly demonstrates that the "frequency domain cross-domain attention mechanism" and "frequency feature grouping strategy" proposed in this invention endow the network architecture with a natural and inherent anti-interference capability.

[0102] (2) Comparison of comprehensive defense capabilities (with confrontation training) When adversarial training (rows marked "Adversarial Training" in the table) is introduced as an enhancement method, the advantages of this invention (CDT) are further amplified. The robust accuracy of the CDT model soars to 58.42%. Compared to the best-performing Swin-Transformer in the spatial domain (51.78%), this invention achieves a 6.64% improvement; compared to the JPEG model in the frequency domain (52.49%), it achieves a 5.93% improvement. This indicates that the architecture of this invention can more efficiently utilize adversarial examples for learning, constructing a more robust defense boundary, significantly outperforming existing technologies.

[0103] (3) Comparison of cleaning precision retention capabilities (key advantages) The most significant advantage lies in the "accuracy of clean samples" and its "decrease." Existing technologies generally suffer from serious "side effects": in order to gain defensive capabilities, the model's recognition rate on normal images drops significantly.

[0104] Existing technical pain points: After adversarial training, the Swin-Transformer's cleanliness accuracy decreased by 12.84% (from 88.08% to 75.24%), and its DeiT decreased by 11.8%. This means that while the model is safer, it has become "dumber" and cannot meet the needs of actual commercial applications.

[0105] Advantages of this invention: After adversarial training, the CDT model of this invention still achieves a clean sample accuracy of 82.27%, making it the only model among all comparison models to break the 80% mark. More importantly, its accuracy decrease is only 5.92%, far lower than the average drop of 10%-12% for other models.

[0106] In summary, the experimental data fully demonstrates that this invention does not simply sacrifice accuracy for security. Thanks to its unique frequency domain feature processing architecture, this invention achieves an industry-leading level of defense (58.42%) while minimizing performance degradation (only 5.92%), truly achieving a balance between high robustness and high accuracy, and possessing extremely high practical application value.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A highly robust image recognition method based on a frequency domain cross-domain attention mechanism, characterized in that, The method specifically includes the following steps: S1: Obtain the raw RGB image; S2: Constructing an image recognition network model, specifically including: Frequency domain transformation and dual embedding module: Performs frequency domain transformation and dual embedding on the original RGB image to generate two sets of feature vector sequences, namely spatial feature vector S and frequency feature vector F; Frequency Feature Adaptive Filtering and Multi-Head Grouping Module: This module applies adaptive filtering and multi-head grouping to the frequency feature vector F to obtain a subset of frequency features. ,in i Representing the i One point of attention; Cross-domain attention computation module: combines spatial feature vector S and frequency feature subset By using a cross-domain attention mechanism, the context feature vector O of the frequency domain robust features is obtained through fusion. S3: Train the model using the projected gradient descent algorithm or the fast gradient sign method to obtain the weights of the robust network model after adversarial fine-tuning. S4: Input the image to be identified into the adversarially trained model and output the classification result and confidence score of the image.

2. The highly robust image recognition method according to claim 1, characterized in that, In step S2, the frequency domain transformation and dual embedding module includes: Frequency domain transformation: Divide the input image x into a P×P non-overlapping grid in each color channel, perform a discrete cosine transform on each grid, and extract the frequency coefficients; Shared spatial embedding: Features are extracted using a sliding window to generate a spatial feature vector S; Frequency embedding: P after discrete cosine transform 2 The frequency coefficients are grouped according to their frequency positions to generate a frequency feature vector F.

3. The highly robust image recognition method according to claim 1, characterized in that, In step S2, the frequency feature adaptive filtering and multi-head grouping module includes adaptive filtering and multi-head grouping strategies.

4. The highly robust image recognition method according to claim 3, characterized in that, In step S2, the adaptive filtering specifically includes: inputting the frequency feature vector F into a multilayer perceptron; according to the preset hyperparameter μ, the multilayer perceptron automatically selects the top μ frequency components that contribute the most to image classification and discards the remaining high-frequency components.

5. The highly robust image recognition method according to claim 4, characterized in that, In step S2, the multi-head grouping strategy specifically includes: assuming the network's multi-head attention mechanism has h heads; dividing the selected μ frequency feature vectors into h subgroups on average; each subgroup... It contains different frequency components.

6. The highly robust image recognition method according to claim 1, characterized in that, In step S2, the cross-domain attention calculation module specifically includes: constructing query terms: generating a query matrix using the spatial location feature vector S. : in, Represents the query vector projection matrix; Constructing key-value pairs: using the spatial feature vector S and the grouped frequency feature vectors Generate key-value pairs together ( The calculation formula is: in, The key vector calculated for attention. The value vector computed for attention. , , , and They serve the same purpose: to transform trainable parameter matrices that change during training based on gradient optimization. Their function is to... and They are projected onto different subspaces for subsequent attention mechanism calculations; Calculate attention weights: in, The formula for calculating the attention mechanism; The scaling factor, equal in size to the feature dimension of the Q matrix, is used for scaling. The results were normalized.

7. The robust image recognition method according to claim 1, characterized in that, In step S3, training the model specifically includes: adversarial example generation: based on the gradient information of the current network, adversarial examples are generated by superimposing restricted perturbations onto the original input image using the projected gradient descent algorithm or the fast gradient sign method. ; Hybrid loss optimization: Input the original clean image and the generated adversarial example into the network simultaneously, calculate the hybrid loss function, and update the network parameters and weights through backpropagation.

8. A system applicable to the highly robust image recognition method according to any one of claims 1 to 7, characterized in that, The system includes: The frequency domain transformation and dual embedding module includes frequency domain transformation, shared spatial embedding and frequency embedding. It performs frequency domain transformation and dual embedding on the original RGB image to generate two sets of feature vector sequences, namely spatial feature vector S and frequency feature vector F. The frequency feature adaptive and filtering multi-head grouping module includes adaptive filtering and multi-head grouping strategies. It processes the frequency feature vector F through adaptive filtering and multi-head grouping to obtain a subset of frequency features. ,in i Representing the i One point of attention; The cross-domain attention processing module achieves deep interaction between spatial semantics and frequency domain details by constructing a cross-domain query mechanism. Specifically, it integrates the spatial feature vector S and the frequency feature subset. By using a cross-domain attention mechanism, the context feature vector O of the frequency domain robust features is obtained through fusion. Adversarial training enhancement module, including adversarial sample generation and hybrid loss optimization; The classification decision output module includes global feature aggregation and classification prediction; the global feature aggregation is to perform global average pooling on the output feature sequence or extract specific categories; the classification prediction is to calculate the probability distribution of each classification label through a fully connected layer.

Images