Fine-grained image classification method based on deep wavelet feature fusion

Abstract

This paper discloses a fine-grained image classification method based on deep wavelet feature fusion. In this method, the input image undergoes data preprocessing and then enters the WTConv module, where it is decomposed into multi-frequency sub-band features. The features are then input into the WaveAttention module, which extracts global and local information through a multi-scale attention mechanism. Finally, the WaveKAN module performs nonlinear refinement on the features and outputs the classification results. This method decomposes the image into multi-frequency sub-bands through wavelet transform, improving the model's ability to extract features of global structure and local details. It also enhances the attention mechanism, enabling it to fuse multi-scale and frequency domain information, and improving fine-grained classification performance. A nonlinear feature refinement module is introduced to optimize the classification head to model complex relationships, thereby improving prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of visual classification, and more specifically to a fine-grained image classification method based on deep wavelet feature fusion. Background Technology

[0002] Fine-grained visual classification (FGVC) is an important research direction in computer vision, aiming to identify objects with subtle inter-class differences but significant intra-class variations, such as different species of fish, birds, or plants. Traditional methods rely on convolutional neural networks (CNNs), such as ResNet, Inception, and DenseNet, which have achieved success in image classification through local feature extraction. However, as task complexity increases, the limitations of CNNs in modeling global contextual information and capturing multi-scale features become increasingly apparent.

[0003] In recent years, Visual Transformers (ViTs) have garnered significant attention for their ability to model long-range dependencies through self-attention mechanisms. ViTs segment images into fixed-size patches, treating them as labeled sequences, which significantly improves image classification performance. Subsequently, knowledge distillation enhances data efficiency, and hierarchical structures are introduced to generate multi-resolution feature maps, making them suitable for dense prediction tasks. However, they still face challenges in fine-grained classification: they excel at global modeling but lack the ability to capture local details and multi-scale features, and they lack explicit frequency domain modeling mechanisms.

[0004] Wavelet transform (WT) is a powerful signal processing tool capable of decomposing signals into multi-frequency and multi-scale representations. It excels in feature extraction, and learning CNN representations on wavelet subbands can improve image restoration performance. Integrating WT into U-Net optimizes the downsampling and upsampling processes. However, the systematic integration of wavelet transform with deep learning models (especially transformers) has not been fully explored. Common technical solutions include:

[0005] Method 1: Image classification based on standard ViTs

[0006] ViT and its variants (such as DeiT and Swin Transformer) model global contextual information through self-attention mechanisms. For example, they segment images into blocks and process them into sequences, improve efficiency through training strategies, and introduce hierarchical structures to generate multi-scale features.

[0007] Method 2: CNN Feature Extraction Based on Wavelet Transform

[0008] Combining wavelet transform with CNNs can, for example, utilize low-resolution wavelet subbands to recover image details and improve efficiency by decomposing features into multi-level subbands.

[0009] Method 3: Hybrid CNN and Transformer Model

[0010] By combining large and small block features with a dual-branch transformer, a multi-scale hierarchical structure is introduced. These methods attempt to integrate the advantages of CNN and ViT.

[0011] Current image classification tasks face challenges related to multi-source and multi-scale characteristics: natural images typically contain complex lighting variations (such as backlit or high dynamic range scenes), cross-scale object distributions (such as buildings ranging from meters to centimeters in remote sensing images), and the coupling of high-frequency noise (such as instrument artifacts in medical images) with low-frequency semantic information (such as the overall layout of a scene). This intertwined nature of multi-frequency features creates significant bottlenecks in traditional visual models for feature decoupling and focusing on key information, specifically manifested as follows:

[0012] (1) The ViTs method is good at modeling the global context, but it is not good at capturing the local details that are crucial in fine-grained classification tasks, which leads to a performance drop when distinguishing subtle differences between classes.

[0013] (2) It performs well in local feature extraction, but it cannot effectively model long-distance dependencies, which limits its applicability in complex scenarios.

[0014] (3) Existing methods lack explicit frequency domain modeling mechanisms and cannot simultaneously capture multi-scale features of global structure and local details. Summary of the Invention

[0015] To address the aforementioned shortcomings in existing technologies, this invention provides a fine-grained image classification method based on deep wavelet feature fusion.

[0016] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0017] A fine-grained image classification method based on deep wavelet feature fusion includes the following steps:

[0018] S1. Construct a fine-grained visual classification network that integrates wavelet transform and visual transform. Input the image to be classified and perform preprocessing to obtain standardized image data.

[0019] S2. Use the wavelet convolution module to perform multi-frequency sub-band decomposition on the standardized image data to generate a feature map that integrates global structure and local details.

[0020] S3. Input the feature map obtained in S2 into the wavelet attention module, compress the dimension of the key value vector through wavelet downsampling, and extract multi-scale features by combining the multi-head self-attention mechanism to obtain the optimized input features;

[0021] S4. Input the optimized input features into the wavelet Kolmogorov-Arnold network module, optimize the features through nonlinear basis functions, and output the classification probability.

[0022] Furthermore, step S2 specifically includes the following steps:

[0023] S21. The input standardized image is decomposed into low-frequency sub-band and high-frequency sub-band using discrete wavelet transform;

[0024] S22. Perform 7*7 large kernel convolution and attention channel mechanism on low frequency subband to enhance global features, and perform 3*3 small kernel convolution and convolution attention channel mechanism on high frequency subband to optimize detailed features;

[0025] S23. The global and detailed features obtained in S22 are reconstructed by inverse discrete wavelet transform to obtain reconstructed features, which are then fused with the features of the 5*5 convolution path to obtain a feature map that integrates global structure and local details.

[0026] Furthermore, the feature map that integrates global structure and local details in S23 is represented as follows:

[0027]

[0028] In the formula, To integrate feature maps that combine global structure with local details, For inverse wavelet transform, Enhanced global structural features For optimized local detail features, For Haar wavelet filters, Output a 5x5 convolution path.

[0029] Furthermore, step S3 specifically includes the following steps:

[0030] S31, Input the feature map obtained from S2 that fuses global structure and local details;

[0031] S32. Downsample the keys and values ​​using wavelet discrete transform to generate wavelet features;

[0032] S33. Perform multi-head attention, generate queries, keys, and values, calculate attention scores, and output weighted scores.

[0033] S34. Reconstruct multi-frequency features from the weighted output of S33 using inverse wavelet discrete transform, and then concatenate the classification tokens to obtain optimized input features.

[0034] Furthermore, the optimized input features in S34 are represented as follows:

[0035]

[0036] In the formula, To incorporate the input features optimized by residual connections, For linear projection of the input features, It is the inverse discrete wavelet transform and:

[0037]

[0038] In the formula, For inverse discrete wavelet transform, For weighted output, For multi-head attention scores, This represents the value of the multi-head attention after downsampling. It is a Haar wavelet filter.

[0039] Furthermore, step S4 specifically includes the following steps:

[0040] S41. Apply the KAN nonlinear transform to the optimized input features obtained in S3. And generate nonlinear features;

[0041] S42. Fuse the nonlinear features with the wavelet features to obtain the fused features;

[0042] S43. Combine and optimize the fusion features with different weights according to the fusion ratio, and generate classification results through linear projection.

[0043] Furthermore, the fusion feature in S42 is represented as follows:

[0044]

[0045] In the formula, As a feature of fusion, Wavelet features;

[0046] The classification result in S43 is represented as follows:

[0047]

[0048] In the formula, For the classification results, For linear projection of fused features.

[0049] The present invention has the following beneficial effects:

[0050] 1) By taking the stiffness and friction of the bearing contact interface as a bridge, the interaction between the tribology and dynamics of the control torque gyroscope is considered, and the prediction results will be more accurate.

[0051] 2) The model takes into account the effects of actual machining roughness, elastohydrodynamic lubrication and frictional heat on bearing contact stiffness and friction coefficient. Compared with traditional Hertzian contact stiffness and empirical friction coefficient, the model is more reasonable and scientific.

[0052] 3) The established tribodynamic model calculates the contact stiffness and friction coefficient of the corresponding dynamic load through interpolation, which can effectively reduce the calculation time of the model. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the fine-grained visual classification method based on wavelet fusion of the present invention.

[0054] Figure 2 This is a flowchart illustrating the overall framework of wavelet fusion in an embodiment of the present invention.

[0055] Figure 3 This is a schematic diagram of the wavelet convolution module structure in an embodiment of the present invention.

[0056] Figure 4 This is a schematic diagram of the wavelet attention module structure according to an embodiment of the present invention.

[0057] Figure 5 This is a schematic diagram of the wavelet Kolmogorov-Arnold network module structure according to an embodiment of the present invention. Detailed Implementation

[0058] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0059] A fine-grained image classification method based on deep wavelet feature fusion, such as Figure 1 As shown, it includes the following steps:

[0060] S1. Construct a fine-grained visual classification network that integrates wavelet transform and visual transform. Input the image to be classified and perform preprocessing to obtain standardized image data.

[0061] WaveFormer is a fine-grained visual classification (FGVC) framework that integrates wavelet transform (WT) and visual transformer (ViT). Through the synergistic effect of three modules—WTConv (wavelet convolution module), WaveAttention (wavelet attention module), and WaveKAN (wavelet Kolmogorov-Arnold network module)—it achieves multi-scale, multi-frequency feature extraction and efficient classification. The technical solution is illustrated in detail below with diagrams. Figure 2 As shown.

[0062] This invention uses the Fish4Knowledge dataset as the experimental benchmark. This dataset contains high-resolution images (average resolution 2560×1920 pixels) of 23 marine species in complex underwater environments (such as different light intensities, turbidity, and background interference). First, the pixel values ​​are normalized to the [0,1] interval to eliminate illumination bias. Second, a multi-scale data augmentation strategy is applied, including random horizontal flipping (probability 50%), ±15° rotation, and 224×224 pixel dynamic cropping, to enhance the robustness of the model to changes in viewing angle. Finally, the image is decomposed into four frequency sub-bands (LL / LH / HL / HH) using Discrete Wavelet Transform (DWT) to construct a multi-resolution feature pyramid, providing a foundation for subsequent joint frequency-spatial domain modeling.

[0063] S2. Use the wavelet convolution module to perform multi-frequency sub-band decomposition on the standardized image data to generate a feature map that integrates global structure and local details.

[0064] like Figure 3 The diagram shows the architecture of the WTConv module. The input features are decomposed into low-frequency (LL) and high-frequency (LH, HL, HH) subbands by DWT. The low-frequency subbands are processed by large kernel convolution and SE module, and the high-frequency subbands are processed by small kernel convolution and CBAM module. Finally, they are reconstructed by IDWT and fused with the convolution path.

[0065] Figure 3 The innovative architecture of the WTConv module is demonstrated in detail. The module decomposes the input image into multi-frequency sub-bands using wavelet transform, extracts global and local features, and optimizes feature fusion to improve representation capabilities. The specific process is as follows:

[0066] (1) Input and decomposition: Input feature map (B, C, H, and W represent batch size, number of channels, height, and width, respectively). The low-frequency subbands are decomposed using DWT (implemented via 2D convolution F.conv2d, with a stride of 2, halved spatial resolution, and 4 times the number of channels). (Global Information) and High-Frequency Subband (Detailed information) Haar wavelet decomposition was used. For sub-band:

[0067]

[0068] In the formula, wavelet = 'haar': Haar wavelet filter (low-pass: Qualcomm: To achieve lossless decomposition

[0069] (2) Feature optimization: for low-frequency subbands The global features are enhanced by applying 7x7 convolutions and SE modules, as shown below:

[0070] right application Convolution and SE modules:

[0071]

[0072] After splicing the high-frequency subbands, 3x3 convolution and CBAM modules are used to optimize the detailed features, as shown below:

[0073]

[0074] To smooth high-frequency subband fluctuations, multi-level DWT preprocessing is used, which is similar to removing noise components by decomposition, making high-frequency details clearer and reducing noise interference.

[0075] (3) Fusion and output: Features are reconstructed using IDWT (implemented by 2D transposed convolution F.conv_transpose2d with a stride of 2, doubling the spatial resolution). and features of the 5x5 convolution path Fusion, represented as:

[0076] right application convolution:

[0077]

[0078] Output The fusion weights are optimized by testing different convolutional kernel sizes (such as 3x3, 7x7, and 9x9) to ensure a balance between global and local features.

[0079] Reconstruct and merge features:

[0080]

[0081] Output:

[0082] Error Analysis and Optimization: The output error of the WTConv module is defined as follows:

[0083]

[0084] in, and These represent the module output and the true features at time t, respectively.

[0085] S3. Input the feature map obtained in S2 into the wavelet attention module, compress the dimension of the key value vector through wavelet downsampling, and extract multi-scale features by combining the multi-head self-attention mechanism to obtain the optimized input features;

[0086] like Figure 4 As shown, the WaveAttention module is integrated into the overall architecture. It downsamples the keys and values ​​through DWT, extracts features by combining a multi-head attention mechanism, and finally reconstructs the output through IDWT.

[0087] The WaveAttention module enhances global and local feature extraction by fusing multi-scale and frequency domain information through wavelet downsampling and multi-head attention mechanisms. The specific process is as follows:

[0088] Input to WTConv module output Wavelet features are generated by downsampling the key (K) and value (V) using DWT. This reduces computational complexity while preserving frequency information.

[0089] Perform multi-head attention to generate a query (Q), key (K), and value (V), represented as follows:

[0090]

[0091] in, The learnable projection matrix representing the query, key, and value will 3D feature mapping to dimension.

[0092] Simultaneously calculate the attention score and output it in a weighted manner, as follows:

[0093]

[0094]

[0095] in, .

[0096] To improve performance, different optimization algorithms (such as RMSProp, Adam, and SGD with Momentum) were tested and attention weights were adjusted. Adam typically performed best. Multi-frequency features were reconstructed using IDWT, concatenated back to the classification token, and the output was... , represented as:

[0097]

[0098] Concatenate the category tokens and output the results. This is represented as a combined residual join:

[0099]

[0100] The number of attention heads was experimentally optimized to minimize feature loss. The output error of the WaveAttention module is defined as:

[0101] .

[0102] S4. Input the optimized input features into the wavelet Kolmogorov-Arnold network module, optimize the features through nonlinear basis functions, and output the classification probability.

[0103] like Figure 5 The diagram shows the WaveKAN module architecture. Input features are fused with wavelet features through a KAN nonlinear transformation, and finally, the classification result is output through linear projection. The WaveKAN module optimizes classification performance and improves prediction accuracy through nonlinear refinement and feature fusion. Specifically, it processes the WaveAttention output... Application of KAN nonlinear transformation Generate features. With wavelet features Integration, to obtain The fusion ratio was optimized by testing different weight combinations. Classification results were generated using linear projection. To prevent overfitting, L2 regularization is applied to the KAN weights.

[0104] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0105] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0107] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

[0108] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A fine-grained image classification method based on deep wavelet feature fusion, characterized in that, Includes the following steps: S1. Construct a fine-grained visual classification network that integrates wavelet transform and visual transform. Input the image to be classified and perform preprocessing to obtain standardized image data. S2. Perform multi-frequency sub-band decomposition on the standardized image data using a wavelet convolution module to generate a feature map that fuses global structure and local details. This includes the following steps: S21. The input standardized image is decomposed into low-frequency sub-band and high-frequency sub-band using discrete wavelet transform; S22. Perform 7*7 large kernel convolution and attention channel mechanism on low frequency subband to enhance global features, and perform 3*3 small kernel convolution and convolution attention channel mechanism on high frequency subband to optimize detailed features; S23. The global and detailed features obtained in S22 are reconstructed using inverse discrete wavelet transform to obtain reconstructed features, which are then fused with the features from the 5*5 convolution path to obtain a feature map that integrates global structure and local details. The feature map that integrates global structure and local details is represented as follows: ； In the formula, To integrate feature maps that combine global structure with local details, For inverse wavelet transform, Enhanced global structural features For optimized local detail features, For Haar wavelet filters, Output a 5x5 convolution path; S3. Input the feature map obtained in S2 into the wavelet attention module, compress the dimension of the key value vector through wavelet downsampling, and extract multi-scale features by combining the multi-head self-attention mechanism to obtain the optimized input features; S4. Input the optimized input features into the wavelet Kolmogorov-Arnold network module, optimize the features through nonlinear basis functions, and output the classification probability.

2. The fine-grained image classification method based on deep wavelet feature fusion according to claim 1, characterized in that, S3 specifically includes the following steps: S31, Input the feature map obtained from S2 that fuses global structure and local details; S32. Downsample the keys and values ​​using wavelet discrete transform to generate wavelet features; S33. Perform multi-head attention, generate queries, keys, and values, calculate attention scores, and output weighted scores. S34. Reconstruct multi-frequency features from the weighted output of S33 using inverse wavelet discrete transform, and concatenate them into classification tokens to obtain optimized input features.

3. The fine-grained image classification method based on deep wavelet feature fusion according to claim 2, characterized in that, The optimized input features in S34 are represented as follows: ； In the formula, To incorporate the input features optimized by residual connections, For linear projection of the input features, It is the inverse discrete wavelet transform and: ； In the formula, For inverse discrete wavelet transform, For weighted output, For multi-head attention scores, This represents the value of the multi-head attention after downsampling. It is a Haar wavelet filter.

4. The fine-grained image classification method based on deep wavelet feature fusion according to claim 1, characterized in that, S4 specifically includes the following steps: S41. Apply the KAN nonlinear transformation to the optimized input features obtained in S3 and generate nonlinear features. ; S42. Fuse the nonlinear features with the wavelet features to obtain the fused features; S43. Combine and optimize the fusion features with different weights according to the fusion ratio, and generate classification results through linear projection.

5. The fine-grained image classification method based on deep wavelet feature fusion according to claim 4, characterized in that, The fusion feature in S42 is represented as follows: ； In the formula, As a feature of fusion, Wavelet features; The classification result in S43 is represented as follows: ； In the formula, For the classification results, For linear projection of fused features.

Images