Fine-grained image classification method based on deep wavelet feature fusion

Abstract

The invention discloses a fine-grained image classification method based on deep wavelet feature fusion, and the method comprises the steps: carrying out the data preprocessing of an input image, enabling the input image to enter a WTConv module, and decomposing the input image into multi-frequency sub-band features; inputting the features into a WaveAttention module, and extracting global and local information through a multi-scale attention mechanism; and finally, the WaveKAN module carries out nonlinear refining on the features and outputs a classification result. According to the method, the image is decomposed into multi-frequency sub-bands through wavelet transformation, the feature extraction capability of the model for global structures and local details is improved, the attention mechanism is enhanced, multi-scale and frequency domain information can be fused, and the fine-grained classification performance is improved. A nonlinear feature refining module is introduced, and a classification head is optimized to model a complex relation, so that the prediction precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of visual classification, and in particular to a fine-grained image classification method based on deep wavelet feature fusion. Background Art

[0002] Fine-grained visual classification (FGVC) is an important research direction in computer vision. Its goal is to identify objects with subtle inter-class differences but large intra-class variations, such as different species of fish, birds, or plants. Traditional methods rely on convolutional neural networks (CNNs), such as ResNet, Inception, and DenseNet, which have achieved success in image classification by extracting local features. However, as the complexity of the task increases, the limitations of CNNs in modeling global context and capturing multi-scale features become increasingly apparent.

[0003] In recent years, visual transformers (ViTs) have garnered widespread attention for their ability to model long-range dependencies via self-attention mechanisms. ViTs segment images into fixed-size patches and process them as labeled sequences, significantly improving image classification performance. Subsequently, knowledge distillation was used to improve data efficiency, introducing a hierarchical structure to generate multi-resolution feature maps suitable for dense prediction tasks. However, they still face challenges in fine-grained classification: they excel at global modeling but lack the ability to capture local details and multi-scale features, and lack explicit frequency domain modeling mechanisms.

[0004] The wavelet transform (WT) is a powerful signal processing tool that can decompose signals into multi-frequency and multi-scale representations. It excels in feature extraction. Learning CNN representations on wavelet subbands can improve image restoration performance. Integrating WT into U-Net optimizes downsampling and upsampling. However, the systematic integration of the wavelet transform with deep learning models (particularly transformers) has not been fully explored. Common technical solutions include:

[0005] Method 1: Image classification based on standard ViTs

[0006] ViT and its variants (such as DeiT and Swin Transformer) model global contextual information through a self-attention mechanism. For example, images are segmented into blocks and processed as sequences. Training strategies improve efficiency and a hierarchical structure is introduced to generate multi-scale features.

[0007] Method 2: CNN feature extraction based on wavelet transform

[0008] Combining wavelet transform with CNN, for example, using low-resolution wavelet subbands to restore image details and improving efficiency by decomposing features into multi-level subbands.

[0009] Method 3: Hybrid CNN and Transformer Model

[0010] These methods attempt to combine the advantages of CNN and ViT by combining large and small block features through a dual-branch transformer and introducing a multi-scale hierarchical structure.

[0011] Current image classification tasks face multi-source and multi-scale challenges: natural images often contain complex illumination variations (such as backlit or high dynamic range scenes), cross-scale object distribution (such as buildings ranging from meters to centimeters in remote sensing images), and the coupling of high-frequency noise (such as instrument artifacts in medical images) with low-frequency semantic information (such as the overall layout of the scene). This interweaving of multi-band features creates significant bottlenecks in traditional vision models in terms of feature decoupling and key information focus, as shown in the following:

[0012] (1) The ViTs method is good at modeling global context, but it fails to capture local details that are crucial in fine-grained classification tasks, resulting in poor performance in distinguishing subtle inter-class differences.

[0013] (2) It performs well in local feature extraction, but cannot effectively model long-distance dependencies, which limits its applicability in complex scenes.

[0014] (3) Existing methods lack an explicit frequency domain modeling mechanism and are unable to simultaneously capture the multi-scale features of global structure and local details. Summary of the Invention

[0015] In view of the above-mentioned deficiencies in the prior art, the present invention provides a fine-grained image classification method based on deep wavelet feature fusion.

[0016] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0017] A fine-grained image classification method based on deep wavelet feature fusion includes the following steps:

[0018] S1. Build a fine-grained visual classification network that integrates wavelet transform and visual transform, input the image to be classified and perform preprocessing to obtain standardized image data;

[0019] S2. performing multi-frequency sub-band decomposition on the standardized image data using a wavelet convolution module to generate a feature map that integrates global structure and local details;

[0020] S3, input the feature map obtained in S2 into the wavelet attention module, compress the key value vector dimension through wavelet downsampling, and extract multi-scale features in combination with the multi-head self-attention mechanism to obtain the optimized input features;

[0021] S4. Input the optimized input features into the wavelet Kolmogorov-Arnold network module, optimize the features through nonlinear basis functions and output the classification probability.

[0022] Furthermore, the S2 specifically includes the following steps:

[0023] S21, decomposing the input normalized image into low-frequency sub-bands and high-frequency sub-bands using discrete wavelet transform;

[0024] S22: Perform 7*7 large kernel convolution and attention channel mechanism on the low-frequency sub-band to enhance the global features, and perform 3*3 small kernel convolution and convolution attention channel mechanism on the high-frequency sub-band to optimize the detail features;

[0025] S23, reconstruct the global features and detail features obtained in S22 by inverse discrete wavelet transform to obtain reconstructed features, and fuse them with the features of the 5*5 convolution path to obtain a feature map that fuses the global structure and local details.

[0026] Furthermore, the feature map that fuses the global structure and local details in S23 is expressed as:

[0027]

[0028] Where, To fuse the feature maps of global structure and local details, is the inverse wavelet transform, Enhanced global structural features, is the optimized local detail feature, is the Haar wavelet filter, Output of the 5*5 convolution path.

[0029] Furthermore, the S3 specifically includes the following steps:

[0030] S31, input S2 to obtain the feature map of the fusion of global structure and local details;

[0031] S32, downsampling the keys and values ​​by wavelet discrete transform to generate wavelet features;

[0032] S33, perform multi-head attention, generate query, key, value, calculate attention score and weighted output;

[0033] S34. Reconstruct the multi-frequency features of the weighted output results of S33 through inverse wavelet discrete transform, and splice the classification tokens to obtain the optimized input features.

[0034] Furthermore, the input features after optimization in S34 are expressed as:

[0035]

[0036] Where, To combine the input features after residual connection optimization, is the linear projection of the input features, is the inverse discrete wavelet transform and:

[0037]

[0038] Where, is the inverse discrete wavelet transform, is the weighted output, is the multi-head attention score, is the value of multi-head attention after downsampling, is the Haar wavelet filter.

[0039] Furthermore, the S4 specifically includes the following steps:

[0040] S41. Apply KAN nonlinear transformation to the optimized input features obtained in S3 and generate nonlinear features;

[0041] S42, fusing the nonlinear feature with the wavelet feature to obtain a fused feature;

[0042] S43. Optimize the combination of different weights of the fusion features according to the fusion ratio, and generate the classification results through linear projection.

[0043] Furthermore, the fusion feature in S42 is expressed as:

[0044]

[0045] Where, To fusion features, Wavelet features;

[0046] The classification result in S43 is expressed as:

[0047]

[0048] Where, is the classification result, is the linear projection of the fused features.

[0049] The present invention has the following beneficial effects:

[0050] 1) Taking the bearing contact interface stiffness and friction as a bridge, the interaction between the tribology and dynamics of the control torque gyro is considered, and the prediction results will be more accurate.

[0051] 2) The influence of real machining roughness, elastohydrodynamic lubrication and tribothermal effect on bearing contact stiffness and friction coefficient is taken into account. Compared with the traditional Hertz contact stiffness and empirical friction coefficient, the constructed model is more reasonable and scientific.

[0052] 3) The established friction dynamics model calculates the contact stiffness and friction coefficient corresponding to the dynamic load through interpolation method, which can effectively reduce the calculation time of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of the process of the wavelet fusion fine-grained visual classification method of the present invention.

[0054] Figure 2 This is a flow chart of the overall framework of wavelet fusion according to an embodiment of the present invention.

[0055] Figure 3 Schematic diagram of the wavelet convolution module structure of an embodiment of the present invention.

[0056] Figure 4 Schematic diagram of the wavelet attention module structure according to an embodiment of the present invention.

[0057] Figure 5 Schematic diagram of the structure of the wavelet Kolmogorov-Arnold network module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0059] A fine-grained image classification method based on deep wavelet feature fusion, such as Figure 1 As shown, the following steps are included:

[0060] S1. Build a fine-grained visual classification network that integrates wavelet transform and visual transform, input the image to be classified and perform preprocessing to obtain standardized image data;

[0061] WaveFormer is a fine-grained visual classification (FGVC) framework that integrates wavelet transform (WT) and visual transformer (ViT). It achieves multi-scale and multi-frequency feature extraction and efficient classification through the synergy of three modules: WTConv (wavelet convolution module), WaveAttention (wavelet attention module), and WaveKAN (wavelet Kolmogorov-Arnold network module). The following is a detailed technical solution with a schematic diagram. Figure 2 shown.

[0062] This paper uses the Fish4Knowledge dataset as an experimental benchmark. This dataset contains high-resolution images (average resolution 2560×1920 pixels) of 23 marine species in complex underwater environments (such as different light intensities, turbidity and background interference). First, the pixel values ​​are normalized to the interval [0,1] to eliminate illumination bias. Secondly, a multi-scale data augmentation strategy is applied, including random horizontal flipping (with a probability of 50%), ±15° rotation and 224×224 pixel dynamic cropping, to enhance the model's robustness to perspective changes. Finally, the discrete wavelet transform (DWT) is used to decompose the image into four frequency subbands: LL / LH / HL / HH, and a multi-resolution feature pyramid is constructed, which provides a basis for subsequent frequency-domain and spatial-domain joint modeling.

[0063] S2. performing multi-frequency sub-band decomposition on the standardized image data using a wavelet convolution module to generate a feature map that integrates global structure and local details;

[0064] like Figure 3 The figure shows the WTConv module architecture. The input features are decomposed into low-frequency (LL) and high-frequency (LH, HL, HH) subbands using the DWT. The low-frequency subbands are processed by large-kernel convolution and the SE module, while the high-frequency subbands are processed by small-kernel convolution and the CBAM module. Finally, they are reconstructed using the IDWT and fused with the convolution path.

[0065] Figure 3 The innovative architecture design of the WTConv module is demonstrated in detail. The module decomposes the input image into multiple frequency subbands through wavelet transform, extracts global and local features, and optimizes feature fusion to improve representation capabilities. The specific process is as follows:

[0066] (1) Input and decomposition: Input feature map (B, C, H, W are batch size, number of channels, height, and width, respectively), decomposed into low-frequency subbands using DWT (implemented via 2D convolution F.conv2d with a stride of 2, halving the spatial resolution and quadrupling the number of channels) (global information) and high frequency sub-bands (Details), using Haar wavelet decomposition For subband:

[0067]

[0068] Where wavelet = 'haar': Haar wavelet filter (low pass: , Qualcomm: ), to achieve lossless decomposition

[0069] (2) Feature optimization: for low-frequency sub-band Apply 7x7 convolution and SE module to enhance global features, expressed as:

[0070] right application Convolution and SE modules:

[0071]

[0072] After splicing the high-frequency sub-bands, 3x3 convolution and CBAM modules are used to optimize the detail features, which can be expressed as:

[0073]

[0074] In order to smooth the high-frequency sub-band fluctuations, multi-level DWT preprocessing is used, which is similar to removing noise components by decomposition, making high-frequency details clearer and reducing noise interference.

[0075] (3) Fusion and output: Reconstruct features through IDWT (implemented through 2D transposed convolution F.conv_transpose2d, with a stride of 2 and doubled spatial resolution). and features with 5x5 convolutional paths Fusion, expressed as:

[0076] right application convolution:

[0077]

[0078] Output The fusion weights are optimized by testing different convolution kernel sizes (such as 3x3, 7x7, and 9x9) to ensure the balance of global and local features.

[0079] Reconstruct features and fuse them:

[0080]

[0081] Output:

[0082] Error analysis and optimization: The output error of the WTConv module is defined as:

[0083]

[0084] in, and are the module output and true features at time t respectively.

[0085] S3, input the feature map obtained in S2 into the wavelet attention module, compress the key value vector dimension through wavelet downsampling, and extract multi-scale features in combination with the multi-head self-attention mechanism to obtain the optimized input features;

[0086] like Figure 4 As shown in the figure, the WaveAttention module is integrated into the overall architecture, downsampling the keys and values ​​through DWT, extracting features in combination with the multi-head attention mechanism, and finally reconstructing the output through IDWT.

[0087] The WaveAttention module fuses multi-scale and frequency domain information through wavelet downsampling and multi-head attention mechanism to enhance global and local feature extraction. The specific process is as follows:

[0088] Input WTConv module output , downsample the key (K) and value (V) through DWT to generate wavelet features , reducing computational complexity and preserving frequency information.

[0089] Perform multi-head attention to generate query (Q), key (K), value (V), which can be expressed as:

[0090]

[0091] in, Representing the learnable projection matrix of query, key, and value, Dimensional features are mapped to dimension.

[0092] At the same time, the attention score is calculated and the weighted output is expressed as:

[0093]

[0094]

[0095] in, .

[0096] To improve the effect, different optimization algorithms (such as RMSProp, Adam, SGD with Momentum) were tested to adjust the attention weights. Adam usually performed the best. Multi-frequency features were reconstructed through IDWT, and the classification tokens were spliced ​​back and output. , expressed as:

[0097]

[0098] Splice back to the classification token and output , expressed as a combined residual connection:

[0099]

[0100] The number of attention heads is optimized through experiments to minimize feature loss. The output error of the WaveAttention module is defined as:

[0101] .

[0102] S4. Input the optimized input features into the wavelet Kolmogorov-Arnold network module, optimize the features through nonlinear basis functions and output the classification probability.

[0103] like Figure 5 The WaveKAN module architecture is shown in Figure 1. The input features are transformed by KAN nonlinearly, fused with wavelet features, and finally the classification results are output through linear projection. The WaveKAN module optimizes classification performance and improves prediction accuracy through nonlinear refinement and feature fusion. The specific process is: Apply KAN nonlinear transformation , generate features. and wavelet features Fusion, get The fusion ratio is optimized by testing different weight combinations. Classification results are generated by linear projection To prevent overfitting, L2 regularization is applied to the KAN weights.

[0104] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0107] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

[0108] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A fine-grained image classification method based on deep wavelet feature fusion, characterized in that: The steps include: S1. Build a fine-grained visual classification network that integrates wavelet transform and visual transform, input the image to be classified and perform preprocessing to obtain standardized image data; S2. performing multi-frequency sub-band decomposition on the standardized image data using a wavelet convolution module to generate a feature map that integrates global structure and local details; S3, input the feature map obtained in S2 into the wavelet attention module, compress the key value vector dimension through wavelet downsampling, and extract multi-scale features in combination with the multi-head self-attention mechanism to obtain the optimized input features; S4. Input the optimized input features into the wavelet Kolmogorov-Arnold network module, optimize the features through nonlinear basis functions and output the classification probability.

2. The fine-grained image classification method based on deep wavelet feature fusion according to claim 1 is characterized in that: The S2 specifically includes the following steps: S21, decomposing the input normalized image into low-frequency sub-bands and high-frequency sub-bands using discrete wavelet transform; S22: Perform 7*7 large kernel convolution and attention channel mechanism on the low-frequency sub-band to enhance the global features, and perform 3*3 small kernel convolution and convolution attention channel mechanism on the high-frequency sub-band to optimize the detail features; S23, reconstruct the global features and detail features obtained in S22 by inverse discrete wavelet transform to obtain reconstructed features, and fuse them with the features of the 5*5 convolution path to obtain a feature map that fuses the global structure and local details.

3. The fine-grained image classification method based on deep wavelet feature fusion according to claim 2 is characterized in that: The feature map of the fusion of global structure and local details in S23 is expressed as: ； Where, To fuse the feature maps of global structure and local details, is the inverse wavelet transform, Enhanced global structural features, is the optimized local detail feature, is the Haar wavelet filter, Output of the 5*5 convolution path.

4. The fine-grained image classification method based on deep wavelet feature fusion according to claim 1 is characterized in that: The S3 specifically includes the following steps: S31, input S2 to obtain the feature map of the fusion of global structure and local details; S32, downsampling the keys and values ​​by wavelet discrete transform to generate wavelet features; S33, perform multi-head attention, generate query, key, value, calculate attention score and weighted output; S34. Reconstruct the multi-frequency features of the weighted output results of S33 through inverse wavelet discrete transform, and splice the classification tokens to obtain the optimized input features.

5. The fine-grained image classification method based on deep wavelet feature fusion according to claim 4 is characterized in that: The optimized input features in S34 are expressed as: ； Where, To combine the input features after residual connection optimization, is the linear projection of the input features, is the inverse discrete wavelet transform and: ； Where, is the inverse discrete wavelet transform, is the weighted output, is the multi-head attention score, is the value of the multi-head attention after downsampling, is the Haar wavelet filter.

6. The fine-grained image classification method based on deep wavelet feature fusion according to claim 1 is characterized in that: The S4 specifically includes the following steps: S41. Apply KAN nonlinear transformation to the optimized input features obtained in S3 and generate nonlinear features; S42, fusing the nonlinear feature with the wavelet feature to obtain a fused feature; S43. Optimize the combination of different weights of the fusion features according to the fusion ratio, and generate the classification results through linear projection.

7. The fine-grained image classification method based on deep wavelet feature fusion according to claim 6 is characterized in that: The fusion feature in S42 is expressed as: ； Where, To fusion features, Wavelet features; The classification result in S43 is expressed as: ； Where, is the classification result, is the linear projection of the fused features.

Images