Image classification method and device based on wavelet convolution and adaptive gating fusion

Abstract

The invention discloses an image classification method and device based on wavelet convolution and adaptive gating fusion, and the method comprises the steps: inputting a preprocessed image into a preliminary feature extraction module, extracting a preliminary feature, inputting the preliminary feature into a dual-branch feature extraction module, and carrying out the global branch and local branch, so as to obtain a final feature; the global features are extracted by adopting wavelet convolution, the local features are extracted by adopting standard convolution in the local branches, and then the global features and the local features are subjected to adaptive gating fusion to obtain intermediate fusion features; and adjusting and optimizing the intermediate fusion features, performing residual connection with the initial features to obtain final fusion features, and finally inputting the final fusion features into a classification module to obtain a final classification result. According to the method, global image information can be captured by using a large receptive field to eliminate the influence of background interference and illumination variation, and tiny differences such as textures, edges and colors in a focus area can be captured by extracting local detail information.

Description

Technical Field

[0001] The present application belongs to the field of digital image processing technology, and in particular relates to an image classification method and device based on wavelet convolution and adaptive gating fusion. Background Art

[0002] Skin diseases currently pose a significant challenge in global public health, and their early diagnosis and accurate classification are crucial for clinical treatment and patient prognosis. With the rapid development of deep learning in computer vision in recent years, image analysis methods based on convolutional neural networks (CNNs) have been widely applied to the detection and classification of skin disease images. For example, AlexNet, VGG, ResNet, and their derivatives excel in feature extraction, utilizing stacked layers of convolutional layers to capture local texture, edge, and color information, thereby enabling automated identification of lesion areas. However, these models are primarily applied to images acquired under professional dermatoscopes, focusing on fine-grained features that reveal subtle differences within lesion areas while neglecting the importance of global information in real-world settings. In real-world scenarios, especially those captured under everyday conditions, skin disease images often feature complex backgrounds, uneven lighting, and variable resolution and camera angles. This results in unclear distinctions between lesions and surrounding normal skin, and the presence of confounding factors such as artifacts, hair, and shadows. These global factors can interfere with lesion localization and feature extraction, leading to misclassification by traditional CNN models when distinguishing malignant from benign lesions.

[0003] In this context, some studies have attempted to introduce multi-scale data augmentation strategies to reduce the interference caused by environmental factors in the judgment process, and enrich training data through methods such as image rotation, scaling, flipping, and local random discarding, thereby improving the model's adaptability to various environmental factors. However, these methods can only alleviate the problem at the data level and are difficult to fundamentally improve the limitations of feature extraction.

[0004] Because single feature extraction strategies struggle to distinguish subtly different lesion regions, some research has begun to incorporate multi-scale feature fusion and attention mechanisms to enhance the ability to extract global information. For example, the Inception family of models utilizes parallel branch designs to extract features at different scales, while ResNeXt and EfficientNet optimize network depth and width to expand the model's receptive field and capture global semantic information. Furthermore, attention modules such as SENet and CBAM are used to enhance the response of key regions, improving the representation of local features. However, these methods typically only fuse local and global information through simple concatenation at the end of the network, failing to fully address feature blurring caused by background interference. Furthermore, with the introduction of the Visual Transformer (ViT) architecture, the self-attention mechanism enables the model to capture global information across the entire image in a single operation. However, the Transformer model is often weak in detail representation and computationally complex, making it difficult to directly apply to high-resolution scenarios such as dermatological images. Furthermore, while the Transformer architecture possesses the ability to capture global information, its self-attention mechanism lacks specificity when processing local details. Some studies have attempted to combine CNNs with Transformers, either in parallel or serial fashion, to achieve complementary global and local features. However, these approaches often suffer from simple fusion methods and insufficient interaction between local and global information, resulting in limited classification performance in complex real-world scenarios. Furthermore, some researchers have attempted to mimic the Transformer's ability to capture global information by expanding the receptive field using dilated convolutions and stacked deep convolutions. For example, TCNN uses dilated convolutions to enhance the model's global context capture. RepVGG employs a structural reparameterization strategy, employing a multi-branch design during training to enrich feature representations and fusing these branches into a single convolutional layer during inference, achieving a larger receptive field while maintaining computational efficiency. RepLKNet directly expands the network's receptive field by convolving the input image with large convolutional kernels, thereby capturing a wider range of contextual information. However, these approaches require high network parameters and computational resources and can easily blur or partially lose local feature information. This presents a significant challenge for applications that require precise capture of local color and texture variations in skin lesions.

[0005] On the other hand, while some existing research based on attention mechanisms and cross-domain learning has improved the model's processing of local and global information to a certain extent, their designs often employ static fusion strategies, failing to implement dynamic weight adjustments, resulting in less than ideal fusion of local and global information. For example, the FPN architecture fuses feature maps from different levels by element-by-element addition; while BiFPN achieves multiple weighted fusions of features at different scales through the design of a bidirectional feature pyramid network. This static fusion approach, to a certain extent, limits the network's ability to comprehensively utilize local fine-grained features and global information in complex scenarios. This is particularly true in dermatology images, where background interference, lighting, and color fluctuations can lead to the omission of key information or improper weighting, thereby reducing classification accuracy and robustness.

[0006] In summary, current technology faces two core problems in the field of skin disease image classification: first, insufficient global information capture leads to significant influence of factors such as lighting and background interference in practical applications; second, the method of extracting local fine-grained features and fusing them with global features is not sophisticated enough, making it difficult to simultaneously take into account the extraction of large receptive fields and subtle local information. Summary of the Invention

[0007] The purpose of this application is to provide an image classification method and device based on wavelet convolution and adaptive gating fusion, in order to solve the above problems raised in the background technology and the shortcomings of the existing technology. An image classification method based on wavelet convolution and adaptive gating fusion, comprising: Preprocess the collected images; The preprocessed image is input into the preliminary feature extraction module to extract preliminary features; The preliminary features are input into the dual-branch feature extraction module, passing through the global branch and the local branch respectively. The global features are extracted using wavelet convolution, and the local branches are extracted using standard convolution. The global features and local features are then adaptively gated and fused to obtain intermediate fused features. The intermediate fused features are adjusted and optimized, and then residually connected with the preliminary features to obtain the final fused features. The final fusion features are input into the classification module to obtain the final classification results.

[0008] Furthermore, the preprocessing of the collected images includes: Enhance image contrast; Resize the image to a uniform size and normalize it; Perform noise reduction on the image.

[0009] Furthermore, the preliminary feature extraction module includes a convolution layer, a normalization layer and an activation function layer.

[0010] Furthermore, the adaptive gating fusion of the global features and the local features to obtain the intermediate fusion features includes: Perform global response normalization on global features and local features respectively; A learnable scalar parameter is introduced to dynamically weight the two features to generate intermediate fusion features.

[0011] Furthermore, the adjusting and optimizing of the intermediate fusion features includes: The intermediate fusion features are input into the convolution layer for channel mixing, and then nonlinearity is introduced through the activation function, and then the first feature is obtained through the second convolution layer; Input the first feature into the LayerScale layer to scale the channel-level features to obtain the second feature; The second feature is randomly discarded to obtain the adjusted and optimized feature.

[0012] The present application also proposes an image classification device based on wavelet convolution and adaptive gate fusion, which includes a processor and a memory storing a plurality of computer instructions. When the computer instructions are executed by the processor, the steps of the above method are implemented.

[0013] This application proposes an image classification method and device based on the fusion of wavelet convolution and adaptive gating, which can not only use a large receptive field to capture global image information to eliminate the influence of background interference and illumination changes, but also capture subtle differences in texture, edges, and colors in the lesion area by extracting local detail information. In the global branch, wavelet convolution is used to perform frequency domain decomposition to extract low-frequency global morphology and high-frequency detail information, thereby expanding the receptive field; in the local branch, a standard 3×3 convolution is used to focus on local feature extraction to ensure accurate capture of detail information; at the same time, a lightweight adaptive gating mechanism is used to achieve dynamic interaction and fusion of global and local features, effectively improving the classification accuracy and generalization ability of the model in complex environments. The introduction of this new method provides a new technical idea for solving the problems of background interference and insufficient fine-grained feature extraction in the current classification of skin disease images, and provides reliable technical support for actual clinical auxiliary diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flowchart of the image classification method based on wavelet convolution and adaptive gating fusion in this application.

[0015] Figure 2 This is a schematic diagram of the dual-branch feature extraction module structure of this application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0017] The core of this application is to propose an image classification method based on wavelet convolution and adaptive gated fusion. Through frequency-spatial domain collaborative feature extraction, adaptive fusion and hierarchical data enhancement strategy, it solves the problems of insufficient multi-scale feature extraction and significant background interference in skin disease images under complex environments, and significantly improves classification accuracy and generalization ability.

[0018] One embodiment of the present application, such as Figure 1 As shown in the figure, an image classification method based on wavelet convolution and adaptive gating fusion is proposed, including: Step S1: pre-process the collected image.

[0019] In this step, images are collected, which can be captured by mobile phone or professional equipment, such as collecting skin images, and then preprocessing them.

[0020] Specifically, preprocessing includes: Step 1.1.1. Enhance image contrast.

[0021] To enhance the image contrast, the histogram equalization method or the limited contrast adaptive histogram equalization (CLAHE) method can be used to avoid excessive enhancement of noise or details while improving the visual quality of the image. We will not go into details here.

[0022] Step 1.1.2: Resize the image to a uniform size and normalize it.

[0023] This step ensures that subsequent operations are performed at the same scale and standard. First, the image format is adjusted to match the network input format. A uniform resizing operation is then applied to the original image, and a normalization algorithm is used to map pixel values ​​to a standard range to eliminate the effects of differences in image acquisition equipment and environment.

[0024] Step 1.1.3: Perform noise reduction on the image.

[0025] Finally, the Gaussian filtering algorithm is used to pre-denoise the image to ensure that the detailed information in the image will not be distorted by noise interference.

[0026] It should be noted that since this application constructs an image classification network, it is necessary to train the constructed image classification network. Therefore, it is necessary to construct training samples. The training samples can use existing public data sets or generate training samples by preprocessing the collected images.

[0027] Training samples are generated by preprocessing the captured images, typically starting with preprocessing and labeling, followed by data augmentation. Data augmentation is performed on the preprocessed images to improve the model's robustness to image diversity and noise in complex scenes through a variety of data augmentation strategies, alleviating issues like insufficient data samples and uneven classification, and improving the model's generalization capabilities.

[0028] Data enhancement can usually be done in the following steps: Step 1.2.1. Use the OpenCV vision library to load the image file and use image scaling and cropping operations to adjust the image to the target size while ensuring that key details are preserved. Step 1.2.2: Generate multi-view image samples using rotation operations (limited to a rotation angle of ±15 degrees) and horizontal flipping, thereby effectively expanding the sample space. Step 1.2.3: Apply random adjustments to brightness and contrast to simulate image acquisition scenarios under different lighting conditions to ensure that the model has high adaptability to lighting changes. Step 1.2.4: Use a local random dropout strategy to randomly block out parts of the image to simulate background interference and local missingness, forcing the model to learn redundant and stable feature representations. Step 1.2.5: Convert the image after the above multiple data augmentation processing into a tensor format suitable for deep learning model input, so as to meet the shape requirements of subsequent network input.

[0029] Step S2: input the preprocessed image into the preliminary feature extraction module to extract preliminary features.

[0030] This step performs preliminary convolution operations and dimension conversion on the preprocessed image to provide adaptation data for subsequent dual-branch feature extraction.

[0031] Specifically, the preliminary feature extraction module uses a convolutional neural network, including a convolution layer (Conv2d), a normalization layer (Layer Normalization), and an activation function layer (GELU activation function) to extract preliminary features.

[0032] The convolution layer maps the image from the original input space to a high-dimensional feature space. After the convolution operation, the normalization method and GELU activation function are applied to ensure that the feature data maintains a stable numerical distribution for subsequent dual-branch feature extraction.

[0033] Step S3: Input the preliminary features into the dual-branch feature extraction module, pass through the global branch and the local branch respectively, the global features use wavelet convolution to extract global features, and the local branches use standard convolution to extract local features, then adaptively gate the global features and the local features to obtain intermediate fusion features, and adjust and optimize the intermediate fusion features and perform residual connection with the preliminary features to obtain the final fusion features.

[0034] The dual-branch feature extraction module in this embodiment is as follows: Figure 2 As shown in the figure, the preliminary features extracted by the preliminary feature extraction module are input into the dual-branch feature extraction module, and feature extraction is performed in each of the two branches. The global branch uses wavelet convolution (WTConv) to achieve frequency domain decomposition using a fixed wavelet transform. The global branch then extracts global morphological and detail features through a combination of 1×1 and 3×3 convolutional layers to obtain global features. The local branch uses standard 3×3 convolution to focus on capturing local edge and texture information in the image to obtain local features.

[0035] Specifically, in the global branch, wavelet convolution (WTConv) is used to perform frequency domain decomposition on the input feature map. Wavelet transforms are first used to decompose the input into low-frequency and high-frequency components. The low-frequency components primarily extract the overall morphological information characteristics of the image, while the high-frequency components primarily capture local texture and edge details. This approach enables the network to obtain a larger receptive field, thereby expanding the range of feature capture. Finally, the feature map processed by wavelet convolution is sequentially normalized using LayerNorm and transformed using the GELU activation function to generate a global representation with global semantic information, providing a high-quality global information representation for subsequent fusion.

[0036] In the local branch, a standard 3×3 convolution kernel is used to convolve the input features to extract local texture and edge details in the image. The feature map generated by the convolution is then transformed using LayerNorm normalization and GELU activation to enhance the expression of local features. This ensures the numerical stability of local information and provides detailed local features for fine-grained lesion classification in the subsequent fusion step.

[0037] Then, the global and local features are adaptively gated and fused to produce intermediate fused features. This embodiment achieves dynamic fusion through lightweight adaptive gated fusion. The goal is to simultaneously extract global semantic information and local detail features of the image and process them separately, achieving dynamic fusion of two-branch features and ultimately forming a comprehensive feature representation suitable for classification tasks.

[0038] In a specific embodiment, the adaptive gating fusion includes: Step 3.1.1: Perform global response normalization on global features and local features respectively.

[0039] This step performs global response normalization on global features and local features respectively to eliminate the difference in numerical amplitude between different branches and ensure that the two are compared on the same scale. The global response normalization process is shown in Formula 1: (1) Among them F g and F l where ϵ represents the feature inputs of global and local features, respectively, and ϵ is a numerical stability constant. The final output is the normalized features. Normalization distributes the feature contributions of different branches proportionally, preventing a branch from dominating the fusion process due to excessive activation strength.

[0040] Step 3.1.2: Introduce a learnable scalar parameter to dynamically weight the two features to generate an intermediate fusion feature.

[0041] This step realizes the effective interaction and fusion of global information and local detail information. The fusion process is shown in Formula 2: (2) and For the normalized features, the scalar parameter α can be learned to achieve dynamic weights. α is continuously adjusted through back propagation during the training process to dynamically balance the contribution ratio of global features and local features.

[0042] Then adjust and optimize the intermediate fusion features, including: Step 3.2.1: Input the intermediate fusion features into the convolution layer for channel mixing, then introduce nonlinearity through the activation function, and then pass it through the second convolution layer to obtain the first feature.

[0043] The intermediate fusion features are input into a 1×1 convolutional layer for channel mixing, and then nonlinearity is introduced through the GELU activation function. Then, the features are passed through a second 1×1 convolutional layer to enhance the feature expression ability and refine the feature distribution to obtain the first feature.

[0044] Step 3.2.2: Input the first feature into the LayerScale layer, scale the channel-level features, and obtain the second feature.

[0045] This embodiment introduces the LayerScale layer, which multiplies each channel by a learnable scaling factor γ to scale the channel-level features and gradually learn the importance of each channel. This achieves a process from smooth transition to capability enhancement, helps stabilize deep network training, improves the robustness of feature expression, and obtains the second feature.

[0046] Step 3.2.3: Perform random discarding of paths on the second feature to obtain the adjusted and optimized feature.

[0047] This step randomly discards paths to achieve regularization and improve the generalization performance of the model.

[0048] Finally, the adjusted and optimized features are residually connected with the preliminary features to generate the final fused features.

[0049] Step S4: input the final fusion features into the classification module to obtain the final classification results.

[0050] The classification module of this embodiment includes a fully connected layer, a normalization layer, a random dropout layer, and an activation function layer. The final fusion feature is input into the fully connected layer. The output dimension is determined according to the requirements of the classification task. A normalization layer and a random dropout layer are added before the fully connected layer to reduce the risk of overfitting and improve the robustness of the model. Then, the output of the fully connected layer is mapped to a specific classification probability through the activation function to obtain the classification result.

[0051] In the current research on skin disease image classification, in actual application scenarios, there is not only the problem of extracting fine-grained features of subtle differences within the lesion area, but also the problem of significant changes in global information such as color difference and contrast between the variable background and the lesion area. These global factors will interfere with the lesion positioning and feature extraction in the image, causing the model to misjudge when distinguishing between malignant and benign lesions. Effective skin disease image classification requires the model to pay attention to both the global information of the overall image and the local fine-grained features within the lesion area. However, the simple superposition or splicing of local information and global information in the existing methods cannot achieve sufficient interaction and fusion between the two, resulting in the model being unstable and prone to misjudgment when facing complex backgrounds, cross-skin color and other changing scenes. The technical solution of this application adopts wavelet convolution technology through global branches, and uses frequency domain decomposition to simultaneously extract low-frequency global morphology and high-frequency detail features, while using standard convolution through local branches to focus on local texture information. In addition, in order to better realize the dynamic interaction and fusion of global and local features, a lightweight adaptive gated fusion is introduced to improve the accuracy of distinguishing fine-grained lesions and significantly improve the classification robustness in complex environments.

[0052] In order to verify the effectiveness of the method proposed in this application, based on the DDI (Diverse Dermatology Images) dataset released by the Stanford University Clinical Center, the classification performance of the commonly used feature extraction network and the technical solution of this application (WT-ConvNeXt) on complex background and multi-skin color samples was compared to verify the applicability and robustness of the structure of this application for real-world skin lesion detection.

[0053] The comparison networks include InceptionV3, DenseNet121, ShuffleNetV2, SEResNeXt, EfficientNetV2, ConvNeXt, and the network of this application. All models are evaluated for Accuracy and overall 50% AUC under the same hardware and hyperparameter conditions to ensure the repeatability and comparability of the experiments. The experimental results are shown in Table 1: Table 1

[0054] Compared with other networks, although the accuracy of the network of the present application has only been slightly improved, the AUC value has been significantly improved compared with other models. This shows that when facing tasks such as the DDI dataset with complex backgrounds and multi-dimensional imbalances, the network of the present invention not only shows better generalization ability in cross-skin color detection and subtle lesion recognition, but also to a certain extent reduces the situation where dark-skinned benign lesions are misclassified as malignant, reflecting its stronger robustness in skin disease image detection.

[0055] This application achieves collaborative feature extraction in the frequency and spatial domains, and has more accurate characterization capabilities for lesion areas in skin disease images under complex backgrounds and changing lighting conditions. It fills the gaps in existing skin disease image classification in real-world environments and the dynamic weight allocation of global and local features for skin diseases. Unlike fine-grained classification methods that rely solely on spatial domain convolution to extract lesion details, and global information extraction schemes that only expand the receptive field through dilated convolution or large convolution kernels, this application simultaneously introduces frequency domain wavelet convolution and standard 3×3 spatial convolution branches, and dynamically balances global and local feature weights through a lightweight adaptive gating mechanism, solving the problem of either losing details or being insufficient to eliminate background interference in existing static multi-scale fusion.

[0056] In another embodiment, the present application also provides an image classification device based on wavelet convolution and adaptive gated fusion, comprising a processor and a memory storing a plurality of computer instructions, wherein the computer instructions implement the steps of the above method when executed by the processor.

[0057] Regarding the specific definition of the image classification device based on wavelet convolution and adaptive gate fusion, please refer to the definition of the image classification method based on wavelet convolution and adaptive gate fusion above, which will not be repeated here. The above-mentioned image classification device based on wavelet convolution and adaptive gate fusion can be implemented in whole or in part by software, hardware and their combination. It can be embedded in or independent of the processor in the computer device in hardware form, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0058] The memory and processor are electrically connected, directly or indirectly, to enable data transmission or interaction. For example, these elements may be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program executable on the processor. The processor executes the computer program stored in the memory to implement the image classification method based on wavelet convolution and adaptive gated fusion in an embodiment of the present invention.

[0059] The memory may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory is used to store a program, and the processor executes the program after receiving an execution instruction.

[0060] The processor may be an integrated circuit chip with data processing capabilities. The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor.

[0061] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. An image classification method based on wavelet convolution and adaptive gating fusion, characterized in that: The image classification method based on wavelet convolution and adaptive gating fusion includes: Preprocess the collected images; The preprocessed image is input into the preliminary feature extraction module to extract preliminary features; The preliminary features are input into the dual-branch feature extraction module, passing through the global branch and the local branch respectively. The global features are extracted using wavelet convolution, and the local branches are extracted using standard convolution. The global features and local features are then adaptively gated and fused to obtain intermediate fused features. The intermediate fused features are adjusted and optimized, and then residually connected with the preliminary features to obtain the final fused features. The final fusion features are input into the classification module to obtain the final classification results.

2. The image classification method based on wavelet convolution and adaptive gating fusion according to claim 1 is characterized in that: The preprocessing of the collected images includes: Enhance image contrast; Resize the image to a uniform size and normalize it; Perform noise reduction on the image.

3. The image classification method based on wavelet convolution and adaptive gating fusion according to claim 1 is characterized in that: The preliminary feature extraction module includes a convolution layer, a normalization layer and an activation function layer.

4. The image classification method based on wavelet convolution and adaptive gating fusion according to claim 1, characterized in that: The adaptive gating fusion of global features and local features to obtain intermediate fusion features includes: Perform global response normalization on global features and local features respectively; A learnable scalar parameter is introduced to dynamically weight the two features to generate intermediate fusion features.

5. The image classification method based on wavelet convolution and adaptive gating fusion according to claim 1 is characterized in that: The adjusting and optimizing of the intermediate fusion features includes: The intermediate fusion features are input into the convolution layer for channel mixing, and then nonlinearity is introduced through the activation function, and then the first feature is obtained through the second convolution layer; Input the first feature into the LayerScale layer to scale the channel-level features to obtain the second feature; The second feature is randomly discarded to obtain the adjusted and optimized feature.

6. An image classification device based on wavelet convolution and adaptive gating fusion, comprising a processor and a memory storing a plurality of computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images