An Automated Classification System and Method for Common Non-Melanocytic Tumors Based on CNN
By using a CNN-based automatic classification system, which generates training data using a diffusion model and a multi-scale global attention enhancement module, the problems of data imbalance and insufficient fine-grained feature extraction in skin cancer diagnosis are solved. This achieves high-accuracy classification of skin lesions and visual assisted diagnosis, making it suitable for primary healthcare institutions.
Patent Information
- Application Number
- CN202611245394.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-17
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies for skin cancer diagnosis suffer from problems such as data class imbalance, insufficient fine-grained feature extraction, severe image interference, and weak model interpretability, resulting in low accuracy in skin lesion identification and difficulty in achieving non-invasive and accurate classification.
An automatic classification system based on CNN is adopted. Training data is generated by freezing the pre-trained latent diffusion model and LoRA adapter. Combined with a multi-scale global attention enhancement module and Grad-CAM visualization, image preprocessing and multi-stage optimization training are performed to construct the EfficientNetV2-S classification network, realizing fine-grained feature extraction and visualization-assisted diagnosis.
It significantly improves the recognition rate of small sample categories and the overall classification accuracy, enhances the model's ability to capture subtle lesion features, and improves the transparency and interpretability of diagnosis through visualization. It is easy to deploy and operate in primary healthcare institutions.
Smart Images

Figure CN123118486A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of medical image processing and computer-aided diagnosis technology, and in particular to an automatic classification system and method for common non-melanocyte tumors based on CNN. Background Technology
[0002] As the largest organ in the human body, the skin is constantly exposed to the external environment and is susceptible to various factors such as ultraviolet radiation, aging, and environmental stimuli, which can induce a variety of skin diseases. In recent decades, the global incidence of skin cancer has continued to rise, becoming one of the most serious malignant tumors threatening human health. The World Health Organization (WHO) clearly states that about one-third of all cancer cases are skin cancer, and its disease burden is increasing significantly year by year. In the United States alone, millions of new skin cancer patients are diagnosed each year. Basal cell carcinoma (BCC) and squamous cell carcinoma (SCC) are the most common types of non-melanoma skin cancer. Actinic keratosis (AK), as a typical precancerous lesion, is highly likely to develop into squamous SCC if not intervened in time. Benign keratosis (BKL) is a general term used in dermatology to describe a group of common benign epidermal proliferative lesions, mainly covering seborrheic keratosis (SK), lichen planus keratosis (LPLK), and solar lentigines (SL). It is particularly suitable as a diagnostic label when the clinical or dermoscopic features of the three overlap and it is difficult to accurately classify them. In clinical practice, benign keratosis can sometimes be very similar in appearance to malignant lesions, making them easily confused.
[0003] In traditional diagnostic models, the identification of skin lesions mainly relies on visual observation by doctors, dermoscopy, and pathological biopsy. Dermoscopy diagnosis is highly dependent on physician experience; the accuracy rate can be as low as 58.7% for inexperienced physicians. While pathological biopsy is the gold standard, it is highly invasive, cumbersome, and time-consuming. Given the current uneven distribution of medical resources and the limited screening capacity of primary healthcare institutions, manual diagnosis is easily affected by subjective judgment, image quality, and lesion similarity, resulting in an overall accuracy rate of only 75% to 84%, with significant problems of missed diagnoses, misdiagnoses, and overtreatment. Therefore, achieving early, non-invasive, and accurate classification of four key lesions—actinized keratosis, basal cell carcinoma, benign keratosis, and squamous cell carcinoma—has become an urgent need to improve the level of skin cancer diagnosis and treatment.
[0004] In recent years, deep learning technology, especially convolutional neural networks (CNNs), has demonstrated great potential in medical image analysis. Existing technologies, such as ResNet and VGG, have achieved breakthroughs in classification accuracy, but suffer from insufficient fine-grained feature extraction capabilities. Lightweight models like MobileNet, while computationally efficient, experience significant accuracy loss when processing complex medical images. Although the EfficientNet series has advantages in feature learning, its direct application in this field still faces the following major challenges: (1) Imbalanced data categories: In public datasets, the number of samples of key lesions such as AK and SCC is far less than that of other categories such as BCC and BKL, which leads to bias in model training, low recognition rate of small sample categories, and seriously affects the clinical applicability of the model. (2) Insufficient fine-grained feature extraction: The four types of skin lesions differ very slightly in color, texture and boundary, which is a typical fine-grained classification task. Existing models have difficulty effectively capturing these key discriminative features, which leads to similar lesions (such as AK and SCC) being easily misclassified; (3) Severe image interference: Clinical dermoscopy images are generally affected by interference such as hair on the body surface, random noise, and punctate spots, which directly obscure the effective features of the lesion area and further increase the difficulty of identification; (4) The model has weak interpretability and low system integration: As an auxiliary diagnostic tool, the existing system often only outputs classification results and lacks visualization of the diagnostic basis; at the same time, there is a lack of a clinical auxiliary system that integrates data augmentation, high-precision classification and result visualization functions in an easy-to-use manner. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention provides an automatic classification system and method for common non-melanocyte tumors based on CNN, applicable to the automatic identification and classification of four common non-melanocyte tumors: basal cell carcinoma, squamous cell carcinoma, actinic keratosis, and benign keratosis. It also aims to solve the following technical problems: (1) How to effectively expand the training data of small sample categories (AK and SCC) to alleviate the problem of class imbalance in the dataset and improve the model's ability to identify such high-risk lesions; (2) How to enhance the model’s ability to extract subtle color, texture and morphological differences in skin lesion images, thereby improving the fine-grained discrimination accuracy of four similar lesions (especially AK and SCC); (3) How to effectively remove interference such as hair and noise in dermoscopy images, improve the image input quality, and enable the model to focus on the core features of the lesion; (4) How to improve the reliability of model classification results and enhance the transparency and interpretability of its decision-making process through visualization, while integrating the above technologies into an end-to-end system that is easy to operate and can be deployed independently.
[0006] Specifically, the present invention is implemented using the following technical solutions: Firstly, an automated classification system for common non-melanocyte tumors based on CNN, comprising: Sample generation module: Freeze the VAE encoder, CLIP text encoder, and original U-Net weights of the pre-trained latent diffusion model. Add and train the LoRA adapter to the attention layer of U-Net to learn the typical appearance, characteristic color texture, local edge contours, and subtle structural differences of real lesions. Introduce the latent diffusion model and combine it with low-rank adaptive technology to generate and augment images for the AK and SCC categories with scarce sample sizes. Combine the generated synthetic images with the original images according to a preset ratio to form an enhanced training set. Image preprocessing module: Configures a combined preprocessing algorithm to perform scale normalization, spatial noise filtering, intelligent hair interference removal and standardization on the input dermoscopy image in sequence, and outputs a preprocessed image in standard tensor format; The classification module is constructed as an improved EfficientNetV2-S classification network as the subject classification and recognition model. The subject classification and recognition model includes a feature extraction backbone network, a multi-scale global attention enhancement module, and a classification head. The multi-scale global attention enhancement module includes four parallel convolutional branches for extracting multi-scale features of different receptive fields. The kernel sizes of the four convolutional branches are 1×1, 3×3, 5×5, and 7×7, respectively. The feature maps output by each convolutional branch are concatenated along the channel dimension and then recalibrated through a channel attention mechanism. Optimize the training module: Configure a three-stage progressive fine-tuning training strategy to train the subject classification and recognition model, while superimposing multiple optimization strategies; The auxiliary diagnostic module adopts a B / S architecture, receives dermoscopic images uploaded by users, and calls the image preprocessing module and the recognition and classification module respectively to perform automatic classification and recognition. It combines the Grad-CAM algorithm to generate a heat map of the lesion area and outputs the classification results and visual diagnostic basis.
[0007] Specifically, in the sample generation module, the latent diffusion model adopts a diffusion model based on Stable Diffusion 1.5, and the ratio of the number of original data to the number of synthesized images is 7:3. The generation of the synthesized images includes: using text prompts as conditional input, extracting semantic features through CLIP text encoder, combining random latent variables, time step information and conditional U-Net for stepwise denoising, and finally generating skin disease images with clear category semantic features through VAE decoder; and using three objective evaluation indicators, FID, KID and IS, to evaluate the quality of the generated images.
[0008] Specifically, the enhanced training set obtained in the sample generation module is constructed based on the ISIC2019 dataset, consisting of 7442 dermoscopy images, which are divided into a training set of 5207 images, a validation set of 1486 images, and a test set of 749 images in a 7:2:1 ratio; among them, 259 synthetic images are generated for the AK category and 188 synthetic images are generated for the SCC category, for a total of 5654 images in the enhanced training set; and cross-dataset validation is performed using the MILK10K external validation set.
[0009] Specifically, the combined preprocessing algorithm of the image preprocessing module includes: Scale normalization was performed, scaling all dermoscopy images to a uniform 384×384 pixels to eliminate resolution differences between different acquisition devices. Spatial noise is filtered out, and a Gaussian filtering algorithm is used to smooth and reduce noise in the dermoscopy image, suppressing shooting noise and punctate spots, while protecting the edge contour of the lesion; wherein, the convolution kernel size of the Gaussian filtering algorithm is 5×5, and the standard deviation σ=1.0; Intelligent removal of hair interference is achieved by obtaining a single-channel image through standard grayscale transformation. Morphological closing operations are used to connect broken hair regions. The kernel size of the morphological closing operation is 9×9. A precise hair mask is generated through grayscale difference operations, and a grayscale threshold T=30 is set to distinguish hair from normal skin or lesion areas. The Telea image restoration algorithm is used to fill and repair the areas occluded by hair layer by layer based on the known pixel information of the neighborhood of the mask region through a fast walking method. Standardization processing: The RGB image is converted to the standard tensor format of the deep learning framework. Standardization is then performed based on the distribution of the subject classification model in the aforementioned recognition module, aligning it with the normalization rules of the backbone network of the subject classification model. The normalization formula is expressed as: ; in, , .
[0010] Specifically, in the recognition and classification module, the feature extraction backbone network adopts the EfficientNetV2-S classification network pre-trained on ImageNet, while removing the average pooling layer and fully connected classification layer at its end, retaining the complete convolutional feature extraction part; the EfficientNetV2-S classification network also introduces progressive learning and mixed precision training to achieve an optimal balance between network depth, width and input resolution; The multi-scale global attention enhancement module is connected to the end of the feature extraction backbone network. The stride of the four parallel convolutional branches is 1, and the padding is 0, 1, 2, and 3 respectively. The classification head comprises a globally average pooling layer, a flattening layer, a batch normalization layer, a dropout layer, a first fully connected layer, a ReLU activation function, a second fully connected layer, a ReLU activation function, and a third fully connected layer, which are sequentially connected to output the probability distribution of four types of lesions. The dropout rate of the dropout layer is set to 0.5. The dimensions of the first fully connected layer are 1280×1024, the dimensions of the second fully connected layer are 1024×512, and the dimensions of the third fully connected layer are 512×4.
[0011] Specifically, in the multi-scale global attention enhancement module, the channel attention mechanism generates channel weights through global average pooling and two fully connected layers with ReLU activation functions in the intermediate layer, and multiplies them element-wise with the concatenated multi-scale feature map to achieve feature recalibration, so that the model focuses on key lesion areas and suppresses background noise and redundant information; wherein, the channel compression ratio of the channel attention mechanism is set to 16.
[0012] Specifically, the three-stage progressive fine-tuning training strategy in the optimized training module includes: In the first stage, all parameters of the feature extraction backbone network are frozen, and only the multi-scale global attention enhancement module and the classification head are trained to enable the new module to quickly adapt to the skin lesion classification task. In the second stage, the parameters of the last 50% of the layers corresponding to the high-level semantic feature extraction part of the backbone network are unfrozen, and the corresponding unfrozen layers, multi-scale global attention enhancement modules, and classification heads are jointly trained.
[0013] The third stage involves unfreezing all network parameters for training. The first phase of training consists of 90 epochs with a learning rate of 1×10⁻⁶. -3 The second phase of training consists of 100 epochs, with a learning rate set to 1×10⁻⁶. -4 The third phase of training consists of 100 epochs, with a learning rate set to 5 x 10^6. -6 An early stopping strategy is used in each training phase to verify that the loss does not decrease after 15 rounds, and the model weights with the lowest verification loss are saved.
[0014] Specifically, the multiple optimization strategies superimposed in the optimization training module include: Class Balanced Sampling: A class balanced sampling strategy is adopted when loading training data. The original number of samples of the four classes in the training set is counted and a sampling target is set. By calculating the sampling weight of each sample, the WeightedRandomSampler data sampler is used for sampling with replacement. The sampling target is 2000 samples of class AK, maintaining the original number of class BCC, 2000 samples of class BKL, and 2000 samples of class SCC, to ensure that the smaller sample classes have more learning opportunities in each round of training. Online data augmentation: Randomly flips horizontally, randomly flips vertically, randomly rotates, color-perturbs, and randomly erases the input image; Mixed loss function: Focal Loss is used as the basic loss function with parameter γ=2.0, combined with label smoothing factor ε=0.1; at the same time, MixUp algorithm is introduced to perform linear mixing of samples with parameter α=0.15, which enhances the model's generalization ability and alleviates the problems of model overconfidence and overfitting. Optimizer configuration: Adam optimizer is used in all stages, with a weight decay of 1×10. -5 It also uses torch.cuda.amp for automatic mixed-precision training to reduce GPU memory usage.
[0015] Specifically, the auxiliary diagnostic module is configured as follows: The system receives dermoscopy images uploaded by users through the front-end interactive interface and automatically calls the image preprocessing module to complete scale normalization, Gaussian noise reduction, intelligent hair removal and standardization processing. The backend calls the recognition and classification module to perform inference, obtains the four-class prediction results and prediction confidence, and uses the Grad-CAM algorithm to backpropagate the gradient information corresponding to the final classification result to the feature map output by the multi-scale global attention enhancement module, generating a class activation heatmap and superimposing it back onto the original image. The front-end interface synchronously displays disease categories, model prediction confidence, Grad-CAM heatmaps, and standardized disease analysis and diagnostic suggestions retrieved from a preset database, and supports one-click saving of diagnostic records and reports; the preset diagnostic database adopts a hard-coded key-value pair structured storage model, with disease category labels as the primary key, to establish a mapping relationship between clinical manifestation descriptions and treatment suggestions; The front-end uses HTML and JavaScript to build the interactive interface, while the back-end inference service is based on the Flask framework.
[0016] On the other hand, a CNN-based automatic classification method for common non-melanocyte tumors includes the following steps: Step S1: Obtain the original dermoscopy image dataset, freeze the VAE encoder, CLIP text encoder, and original U-Net weights of the pre-trained latent diffusion model, add and train the LoRA adapter to the attention layer of the U-Net to learn the typical appearance, characteristic color texture, local edge contours, and subtle structural differences of real lesions; based on the latent diffusion model and combined with low-rank adaptive technology, perform image generation and augmentation on the AK and SCC categories with sparse sample sizes, combine the generated synthetic images with the original images according to a preset ratio to construct an enhanced training set; Step S2: Perform scale normalization, spatial noise filtering, intelligent hair interference removal, and standardization processing sequentially on the images in the enhanced training set, and output a preprocessed image in standard tensor format; wherein, the intelligent hair interference removal specifically includes: obtaining a single-channel image through standard grayscale transformation, connecting the broken hair regions using morphological closing operations with a kernel size of 9×9; generating an accurate hair mask through grayscale difference operations, setting a grayscale threshold T=30; and using the Telea image restoration algorithm to restore the areas occluded by hair through a fast walking method; Step S3: Construct an improved EfficientNetV2-S classification network as the main classification and recognition model. A three-stage progressive fine-tuning strategy is used to train the classification network. During training, class-balanced sampling, online data augmentation, and a hybrid loss function are superimposed for optimization. The Adam optimizer is used in all stages, and automatic mixed precision training is employed to accelerate the training process. The subject classification and recognition model includes a feature extraction backbone network, a multi-scale global attention enhancement module, and a classification head. The multi-scale global attention enhancement module includes four parallel convolutional branches with kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively. The feature maps output by each convolutional branch are concatenated along the channel dimension and then recalibrated through a channel attention mechanism. In the three-stage progressive fine-tuning strategy, the first stage freezes all parameters of the feature extraction backbone network and trains only the multi-scale global attention enhancement module and the classification head; the second stage unfreezes the parameters of the last 50% of the backbone network and trains them together; and the third stage unfreezes all network parameters and trains them together. The multiple optimization strategies include: using WeightedRandomSampler for class balancing sampling, with a sampling target of 2000 samples for class AK, maintaining the original number for class BCC, 2000 samples for class BKL, and 2000 samples for class SCC; using Focal Loss as the base loss function with parameter γ=2.0, combined with a label smoothing factor ε=0.1; introducing the MixUp algorithm for linear mixing of samples with parameter α=0.15; using the Adam optimizer in all stages and employing automatic mixed precision training; and using an early stopping strategy in each training stage, terminating if the validation loss does not decrease after 15 rounds, and saving the model weights with the lowest validation loss. Step S4: The trained classification network is encapsulated into an auxiliary diagnostic system. It receives dermoscopy images uploaded by users, performs preprocessing steps, calls the classification network for inference, outputs four-class prediction results and confidence scores, and uses the Grad-CAM algorithm to generate a heat map of the lesion area for visualization. It simultaneously displays the disease category, model prediction confidence score, Grad-CAM heat map, and standardized disease analysis and diagnostic suggestions retrieved from the preset database. It also supports one-click saving of diagnostic records and reports. The auxiliary diagnostic system adopts a B / S architecture, with the front-end using HTML+JavaScript to build the interactive interface and the back-end inference service based on the Flask framework.
[0017] The beneficial effects of this invention are as follows: (1) Effectively alleviates the problem of data imbalance: By using the diffusion model sample generation technology based on Stable Diffusion 1.5 combined with LoRA, under the optimal ratio of 7:3 between the original data and the generated data, the recall rate of the AK category increased from 70.45% to 82.95%, an increase of 12.50%; the precision of the SCC category increased significantly from 76.56% to 85.25%, an increase of 8.69%; and the overall classification accuracy increased from 88.79% to 92.79%, an increase of 4.00%. At the same time, the quality of the generated images was verified by objective evaluation indicators (FID: AK is 56.37, SCC is 64.12; KID: AK is 0.0132, SCC is 0.0157; IS: AK is 4.28, SCC is 4.08), proving that the generated images are highly consistent with the real images in terms of feature distribution, and do not introduce obvious noise or cause overfitting.
[0018] (2) Significantly improves fine-grained classification accuracy: By designing a multi-scale global attention enhancement module (GMAB), combined with parallel convolutional branches of four different receptive fields (1×1, 3×3, 5×5, 7×7) and a global channel attention mechanism, the model's ability to capture subtle features of lesions is enhanced. Comparative experiments on the same augmented dataset show that MSAE-Net achieves an overall accuracy of 92.79%, a recall of 92.79%, and an F1 score of 92.75%, which are 2.27% higher than ResNet50 (accuracy 90.52%) and 4.27% higher than MobileNetV2 (accuracy 88.52%), respectively, demonstrating higher and more stable overall classification accuracy.
[0019] (3) Image preprocessing significantly improves input quality: By combining morphological operations, gray-level difference, edge detection and Telea image inpainting preprocessing algorithm, hair occlusion and random noise interference in dermoscopy images are effectively removed. Comparative experiments show that the overall accuracy of the model increases from 91.32% to 92.79% after the introduction of preprocessing, which verifies the necessity of the preprocessing module in purifying images and improving model performance.
[0020] (4) Enhanced model reliability and interpretability, with high practicality and ease of use: It integrates Grad-CAM visualization function, which can intuitively display the lesion areas that the model focuses on when making decisions, providing doctors with objective and visual auxiliary reference. This invention encapsulates the entire process of data augmentation, image preprocessing, high-precision classification, result visualization and report generation into a simple auxiliary diagnostic system. It adopts a B / S architecture. In actual use, doctors only need to upload images through a browser to obtain diagnostic references. The operation is simple, and the model has low dependence on high-performance graphics cards, making it easy to deploy and promote in standard medical computer environments and primary medical institutions. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.
[0022] Figure 1 This is a schematic diagram of the diffusion model architecture in an embodiment of the present invention; Figure 2 This is a schematic diagram of the subject classification and recognition model architecture in an embodiment of the present invention; Figure 3 This is a diagram of the attention structure of the multi-scale global attention enhancement module in an embodiment of the present invention. Figure 4 This is a schematic diagram illustrating the construction of the classification system in an embodiment of the present invention; Figure 5 This is a schematic diagram of the initial user interface of the system in an embodiment of the present invention; Figure 6 This is an example diagram of the system user interface in an embodiment of the present invention. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0024] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0025] The following is in conjunction with the appendix Figures 1-6 The following describes some embodiments of the present invention in detail. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0026] This invention proposes an automatic classification system and method for common non-melanocyte tumors based on CNN, with the core being the MSAE-Net deep learning framework. In a preferred embodiment, the system modules are composed as follows: Sample generation module: Freeze the VAE encoder, CLIP text encoder, and original U-Net weights of the pre-trained latent diffusion model. Add and train the LoRA adapter to the attention layer of U-Net to learn the typical appearance, characteristic color texture, local edge contours, and subtle structural differences of real lesions. Introduce the latent diffusion model and combine it with low-rank adaptive technology to generate and augment images for the AK and SCC categories with scarce sample sizes. Combine the generated synthetic images with the original images according to a preset ratio to form an enhanced training set. Image preprocessing module: Configures a combined preprocessing algorithm to perform scale normalization, spatial noise filtering, intelligent hair interference removal and standardization on the input dermoscopy image in sequence, and outputs a preprocessed image in standard tensor format; The classification module is an improved EfficientNetV2-S classification network, which is used as the subject classification and recognition model. The subject classification and recognition model includes a feature extraction backbone network, a multi-scale global attention enhancement module, and a classification head. The multi-scale global attention enhancement module includes four parallel convolutional branches for extracting multi-scale features of different receptive fields. The kernel sizes of the four convolutional branches are 1×1, 3×3, 5×5, and 7×7, respectively. The feature maps output by each convolutional branch are concatenated along the channel dimension and then recalibrated through a channel attention mechanism. Optimize the training module: Configure a three-stage progressive fine-tuning training strategy to train the subject classification and recognition model, while superimposing multiple optimization strategies; The auxiliary diagnostic module adopts a B / S architecture, receives dermoscopic images uploaded by users, and calls the image preprocessing module and the recognition and classification module respectively to perform automatic classification and recognition. It combines the Grad-CAM algorithm to generate a heat map of the lesion area and outputs the classification results and visual diagnostic basis.
[0027] The following is a detailed description of each module.
[0028] (I) Sample Generation Module This module introduces a diffusion model based on Stable Diffusion 1.5 and combines it with Low-Rank Adaptive (LoRA) technology to generate and augment images for the AK and SCC categories, which have scarce sample sizes. Specifically, the VAE encoder, CLIP text encoder, and original U-Net weights of the pre-trained diffusion model are frozen. Only the attention layer of the U-Net is added and trained with a LoRA adapter to learn the core features of real lesions (including typical appearance, characteristic color texture, local edge contours, and subtle structural differences). The model uses text prompts as conditional input, extracts semantic features through the CLIP text encoder, and completes a stepwise denoising process by combining random latent variables, time step information, and conditional U-Net. Finally, the VAE decoder generates skin disease images with clear category semantic features. The generated synthetic images and the original images together constitute the augmented training set in a specific ratio. This approach effectively controls the generation noise and overfitting risk while alleviating class imbalance.
[0029] (I) Image Preprocessing Module: Addressing the three typical interferences commonly found in dermoscopy images—hair occlusion, random noise, and dot-like spots—this solution designs a combined preprocessing algorithm integrating morphological operations, gray-level difference, edge detection, and image inpainting. Specifically, it includes: (1) Scale normalization: All images are uniformly scaled to 384×384 pixels to eliminate the resolution differences between different acquisition devices; (2) Spatial noise filtering: Gaussian filtering algorithm is used to smooth and reduce noise in the image, effectively suppressing shooting noise and dotted spots while protecting the edge contour of the lesion to the greatest extent. (3) Intelligent removal of hair interference: First, a single-channel image is obtained through standard grayscale transformation, and the broken hair area is connected by morphological closing operation; then, a precise hair mask is generated by grayscale difference operation to distinguish hair from normal skin / lesion area; finally, the Telea image restoration algorithm is used to restore the area covered by hair, and avoids accidental deletion of lesion texture and edge details on the premise of removing interference.
[0030] (4) Standardization processing: Convert the RGB image into the standard tensor format of the deep learning framework. The normalization process strictly follows the distribution of the EfficientNetV2-S pre-training dataset, and the normalization formula is as follows: ; in, , It is perfectly aligned with the normalization rules of the ImageNet pre-trained weights in the backbone network.
[0031] (III) Recognition and Classification Module This module uses the MSAE-Net deep learning framework, specifically: constructing an improved EfficientNetV2-S classification network as the main classification and recognition model, with the following structure: (1) Feature extraction backbone network: EfficientNetV2-S pre-trained on ImageNet is used as the feature extraction backbone. The average pooling layer and fully connected classification layer at the end are removed, and the complete convolutional feature extraction part is retained. EfficientNetV2-S introduces core technologies such as progressive learning and mixed precision training, which can achieve the optimal balance between network depth, width and input resolution.
[0032] (2) Multi-scale Global Attention Enhancement Module (GMAB): This module is connected at the end of the backbone network. This module contains four parallel convolutional branches for extracting multi-scale features of different receptive fields. The feature maps output by each branch are concatenated along the channel dimension and then recalibrated through a channel attention mechanism. This attention mechanism generates channel weights through global average pooling and two fully connected layers (with ReLU activation in the middle layer), and multiplies them element-wise with the concatenated multi-scale feature maps to achieve feature recalibration, enabling the model to focus on key lesion areas and suppress background noise and redundant information.
[0033] (3) Classification Head: It consists of a global average pooling layer, a flattening layer, a batch normalization layer, a dropout layer, a first fully connected layer, a ReLU activation function, a second fully connected layer, a ReLU activation function, and a third fully connected layer connected in sequence, and finally outputs the probability distribution of four types of lesions.
[0034] (iv) Optimize the training module This module employs a three-stage progressive fine-tuning training strategy to train the EfficientNetV2-S classification network (subject classification and recognition model) for the recognition and classification module, and superimposes multiple optimization strategies.
[0035] In the three-stage progressive fine-tuning training strategy, the subject classification and recognition model directly loads the official ImageNet pre-trained weights and fine-tunes them layer by layer in three stages. Specifically, this includes: Phase 1: Freeze all parameters of the EfficientNetV2-S backbone network and train only the GMAB module and classification head to enable the new module to quickly adapt to the skin lesion classification task.
[0036] The second stage involves unfreezing the last 50% of the layer parameters of the backbone network (corresponding to the high-level semantic feature extraction part) and jointly training the unfrozen layer, the GMAB module, and the classification head.
[0037] Phase 3: Unfreeze all network parameters. Each training phase employs an early stopping strategy (terminating if the validation loss does not decrease after 15 epochs), and saves the model weights with the lowest validation loss.
[0038] Multiple optimization strategies include: Class Balanced Sampling: A class balanced sampling strategy is adopted when loading training data. The original number of samples of the four classes in the training set is counted. The sampling target is set as 2000 samples of class AK, maintaining the original number of class BCC, 2000 samples of class BKL, and 2000 samples of class SCC. By calculating the sampling weight of each sample, weightedRandomSampler is used for sampling with replacement to ensure that the smaller sample class has a more sufficient learning opportunity in each round of training.
[0039] Online data augmentation: Performs random horizontal flipping, random vertical flipping, random rotation, color perturbation (brightness, contrast, saturation perturbation amplitude), and random erasing on the input image.
[0040] Mixed loss function: Focal Loss is used as the base loss function to reduce the dominant role of easily distinguishable samples in the loss function, making the model pay more attention to difficult-to-distinguish samples; at the same time, the MixUp algorithm is introduced to perform linear mixing of samples, which enhances the model's generalization ability and effectively alleviates the problems of model overconfidence and overfitting.
[0041] Optimizer configuration: The Adam optimizer is used in all stages, and automatic mixed precision training (torch.cuda.amp) is used to accelerate the training process and reduce GPU memory usage.
[0042] (V) The auxiliary diagnosis module encapsulates the above MSAE-Net model into a complete auxiliary diagnosis system. The system adopts a B / S architecture, with the front-end using HTML+JavaScript to build the interactive interface, and the back-end inference service based on the Flask framework. The system operation flow includes: (1) Image upload and preprocessing: The system receives dermoscopy images uploaded by users through the front-end interactive interface and automatically calls the preprocessing module to complete scale normalization, Gaussian noise reduction, intelligent hair removal and standardization.
[0043] (2) Automatic identification and visualization: The backend calls the MSAE-Net model (identification and classification module) to perform inference, obtain the four-class prediction results and prediction confidence, and at the same time uses the Grad-CAM algorithm to generate a heat map of the lesion area; the gradient information corresponding to the final classification result is backpropagated to the feature map output by the GMAB module to generate a class activation heat map and superimpose it back onto the original image to realize the visualization of the classification basis.
[0044] (3) Results output: The system displays disease category, model prediction confidence, Grad-CAM heat map and standardized disease analysis and diagnostic suggestions retrieved from the preset database on the front-end interface, and supports one-click saving of diagnostic records and reports.
[0045] Based on the design of the above modules and network model, the construction process of this classification system is as follows: Figure 4 As shown.
[0046] This solution also provides specific implementation examples to illustrate the implementation of each module in detail: Example 1: Dataset Construction and Sample Augmentation This example uses the public dataset ISIC2019, comprising 7442 dermoscopy images covering four classes: AK, BCC, BKL, and SCC. These images are divided into a training set (5207 images), a validation set (1486 images), and a test set (749 images) in a 7:2:1 ratio. The original training set contains 606 images of class AK, 2326 images of class BCC, 1836 images of class BKL, and 439 images of class SCC. For the smaller sample classes (AK and SCC) in the training set, the Stable Diffusion 1.5 model combined with LoRA is used for image generation. Text descriptions (such as "actinic keratosis dermoscopy" and "squamous cell carcinoma dermoscopy") are used as conditional input, and semantic features are extracted using the CLIP text encoder. A LoRA adapter is inserted into the cross-attention layer of the U-Net for fine-tuning, learning to generate images highly consistent with the features of real lesions.
[0047] In this embodiment, the image generation of the Stable Diffusion 1.5 model (diffusion model) is as follows: Figure 1 As shown, it includes a VAE encoder, a CLIP text encoder, a U-Net with a LoRA adapter, and a VAE decoder.
[0048] After multiple ablation experiments, the optimal ratio of original data to generated images was determined to be 7:3, resulting in 259 AK images and 188 SCC images. All generated images were merged with the original training set to form an augmented training set, totaling 5654 images (865 AK images, 2326 BCC images, 1836 BKL images, and 627 SCC images). To comprehensively measure the quality of the generated images, this embodiment also uses three objective evaluation metrics: FID (Fréchet Inception Distance), KID (Kernel Inception Distance), and IS (Inception Score). FID measures the overall distribution difference between the real and generated images in the feature space. KID introduces the maximum mean difference and kernel function to supplement the evaluation of the generated image distribution under small sample conditions. IS evaluates the sharpness and diversity of the generated images by calculating the KL divergence between the category prediction distribution and the marginal distribution. Calculations show that the FID of the image generated by AK is 56.37, the KID is 0.0132, and the IS is 4.28; while the FID of the image generated by SCC is 64.12, the KID is 0.0157, and the IS is 4.08, indicating that the generated images are highly consistent with the real images in terms of feature distribution.
[0049] In this embodiment, the scheme also designed a comparison of different diffusion models. Under the same data partitioning, number of generated samples, and classification training settings, the classification performance showed significant differences after adding data generated by different diffusion models, as shown in Table 1. Among them, SD1.5 + LoRA achieved Accuracy, Recall, F1, and AUC of 92.79%, 92.79%, 92.75%, and 0.9734, respectively, indicating that the comprehensive classification result was optimal. Compared with the Real-only baseline, Accuracy and F1 improved by 2.80 and 2.81 percentage points, respectively; compared with other diffusion models, SD1.5 + LoRA also maintained its leading position in all indicators. Compared with the Real-only baseline, the classification performance improved to some extent after adding data generated by SD2.0 and SD3.0, but the improvement was still lower than that of SD1.5 + LoRA. The Accuracy and F1 of the enhanced SDXL-base were both lower than those of Real-only, indicating that updating the scale and version of the diffusion model does not necessarily lead to a more effective classification enhancement effect. In summary, the data generated by SD1.5 + LoRA is better suited to the current skin disease classification task. Therefore, this embodiment selects SD1.5 + LoRA as the subsequent generation enhancement model.
[0050] Table 1 Comparison data of different diffusion models Example 2: Specific implementation of image preprocessing. Perform the following preprocessing operations on all images in the augmentation training set: (1) Scale normalization: All images are scaled to 384×384 pixels to match the input specifications of the EfficientNetV2-S network.
[0051] (2) Spatial noise removal: Gaussian filtering algorithm (kernel size 5×5, standard deviation σ=1.0) is used to smooth and reduce noise in the image.
[0052] (3) Intelligent removal of hair interference: First, the RGB image is converted into a grayscale image, and the broken hair area is connected by morphological closing operation (kernel size is 9×9); then, a precise binary hair mask is generated by grayscale difference operation (setting grayscale threshold T=30); finally, the Telea image restoration algorithm is used to fill and repair the area covered by hair layer by layer based on the known pixel information of the neighborhood of the mask area, and restore the texture of the lesion.
[0053] (4) Normalization: Convert the RGB image to the standard tensor format (C×H×W) of the PyTorch framework, and normalize it according to the formula: ,in , .
[0054] Example 3: Implementation and Training of the MSAE-Net Model (An Improved EfficientNetV2-S Classification Network) In this embodiment, the MSAE-Net classification network is constructed as follows: Figure 2 As shown, the model architecture includes: (1) Backbone network: EfficientNetV2-S pre-trained on ImageNet is used, and the top average pooling layer and fully connected layer are removed, while the complete convolutional feature extraction part is retained.
[0055] (2) Multi-scale Global Attention Augmentation Module (GMAB): Connect it to the end of the backbone network. For example... Figure 3 As shown, this module contains four parallel convolutional layers with kernel sizes of 1×1, 3×3, 5×5, and 7×7, all with a stride of 1 and padding of 0, 1, 2, and 3, respectively. The feature maps output from each branch are concatenated along the channel dimension and then input into a global channel attention module. This module generates channel weights through global average pooling and two convolutional layers (with ReLU activation in the middle layer and a compression ratio of 16), and multiplies them element-wise with the input feature map to achieve feature recalibration.
[0056] In this embodiment, the MSAE-Net model of this scheme is compared and tested using an external validation set on the MILK10K dataset. All data do not overlap with ISIC2019, and all training strategies are consistent with the ISIC2019 dataset. The MILK10K dataset contains 7684 images (AK, SCC, BCC, BKL), which are divided into training, validation, and test sets in a ratio of 7:2:1. The training set contains 424 images of AK, 662 images of SCC, 3530 images of BCC, and 761 images of BKL. The comparison and test results are shown in Table 2.
[0057] Table 2 Test Results of External Validation Set (Non-ISIC2019) In this embodiment, to investigate the impact of the bottleneck dimension of the GMAB channel attention branch on the model's classification performance, the channel compression ratio r was set to 4, 8, 16, 32, and 64, respectively. Comparative experiments were conducted while maintaining other experimental conditions. The results are shown in Table 3. As the channel compression ratio increased from 4 to 16, the model's accuracy and F1 score generally showed an upward trend. When r=16, the model's accuracy, recall, and F1 scores reached 0.9279, 0.9279, and 0.9275, respectively, achieving optimal results. Although the AUC at r=8 was slightly higher than that at r=16, the difference was only 0.0018; and the accuracy and F1 scores at r=16 were 0.0174 and 0.0184 higher than those at r=8, respectively, while the number of attention branch parameters was also less. This indicates that r=16 maintained strong cross-threshold discrimination ability while achieving better final classification performance and lower model complexity. When the compression ratio was further increased to 32 and 64, the overall performance of the model decreased. Therefore, this paper ultimately sets the GMAB channel compression ratio to 16.
[0058] Table 3 Performance Comparison of Multi-Scale Global Attention Enhancement Module Models under Different Compression Ratios (3) Classification head: It consists of a global average pooling layer, a flattening layer, a batch normalization layer, a dropout layer (dropout rate set to 0.5), a first fully connected layer (1280→1024), a ReLU activation function, a second fully connected layer (1024→512), a ReLU activation function, and a third fully connected layer (512→4) connected in sequence, and finally outputs a 4-dimensional vector.
[0059] In this embodiment, to analyze the impact of Dropout on the classification performance of the classification head model, the GMAB channel compression ratio was fixed at 16. This scheme also designed to set the Dropout rate to 0, 0.1, 0.2, 0.3, 0.4, 0.5 and 0.6 respectively, while keeping the other training parameters the same. The performance comparison results are shown in Table 4.
[0060] Table 4. Performance Comparison of Classification Head Models under Different Dropout Rates Experimental results show that when the dropout rate is 0.5, the model's Accuracy, Weighted Recall, and Weighted-F1 scores reach 0.9279, 0.9279, and 0.9275 respectively, all being the highest values among all groups. This indicates that appropriate random deactivation can improve the model's final classification performance. Without dropout, although the model achieves the highest AUC, its Accuracy and F1 scores are lower than those with a dropout rate of 0.5, indicating that the higher cross-threshold ranking ability does not fully translate into more accurate class determination. When the dropout rate is further increased to 0.6, the model performance declines due to the loss of effective features caused by excessive random deactivation. Considering the final classification metrics such as Accuracy and F1, this embodiment sets the dropout rate to 0.5.
[0061] Model training process: In this embodiment, the network directly loads the official ImageNet pre-trained weights and adopts a three-stage progressive fine-tuning strategy, as follows: Data Loading: The training set uses the augmented training set (5654 images) constructed using Implementation Method 1. Combined with the preprocessing workflow described in Implementation Method 2, a class-balanced sampling strategy is employed to ensure a balanced proportion of samples from each class in each batch of data. The training set sampling targets are: 2000 samples for class AK, 2326 samples for class BCC (maintaining the original number), 2000 samples for class BKL, and 2000 samples for class SCC, totaling approximately 8326 samples. The validation set uses the original validation set (1486 images), undergoing only scale normalization and standardization processing, without data augmentation or sampling.
[0062] Phase 1 (Frozen Backbone Training): Freeze all parameters of the EfficientNetV2-S backbone network (requires_grad=False), and train only the GMAB module and classification head. The learning rate is set to 1×10⁻⁶. -3 The optimizer uses Adam (weight decay 1×10). -5The model was trained for 90 epochs. An early stopping strategy was adopted (the model was terminated if the validation loss did not decrease after 15 epochs), and the model weights with the lowest validation loss were saved.
[0063] Phase Two (Unfreezing High-Level Training): Reload the optimal model weights saved from Phase One. Unfreeze the parameters of the last 50% of the backbone network (i.e., the high-level semantic feature extraction part), and jointly train the unfrozen layers, the GMAB module, and the classification head. The learning rate is set to 1×10⁻⁶. -4 Train for 100 epochs, using an early stopping strategy (terminating if the validation loss does not decrease after 15 epochs), and save the model weights with the lowest validation loss.
[0064] Phase 3 (Global Fine-tuning): Reload the optimal model weights saved in Phase 2. Unfreeze all network parameters (requires_grad=True), and adjust the settings at 5×10⁻⁶. -6 The learning rate is globally fine-tuned, and the training is performed for 100 epochs. An early stopping strategy is adopted (the model is terminated if the validation loss does not decrease after 15 epochs), and the model weights with the lowest validation loss are saved as the final model.
[0065] 3. Loss Function and Regularization: Focal Loss: Focal Loss is used as the basic loss function with parameter γ=2.0. Combined with label smoothing (smoothing factor ε=0.1), the dominant role of easily distinguishable samples in the loss function is reduced, making the model pay more attention to difficult-to-distinguish samples.
[0066] MixUp Enhancement: The MixUp algorithm (α=0.15) is introduced to perform linear mixing of samples. During training, samples within each batch are randomly paired and linearly interpolated, and the loss function is adjusted accordingly to a weighted combination of mixed labels to enhance the model's generalization ability.
[0067] Mixed precision training: Using torch.cuda.amp with the GradScaler gradient scaler to accelerate training, reduce GPU memory usage, and improve training efficiency.
[0068] Implementation Method 4: Deployment and Application of the Auxiliary Diagnostic System (Auxiliary Diagnostic Module) The trained MSAE-Net model is integrated into the assisted diagnostic system. The system adopts a B / S architecture, with the front end built using HTML and JavaScript, and the back-end inference service based on the Flask framework.
[0069] The specific operating procedure is as follows: The doctor enters the user interface (e.g., ... Figure 5Click the "Upload Image" button and select the dermoscopy image to be diagnosed. After the image is uploaded, a thumbnail is displayed on the interface. The doctor clicks the "Preprocessing" button (the system automatically executes the complete preprocessing process described in Implementation Method Two), and then clicks the "Start Recognition" button.
[0070] After receiving the preprocessed image, the backend inputs it into the MSAE-Net model for inference. The model outputs probability distributions for four categories, taking the category with the highest probability as the final diagnosis result, and simultaneously calculating the confidence score of this prediction. Furthermore, the system calls the Grad-CAM algorithm to backpropagate the gradient information corresponding to the final classification result to the feature map output by the GMAB module, generating a class activation heatmap, which is then overlaid onto the original image to highlight the lesion areas that contribute most to the classification decision. Finally, the frontend interface (such as...) Figure 6 The system will simultaneously display: disease name, model prediction confidence percentage, lesion heatmap, and standardized disease analysis and diagnostic suggestions retrieved from a pre-set database based on the classification results. Doctors can save the current diagnosis record and report with one click. The entire process requires no complex operations, achieving full automation from image input to diagnostic suggestion output.
[0071] In this embodiment, the built-in Preset Diagnostic Database of this auxiliary diagnostic system is a lightweight, highly maintainable structured medical knowledge base. Its core function is to provide standardized, clinically reviewed interpretations of conditions and treatment suggestions for the four-class classification output of the MSAE-Net model. This database is implemented using a hard-coded front-end approach, and all knowledge entries are reviewed and revised by dermatologists to ensure the clinical accuracy and security of the content.
[0072] The database is constructed using a hard-coded, key-value pair structured storage model. It uses disease category labels as the primary key to establish a precise mapping between these labels and clinical manifestations, as well as treatment recommendations. Each knowledge record contains the following fixed fields: .
[0073] Based on the above system design, this solution also proposes an automatic classification method for common non-melanocyte tumors based on CNN, including the following steps: Step S1: Obtain the original dermoscopy image dataset, freeze the VAE encoder, CLIP text encoder, and original U-Net weights of the pre-trained latent diffusion model, add and train the LoRA adapter to the attention layer of the U-Net to learn the typical appearance, characteristic color texture, local edge contours, and subtle structural differences of real lesions; based on the latent diffusion model and combined with low-rank adaptive technology, perform image generation and augmentation on the AK and SCC categories with sparse sample sizes, combine the generated synthetic images with the original images according to a preset ratio to construct an enhanced training set; Step S2: Perform scale normalization, spatial noise filtering, intelligent hair interference removal, and standardization on the images in the enhanced training set in sequence, and output a preprocessed image in standard tensor format; wherein, intelligent hair interference removal specifically includes: obtaining a single-channel image through standard grayscale transformation, connecting the broken hair regions using morphological closing operation with a kernel size of 9×9; generating an accurate hair mask through grayscale difference operation, setting a grayscale threshold T=30; and using the Telea image inpainting algorithm to restore the areas occluded by hair through a fast walking method. Step S3: Construct an improved EfficientNetV2-S classification network as the main classification and recognition model. A three-stage progressive fine-tuning strategy is used to train the classification network. During training, class-balanced sampling, online data augmentation, and a hybrid loss function are superimposed for optimization. The Adam optimizer is used in all stages, and automatic mixed precision training is employed to accelerate the training process. The subject classification and recognition model includes a feature extraction backbone network, a multi-scale global attention enhancement module, and a classification head. The multi-scale global attention enhancement module includes four parallel convolutional branches with kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively. The feature maps output by each convolutional branch are concatenated along the channel dimension and then recalibrated through a channel attention mechanism. In the three-stage progressive fine-tuning strategy, the first stage freezes all parameters of the feature extraction backbone network and trains only the multi-scale global attention enhancement module and the classification head. The second stage unfreezes the parameters of the last 50% of the backbone network and trains them together. The third stage unfreezes all network parameters and trains them together. The multiple optimization strategies include: using WeightedRandomSampler for class balancing sampling, with a sampling target of 2000 samples for class AK, maintaining the original number for class BCC, 2000 samples for class BKL, and 2000 samples for class SCC; using Focal Loss as the base loss function with parameter γ=2.0, combined with a label smoothing factor ε=0.1; introducing the MixUp algorithm for linear mixing of samples with parameter α=0.15; using the Adam optimizer in all stages and employing automatic mixed precision training; and using an early stopping strategy in each training stage, terminating if the validation loss does not decrease after 15 rounds, and saving the model weights with the lowest validation loss. Step S4: The trained classification network is encapsulated into an auxiliary diagnostic system. It receives dermoscopy images uploaded by users, performs preprocessing steps, calls the classification network for inference, outputs four-class prediction results and confidence scores, and uses the Grad-CAM algorithm to generate a heat map of the lesion area for visualization. It simultaneously displays the disease category, model prediction confidence score, Grad-CAM heat map, and standardized disease analysis and diagnostic suggestions retrieved from the preset database, and supports one-click saving of diagnostic records and reports.
[0074] For the foregoing embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.
[0075] The above embodiments describe the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the invention should be within the protection scope of the appended claims.
Claims
1. An automated classification system for common non-melanocyte tumors based on CNN, characterized in that, include: Sample generation module: Freeze the VAE encoder, CLIP text encoder, and original U-Net weights of the pre-trained latent diffusion model. Add and train the LoRA adapter to the attention layer of U-Net to learn the typical appearance, characteristic color texture, local edge contours, and subtle structural differences of real lesions. Introduce the latent diffusion model and combine it with low-rank adaptive technology to generate and augment images for the AK and SCC categories with scarce sample sizes. Combine the generated synthetic images with the original images according to a preset ratio to form an enhanced training set. Image preprocessing module: Configures a combined preprocessing algorithm to perform scale normalization, spatial noise filtering, intelligent hair interference removal and standardization on the input dermoscopy image in sequence, and outputs a preprocessed image in standard tensor format; The classification module is constructed as an improved EfficientNetV2-S classification network as the subject classification and recognition model. The subject classification and recognition model includes a feature extraction backbone network, a multi-scale global attention enhancement module, and a classification head. The multi-scale global attention enhancement module includes four parallel convolutional branches for extracting multi-scale features of different receptive fields. The kernel sizes of the four convolutional branches are 1×1, 3×3, 5×5, and 7×7, respectively. The feature maps output by each convolutional branch are concatenated along the channel dimension and then recalibrated through a channel attention mechanism. Optimize the training module: Configure a three-stage progressive fine-tuning training strategy to train the subject classification and recognition model, while superimposing multiple optimization strategies; The auxiliary diagnostic module adopts a B / S architecture, receives dermoscopic images uploaded by users, and calls the image preprocessing module and the recognition and classification module respectively to perform automatic classification and recognition. It combines the Grad-CAM algorithm to generate a heat map of the lesion area and outputs the classification results and visual diagnostic basis.
2. The CNN-based automatic classification system for common non-melanocyte tumors as described in claim 1, characterized in that, In the sample generation module, the latent diffusion model adopts a diffusion model based on Stable Diffusion 1.5, with the ratio of raw data to synthesized images being 7:
3. The generation of the synthesized images includes: using text prompts as conditional input, extracting semantic features through a CLIP text encoder, performing stepwise denoising by combining random latent variables, time step information, and a conditional U-Net, and finally generating skin disease images with clear category semantic features through a VAE decoder; and using three objective evaluation metrics, FID, KID, and IS, to evaluate the quality of the generated images.
3. The CNN-based automatic classification system for common non-melanocyte tumors as described in claim 2, characterized in that, The enhanced training set obtained in the sample generation module is constructed based on the ISIC2019 dataset, consisting of 7442 dermoscopy images, which are divided into a training set of 5207 images, a validation set of 1486 images, and a test set of 749 images in a 7:2:1 ratio. Among them, 259 synthetic images are generated for the AK category and 188 synthetic images are generated for the SCC category, for a total of 5654 images in the enhanced training set. Cross-dataset validation is performed using the MILK10K external validation set.
4. The CNN-based automatic classification system for common non-melanocyte tumors as described in claim 1, characterized in that, The combined preprocessing algorithm of the image preprocessing module specifically includes: Scale normalization was performed, scaling all dermoscopy images to a uniform 384×384 pixels to eliminate resolution differences between different acquisition devices. Spatial noise is filtered out, and a Gaussian filtering algorithm is used to smooth and reduce noise in the dermoscopy image, suppressing shooting noise and punctate spots, while protecting the edge contour of the lesion; wherein, the convolution kernel size of the Gaussian filtering algorithm is 5×5, and the standard deviation σ=1.0; Intelligent removal of hair interference is achieved by obtaining a single-channel image through standard grayscale transformation. Morphological closing operations are used to connect broken hair regions. The kernel size of the morphological closing operation is 9×9. A precise hair mask is generated through grayscale difference operations, and a grayscale threshold T=30 is set to distinguish hair from normal skin or lesion areas. The Telea image restoration algorithm is used to fill and repair the areas occluded by hair layer by layer based on the known pixel information of the neighborhood of the mask region through a fast walking method. Standardization processing: The RGB image is converted to the standard tensor format of the deep learning framework. Standardization is then performed based on the distribution of the subject classification model in the aforementioned recognition module, aligning it with the normalization rules of the backbone network of the subject classification model. The normalization formula is expressed as: ; in, , .
5. The CNN-based automatic classification system for common non-melanocyte tumors as described in claim 1, characterized in that, In the recognition and classification module, the feature extraction backbone network adopts the EfficientNetV2-S classification network pre-trained on ImageNet, while removing the average pooling layer and fully connected classification layer at its end, retaining the complete convolutional feature extraction part; the EfficientNetV2-S classification network also introduces progressive learning and mixed precision training to achieve an optimal balance between network depth, width and input resolution. The multi-scale global attention enhancement module is connected to the end of the feature extraction backbone network. The stride of the four parallel convolutional branches is 1, and the padding is 0, 1, 2, and 3 respectively. The classification head includes a globally average pooling layer, a flattening layer, a batch normalization layer, a dropout layer, a first fully connected layer, a ReLU activation function, a second fully connected layer, a ReLU activation function, and a third fully connected layer, which ultimately output the probability distribution of four types of lesions. Wherein, the dropout rate of the Dropout layer is set to 0.5, the dimension of the first fully connected layer is 1280×1024, the dimension of the second fully connected layer is 1024×512, and the dimension of the third fully connected layer is 512×4.
6. The CNN-based automatic classification system for common non-melanocyte tumors as described in claim 5, characterized in that, In the multi-scale global attention enhancement module, the channel attention mechanism generates channel weights through global average pooling and two fully connected layers with ReLU activation functions in the intermediate layer, and multiplies them element-wise with the concatenated multi-scale feature map to achieve feature recalibration, so that the model focuses on key lesion areas and suppresses background noise and redundant information; wherein, the channel compression ratio of the channel attention mechanism is set to 16.
7. The CNN-based automatic classification system for common non-melanocyte tumors as described in claim 1, characterized in that, The three-stage progressive fine-tuning training strategy in the optimized training module specifically includes: In the first stage, all parameters of the feature extraction backbone network are frozen, and only the multi-scale global attention enhancement module and the classification head are trained to enable the new module to quickly adapt to the skin lesion classification task. In the second stage, the parameters of the last 50% of the high-level semantic feature extraction part of the backbone network are unfrozen, and the corresponding unfrozen layer, multi-scale global attention enhancement module and classification head are jointly trained. The third stage involves unfreezing all network parameters for training. The first phase of training consists of 90 epochs with a learning rate of 1×10⁻⁶. -3 The second phase of training consists of 100 epochs, with a learning rate set to 1×10⁻⁶. -4 The third phase of training consists of 100 epochs, with a learning rate set to 5 x 10^6. -6 An early stopping strategy is used in each training phase to verify that the loss does not decrease after 15 rounds, and the model weights with the lowest verification loss are saved.
8. The CNN-based automatic classification system for common non-melanocyte tumors as described in claim 7, characterized in that, The multiple optimization strategies superimposed in the optimized training module include: Class Balanced Sampling: A class balanced sampling strategy is adopted when loading training data. The original number of samples of the four classes in the training set is counted and a sampling target is set. By calculating the sampling weight of each sample, the WeightedRandomSampler data sampler is used for sampling with replacement. The sampling target is 2000 samples of class AK, maintaining the original number of class BCC, 2000 samples of class BKL, and 2000 samples of class SCC, to ensure that the smaller sample classes have more learning opportunities in each round of training. Online data augmentation: Randomly flips horizontally, randomly flips vertically, randomly rotates, color-perturbs, and randomly erases the input image; Mixed loss function: Focal Loss is used as the basic loss function with parameter γ=2.0, combined with label smoothing factor ε=0.1; at the same time, MixUp algorithm is introduced to perform linear mixing of samples with parameter α=0.15, which enhances the model's generalization ability and alleviates the problems of model overconfidence and overfitting. Optimizer configuration: Adam optimizer is used in all stages, with a weight decay of 1×10. -5 It also uses torch.cuda.amp for automatic mixed-precision training to reduce GPU memory usage.
9. The CNN-based automatic classification system for common non-melanocyte tumors as described in claim 1, characterized in that, The auxiliary diagnostic module is specifically configured as follows: The system receives dermoscopy images uploaded by users through the front-end interactive interface and automatically calls the image preprocessing module to complete scale normalization, Gaussian noise reduction, intelligent hair removal and standardization processing. The backend calls the recognition and classification module to perform inference, obtains the four-class prediction results and prediction confidence, and uses the Grad-CAM algorithm to backpropagate the gradient information corresponding to the final classification result to the feature map output by the multi-scale global attention enhancement module, generating a class activation heatmap and superimposing it back onto the original image. The front-end interface synchronously displays disease categories, model prediction confidence, Grad-CAM heatmaps, and standardized disease analysis and diagnostic suggestions retrieved from a preset database, and supports one-click saving of diagnostic records and reports; the preset diagnostic database adopts a hard-coded key-value pair structured storage model, with disease category labels as the primary key, to establish a mapping relationship between clinical manifestation descriptions and treatment suggestions; The front-end uses HTML and JavaScript to build the interactive interface, while the back-end inference service is based on the Flask framework.
10. A CNN-based automatic classification method for common non-melanocyte tumors, implemented based on the CNN-based automatic classification system for common non-melanocyte tumors as described in any one of claims 1 to 9, characterized in that, Includes the following steps: Step S1: Obtain the original dermoscopy image dataset, freeze the VAE encoder, CLIP text encoder, and original U-Net weights of the pre-trained latent diffusion model, add and train the LoRA adapter to the attention layer of the U-Net to learn the typical appearance, characteristic color texture, local edge contours, and subtle structural differences of real lesions; based on the latent diffusion model and combined with low-rank adaptive technology, perform image generation and augmentation on the AK and SCC categories with sparse sample sizes, combine the generated synthetic images with the original images according to a preset ratio to construct an enhanced training set; Step S2: Perform scale normalization, spatial noise filtering, intelligent hair interference removal, and standardization processing sequentially on the images in the enhanced training set, and output a preprocessed image in standard tensor format; wherein, the intelligent hair interference removal specifically includes: obtaining a single-channel image through standard grayscale transformation, connecting the broken hair regions using morphological closing operations with a kernel size of 9×9; generating an accurate hair mask through grayscale difference operations, setting a grayscale threshold T=30; and using the Telea image restoration algorithm to restore the areas occluded by hair through a fast walking method; Step S3: Construct an improved EfficientNetV2-S classification network as the main classification and recognition model. A three-stage progressive fine-tuning strategy is used to train the classification network. During training, class-balanced sampling, online data augmentation, and a hybrid loss function are superimposed for optimization. The Adam optimizer is used in all stages, and automatic mixed precision training is employed to accelerate the training process. The subject classification and recognition model includes a feature extraction backbone network, a multi-scale global attention enhancement module, and a classification head. The multi-scale global attention enhancement module includes four parallel convolutional branches with kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively. The feature maps output by each convolutional branch are concatenated along the channel dimension and then recalibrated through a channel attention mechanism. In the three-stage progressive fine-tuning strategy, the first stage freezes all parameters of the feature extraction backbone network and trains only the multi-scale global attention enhancement module and the classification head; the second stage unfreezes the parameters of the last 50% of the backbone network and trains them together; and the third stage unfreezes all network parameters and trains them together. The multiple optimization strategies include: using WeightedRandomSampler for class balancing sampling, with a sampling target of 2000 samples for class AK, maintaining the original number for class BCC, 2000 samples for class BKL, and 2000 samples for class SCC; using Focal Loss as the base loss function with parameter γ=2.0, combined with a label smoothing factor ε=0.1; introducing the MixUp algorithm for linear mixing of samples with parameter α=0.15; using the Adam optimizer in all stages and employing automatic mixed precision training; and using an early stopping strategy in each training stage, terminating if the validation loss does not decrease after 15 rounds, and saving the model weights with the lowest validation loss. Step S4: The trained classification network is encapsulated into an auxiliary diagnostic system. It receives dermoscopy images uploaded by users, performs preprocessing steps, calls the classification network for inference, outputs four-class prediction results and confidence scores, and uses the Grad-CAM algorithm to generate a heat map of the lesion area for visualization. It simultaneously displays the disease category, model prediction confidence score, Grad-CAM heat map, and standardized disease analysis and diagnostic suggestions retrieved from the preset database, and supports one-click saving of diagnostic records and reports.
