A pathological image enhancement and classification method based on improved DCGAN training strategy

Abstract

This paper discloses a pathology image enhancement and classification method based on an improved DCGAN training strategy. This method generates high-fidelity pathology images using a limited number of labeled samples. The generated images are then added to the original data to improve the quality of the original dataset. The images are then classified using a lightweight CRC pathology image classification network, MRD-Net. This approach significantly improves the quality of the original dataset, effectively alleviating the difficulties of acquiring medical imaging data and the high cost of annotation. While ensuring classification accuracy, it significantly reduces the number of model parameters and significantly improves classification efficiency.

Description

Technical Field

[0001] The present invention relates to the field of pathological image-assisted diagnosis technology, and in particular to a pathological image enhancement and classification method based on an improved DCGAN training strategy. Background Art

[0002] Colorectal cancer (CRC) is the third most common malignancy worldwide, ranking second among women and third among men, and has a high mortality rate. According to statistics, approximately 1.93 million people were diagnosed with CRC worldwide in 2020, accounting for 10% of all cancer cases. In the same year, CRC-related deaths reached 935,000, accounting for 9.7% of all cancer-related deaths. Clinical studies have shown that the prognosis of CRC is closely related to the tumor's lymph node metastasis stage. The five-year survival rate for patients with early-stage CRC is close to 90%, while the survival rate for patients with advanced CRC drops sharply to 13%. Therefore, early detection and diagnosis are crucial for timely intervention and treatment of CRC, significantly improving patient survival.

[0003] Currently, the diagnosis of CRC mainly relies on colonoscopy and histopathological analysis. Although colonoscopy can provide intuitive imaging information, due to the complex intestinal environment, colonoscopy may face problems such as low image contrast and optical interference from polyps, resulting in missed detection or misdiagnosis of lesions. Relatively speaking, histopathological examination is considered the "gold standard" for CRC diagnosis. Through sampling and analysis at the cellular level, it can provide more accurate diagnostic results. However, the manual classification and segmentation of pathological images not only requires a high level of professional skills, but is also time-consuming and labor-intensive, and can easily miss tiny lesions. At the same time, diagnostic results may also be affected by subjective differences between pathologists, resulting in inconsistent interpretations.

[0004] In recent years, deep learning technology has been widely used in medical image analysis and has achieved remarkable results. Deep learning, in particular, can automatically learn both low-level and high-level features from images, thereby helping pathologists improve diagnostic efficiency. However, in the construction of deep learning classification models for CRC, data insufficiency and class imbalance persist, severely impacting the model's accuracy for minority samples. Furthermore, while traditional data augmentation methods, such as flipping, mirroring, scaling, and cropping, can expand the size of the training set, they fail to introduce new information or enhance the diversity of sample distributions, limiting their effectiveness in addressing data scarcity and class imbalance. Furthermore, despite breakthroughs in accuracy, existing classification models based on convolutional neural networks (CNNs) still face challenges in practical applications, such as high model complexity, high computational overhead, and a high risk of overfitting. These factors limit their practical application, particularly in resource-limited settings. Achieving efficient and cost-effective automated diagnosis remains an unmet challenge. While some studies have optimized CNNs by introducing attention mechanisms or network compression methods, achieving a balance between high performance and low complexity remains a technical challenge.

[0005] Therefore, how to use deep learning technology, especially generative adversarial networks (GAN), to effectively enhance training data and improve the performance of CRC classification models while solving the problems of data scarcity and class imbalance remains a key issue in current research. Summary of the Invention

[0006] To address the above technical problems, the present invention provides a pathological image enhancement and classification method based on an improved DCGAN training strategy. An improved DCGAN training method is proposed based on deep convolutional generative adversarial networks (DCGAN) for pathological image synthesis. The synthesized images are then used to expand the dataset. Finally, a lightweight classification network, MRD-Net (modified residual depthwise separable convolutional neural net), is proposed based on CNN. This network can efficiently implement medical image classification and quickly and accurately classify CRC pathological images.

[0007] The present invention is achieved by adopting the following technical solutions:

[0008] A pathological image enhancement and classification method based on an improved DCGAN training strategy includes the following steps:

[0009] Step S1: Generate high-fidelity pathological images under limited labeled sample conditions through an improved DCGAN training strategy;

[0010] Step S2: Add the generated image to the original data to improve the quality of the original dataset;

[0011] Step S3: Classify the image using the lightweight CRC pathology image classification network MRD-Net model.

[0012] Specifically, the improved DCGAN training strategy is based on the DCGAN network architecture, including a generator, a discriminator and a convolutional layer, and modifies the number of layers and channels of the generator and the discriminator. The generator and the discriminator both include six convolutional blocks.

[0013] Specifically, the generator of the improved DCGAN training strategy introduces Gaussian noise with a mean of 0 and a variance of 1 into the input data at the beginning of training, then gradually reduces the noise variance through an exponential decay strategy, and introduces DiffAugment enhancement data to both real data and generated data.

[0014] Specifically, the lightweight CRC pathology image classification network MRD-Net model includes a feature extractor module and a classifier module. The feature extractor module consists of five convolutional blocks with channel numbers of 32, 96, 192, 384 and 512, respectively. The first convolutional block uses two 3×3 convolutional layers for initial feature extraction, followed by Batch Normalization, ReLU activation function and MaxPool layer for dimensionality reduction. The second to fifth convolutional blocks introduce an improved DSC module.

[0015] Specifically, the ratio of the second to fifth convolution block modules is 1:1:3:1.

[0016] Specifically, the improved DSC module is specifically configured as follows:

[0017] A 1×1 convolution is used to expand the channel dimension and introduce nonlinear mapping, followed by a 3×3 depth convolution to extract spatial features, and then a 1×1 convolution is used for channel fusion;

[0018] Add the channel fusion result to the output of the first 1×1 convolution and generate the final output through the ReLU activation function;

[0019] The feature extraction part finally compresses the spatial features into a fixed global representation through an AdaptiveMaxPool layer for use by the classifier module.

[0020] Specifically, the classifier module includes two fully connected layers. The first fully connected layer reduces the 512-dimensional features to 256 dimensions and uses the ReLU activation function and Dropout to achieve nonlinear mapping and regularization. The second fully connected layer outputs the final classification results of the pathological image.

[0021] The beneficial effects of the present invention are as follows: the present invention proposes a method for CRC pathology image-assisted diagnosis that combines a generative adversarial network with a lightweight classification model, which has high practical value and broad application prospects. The CRC-assisted classification method constructed in this solution includes three main steps: image generation, data fusion, and pathology image classification. First, through an improved DCGAN training method, high-fidelity pathology images can be generated under limited annotated sample conditions. Second, the generated images are added to the original data, significantly improving the quality of the original dataset and effectively alleviating the problems of difficulty in acquiring medical imaging data and high annotation costs. Finally, through the lightweight classification network MRD-Net, the number of model parameters is significantly reduced (to only 2.18% of VGG and 8.59% of ResNet-50) while ensuring classification accuracy, significantly improving classification efficiency. The CRC-assisted classification method constructed in this solution has significant advantages in generation quality, classification performance, and resource adaptability. It can provide clinicians with efficient and reliable auxiliary diagnosis support and is suitable for early screening and preliminary diagnosis of CRC. It has good clinical practical value and industrial promotion potential, and solves the following technical problems:

[0022] (1) Solved the problem of low quality pathological image generation when annotated samples are limited. A new DCGAN training method is proposed. By introducing a dynamic training mechanism and a generated image quality assessment strategy, high-fidelity CRC pathological image synthesis is achieved. This significantly improves the scalability and data quality of limited annotated datasets, providing a richer and more effective sample basis for the training of subsequent classification models.

[0023] (2) The problem of large number of parameters and high computational cost in existing classification models is solved. Based on the VGG-19 network structure, combined with the DSC in MobileNet and the residual skip connection of ResNet, a lightweight CRC classification network MRD-Net is designed. While maintaining high classification accuracy, the number of model parameters is reduced to 2.18% of the VGG network and 8.59% of the ResNet-50 network, significantly improving the computational efficiency of the model.

[0024] (3) The problem of lack of effective coupling between data augmentation methods and classification model performance improvement is solved. By fusing the generated high-quality pathology image samples with the original dataset, a richer dataset is constructed for training the classification network. The verification results on the NCT-CRC-HE-100K and NCT-CRC-VAL-7K datasets show that the training strategy combined with the proposed lightweight classification model significantly improves the classification accuracy of CRC pathology images, further demonstrating the feasibility and superiority of the proposed method in actual medical image analysis tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0026] Figure 1 Schematic diagram of the process of pathological image enhancement and classification method based on the improved DCGAN training strategy in an embodiment of the present invention;

[0027] Figure 2 This is a structural diagram of the improved DCGAN in an embodiment of the present invention;

[0028] Figure 3 This is a model diagram of the lightweight CRC pathology image classification network MRD-Net in an embodiment of the present invention;

[0029] Figure 4 This is a comparison diagram between the generated image and the real image in the embodiment of the present invention. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0031] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0032] The following is combined with Figure 1-4, some embodiments of the present invention are described in detail. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0033] The present invention proposes a pathological image enhancement and classification method based on an improved DCGAN training strategy. In a preferred embodiment, the method includes:

[0034] Step S1: Generate high-fidelity pathological images under limited labeled sample conditions through an improved DCGAN training strategy;

[0035] Step S2: Add the generated image to the original data to improve the quality of the original dataset;

[0036] Step S3: Classify the image using the lightweight CRC pathology image classification network MRD-Net model.

[0037] The following describes the training strategy and model architecture in this solution in conjunction with specific embodiments:

[0038] This paper proposes a novel two-stage approach for the classification of CRC pathology images, involving image data augmentation and classification models. First, a novel data augmentation framework based on DCGAN was developed, focusing on improvements to Gaussian noise and Differentiable Augmentation (DiffAugment) strategies to generate high-quality histopathology images from limited annotated data. This novel data augmentation method effectively expands limited training datasets, overcomes the scarcity of annotated data, and improves the quality of generated images, providing a richer sample for subsequent classification tasks.

[0039] Secondly, this solution designed a lightweight convolutional neural network classification method, MRD-Net. This network combines the advantages of the VGG-19 architecture with depthwise separable convolutions (DSC) from MobileNet and residual skip connections inspired by ResNet. This design effectively reduces the model's parameter complexity and computational overhead while maintaining classification accuracy. This innovative architectural design not only ensures model lightweightness but also achieves a good balance between performance and computational efficiency, making it particularly suitable for environments with limited computing resources.

[0040] Finally, this technical solution was validated on the NCT-CRC-HE-100K and NCT-CRC-VAL-7K datasets, demonstrating that the proposed generative adversarial approach significantly improved the quality of the datasets. Combined with the lightweight MRD-Net, it more efficiently improved CRC classification performance. Compared to traditional data augmentation methods based on generative adversarial networks, this approach outperformed both training efficiency and image generation quality, achieving both lightweight and high performance in classification.

[0041] In this embodiment, this solution proposes a DCGAN training strategy for CRC pathology image generation and a lightweight classification model for CRC pathology image classification to achieve more efficient CRC pathology image classification. In existing technical solutions, the training time required for the GAN series network is long and the training data required is large. At the same time, although the deep learning model has achieved good performance in CRC tissue pathology image classification, it still faces problems such as high computational complexity, high risk of overfitting, and strong dependence on labeled data. Therefore, this solution proposes the following design:

[0042] (1) A new DCGAN training method is proposed to generate high-quality CRC tissue pathology images from a limited annotated dataset;

[0043] (2) A new lightweight CRC pathology image classification network MRD-Net is proposed. Based on the VGG-19 structure, it combines the depth-separable convolution of Mobile-Net and the residual jump connection of ResNet to achieve high-performance CRC tissue pathology image classification. The overall process of the proposed method is as follows Figure 1 shown.

[0044] The specific technical route for the DCGAN training method (training strategy) proposed in this application is as follows:

[0045] DCGAN relies on adversarial training mechanisms to generate high-quality synthetic images. DCGAN introduces convolutional layers in the generator and discriminator, which significantly improves the stability and performance of training compared to early GAN models. However, the original DCGAN architecture was designed based on 64×64 pixel images, and the network structure was relatively simple. If it is directly applied to medical image generation tasks, it will face problems such as gradient vanishing, gradient explosion, and training instability during training. This solution proposes a more efficient training method, which retains the original DCGAN core structure and makes targeted adjustments to the network architecture and training strategy. The structure of the improved DCGAN is as follows: Figure 2 shown.

[0046] First, to adapt to the generation and discrimination of 224×224 resolution images, this solution modifies the number of layers and channels in the generator and discriminator to enhance the network's capacity and expressiveness. Furthermore, to improve the generator's feature learning capabilities and reduce the risk of model collapse, Gaussian noise with a mean of 0 and a variance of 1 is introduced into the input data at the beginning of training. An exponential decay strategy is then used to gradually reduce the noise variance, preventing the discriminator from converging too quickly and ensuring sufficient training of the generator. To further enhance the stability of generative adversarial training, DiffAugment is introduced to both real and generated data, including operations such as translation, cutout, and color, to enhance the model's generalization capabilities and reduce its reliance on large-scale training data. Subsequently, we named the DCGAN that incorporates this training strategy DCGAN+.

[0047] The specific technical route of the lightweight CRC pathology image classification network MRD-Net model proposed in this solution is as follows:

[0048] In the study of CRC tissue pathology image classification, existing studies have verified the effectiveness of CNN on the NCT-CRC-HE-100K dataset. Among them, although VGG-19 has the best classification performance, it has a high computational complexity; and although MobileNet has the advantage of being lightweight, its classification performance is relatively weak. In order to strike a balance between classification performance and model complexity, this scheme proposes a new lightweight classification network MRD-Net with enhanced feature extraction capabilities and multi-scale feature learning capabilities. The structure diagram of the proposed network MRD-Net is shown in the figure below. Figure 3 shown.

[0049] The classification network in this solution primarily consists of two parts: a feature extractor and a classifier. The feature extractor consists of five convolutional blocks (Blocks 1 to 5), with 32, 96, 192, 384, and 512 channels, respectively. This design balances feature representation with computational efficiency, while also supporting multi-scale feature learning. Block 1 uses two 3×3 convolutional layers for initial feature extraction, followed by Batch Normalization, ReLU activation, and MaxPool for dimensionality reduction. Blocks 2 to 5 incorporate an improved DSC module, with a module ratio of 1:1:3:1, similar to ConvNeXt, to reduce redundant neurons and improve computational efficiency.

[0050] The improved DSC module first expands the channel dimension and introduces nonlinear mapping through a 1×1 convolution. It then uses a 3×3 depthwise convolution to extract spatial features, followed by another 1×1 convolution for channel fusion. Unlike the traditional residual path, which directly adds the input and output, this module adds the output of the first 1×1 convolution and the output of the channel fusion. This structure ensures the continuity and stability of the feature flow. The feature extraction part finally compresses the spatial features into a fixed global representation through an AdaptiveMaxPool layer for use by the classifier.

[0051] The classifier consists of two fully connected layers. The first layer reduces the 512-dimensional features to 256 dimensions and uses the ReLU activation function and Dropout (p=0.5) for nonlinear mapping and regularization. The second fully connected layer outputs the final nine-category classification results for the pathology images.

[0052] In this example, to validate the effectiveness of the proposed method, we first verified the effectiveness of the training strategy. We randomly selected 1,000 images from each category in the NCT-CRC-HE-100K dataset for generative adversarial training. During training, the model was saved every 100 epochs. The Fréchet Inception Distance (FID) between the generated and real images was calculated, and the model with the lowest FID was selected as the optimal generator. The generated images were quantitatively evaluated using metrics such as Kernel Inception Distance (KID) and Inception Score (IS). The results are shown in Table 1. As can be seen from the data in the table, the proposed training strategy significantly improved the training stability of the DCGAN and the quality of the generated images. The resulting images were highly similar to the real pathology images in terms of morphology, texture, color distribution, and structural features. In all metrics, our method outperformed the standard DCGAN and WGAN-GP baselines. Compared to the standard DCGAN, our method reduced the FID by an average of 71.43%, the KID by 76.47%, and the IS by 45.30% across all categories. Compared with WGAN-GP, the proposed method reduces FID by 53.69% on average, reduces KID by 67.35%, and improves IS by 11.44%.

[0053] Table 1. Calculation results of objective indicators of generated images before and after using the training strategy

[0054]

[0055] In addition, we also compare the images generated by DCGAN, DCGAN+ and WGAN-GP with real images. Figure 4As can be seen in the results, DCGAN+, incorporating this training strategy, outperforms the unoptimized model in terms of edge detail, cell structure integrity, and color distribution consistency. The resulting images are clearer and more natural, effectively simulating real tissue pathology images. This further demonstrates that the proposed training strategy significantly improves the stability of DCGAN, providing a better solution for generating high-quality clinical data and laying a solid foundation for the subsequent training of pathology image classification models.

[0056] To validate the effectiveness of the classification network, this solution conducted ablation experiments using three common classification models (VGG-19, ResNet-50, and MobileNet-V1). The classification models were trained on the NCT-CRC-HE-100K dataset before and after augmentation and tested on the NCT-CRC-VAL-7K dataset. During training, the NCT-CRC-HE-100K dataset was split into 80% (training set) and 20% (validation set), and the training loss was calculated using the cross-entropy loss function. The model parameters were updated based on the training set loss, and the best model was saved based on the validation set results. An early stopping mechanism monitored the change in loss over a 20-epoch range. If the validation loss did not decrease within 20 epochs, the model was considered to have converged to the ideal range. The performance of the model was evaluated using metrics such as accuracy (Accuracy, Precision, Recall, F1 score, F1), specificity (Spe), and area under the curve (AUC). The results on the test set are shown in Table 2. The proposed network, MRD-Net, achieved a test accuracy of 91.27% after training on the unaugmented data and 92.65% after training on the augmented data. This performance improvement was also observed for ResNet-50, which increased from 91.70% to 93.64%, VGG-19, which increased from 91.30% to 92.79%, and MobileNet-V1, which increased from 87.91% to 92.65%. These results demonstrate that MRD-Net demonstrates a strong performance advantage in the CRC classification task, demonstrating excellent classification results. Furthermore, augmenting the original training set with generated data provides a richer sample size, further improving the overall performance of the classification network. Based on the expanded dataset, MRD-Net demonstrated better classification accuracy and stability compared to MobileNet-V1, verifying its effectiveness and robustness in CRC classification tasks.

[0057] Table 2. Performance comparison of the proposed network MRD-Net and other networks

[0058]

[0059] Under the same experimental conditions, this solution calculated the parameter complexity (number of parameters) and computational complexity (number of computations) of the proposed network, MRD-Net, and three CNN classification networks. As shown in Table 3, MRD-Net's advantage in lightweightness is clearly evident. The proposed network has only 3.04M parameters, equivalent to 8.59% of ResNet-50, 2.18% of VGG-19, and 94.41% of MobileNet-V1. The number of FLOPs is reduced to 691.96M, equivalent to 12.89% of ResNet-50 and 3.51% of VGG-19, comparable to MobileNet-V1, while offering greater stability.

[0060] Table 3. Comparison of the lightweightness of the proposed network and other networks

[0061]

[0062] In summary, this solution first uses an improved DCGAN training method to generate high-fidelity pathology images with limited labeled samples. Secondly, by adding the generated images to the original data, the quality of the original dataset is significantly improved, effectively alleviating the difficulties in acquiring medical imaging data and the high cost of annotation. Finally, by using the lightweight classification network MRD-Net, the number of model parameters is significantly reduced (to only 2.18% of VGG and 8.59% of ResNet-50) while maintaining classification accuracy, significantly improving classification efficiency.

[0063] For the sake of simplicity, the aforementioned embodiments are described as a series of actions. However, those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are preferred embodiments, and the actions involved are not necessarily required by this application.

[0064] The above embodiments describe the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Without departing from the spirit and scope of the present invention, modifications and variations made by those skilled in the art without departing from the spirit and scope of the present invention should be within the scope of protection of the appended claims.

Claims

1. A pathological image enhancement and classification method based on an improved DCGAN training strategy, characterized in that: The following steps are involved: Step S1: Generate high-fidelity pathological images under limited labeled sample conditions through an improved DCGAN training strategy; the improved DCGAN training strategy is based on the DCGAN network architecture, including a generator, a discriminator, and a convolutional layer, and modifies the number of layers and channels of the generator and discriminator. The generator and discriminator each include six convolutional blocks; the generator introduces Gaussian noise with a mean of 0 and a variance of 1 into the input data at the beginning of training, then gradually reduces the noise variance through an exponential decay strategy, and introduces DiffAugment enhancement data to both the real data and the generated data; Step S2: Add the generated image to the original data to improve the quality of the original dataset; Step S3: Classify the image using a lightweight CRC pathology image classification network MRD-Net model; the lightweight CRC pathology image classification network MRD-Net model includes a feature extractor module and a classifier module. The feature extractor module consists of five convolution blocks, whose channel numbers are 32, 96, 192, 384, and 512, respectively. The first convolution block uses two 3×3 convolution layers for initial feature extraction, followed by Batch Normalization, ReLU activation function, and MaxPool layer for dimensionality reduction. The second to fifth convolution blocks introduce an improved DSC module. The improved DSC module is specifically configured as follows: A 1×1 convolution is used to expand the channel dimension and introduce nonlinear mapping, followed by a 3×3 depth convolution to extract spatial features, and then a 1×1 convolution is used for channel fusion; Add the channel fusion result to the output of the first 1×1 convolution and generate the final output through the ReLU activation function; The feature extraction part finally compresses the spatial features into a fixed global representation through an AdaptiveMaxPool layer for use by the classifier module.

2. A pathological image enhancement and classification method based on an improved DCGAN training strategy as claimed in claim 1, characterized in that: The ratio of the second to fifth convolutional block modules is 1:1:3:

1.

3. A pathological image enhancement and classification method based on an improved DCGAN training strategy as claimed in claim 1, characterized in that: The classifier module includes two fully connected layers. The first fully connected layer reduces the 512-dimensional features to 256 dimensions and uses the ReLU activation function and Dropout to achieve nonlinear mapping and regularization. The second fully connected layer outputs the final classification results of the pathological image.

Images