A small sample-oriented diabetic retinopathy classification method and system

Abstract

The application discloses a kind of small sample-oriented diabetic retinopathy classification method and system, belong to image processing and medical field, wherein, classification method is by pre-training stage based on lightweight RexNet channel optimization, fundus image data preprocessing stage, data enhancement stage, model fine-tuning training stage based on transfer learning, system implementation stage composition;The application solves the problem of medical image data sample and patient privacy, breaks the barrier of insufficient DR diagnostic ability in basic community.At the same time, combined with transfer learning, pre-training is carried out in ImageNet to obtain a pre-trained model.The data is enhanced to improve the accuracy of the diabetic retinopathy classification and detection system, reduce the computing cost, assist the basic community in clinical diagnosis of diabetic retinopathy, and conduct prevention and screening diagnosis of diabetic retinopathy.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, and in particular relates to a classification method and system for diabetic retinopathy in small samples. Background Technology

[0002] Diabetic retinopathy (DR) is a serious complication of diabetes, and it is more common in the elderly. The risk of developing DR increases significantly with age. As the disease progresses, patients may experience symptoms such as blurred vision, narrowed visual field, and decreased contrast sensitivity. In more severe cases, retinal neovascularization, vitreous hemorrhage, preretinal hemorrhage, and tractional retinal detachment may occur, ultimately leading to blindness.

[0003] Regular eye exams are particularly important for screening and early diagnosis of diabetic retinopathy (DR) in the elderly population. Regular eye exams are key to detecting DR. Currently, DR screening in my country also faces significant challenges.

[0004] In the medical field, due to the need to protect patient privacy, medical data labeled with personal information is often not publicly available, leading to a degree of isolation of medical data. Furthermore, the high cost and specialized knowledge required for medical data annotation limit the large-scale expansion of medical datasets.

[0005] In terms of treatment, besides traditional laser and drug therapies, emerging medical technologies, such as the application of deep learning algorithms in medical images, offer new possibilities for the detection and diagnosis of DR (Diagnosis Related Groups). Deep learning has been widely used in medical image processing and disease-aided diagnosis, but it typically requires a large amount of medical data. However, fully annotating large amounts of data is both difficult and impractical, and image sample acquisition also presents challenges. Addressing these characteristics of medical data, transfer learning methods in few-shot learning offer a new solution for medical image processing.

[0006] The samples were reviewed and labeled according to the International Clinical Diabetic Retinopathy Disease Severity Scale (ICDRSS). The samples were divided into five categories: No DR, Mild, Moderate, Severe, and Proliferative, corresponding to 0-4 respectively.

[0007] The emergence of diabetic retinopathy (DR) detection systems has a significant effect on promoting healthy aging and bridging the "gap" in primary healthcare resources, assisting in the early detection and diagnosis of DR. Therefore, a classification method and system for DR with small sample sizes needs to be considered. This involves combining a lightweight RexNet model with optimized channel configuration with data augmentation, using transfer learning to primarily address the issues of limited medical image data samples and insufficient DR diagnostic capabilities in primary communities. Summary of the Invention

[0008] The purpose of this invention is to provide a classification method and system for diabetic retinopathy (DR) with small sample sizes, addressing the issues of limited medical image data samples and patient privacy, and breaking down the barriers of insufficient DR diagnostic capabilities in grassroots communities. Simultaneously, it incorporates transfer learning, pre-training on ImageNet to obtain a pre-trained model. Data augmentation is then applied to improve the accuracy of the DR classification and detection system, reduce computational costs, and assist grassroots communities in the clinical diagnosis and prevention screening of DR.

[0009] The technical solution proposed in this invention includes a pre-training stage based on lightweight RexNet channel optimization, fundus image data preprocessing, data augmentation stage, model fine-tuning training stage based on transfer learning, and system implementation stage.

[0010] Pre-training phase based on lightweight RexNet channel optimization:

[0011] By introducing an inverted residual structure and a linear bottleneck, the network channel design is optimized based on MobileNetV2: ① the number of channels in each layer is progressively increased; ② a non-linear activation function, Swish-1, is used for rank increase. A powerful pre-trained model is obtained by pre-training on the large image dataset ImageNet.

[0012] Fundus image data preprocessing stage:

[0013] The APTOS dataset, containing fundus images, was preprocessed. This dataset originated from the APTOS2019 Blindness Detection dataset released by Aravind Hospital in India on the Kaggle platform. The fundus images were collected under various conditions and environments; some images were very dark and difficult to visualize. To reduce the impact of lighting conditions, grayscale processing and cropping were employed. Images were converted to grayscale to reduce color interference. Regions lacking sufficient information were automatically cropped.

[0014] Data augmentation phase:

[0015] Data augmentation was performed on the preprocessed APTOS dataset, including random scaling, cropping, and interpolation of images; random horizontal flipping of images; automatic contrast adjustment; histogram equalization; image flipping and rotation; increased tonal separation; increased exposure; enhanced exposure effects; and increased color saturation, contrast, brightness, and sharpness. Shear transformation and relative translation were applied relative to the X and Y axes. Normalization was performed, and random erasure was applied to the images.

[0016] Data augmentation is performed on the preprocessed APTOS dataset, including Mixup and Cutmix, which generate new samples by combining different images and labels, improving generalization ability. Mixup creates new images by linear interpolation between two images, while the labels are mixed accordingly. Cutmixup cuts and replaces specific regions (usually rectangular regions) of the image, increasing the model's robustness to local image features. The effect of the augmentation can be controlled by setting different parameters.

[0017] Model fine-tuning training phase based on transfer learning:

[0018] Using preprocessed and augmented image data as input, a pre-trained RexNet model is invoked based on transfer learning for fine-tuning. The lightweight RexNet with optimized channel configuration is optimized using the ReLU6 activation function within the inverse linear residual block, best suited for retinal disease applications, and the AdamP optimizer, with adaptive step size adjustment. Cross-entropy loss is used to train all models for 1000 ecpoches at a batch size of 32. A cosine annealing scheduling rate is applied, with an initial learning rate of 0.001 and 20 linear warm-ups. To prevent overfitting, a weight decay rate of 0.05 is used to obtain the DR classification model.

[0019] Compared with existing inventions, the beneficial effects of the present invention are:

[0020] (1) The method proposed in this invention is based on the lightweight framework RexNet and trained on ImageNet to form a powerful pre-trained model. By optimizing the channel configuration, RexNet helps to improve the accuracy of classification and prediction of diabetic retinopathy while maintaining low computational cost.

[0021] (2) The method proposed in this invention uses extensive data augmentation to find the most discriminative local feature representations to extract DR lesions. By combining different images and labels to generate new samples, the generalization ability is improved, thereby enabling assisted diagnosis and predictive treatment.

[0022] (3) The method proposed in this invention is based on model fine-tuning of transfer learning. The input image data is preprocessed and data augmented. Based on the powerful pre-trained model, it can solve the problems of insufficient medical image data samples and insufficient DR diagnosis capabilities in grassroots communities, promote healthy aging and bridge the "gap" of grassroots medical resources, and assist in the early detection and diagnosis of DR. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the method proposed in this invention.

[0024] Figure 2 These are comparison images of data preprocessing before and after the method proposed in this invention, where the left image is before preprocessing and the right image is after preprocessing.

[0025] Figure 3 The result is the result of the processing of the present invention through steps S31-S34, wherein, from left to right, the image is the original image, the image with random mirror scaling and cropping and interpolation, the image with 50% horizontal flipping, the image enhanced by the RandAugment algorithm, the image with normalization operation and the image with random erasure.

[0026] Figure 4 The result of the present invention after step S35 is as follows: from left to right, the images are: randomly selected image 1, randomly selected image 2, and images after applying Mixup and Cutmix data enhancement with a 50% probability.

[0027] Figure 5 This is a diagram of the model structure based on transfer learning in the method proposed in this invention.

[0028] Figure 6 This is the graphical interface of the diabetic retinopathy classification system provided by the present invention. Detailed Implementation

[0029] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.

[0030] like Figure 1 As shown, this invention provides a classification method for diabetic retinopathy based on small sample sizes, with detailed steps as follows:

[0031] Step 1: Pre-training based on lightweight RexNet channel optimization

[0032] This invention first pre-trains the RexNet model on the ImageNet dataset to obtain the initial network parameters for subsequent DR tasks. Pre-training on large-scale datasets allows the model to learn some general features and prior knowledge, reducing dependence on the target training task dataset and improving training efficiency for the target task. The RexNet model is a popular network designed for ImageNet classification tasks. Addressing the representational bottleneck problem of the MobileNetV2 network, a new network design principle is proposed. RexNet employs multiple expansion layers to gradually increase the rank of the output dimension, reducing the layer representational bottleneck caused by leapfrog improvements. Furthermore, based on the higher rank characteristic of the Swish function, this activation function is introduced in both the first convolutional layer and the expansion layer.

[0033] Step 2: Preprocess the fundus image data

[0034] To address the issue of excessive black borders in images, a brightness threshold-based method is employed. Specifically, the color image is converted to grayscale, simplifying the image data and facilitating calculations. A mask matrix is ​​then created based on the obtained grayscale image and the brightness threshold. The mask matrix is ​​applied to the R / G / B color channels of the image for cropping, and the resulting images are recombined to generate a new color image.

[0035] To address irregularities in fundus images, a circular cropping method is employed to conform them to the shape characteristics of the fundus. Specifically, a circular mask matrix M(x,y) is created and bitwise ANDed with the original image I(x,y). The calculation formula is as follows:

[0036] I cropped (x,y)=I(x,y)·M(x,y)

[0037] Among them, I cropped (x,y) represents the pixel value at position (x,y) in the cropped image. M(x,y) is 1 when the distance from pixel (x,y) to the center of the circle is less than or equal to the radius r. The images before and after processing are shown below. Figure 2 As shown.

[0038] Step 3: Perform data augmentation on the preprocessed APTOS dataset.

[0039] First, the image is randomly mirrored, scaled, cropped, and interpolated. Random cropping refers to randomly sampling a rectangular region from the original image, the size of which is within a certain range (e.g., between 0.08 and 1). Then, the sampled region is scaled to a fixed size (224x224) and used as input to the network. Interpolation is a technique for converting low-resolution images to high-resolution images. It approximates missing pixel values ​​in the original image using interpolation algorithms, reconstructing a clearer image. This invention uses bicubic interpolation, which provides a smoother effect during image scaling and reduces image distortion.

[0040] Secondly, randomly flipping the image horizontally (with a 50% probability) increases the randomness of the data. Because it is done randomly, it increases the diversity and generalization ability of the data, reducing the risk of overfitting in the model.

[0041] Next, the RandAugment algorithm is used to enhance the image. This algorithm randomly selects N operations from a predefined set of enhancement operations and applies a fixed amplitude M to each operation. The configuration used in this invention is rand-m9-mstd0.5-inc1, which randomly selects 2 operations from 15 options (invert / flip / contrast enhancement, etc.), and the amplitude of each operation conforms to a normal distribution with a mean of 9 and a standard deviation of 5.

[0042] Next, the image is standardized, adjusting the pixel values ​​to a distribution centered at 0 with unit variance. This helps the model learn and generalize better. The invention also employs a random erasure method, randomly selecting a rectangular region in the image and replacing its pixel values ​​with random or preset values ​​to simulate an incomplete fundus image. The above data augmentation effects based on a single image are as follows: Figure 3 As shown

[0043] Finally, this invention randomly applies the Mixup and Cutmix methods to images in a batch with a 50% probability, and the processed images are as follows: Figure 4 As shown. Mixup can create new samples with smooth transitions, helping the model learn the boundaries between classes and reducing overfitting. Its operation can be represented as:

[0044] x ′ =λx i +(1-λ)x i

[0045] y ′ =λy j +(1-λ)y j

[0046] Where λ is a number randomly sampled from the Beta distribution, typically in the range [0,1].

[0047] Cutmix selects a local portion of the image for "paste" while simultaneously blending the corresponding labels. This method forces the model to learn different parts of the image, improving its ability to recognize local features. Its operation can be represented as:

[0048] x ′ =M⊙x A +(1-M)⊙x B

[0049] y ′ =λy A +(1-λ)y B

[0050] Here, M is a binary mask used to indicate which regions should be preserved from x. A Which regions should come from x? B λ is also a number sampled from the Beta distribution, used to determine the mixing ratio of the labels.

[0051] Step 4: Fine-tune the pre-trained model

[0052] All data processed in step three is randomly divided into training and test sets in a 4:1 ratio. The model's weights and biases are optimized on the training set, while its performance is evaluated on the test set to verify its effectiveness and generalization ability. For neural network models, selecting an appropriate optimizer, loss function, and formulating a reasonable learning rate scheduling strategy plays a crucial role in the model's efficient learning and good performance.

[0053] This invention employs the AdamP optimizer to adjust the learning rate of each parameter. It introduces weight-view momentum and adaptive learning rate scheduling, enabling it to intelligently adjust the learning speed at different training stages. Weight momentum helps the optimizer maintain direction and speed, avoiding detours during optimization; while the adaptive learning rate ensures that each parameter is updated at a rate appropriate to it. This invention sets the relative weight decay parameter of the AdamP optimizer to 0.01 and uses Nesterov momentum.

[0054] Due to Mixup and Cutmix processing, the labeled data is transformed from hard-labeled to soft-labeled. The conventional Softmax cross-loss function is no longer applicable; therefore, this invention employs a soft-label cross-loss function. Its formula is as follows:

[0055]

[0056] Where N is the number of samples, M is the number of classes, and t ij p is the probability of the j-th class among the true labels (soft labels) of the i-th sample. ijIt is the probability that the model predicts the i-th sample belongs to the j-th category.

[0057] For the selection of the learning rate scheduling strategy, this invention employs cosine annealing. A relatively large learning rate is started to avoid the model oscillating around local optima. As training progresses, the learning rate gradually decreases according to a cosine function. This process is repeated periodically. Furthermore, to ensure stable model startup, this invention also employs a warm-up strategy, with an initial learning rate of 0.000001, gradually increased to 0.001, and a warm-up period of 20.

[0058] Furthermore, to further mitigate model overfitting and enhance the representational power of single-layer networks, a spatial dropout operation was incorporated into the model. This operation randomly discards all activation values ​​across the entire feature map. The network model structure used in this invention is as follows: Figure 5 As shown.

[0059] This invention was trained on an Nvidia RTX 3090 GPU with 24GB of memory, and simulation experiments were conducted to evaluate the model's effectiveness. All models were trained for 1000 epochs, with a batch size of 32 and an image output size of 320*320. The accuracy evaluation metrics used were the area under the feature curve (AUC), accuracy (ACC), Kappa score, and F1 score. The experimental results are as follows.

[0060] It is easy to see from the table that the present invention, which uses data augmentation and fine-tuning, outperforms models that only use pre-trained models or models without data augmentation in all four evaluation metrics: AUC, ACC, Kappa, and F1 score.

[0061]

[0062] Step 5: Graphical detection interface.

[0063] This invention also establishes a simple graphical interface to provide users with user-friendly detection operations and stores user detection data for easy tracking of user health status. This graphical system is built based on the Tkinter library.

[0064] This interface provides registration and login functionality. Users log in to the system using a password to perform the test, ensuring the isolation of user data. It also offers a fundus image upload function; users can click the "upload image" button to open their local directory and upload fundus images. After the backend program completes the analysis, it displays the image and the inferred lesion type on the front end. This interface records user usage (lesion type) through a MySQL database, facilitating subsequent disease analysis and tracking. The specific interface is shown below. Figure 6 As shown.

Claims

1. A classification method for diabetic retinopathy in small sample sizes, characterized in that, The classification method includes: S1. In the pre-training stage based on lightweight RexNet channel optimization, an inverse residual structure and linear bottleneck are introduced. Based on MobileNetV2, the network channel design is optimized to obtain a pre-trained model based on lightweight RexNet channel optimization. Among them, the optimized network channel design includes progressively increasing the number of channels in each layer and using the non-linear activation function Swish-1 to increase the rank. S2. In the fundus image data preprocessing stage, the data comes from the publicly available APOTS dataset. Preprocessing is performed on the image data with uneven light sources to obtain the preprocessed dataset. Preprocessing is performed separately for fundus images with redundant black borders and irregularities. For the case of redundant black borders, a brightness threshold-based method is used to solve the problem. Specifically, the color image is converted to a grayscale image to simplify the image data. A mask matrix is ​​established based on the obtained grayscale image and brightness threshold. The mask matrix is ​​then applied to the R / G / B color channels of the image for cropping, and the images are recombined to generate new color images. To address irregularities in fundus images, a circular cropping method is employed to conform to the shape characteristics of the fundus. Specifically, a circular mask matrix M(x,y) is created and bitwise ANDed with the original image I(x,y). The calculation formula is as follows: I cropped (x,y)=I(x,y)·M(x,y) Among them, I cropped (x,y) is the pixel value at position (x,y) of the cropped image. M(x,y) has a value of 1 when the distance from pixel (x,y) to the center of the circle is less than or equal to the radius r. S3, Data Augmentation Stage: Through data augmentation, the most discriminative local feature representations are found to extract diabetic retinopathy and improve the performance of retinal vessel segmentation; new samples are generated by combining different images and labels to improve generalization ability. Step S3 includes randomly scaling, cropping, and interpolating the image; randomly flipping the image horizontally; automatic contrast adjustment; histogram equalization; image flipping; rotation; increasing tonal separation; increasing exposure; increasing color saturation, contrast, brightness, and sharpness; applying shear transformation and relative translation on the XY axes; normalization; and randomly erasing the image. Step S3 also includes Mixup and cutmix, which generate new samples by combining different images and labels; Mixup creates new images by linear interpolation between two images, while the labels are mixed accordingly; Cutmixup cuts and replaces specific regions of the image; Step S3 specifically includes the following steps: S31. Randomly mirror, scale, crop, and interpolate the image; random cropping involves randomly sampling a rectangular region from the original image; then scaling the sampled region to a fixed size of 224x224 as the input to the network; and using bicubic interpolation to provide a smoother effect during image scaling and reduce image distortion. S32. Randomly flip the image horizontally with a 50% probability to increase the diversity and generalization ability of the data and reduce the risk of overfitting of the model. S33 uses the RandAugment algorithm to enhance the image. It randomly selects N operations from a predefined set of enhancement operations and applies a fixed amplitude M to each operation. The configuration used is rand-m9-mstd0.5-inc1, which randomly selects 2 operations from 15 operations, and the amplitude of each operation conforms to a normal distribution with a mean of 9 and a standard deviation of 5. S34 performs a standardization operation on the image, adjusting the pixel values ​​to a distribution centered at 0 with unit variance; and uses a random erasure method to randomly select a rectangular area in the image and replace its pixel values ​​with random or preset values ​​to simulate an incomplete fundus image. S35, with a 50% probability, randomly applies the Mixup and Cutmix methods to images in the batch; Mixup creates new samples with smooth transitions, which helps the model learn the boundaries between categories and reduces overfitting; its operation is represented as: x ′ =λx i +(1-λ)x i and ′ =λy j +(1-λ)y j Where λ is a number randomly sampled from the Beta distribution, in the range [0,1]; Cutmix selects local portions of the image for "paste" while simultaneously blending labels accordingly; this forces the model to learn different parts of the image, improving its ability to recognize local features. Its operation is represented as follows: x ′ =M⊙x A +(1-M)⊙x B and ′ =λy A +(1-λ)y B Here, M is a binary mask used to indicate which regions should be preserved from x. A Which regions should come from x? B λ is a number sampled from the Beta distribution, used to determine the mixing ratio of the labels; S4. In the model fine-tuning training stage based on transfer learning, the pre-processed and data-enhanced image data are used as input. The pre-trained model RexNet is called based on transfer learning to fine-tune the model and obtain a classification model for diabetic retinopathy with small sample sizes.

2. The classification method for diabetic retinopathy based on small sample sizes according to claim 1, characterized in that, In step S1, an inverse residual structure and a linear bottleneck are introduced, and the network channel design is optimized based on MobileNetV2; pre-training is performed on the large image dataset ImageNet to obtain a pre-trained model.

3. The classification method for diabetic retinopathy based on small sample sizes according to claim 1, characterized in that, In step S4, the pre-trained model based on lightweight RexNet channel optimization from step S1, as well as the preprocessing and data augmentation of the APOTS dataset completed in steps 2 and 3, are used as input. The pre-trained RexNet model is called based on transfer learning to fine-tune the model and obtain a classification model for diabetic retinopathy with small sample size.

Images