OCT retina image classification method based on improved ResNet-34 network

Abstract

The invention discloses an OCT retina image classification method based on an improved ResNet-34 network, and relates to the technical field of retina medical image classification, and the method comprises the steps: obtaining a retina OCT image data set, and carrying out the preprocessing of an image; an improved ResNet-34 network model is constructed, a CBAM attention module is embedded in a residual block of the improved ResNet-34 network model, a CBAM-Block structure is formed, a channel attention module and a space attention module are combined in a series connection mode, channel feature response of a focus area is self-adaptively enhanced, and a key space position is focused; dynamically adjusting floating point calculation precision in a network training process by adopting an automatic mixing precision training technology; pre-training weight initialization network parameters based on an ImageNet data set are loaded, and the model is finely adjusted through transfer learning; and training a network by using the preprocessed data set, and outputting a retina pathological state classification result. According to the method, through a triple collaborative optimization mechanism of a CBAM-Block residual structure, automatic mixing precision training and transfer learning, the precision and efficiency of retina OCT image classification are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of retinal medical image classification technology, and in particular to an OCT retinal image classification method based on an improved ResNet-34 network. Background Technology

[0002] Retinal diseases are a leading cause of vision impairment and blindness worldwide. Early and accurate diagnosis is crucial for clinical treatment, as it can slow disease progression and protect patients' vision. Optical coherence tomography (OCT), a non-invasive, high-resolution fundus imaging technique, can clearly reveal the fine structures of each layer of the retina, making it a key tool for retinal disease screening and classification. However, OCT images are complex in structure and rich in inter-layer information, and traditional manual interpretation relying on physician experience has significant drawbacks: differences in physician skill and experience lead to strong subjective diagnosis, easily resulting in misdiagnosis or missed diagnosis; manually reviewing each image is time-consuming and laborious, making it difficult to meet the timeliness requirements of large-scale screening; image acquisition is susceptible to noise interference introduced by equipment, operation, and environmental factors, reducing image quality and increasing the difficulty and error of interpretation. To overcome the limitations of manual interpretation, automated classification methods based on deep learning have become the mainstream research approach. Convolutional neural networks (CNNs) can automatically extract discriminative features from complex OCT images, and after training with a large amount of labeled data, they can achieve automatic disease classification, improving diagnostic objectivity and efficiency.

[0003] Current technologies focus on integrating attention mechanisms, transfer learning, and feature fusion modules into classic CNN architectures to enhance the model's focus on lesion regions and global semantic understanding. Examples include hybrid model architectures combining CNNs and transformers, introducing channel and spatial attention mechanisms, and employing multi-scale feature extraction strategies. While existing methods have significantly improved classification performance, they still face key challenges: complex model structures and large parameter counts require more computational resources for training and inference, leading to overfitting and impacting generalization ability; high computational overhead limits application deployment in primary healthcare institutions, making it difficult to meet the needs of real-time clinical diagnosis; and they are sensitive to data imbalance, resulting in decreased classification performance for a few disease classes. Specifically, while hybrid CNN and transformer structures improve global context modeling capabilities, the dual-branch design increases the number of parameters and reduces inference efficiency; methods combining visual transformers and efficient convolutional modules reduce computational complexity, but modeling local details relies on complex attention mechanisms, resulting in insufficient noise resistance; multi-model feature fusion strategies improve classification accuracy, but integrating multiple models increases storage and computational burdens, hindering real-time clinical deployment.

[0004] Overall, existing OCT image automatic classification methods need improvement in balancing accuracy and efficiency, achieving lightweight models, and enhancing noise robustness. Clinically, there is an urgent need for a classification model that combines high classification accuracy, efficient computational power, a simple structural design, and strong noise resistance to meet the needs of automated diagnosis and provide technical support for the early diagnosis and treatment of retinal diseases. Summary of the Invention

[0005] In view of this, the present invention proposes an OCT retinal image classification method based on an improved ResNet-34 network to solve the problem that existing OCT image classification techniques are difficult to balance accuracy and efficiency.

[0006] The specific technical solution of this invention is as follows: An OCT retinal image classification method based on an improved ResNet-34 network includes: Acquire a dataset of retinal OCT images and preprocess the images, including size normalization, noise suppression, and data augmentation. An improved ResNet-34 network model was constructed, and the residual block was improved into a CBAM-Block structure. This structure incorporates the convolutional block attention mechanism CBAM, which adaptively enhances the channel feature response of the lesion region and focuses on key spatial locations through the concatenation of the channel attention module and the spatial attention module. Automatic mixed-precision training technology is used to dynamically adjust the floating-point calculation precision during network training, using a combination of single-precision floating-point numbers and half-precision floating-point numbers to optimize memory usage and computational efficiency. Load pre-trained weights based on the ImageNet dataset to initialize network parameters, and fine-tune the model through transfer learning; The network was trained using the preprocessed dataset and output classification results for retinal pathological conditions, including choroidal neovascularization, diabetic macular edema, drusen, and normal categories.

[0007] Specifically, the CBAM-Block structure adds the Convolutional Block Attention (CBAM) mechanism to the original residual block, so that the input obtained by the residual block sequentially executes the channel attention module and the spatial attention module. The channel attention module generates channel description vectors in parallel through global average pooling and global max pooling, and generates channel weight maps after learning the inter-channel dependencies through a multilayer perceptron. The spatial attention module generates two-channel feature maps based on the feature maps after channel attention weighting, and performs average pooling and max pooling operations along the channel dimension respectively. It generates a two-dimensional spatial attention weight map through an activation function. The final output is the result of multiplying the original feature map with the channel attention weights channel by channel and then multiplying it with the spatial attention weights position by position.

[0008] Specifically, automatic mixed-precision training includes: during forward computation and backpropagation, convolutional layer weights and activation values ​​are stored using half-precision floating-point numbers to accelerate computation; single-precision floating-point numbers are used in the weight update stage to maintain numerical stability; and dynamic loss scaling technology automatically adjusts gradient magnitudes to solve the gradient vanishing problem under half-precision floating-point numbers.

[0009] Specifically, data augmentation operations in the data preprocessing stage include: random horizontal flipping, vertical flipping, and rotation operations to expand sample diversity; random adjustment of brightness and contrast to simulate differences in the imaging environment; and, in response to the problem of sample imbalance, an oversampling strategy is used to increase the frequency of repeated training for minority class samples.

[0010] Specifically, the transfer learning fine-tuning process is as follows: retain all convolutional layer weights of ResNet-34 except for the fully connected layers; replace the fully connected layers with a four-class classification output structure and reset their weights using random initialization; and perform end-to-end training using the cross-entropy loss function and the adaptive moment estimation optimizer.

[0011] Specifically, the computation process of the channel attention module includes: performing global average pooling and global max pooling on the input feature map to generate two channel description vectors; inputting the two vectors into a multilayer perceptron with shared weights, adding the outputs and generating a channel weight vector through an activation function; and multiplying the channel weight vector with the original feature map channel by channel.

[0012] Specifically, the computation process of the spatial attention module includes: performing max pooling and average pooling on the output feature map of the channel attention module along the channel dimension to generate two spatial feature maps; concatenating the two feature maps along the channel dimension and generating a spatial weight map through a convolutional layer; and multiplying the spatial weight map with the output feature map of the channel attention module pixel by pixel.

[0013] Specifically, the residual structure is improved by embedding a CBAM module at the end of the identity mapping branch of the ResNet-34 basic residual block, which is then improved into a CBAM-Block residual block; the residual addition operation is performed between the CBAM-Block output and the original input.

[0014] Specifically, the fine-tuning operation includes two-stage training: the first stage freezes all network layers except the final fully connected layer and only updates the classification head parameters; the second stage unfreezes all network layers and performs end-to-end joint optimization.

[0015] Specifically, in the classification result generation stage, the Softmax function is used to normalize the output of the fully connected layer, and the classification accuracy of each category is used as the final pathological state diagnosis result.

[0016] The beneficial effects of this invention are as follows: Firstly, in terms of model architecture, through lightweight internal modifications to CBAM-Block, fine-grained feature extraction and fusion capabilities comparable to complex models are achieved without significantly increasing network depth and width, thus achieving a better balance between accuracy and efficiency.

[0017] Secondly, in terms of training optimization, AMP technology is explicitly introduced as the core training acceleration method. This is different from most studies that only focus on improving the network structure while ignoring the efficiency of the training process itself. It directly optimizes resource utilization from the perspective of computational accuracy and is more suitable for processing large-scale medical image data.

[0018] Third, in terms of robustness construction, a collaborative strategy of "preprocessing enhancement (solving noise and imbalance) + transfer learning (providing general feature priors) + attention mechanism (improving feature discrimination power)" is adopted to form a systematic solution, rather than relying solely on data augmentation or a single attention module to deal with data defects. This results in more stable and superior classification performance in noisy environments and scenarios with uneven sample distribution. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the OCT retinal image classification method based on the improved ResNet-34 network of the present invention. Figure 2 This is a schematic diagram of four categories of instance images in the OCT-2017 dataset of this invention, where a is a normal image, b is a CNV patient image, c is a DME patient image, and d is a Drusen patient image; Figure 3 This is a flowchart illustrating the data preprocessing process of the present invention; Figure 4 This is a schematic diagram of the CBAM module of the present invention; Figure 5 This is a schematic diagram of the CBAM-Block structure of the present invention; Figure 6 Here are the confusion matrix and ROC curve of the classification network model of this invention, where a is the confusion matrix and b is the ROC curve; Figure 7 This is a bar chart showing the ablation experimental results of the attention mechanism of this invention. Detailed Implementation

[0021] To make the technical problems, solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0022] This invention proposes an OCT retinal image classification method based on an improved ResNet-34 network. This method is specifically designed for the automatic classification of retinal optical coherence tomography (OCT) images, aiming to efficiently identify four retinal pathological states: CNV, DME, Drusen, and normal. The core of this method lies in the systematic integration of three key improvements: First, a CBAM-Block residual block is designed, embedding the convolutional block attention mechanism (CBAM) into the residual structure of the ResNet-34 network. This allows the network to adaptively focus on key lesion regions during a single forward pass, simultaneously capturing local details and global contextual semantics, thus replacing multi-branch feature extraction or multi-model fusion strategies. Second, an automatic mixed-precision training technique is introduced. By dynamically mixing FP16 and FP32 calculations, memory usage is significantly reduced, training speed is improved, and batch processing capability is enhanced while maintaining numerical stability. Finally, transfer learning is performed using ImageNet-based pre-trained weights, and a targeted preprocessing process is adopted, including median filtering, normalization, and minority class-oriented data augmentation, to jointly enhance the model's robustness and generalization ability to imbalanced data and acquisition noise. The overall flowchart is shown below. Figure 1 As shown, the classification process begins with data preparation, which is then preprocessed and input into an improved network to output category prediction results, ultimately achieving high-precision and high-efficiency clinically applicable classification.

[0023] The dataset used is the OCT-2017 dataset, generated using optical coherence tomography (OCT) technology and specifically designed for automated diagnosis research of retinal diseases. OCT is a non-invasive imaging technique that provides high-resolution images of retinal cross-sections and is widely used in the diagnosis and monitoring of ophthalmic diseases. This dataset includes a total of 84,484 retinal OCT images with a resolution of 496×512 pixels, covering four pathological categories: normal, diabetic macular edema (DME), choroidal neovascularization (CNV), and drusen. The category sample distribution is shown in Table 1. Table 1 shows the sample size for each category in OCT-2017.

[0024] The training set contains 37,197 samples for CNV, 11,340 for DME, 8,608 for Drsen, and 26,307 for Normal. The validation set contains 8 samples for each category, and the test set contains 250 samples for each category. This distribution shows that the number of samples in the DME and Drsen categories is significantly less than that in the CNV and Normal categories, leading to class imbalance, which may affect the model's classification performance. The images in the dataset are from real clinical environments, possessing high diversity and representativeness. Each image has been annotated by professional physicians to ensure data accuracy and reliability. Normal retinal images display typical features of healthy structures. DME (Dampness-Induced Mucus Effusion) manifests as fluid accumulation and swelling within the retina, commonly seen in diabetic patients. CNV (Collapse-Induced Vascular Venous) is characterized by abnormal neovascularization of the choroid. These vessels may penetrate the retinal pigment epithelium (RPE) and enter the subretinal space, leading to fluid leakage, hemorrhage, or scarring, and are commonly seen in age-related macular degeneration (AMD). Drusen formation is associated with dysfunction of RPE cells, which are responsible for clearing metabolic waste from the outer segments of the retina. However, in aging or pathological conditions, the clearance capacity of RPE cells decreases, leading to the accumulation of metabolic waste (such as lipids and proteins) between the Bruch membrane and the RPE. These accumulations may trigger local inflammation. Example images for each category are shown below. Figure 2 As shown. This dataset serves as a high-quality benchmark platform, widely used in tasks such as classification and lesion detection, and its openness supports the training and evaluation of this method.

[0025] In the preprocessing stage, optimizations are made to address inherent defects in the dataset; the process is as follows: Figure 3As shown. First, the white edge removal process addresses invalid edge information caused by irregular acquisition: by setting a dynamic threshold, white edge areas are identified and cropped to eliminate background interference. Second, median filtering is applied to remove noise: because the OCT imaging process is easily affected by equipment noise or tissue shedding, resulting in isolated black spots or abnormal pixels, a median filtering algorithm is used to smooth the image, preserve edge details, and ensure the accuracy of the analysis. Next, the resolution is adjusted to unify the input size: since the original image sizes are inconsistent, a bicubic interpolation algorithm is used to fix the resolution to 224×224×3 (a three-channel color image with a width and height of 224 pixels) to adapt to the input requirements of deep learning networks. Then, normalization is performed: all images are standardized, and the mean values ​​of the RGB channels are set to 0.485, 0.456, and 0.406, respectively. Similarly, the standard deviations are fixed to a certain value to unify the data distribution, facilitating subsequent classification training. Finally, data augmentation was implemented to address the class imbalance problem shown in Table 1: Specifically for the minority classes DME and Drusen, random horizontal flipping, random vertical flipping, scaling, and brightness adjustment were applied to expand sample diversity through geometric and color space transformations, balance data distribution, and improve model generalization ability; augmentation was performed only on the training set, while the validation and test sets remained unchanged to ensure fair evaluation.

[0026] The network architecture is based on the ResNet-34 backbone model and has been improved to a CA-OCT architecture, as shown in Table 2: Table 2 shows the ResNet-34 network model architecture.

[0027] This architecture integrates CBAM within residual blocks and enhances the model's feature extraction capabilities and optimizes computational efficiency through transfer learning and AMP techniques. The overall structure includes an input layer, convolutional layers, pooling layers, and multiple residual block sequences. Specifically, the input is a preprocessed image (224×224×3), which passes through an initial convolutional layer Conv1 (7×7 kernel, 64 channels, stride 2) to output a 112×112 feature map, and then through a 3×3 max pooling layer (stride 2) to output a 56×56 feature map. Subsequently, the serialized residual block groups Conv2_x to Conv5_x process the features: Conv2_x contains 3 CBAM-Block residual blocks, outputting a 56×56 feature map with n=64 channels; Conv3_x contains 4 CBAM-Blocks, outputting a 28×28 feature map with n=128; Conv4_x contains 6 CBAM-Blocks, outputting a 14×14 feature map with n=256; and Conv5_x contains 3 CBAM-Blocks, outputting a 7×7 feature map with n=512. Finally, the feature maps are dimensionality-reduced through a global average pooling layer (1×1), output as a 1000-dimensional vector through a fully connected layer, and then processed by a softmax activation function to generate four probability distributions, completing the classification. This architecture solves the gradient vanishing problem through skip connections, while CBAM enhances the feature extraction capability.

[0028] In deep learning, attention mechanisms are techniques that mimic the human visual focusing mechanism, enabling models to automatically focus their attention on the parts most relevant to the current task when processing input information. Different types of attention mechanisms are used in different application scenarios, primarily categorized as self-attention mechanisms, multi-head self-attention mechanisms, channel attention mechanisms, and spatial attention mechanisms. Self-attention mechanisms effectively capture long-distance dependencies by dynamically calculating and weighting the correlations between sequence elements. Multi-head self-attention mechanisms capture diverse dependencies in different feature subspaces of the input sequence by computing multiple attention heads in parallel, fusing the outputs of each head and performing a linear transformation to enhance the model's representational capabilities. Channel attention mechanisms (such as the SE module in SENet) dynamically adjust the weights of each channel by modeling the dependencies between channels, thereby enhancing feature representation capabilities. Spatial attention mechanisms, by introducing an attention module, enable the model to adaptively learn the importance weights of different spatial regions in the input feature map, thus focusing more on key regions and suppressing irrelevant background information.

[0029] Among them, CBAM is a typical hybrid attention mechanism model, which integrates channel attention and spatial attention modules. The CBAM structure diagram is shown below. Figure 4 As shown, CBAM significantly enhances the convolutional neural network's ability to extract image features by sequentially applying channel attention and spatial attention modules, respectively, to weight and optimize the feature maps from the channel and spatial dimensions. This dual attention mechanism enables the model to more accurately capture important information in images, improving its performance in visual tasks.

[0030] ResNet (Residual Network) is a deep neural network architecture proposed by Microsoft Research Asia. Its core strength lies in solving the vanishing and exploding gradient problems during deep network training through residual connections, allowing for deeper and more powerful networks. Residual networks are characterized by ease of optimization and the ability to improve accuracy by significantly increasing depth. Internally, residual blocks utilize skip connections to mitigate the vanishing gradient problem inherent in increasing depth in deep neural networks.

[0031] The CA-OCT model uses a ResNet-34 network with a 34-layer architecture. CBAM is added to the core module residual block to form a CBAM-Block, as shown in the diagram. Figure 5 As shown, n is the number of channels. The main branch of this residual structure consists of two 3×3 convolutional layers and a CBAM. The dashed line on the right side of the residual structure, directly connecting the input to the output, is the shortcut branch. A 1×1 convolutional layer is also added to the shortcut branch, restoring its height and width to their original values. The feature matrix obtained after a series of operations on the main branch is first passed through the CBAM and then added to the input feature matrix. Finally, it is activated by the ReLU function to obtain the final output.

[0032] Traditional deep networks face the vanishing gradient problem as the number of layers increases, making training difficult. Residual blocks, through skip connections, enable the network to learn the residuals between input and output, reducing optimization difficulty. At the same time, skip connections provide a shortcut for gradient backpropagation, avoiding gradient decay layer by layer during backpropagation. CBAM-Block enhances the model's feature extraction capabilities by simultaneously capturing global and local information through a dual attention mechanism.

[0033] Automatic Mixed Precision (AMP) is a deep learning training optimization technique that significantly improves training efficiency and reduces memory usage by combining single-precision (FP32) and half-precision (FP16) floating-point numbers, while maintaining model accuracy and stability. Its core principle lies in leveraging the high computational efficiency and low memory requirements of FP16, while ensuring the numerical stability of critical computational steps through FP32.

[0034] AMP's core mechanism comprises two parts: mixed-precision computation and dynamic loss scaling. In mixed-precision computation, the model's forward and backward propagation primarily utilizes FP16 to accelerate computation and reduce memory usage. However, the small numerical range of FP16 can easily lead to gradient underflow or overflow, thus affecting training stability. To address this, AMP introduces dynamic loss scaling, multiplying the loss function by a scaling factor to amplify gradient values ​​and prevent underflow, while scaling the gradients back to their original range during weight updates. Furthermore, AMP dynamically adjusts the scaling factor, automatically balancing numerical stability and computational efficiency based on whether gradient values ​​overflow or underflow.

[0035] The main advantages of AMP are as follows: First, FP16 is typically 2-8 times faster than FP32, significantly accelerating the model training process. Second, FP16 uses only half the GPU memory of FP32, allowing AMP to support larger batch sizes or more complex model structures, thereby improving the model's expressive power. Finally, through dynamic loss scaling and mixed-precision computation, AMP can maintain model accuracy while accelerating training, avoiding performance degradation caused by numerical instability.

[0036] AMP combines the efficient computation of FP16 with the numerical stability of FP32, providing significant acceleration and memory optimization for deep learning training, effectively solving the numerical underflow problem of FP16 while maintaining the accuracy of the model.

[0037] Transfer learning is a key technique in deep learning. Its core idea is to accelerate the training process of the target task and improve model performance by transferring knowledge learned from one task (source task) to another related task (target task). In computer vision tasks, transfer learning typically uses a model pre-trained on a large-scale dataset (such as ImageNet) as a starting point, applying it to a new task through fine-tuning or feature extraction. This method is particularly suitable for target tasks with limited data, such as medical image classification and object detection.

[0038] In the ResNet-34 network model, transfer learning typically involves the following steps: First, a ResNet-34 model pre-trained on the ImageNet dataset is loaded to leverage its powerful feature extraction capabilities. Second, the first eight convolutional layers are frozen, preventing weight updates during training and preserving the general features learned by the pre-trained model. This effectively reduces the number of training parameters, lowers computational costs, and avoids overfitting issues that may occur when training on small datasets. Furthermore, since the weights of the pre-trained model have already been optimized on large-scale datasets, transferring its feature extraction capabilities to new tasks can significantly accelerate model convergence on those new tasks.

[0039] The advantage of transfer learning lies in its ability to significantly reduce the amount of data required for the target task while improving the model's generalization ability and training efficiency. Especially in fields where data acquisition is difficult, such as medical image analysis, transfer learning can leverage the powerful feature extraction capabilities of pre-trained models to train high-performance models under limited data conditions. Furthermore, transfer learning can reduce computational costs and training time, making it widely valuable in practical applications.

[0040] In the ResNet-34 network model, the CBAM attention mechanism, Automatic Mixed Precision (AMP), and transfer learning module are incorporated, and improvements are made using algorithms such as cross-entropy loss function and Adam optimizer. This section discusses the experimental setup and empirical evaluation of the proposed network model using training, validation, and testing datasets. Objective performance metrics are used to quantitatively evaluate the model (its performance is visually presented using visualization techniques), and comparative analyses with other methods are conducted to highlight its advantages and disadvantages. To further explore the model's internal mechanisms, interpretability analysis is performed, deeply dissecting the model's decision-making process. Furthermore, ablation experiments are used to verify the contribution of each component to the model's performance, thus confirming the effectiveness of the proposed method.

[0041] In the experiments, the proposed network model was trained and tested on a Windows 10 operating system with 64 GB of memory, an NVIDIA 4090 24 GB GPU, a 2 TB solid-state drive, Python 3.7, and PyTorch 1.10.1 + cu 102. The implementation of NVIDIA CUDA and its CUDNN library significantly improved training time and overall performance. Based on the OCT-2017 dataset, the proposed network model was trained, validated, and tested using the hyperparameters set in Table 3. Based on the evaluation results of the model and data subsets, a set of initial parameter values ​​was selected to optimize the hyperparameter selection.

[0042] Table 3 shows the experimental hyperparameter settings.

[0043] To evaluate the model's generalization ability, initial values ​​were used for training and validation. A grid search system was used to systematically explore parameter combinations and identify the optimal hyperparameters. Model performance was continuously monitored during the search process, and the best-performing hyperparameter configuration was selected to ensure the model's effectiveness and robustness for specific datasets and tasks. During training, the model with the highest accuracy and lowest loss function was saved, and the model with the highest test accuracy was selected as the optimal model through comparison.

[0044] In the field of fundus image classification, accuracy, precision, recall, and F1 call are commonly used to evaluate the classification performance of a model. The formulas for these evaluation metrics are shown below: Wherein, true positive (TP) represents the number of samples correctly predicted as positive; true negative (TN) represents the number of samples correctly predicted as negative; false positive (FP) represents the number of samples incorrectly predicted as positive; and false negative (FN) represents the number of samples incorrectly predicted as negative.

[0045] Additionally, the ROC curve is a graph used to evaluate the performance of a classification model, demonstrating its diagnostic capabilities by comparing its ability to identify positive and negative classes. The ROC curve is plotted with the False Positive Rate (FPR) on the horizontal axis and the True Positive Rate (TPR) on the vertical axis. An ideal classifier's ROC curve will be positioned in the upper left corner, indicating that it achieves the maximum TPR with the maximum FPR.

[0046] The formulas for calculating FPR and TPR are as follows: The area under the ROC curve is the AUC, which ranges from 0 to 1. AUC values ​​closer to 1 indicate better classification performance; an AUC value of 0.5 indicates performance equivalent to random guessing. ROC curves and AUC provide a direct way to evaluate and compare the performance of different models.

[0047] The proposed network model was trained using the OCT-2017 dataset, and evaluated using the classification results on the test set. Table 4 shows the classification results of the network model for the four categories in the dataset.

[0048] Table 4 shows the classification results of the CA-OCT network model.

[0049] As can be seen from the table, the network model proposed in this invention achieved excellent results in the classification experiments for all four categories. The Recall for the CNV category and the AUC for all four categories reached 1, indicating that the network model performed the preset classification task exceptionally well. The Drusen category performed slightly worse than the other three categories, with the lowest Precision, Recall, and F1 score among the four categories. Analysis of the confusion matrix provides deeper insights into the model's classification accuracy, as it reveals the distribution of correctly predicted categories and misclassified categories. Figure 6 The confusion matrix shows that the classification performance in the Drusen category is slightly worse, with 6 photos being misclassified.

[0050] Furthermore, ROC curves are a valuable tool for assessing the trade-off between true positives and false positives at different decision thresholds. Figure 6 It can be observed that the ROC curves for all four categories indicate that the model has the best classification performance, with an AUC of 1 for each category, demonstrating that the model has extremely excellent classification ability.

[0051] In this study, the present invention demonstrates through a series of ablation experiments that the proposed model possesses the best and most efficient configuration. Specifically, ablation experiments were conducted on the attention mechanism, AMP, transfer learning, and network architecture. The robustness of the network architecture was verified by feeding both the original dataset and a noisy dataset into the network model. The specific ablation experiment settings and results are shown below.

[0052] In the ablation experiment of AMP, as shown in Table 5, it can be seen that after introducing the automatic mixing precision module, the memory usage of the model is significantly reduced, the calculation speed is significantly improved, and the classification accuracy of the model is effectively maintained.

[0053] Table 5 shows the ablation test results for AMP.

[0054] Automatic mixed-precision technology significantly improves computational efficiency by combining single-precision and half-precision floating-point numbers, while maintaining numerical stability. This feature is particularly important in medical image processing tasks, as medical images typically have high resolution and require substantial computational resources. Experimental data shows that after adding the AMP module, the model's processing time for each OCT fundus image was significantly reduced from 3.4ms to 4.2ms, while classification performance was further improved, with accuracy increasing from 0.9730 to 0.9890. Furthermore, other evaluation metrics for all four categories also saw significant optimization.

[0055] In the ablation experiment of transfer learning, as shown in Table 6, the experimental results show that after introducing the transfer learning module, the classification accuracy of the model is significantly improved and the processing speed is greatly accelerated.

[0056] Table 6 shows the ablation experiment results for transfer learning.

[0057] Transfer learning significantly accelerates the convergence speed of the model on new tasks and improves its generalization ability on the target dataset by utilizing model weights pre-trained on large-scale datasets. This result demonstrates that transfer learning not only effectively reduces training time but also enhances the model's classification performance on OCT images by preserving the general features of the pre-trained model, improving the classification score from 0.8950 to 0.9890.

[0058] This invention conducts ablation experiments on attention mechanisms: experiments are conducted from five aspects: no attention mechanism, self-attention mechanism, multi-head self-attention mechanism, channel self-attention mechanism, and spatial self-attention mechanism used in this invention. All other network settings remain unchanged. The experimental results are shown in Table 7.

[0059] Table 7 shows the results of ablation experiments targeting the attention mechanism.

[0060] from Figure 7 As can be seen from the bar chart, the CBAM chosen in this invention has significant advantages. Introducing an attention mechanism significantly enhances the model's performance in classification tasks, improving its ability to capture key information by dynamically assigning weights to different input features. Although this mechanism introduces additional computational complexity, leading to increased processing time per sample, this trade-off results in improved overall classification performance, manifested as higher accuracy and stronger generalization ability. The self-attention mechanism also performs well, but it is still inferior to the spatial self-attention mechanism chosen in this invention.

[0061] To verify the robustness of the proposed network model to noise, Gaussian noise was added to the OCT-2017 dataset. The average peak signal-to-noise ratio (PSNR) between the original OCT-2017 dataset and its noisy version was 24.21. The noisy dataset was then fed into a well-trained model for testing, and the training results were compared with the original results. As shown in Table 8, it can be observed that the proposed model can achieve comparable performance on both the noisy and original datasets, demonstrating a certain degree of noise resistance.

[0062] Table 8 shows the ablation experiment results regarding the robustness of the network model.

[0063] Ablation experiments revealed that the CA-OCT network model significantly improved its computational efficiency and classification performance by modifying the original residual structure in the ResNet-34 network to CBAM-Block residual blocks and introducing transfer learning and the AMP algorithm. This also enhanced the model's practicality and robustness in the OCT retinal image classification task.

[0064] To verify the performance advantages of the proposed network classification model, a systematic experimental evaluation was conducted on the OCT-2017 dataset. As shown in Table 9, this invention selected several advanced models as benchmarks and compared and analyzed the core indicators such as classification accuracy of each model.

[0065] Table 9 shows the results of comparison with state-of-the-art models using the same OCT-2017 dataset.

[0066] Experimental results show that, compared with other state-of-the-art methods, the proposed multi-module ensemble model CA-OCT exhibits significant advantages across multiple evaluation dimensions. All five methods cited employ deep learning models for training. HCTNet, a hybrid ConvNet-Transformer network, first extracts low-level features using dense residual blocks, then designs two parallel branches of Transformer and ConvNet to acquire global semantics, and finally uses a feature fusion module based on an adaptive weighting mechanism to fuse the global and local features obtained in the above steps. By incorporating a self-attention mechanism, the Transformer can better integrate context and process the entire sequence in parallel, resulting in higher efficiency. While its results are slightly inferior to the CA-OCT network designed in this invention, it significantly optimizes the classification performance of CNN networks. The MedViT network also employs a network model combining Transformer and CNN. It designs an Efficient Convolution Block (ECB) to capture local features and proposes a Local Transformer Block (LTB). This module, by combining Efficient Self-Attention (ESA) and Multi-Head Convolutional Attention (MHCA), reduces computational complexity while separating and fusing multi-frequency features of the image (low-frequency global context and high-frequency local details), enhancing the model's ability to model long-range dependencies and local structures in medical images. Simultaneously, Patch MomentumChanger (PMC) data augmentation technology effectively improves the model's robustness. CM-CNN first utilizes segmentation masks to enhance feature extraction, while simultaneously preserving lesion and background information through the mask and its complementary mask. Furthermore, by combining the segmentation network U-Net with Class Activation Maps (CAM), the lesion region is highlighted, making segmentation more efficient. This paper also uses the Dice loss function and cross-entropy loss function to optimize global classification performance, which can improve the robustness and accuracy of classification and better complete complex classification tasks. The EOCT model classifies OCT images based on the ResNet-50 network model and the random forest algorithm, and adds dual optimizers of SGD and Adam to help improve the convergence speed of the model, making the ResNet-50 network model perform better. The combination of Adam optimizer and ResNet-50 network model can more effectively capture information in OCT images. It is worth noting that the FN-OCT network performs multi-model feature extraction and feature fusion, combining three classifiers: InceptionV3, Inception-ResNet, and Xception, and adding CBAM after each classifier. At the same time, it uses transfer learning to load ImageNet pre-trained weights.Three different fusion strategies were employed to combine three classifiers, and the final classification results for four types of OCT images—CNV, DME, Drusen, and Normal—were output and compared separately. The optimal classification result achieved an accuracy of 0.9870 and an AUC of 0.9910, which is very close to the classification results of the CA-OCT network designed in this invention. The research approach of multi-model feature extraction and feature fusion, employing different fusion strategies, is worthy of further study and research.

[0067] After comparing with similar research methods, it can be found that most authors in the OCT classification field are good at using multi-task ensemble, feature fusion, and attention mechanisms to optimize network models to complete classification tasks. However, while improving the network structure to achieve the predetermined goal, it also leads to complex model structure, large computational load, and low efficiency. The CA-OCT designed in this invention can effectively optimize resource utilization and simplify the network model as much as possible to improve computational efficiency by preloading ImageNet pre-trained weights and freezing the first 8 layers and adding AMP technology. The addition of AMP technology and transfer learning reduces the processing time of each image by 0.8ms and 0.4ms, respectively, and improves the accuracy by 1.60% and 9.40%, respectively. This is more conducive to CA-OCT to complete classification tasks on large-scale datasets and better meets the requirements of efficient and accurate clinical diagnosis. At the same time, the design of CBAM-Block residual blocks allows the network to adaptively focus on lesion areas and capture both local and global information, improving the network's classification ability and enhancing the model's ability to identify complex fundus lesions (such as Drusen). By simply modifying the residual block structure, the task of fusing global and local features—which is required for the fusion of multi-classifiers and feature fusion in the rest of the network—is accomplished, resulting in superior classification performance. Overall classification accuracy is improved by 0.90%, particularly in the Drusen class, where precision, recall, and F1 score increase from 0.9877, 0.9760, and 0.9758 to 0.9920, 0.9840, and 0.9860, respectively. Furthermore, experiments on noisy datasets demonstrate the strong robustness of the CA-OCT network.

[0068] The proposed model integrates multiple modules, combining CBAM, AMP, and transfer learning algorithms to achieve multi-task learning. This results in a significant advantage over other state-of-the-art models in terms of accuracy and other evaluation metrics. Through comparative learning with other network models, future research can incorporate feature fusion and multi-scale feature extraction, and investigate the fusion of multiple classifiers and fusion methods to further enhance the classification results of the network model. Future research can learn from the CM-CNN (Diao et al., 2023) network, which combines segmentation and classification tasks, using a dual-task collaborative design to achieve explicit information interaction between classification and segmentation, realizing end-to-end optimization. Simultaneously, it can learn from the FN-OCT network, which combines multiple classifiers, and investigate fusion strategies to combine the advantages of different classifiers to better complete the classification task.

[0069] The CA-OCT network model proposed in this invention is an improvement on ResNet-34. Through CBAM, automatic mixed precision, and transfer learning modules, it achieves classification of retinal OCT images. Using the OCT-2017 dataset, it performs four-class classification of images of choroidal neovascularization (CNV), diabetic macular edema (DME), drusen, and normal retina. Experimental results show that the proposed improved model achieves an overall classification accuracy of 98.90% on the OCT-2017 dataset, with an AUC value of 1.0 for each class, indicating perfect classification performance in distinguishing between positive and negative samples. In particular, the model achieves a recall rate of 1.0 for the choroidal neovascularization (CNV) class, indicating that all CNV samples are correctly identified with no missed diagnoses. This result verifies the model's efficiency and robustness in retinal OCT image classification tasks, providing reliable technical support for the automated diagnosis of ophthalmic diseases. This model not only outperforms existing models on multiple evaluation metrics but also provides strong technical support for medical image processing tasks, showing broad clinical application prospects.

[0070] The beneficial effects of this invention are as follows: First, the classification model of this invention achieves extremely high classification accuracy and excellent recognition ability for subtle lesions. On the publicly available OCT-2017 dataset, the model achieved an overall accuracy of 98.90%, with an AUC of 1.00 for all classes, and a recall rate of 100% for choroidal neovascularization (CNV). This performance is mainly attributed to the proposed CBAM-Block residual block design. This module integrates channel attention and spatial attention mechanisms into the basic residual structure, enabling the network to adaptively enhance the channel features of the lesion region and focus on key spatial locations during forward propagation, thereby simultaneously capturing local detail textures and global semantic associations of the image. This effectively improves the ability to distinguish lesions with complex structures and subtle features (such as drusen), avoiding the problem of incomplete feature capture that may exist in traditional networks or single attention mechanisms.

[0071] Secondly, the classification model of this invention maintains high accuracy while possessing excellent training and computational efficiency. By introducing Automatic Mixed Precision (AMP) training technology, the model uses a combination of FP16 and FP32 floating-point numbers during training, significantly improving computational speed and reducing GPU memory usage. This not only makes the training process faster but also allows for larger batch sizes, thereby improving GPU resource utilization and model training throughput. This feature directly solves the problems of long training times and high hardware requirements caused by the large number of parameters in existing complex models (such as multi-branch fusion or multi-model ensemble), making the model easier to deploy and iterate in real-world clinical environments.

[0072] Finally, the classification model of this invention exhibits strong robustness and generalization ability. Despite inherent class imbalance in the dataset and noise interference common in medical images, the model remains stable. This performance stems from the synergistic effect of transfer learning strategies and targeted preprocessing. Initialization using weights trained on ImageNet and providing general visual feature priors, combined with median filtering for denoising, dynamic thresholding for white border removal, and geometric and color space data augmentation for minority classes, collectively enhances the model's adaptability to data distribution shifts and noise. This makes the present invention more reliable than methods relying solely on data augmentation or complex network regularization when faced with clinical data of varying quality.

[0073] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for OCT retinal image classification based on an improved ResNet-34 network, characterized in that, include: Acquire a dataset of retinal OCT images and preprocess the images, including size normalization, noise suppression, and data augmentation. An improved ResNet-34 network model was constructed, and the residual block was improved into a CBAM-Block structure. This structure incorporates the convolutional block attention mechanism CBAM, which adaptively enhances the channel feature response of the lesion region and focuses on key spatial locations through the concatenation of the channel attention module and the spatial attention module. Automatic mixed-precision training technology is used to dynamically adjust the floating-point calculation precision during network training, using a combination of single-precision floating-point numbers and half-precision floating-point numbers to optimize memory usage and computational efficiency. Load pre-trained weights based on the ImageNet dataset to initialize network parameters, and fine-tune the model through transfer learning; The network was trained using the preprocessed dataset and output classification results for retinal pathological conditions, including choroidal neovascularization, diabetic macular edema, drusen, and normal categories.

2. The OCT retinal image classification method as described in claim 1, characterized in that, The CBAM-Block structure improves upon the original residual block structure by incorporating the Convolutional Block Attention (CBAM) mechanism, combining the channel attention module and the spatial attention module in a cascaded manner. The channel attention module generates channel description vectors in parallel through global average pooling and global max pooling, and then generates channel weight maps after learning the inter-channel dependencies using a multilayer perceptron. The spatial attention module generates two-channel feature maps based on the feature maps weighted by channel attention by performing average pooling and max pooling operations along the channel dimension, and generates a two-dimensional spatial attention weight map through an activation function. The final output is the result of multiplying the original feature map by the channel attention weights channel by channel, and then multiplying it by the spatial attention weights position by position.

3. The OCT retinal image classification method as described in claim 1, characterized in that, The automatic mixed-precision training specifically includes: during forward computation and backward propagation, the weights and activation values ​​of the convolutional layers are stored using half-precision floating-point numbers to accelerate computation; single-precision floating-point numbers are used in the weight update stage to maintain numerical stability; and dynamic loss scaling technology automatically adjusts the gradient magnitude to solve the gradient vanishing problem under half-precision floating-point numbers.

4. The OCT retinal image classification method as described in claim 1, characterized in that, Data augmentation operations in the data preprocessing stage include: random horizontal flipping, vertical flipping, and rotation operations to increase sample diversity; random adjustment of brightness and contrast to simulate differences in the imaging environment; and oversampling strategies to increase the frequency of repeated training for minority class samples to address the problem of sample imbalance.

5. The OCT retinal image classification method as described in claim 1, characterized in that, The transfer learning fine-tuning process is as follows: retain all convolutional layer weights of ResNet-34 except for the fully connected layers; replace the fully connected layers with a four-class classification output structure and reset their weights using random initialization; and perform end-to-end training using the cross-entropy loss function and the adaptive moment estimation optimizer.

6. The OCT retinal image classification method as described in claim 1, characterized in that, The computation process of the channel attention module includes: performing global average pooling and global max pooling on the input feature map to generate two channel description vectors; inputting the two vectors into a multilayer perceptron with shared weights, adding the outputs and generating a channel weight vector through an activation function; and multiplying the channel weight vector with the original feature map channel by channel.

7. The OCT retinal image classification method as described in claim 1, characterized in that, The computation process of the spatial attention module includes: performing max pooling and average pooling on the output feature map of the channel attention module along the channel dimension to generate two spatial feature maps; concatenating the two feature maps along the channel dimension and generating a spatial weight map through a convolutional layer; and multiplying the spatial weight map with the output feature map of the channel attention module pixel by pixel.

8. The OCT retinal image classification method as described in claim 1, characterized in that, The residual structure is improved by embedding a CBAM module at the end of the identity mapping branch of the ResNet-34 basic residual block, which is then improved into a CBAM-Block residual block; the residual addition operation is performed between the CBAM-Block output and the original input.

9. The OCT retinal image classification method as described in claim 5, characterized in that, The fine-tuning operation consists of two training phases: the first phase freezes all network layers except the final fully connected layer and only updates the classification head parameters; the second phase unfreezes all network layers and performs end-to-end joint optimization.

10. The OCT retinal image classification method as described in claim 1, characterized in that, In the classification result generation stage, the Softmax function is used to normalize the output of the fully connected layer, and the classification accuracy of each category is used as the final pathological state diagnosis result.

Images