Diabetic retinopathy image classification method and system based on deep learning

Abstract

The invention provides a diabetic retinopathy image classification method and system based on deep learning. The method comprises the following steps: screening qualified image data; constructing a dynamic adaptive feature extraction network, and obtaining a multi-scale feature map containing spatio-temporal features; performing cross-dimension feature fusion based on the multi-scale feature graph, constructing a heterogeneous graph structure, fusing multi-modal features, and obtaining feature representation fusing time-space-channel-modal information; and performing uncertainty perception classification based on the feature representation, outputting a category probability and a Dirichlet evidence parameter through a double-end classification structure, and triggering an active learning mechanism when a preset threshold value is exceeded in combination with a composite loss function to obtain a lesion classification result and credibility evaluation. According to the method, collaborative optimization is carried out from four aspects of data quality, feature expression, multi-modal fusion and uncertainty modeling, and the classification accuracy of the diabetic retinopathy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to a method and system for classifying diabetic retinopathy images based on deep learning. Background Art

[0002] Diabetic retinopathy (DR) is the most common complication of diabetic retinopathy and has become one of the three major chronic non-communicable diseases threatening human health. Currently, the disease can only be controlled and treated to curb its progression, but there is no cure. Patients with DR exhibit distinct pathological features at different stages of the disease, including microaneurysms, hemorrhages, and exudates (both hard and soft exudates). Retinal microaneurysms appear in the early stages of diabetic retinopathy, accompanied by abnormal retinal vascular leakage. Exudates can be either hard or soft. Hard exudates are composed of lipid and protein leaking from dilated capillaries and microaneurysms, depositing in the outer and subretinal layers as well-defined plaques. Soft, deep exudates manifest clinically as irregular, ill-defined, cotton-wool or villous retinal exudates of varying sizes. If the condition persists for a long time, some blood vessels may become blocked or closed, preventing nutrients from reaching the retina, causing extensive damage and ultimately leading to blindness.

[0003] In international clinical diagnosis, diabetic retinopathy is generally divided into five stages: no DR, mild NPDR, moderate NPDR, severe NPDR, and proliferative DR. Currently, DR is primarily diagnosed using color fundus photography (CFP), with a physician providing a diagnosis and treatment recommendation based on the results. This requires extensive experience, as DR diagnosis relies heavily on both subjective and technical expertise. Inadequate expertise or prolonged work can lead to misdiagnosis or missed diagnoses, resulting in inaccurate results. Furthermore, limited medical care in some regions prevents many patients from receiving timely diagnosis and treatment, leading to worsening conditions and serious consequences. Summary of the Invention

[0004] The present invention aims to at least solve the technical problem of inaccurate diagnostic results in the prior art, and particularly innovatively proposes a deep learning-based diabetic retinopathy image classification method and system.

[0005] In order to achieve the above-mentioned object of the present invention, the present invention provides a method for classifying diabetic retinopathy images based on deep learning, the method comprising: S1. Using a no-reference image quality assessment algorithm to screen qualified original fundus images, and dynamically preprocessing the original fundus images to obtain dynamically preprocessed image data; S2. Constructing a dynamic adaptive feature extraction network based on the image data, the dynamic adaptive feature extraction network includes a dynamic convolution module based on an improved ConvNeXt-Lite backbone, integrating dynamic separable convolution and receptive field adaptation mechanisms, and configuring a lesion-sensitive path selector to dynamically adjust the computational path through a lightweight classifier array to obtain a multi-scale feature map containing spatiotemporal features; S3. Implementing cross-dimensional feature fusion based on the multi-scale feature map of the spatiotemporal features. The fusion process includes achieving joint weighting of the spatial, channel, and temporal dimensions through a three-dimensional attention matrix, constructing a heterogeneous graph structure that fuses multimodal features of the RGB image, the vascular segmentation map, and the lesion probability map, and using a dynamic graph convolutional network for inter-modal information transfer to obtain a feature representation that fuses the spatiotemporal, channel, and modal information. S4. Perform uncertainty-aware classification based on the feature representation of the fused spatiotemporal-channel-modal information. The classification process includes simultaneously outputting category probabilities and Dirichlet evidence parameters through a dual-headed classification structure, constructing a composite loss function based on Kullback-Leibler divergence and probability variance, and triggering an active learning mechanism when the prediction uncertainty exceeds a preset threshold to obtain lesion classification results and credibility assessment.

[0006] In another aspect, the present invention further provides a deep learning-based diabetic retinopathy image classification system, the system comprising the deep learning-based diabetic retinopathy image classification method; the system further comprising: An acquisition module, used for acquiring diabetic retinopathy images; A preprocessing module, configured to preprocess the image acquired by the acquisition module to obtain preprocessed image data; A feature extraction module is used to extract, based on the preprocessed image data, a multi-scale feature map representation containing spatiotemporal features and a feature representation that integrates spatiotemporal-channel-modal information using the deep learning-based diabetic retinopathy image classification method; A classification module is used to construct a dual-head classification structure based on the feature representation of the fused spatiotemporal-channel-modal information to obtain lesion classification results and credibility assessment; The output module is used to output the lesion classification result and credibility assessment.

[0007] The beneficial effects of the present invention are as follows: the present invention screens qualified images through reference-free image quality assessment, combines a dynamic adaptive feature extraction network (integrating dynamic convolution and receptive field adaptation mechanism) to enhance adaptability to lesion shape, and then fuses multimodal features through a three-dimensional attention matrix and a heterogeneous graph structure. Finally, the present invention adopts uncertainty-aware classification (a dual-headed structure outputs category probabilities and evidence parameters, combined with a composite loss function to optimize accuracy) and an active learning mechanism to collaboratively optimize data quality, feature expression, multimodal fusion, and uncertainty modeling, thereby significantly improving the accuracy of diabetic retinopathy classification.

[0008] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which: Figure 1 This is a flow chart of a deep learning-based diabetic retinopathy image classification method of the present invention. DETAILED DESCRIPTION

[0010] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0011] Example 1 like Figure 1 As shown, a diabetic retinopathy image classification method based on deep learning, the method comprising: S1. Using a no-reference image quality assessment algorithm to screen qualified original fundus images, and dynamically preprocessing the original fundus images to obtain dynamically preprocessed image data; It should be noted that in step S1, the non-reference image quality assessment algorithm can automatically assess the image quality without comparing it with other images, thereby screening out clear, blur-free, noise-free, high-quality fundus images. The dynamic preprocessing step performs adaptive preprocessing operations on the image based on the image quality assessment results, such as brightness adjustment, contrast enhancement, etc., to further improve the image quality. In this embodiment, the dynamic preprocessing is specifically adaptive histogram equalization and gamma correction. Adaptive histogram equalization can perform histogram equalization on local areas of the image to enhance the local contrast of the image; gamma correction is used to adjust the brightness of the image to make the grayscale distribution of the image more uniform and improve the visual effect of the image.

[0012] S2. Constructing a dynamic adaptive feature extraction network based on the image data, the dynamic adaptive feature extraction network includes a dynamic convolution module based on an improved ConvNeXt-Lite backbone, integrating dynamic separable convolution and receptive field adaptation mechanisms, and configuring a lesion-sensitive path selector to dynamically adjust the computational path through a lightweight classifier array to obtain a multi-scale feature map containing spatiotemporal features; In step S2, it is important to note that the dynamic convolution module, through the integration of dynamic separable convolution, can dynamically adjust the convolution kernel parameters based on image features, thereby improving the flexibility and accuracy of feature extraction. The receptive field adaptation mechanism adaptively adjusts the receptive field size based on the size and shape of the lesion area, ensuring that the network can capture the key characteristics of the lesion. In addition, the lesion-sensitive path selector dynamically selects the optimal computational path based on the image lesion characteristics through an array of lightweight classifiers, further improving the efficiency and accuracy of feature extraction.

[0013] S3. Implementing cross-dimensional feature fusion based on the multi-scale feature map of the spatiotemporal features. The fusion process includes achieving joint weighting of the spatial, channel, and temporal dimensions through a three-dimensional attention matrix, constructing a heterogeneous graph structure that fuses multimodal features of the RGB image, the vascular segmentation map, and the lesion probability map, and using a dynamic graph convolutional network for inter-modal information transfer to obtain a feature representation that fuses the spatiotemporal, channel, and modal information. What needs to be explained in detail in step S3 is that, through the three-dimensional attention matrix, the present invention achieves joint weighting of the feature map in the three dimensions of space, channel, and time, thereby enhancing the network's sensitivity and robustness to lesion features. Weighting in the spatial dimension helps the network focus on the lesion area and ignore background noise; weighting in the channel dimension can highlight the feature channels related to the lesion and suppress irrelevant channels; weighting in the temporal dimension (although not directly involved in single image classification, it is particularly important in video analysis or sequence image processing, and is assumed here as a potential application for extension to continuous frames or time series processing) helps capture the changing trends of lesions over time. This cross-dimensional feature fusion strategy significantly improves the richness and discriminative power of feature representation.

[0014] The construction of the heterogeneous graph structure further integrates information from different modalities, including RGB images, vascular segmentation maps, and lesion probability maps. The RGB image provides the original visual information, the vascular segmentation map highlights the vascular structure, and the lesion probability map reflects the preliminary prediction results of the lesion. By fusing these multimodal features in a heterogeneous graph and using a dynamic graph convolutional network to transfer information between modalities, the present invention achieves deep interaction and complementarity of cross-modal features, thereby obtaining a more comprehensive and accurate feature representation. Step S3 not only enhances the network's ability to extract lesion features through cross-dimensional feature fusion and multimodal feature fusion of heterogeneous graph structures, but also improves the accuracy and reliability of classification.

[0015] S4. Perform uncertainty-aware classification based on the feature representation of the fused spatiotemporal-channel-modal information. The classification process includes simultaneously outputting category probabilities and Dirichlet evidence parameters through a dual-headed classification structure, constructing a composite loss function based on Kullback-Leibler divergence and probability variance, and triggering an active learning mechanism when the prediction uncertainty exceeds a preset threshold to obtain lesion classification results and credibility assessment.

[0016] It is important to note in detail in step S4 that the design of the dual-head classification architecture is inspired by the multi-task learning concept in deep learning, aiming to simultaneously optimize classification accuracy and prediction uncertainty. The category probability distribution is obtained using the Softmax activation function, representing the probability of a sample belonging to each category. The Dirichlet evidence parameter, on the other hand, provides a representation of the uncertainty of the prediction result, reflecting the degree of dispersion in the prediction probability distribution. When the prediction uncertainty is high, i.e., the variance of the Dirichlet evidence parameter is large, this indicates that the model lacks confidence in the classification of the current sample, and the active learning mechanism needs to be triggered for further processing.

[0017] Active learning is an iterative learning method that allows the model to select the most valuable samples for manual review or dynamic weight adjustment during training. In this method, when a predicted uncertainty metric (such as the variance of the Dirichlet evidence parameter) exceeds a preset threshold, the model considers the current sample to have a high uncertainty value and selects these samples for active learning. Through manual review or dynamic weight adjustment, model parameters can be updated, thereby improving the classification accuracy and reliability of the model for high-uncertainty samples.

[0018] Furthermore, the present invention employs a composite loss function to optimize model performance. This composite loss function combines the Kullback-Leibler divergence loss, which measures the difference between the predicted probability distribution and the true label distribution, to optimize classification accuracy, and the probability variance loss, which quantifies the uncertainty of the prediction results, to optimize reliability. By balancing the impact of these two types of losses through a weighted summation, the final optimization target is obtained, thereby guiding the model training process.

[0019] In summary, this invention achieves high-accuracy and high-confidence classification of diabetic retinopathy images by constructing a dual-head classification structure, introducing an active learning mechanism, and using a composite loss function. This not only improves the efficiency and accuracy of lesion detection, but also improves the accuracy of lesion detection.

[0020] The principle behind this embodiment's deep learning-based diabetic retinopathy image classification method is as follows: First, high-quality fundus images are selected using a no-reference image quality assessment algorithm. Subsequently, deep feature extraction is performed on the image data using a dynamic adaptive feature extraction network. This network integrates dynamic convolution and receptive field adaptation mechanisms, adaptively adjusting feature extraction strategies based on image content to more effectively capture lesion-related feature information. During the feature extraction phase, not only is the spatiotemporal characteristics of the image considered, but multi-scale feature maps are also used to capture lesion details at different scales.

[0021] Next, a cross-dimensional feature fusion strategy was implemented, jointly weighting features from the spatial, channel, and temporal dimensions, and utilizing a three-dimensional attention matrix to enhance the representation of key features. By constructing a heterogeneous graph structure, the multimodal features of RGB images, vascular segmentation maps, and lesion probability maps were integrated, further enhancing the richness and accuracy of feature representation. A dynamic graph convolutional network was employed for intermodal information transfer, achieving deep fusion of multimodal features and obtaining a feature representation that integrates spatiotemporal, channel, and modal information.

[0022] Finally, when performing uncertainty-aware classification, a dual-headed classification architecture simultaneously outputs class probabilities and Dirichlet evidence parameters. This not only provides lesion classification results but also quantifies the uncertainty of the predictions. By combining the Kullback-Leibler divergence and probability variance to construct a composite loss function, both classification accuracy and reliability are optimized. When prediction uncertainty exceeds a preset threshold, an active learning mechanism is triggered, manually reviewing or dynamically adjusting the weights of high-uncertainty samples. This continuously updates model parameters and improves the accuracy and reliability of lesion classification.

[0023] As an optional embodiment of the present invention, optionally, in step S1, the no-reference image quality assessment algorithm includes: in, represents the quality assessment score, represents the natural base, 、 、 and represents the learnable parameters, Indicates the image clarity index, represents the maximum observed value of the clarity index on the training set, represents the contrast index, represents the maximum observed value of the contrast index on the training set, Indicates the noise level index, represents the maximum observed value of the noise index in the training set, Indicates the illumination uniformity index. represents the maximum observed value of the illumination uniformity index in the training set, Indicates the height of the image, Indicates the width of the image, represents the gradient response of the Sobel operator in the x direction, represents the gradient response of the Sobel operator in the y direction, represents a high-pass filter, represents the convolution operation, Indicates that the input fundus image is at coordinates The gray value at Represents the meshing parameters for illumination uniformity analysis, represents the local mean of the k-th grid, represents the global mean of the image, represents the local variance of the k-th grid.

[0024] As an optional embodiment of the present invention, optionally, in step S2, a multi-scale feature map representation containing spatiotemporal features is obtained: S201, constructing an improved ConvNeXt-Lite backbone network based on the preprocessed image data, integrating a dynamic separable convolution module to obtain an initial feature map; It is important to note in detail in step S201 that the improved ConvNeXt-Lite backbone network utilizes a lightweight design. By reducing the number of parameters and computational complexity, this improves the model's operational efficiency while maintaining feature extraction capabilities. The dynamic separable convolution module dynamically adjusts the parameters of the convolution kernel based on the features of the input image, enabling precise capture of image features. This module adaptively learns the importance of different features, thereby giving them greater attention during feature extraction and enhancing the model's adaptability to lesion shapes.

[0025] S202, implementing a receptive field adaptation mechanism based on the initial feature map, embedding a deformable convolution module in a shallow network, inputting the initial feature map, and obtaining a feature map adapted to the lesion shape; What needs to be explained in detail in step S202 is that the receptive field adaptation mechanism, by introducing a deformable convolution module, can adaptively adjust the receptive field size according to the shape of the lesion, thereby more effectively capturing the characteristic information related to the lesion. The deformable convolution module can locally deform the feature map to adapt to the changes in the lesion shape while maintaining spatial resolution. This adaptive mechanism enables the model to better learn the shape characteristics of the lesion, improving the accuracy and robustness of feature expression.

[0026] S203, implementing a receptive field adaptation mechanism based on the feature map adapted to the lesion shape, embedding a dilated separable convolution module in the deep network, inputting the feature map adapted to the lesion shape, and obtaining a feature map with an expanded receptive field; What needs to be explained in detail in step S203 is that the dilated separable convolution module can effectively expand the receptive field without increasing the computational complexity, thereby capturing a wider range of contextual information. By embedding the deformable convolution module and the dilated separable convolution module in the network at different levels, multi-scale capture of lesion features is achieved, improving the richness and accuracy of feature expression. At the same time, the receptive field adaptation mechanism enables the model to adaptively adjust the receptive field size according to the different lesion shapes, further enhancing the model's adaptability to lesion shapes.

[0027] S204: configuring a lesion-sensitive path selector based on the feature map after the receptive field is expanded, evaluating the quality of the feature map through a lightweight classifier array, inputting the feature map after the receptive field is expanded, and obtaining a dynamically adjusted calculation path; What needs to be explained in detail in step S204 is that the lesion-sensitive path selector is designed to dynamically select the optimal calculation path to improve the efficiency and accuracy of feature extraction. It analyzes the feature map after expanding the receptive field and uses a lightweight classifier array to evaluate the quality of the feature map. Based on the evaluation results, the lesion-sensitive path selector can intelligently select the most appropriate calculation path, thereby accurately capturing the lesion characteristics. This dynamic adjustment mechanism enables the model to respond flexibly in different situations, improving the overall classification performance. By configuring the lesion-sensitive path selector, the present invention further enhances the sensitivity and robustness of the model to lesion characteristics.

[0028] S205 , performing multi-scale feature fusion based on the dynamically adjusted calculation path, constructing a feature pyramid and fusing spatiotemporal information, inputting the dynamically adjusted calculation path, and obtaining a multi-scale feature map containing spatiotemporal features.

[0029] What needs to be explained in detail in step S205 is that the construction of the feature pyramid is to achieve the fusion of multi-scale features by upsampling, downsampling or direct connection of feature maps of different scales. This fusion strategy not only retains the spatial information of the image, but also integrates the details of lesions at different scales, thereby improving the comprehensiveness and accuracy of feature expression. At the same time, by introducing the spatiotemporal information fusion mechanism, the present invention can perform correlation analysis on lesion features at different time points, capture the changing trend of lesions over time, and further enhance the dynamics and robustness of feature expression. Through the processing of step S205, the present invention obtains a multi-scale feature map representation containing rich spatiotemporal features.

[0030] As an optional embodiment of the present invention, optionally, in step S205, the expression for obtaining the multi-scale feature map containing spatiotemporal features is: in, Represents a multi-scale feature map containing spatiotemporal features, Represents a 2D convolutional layer, which is used to further integrate spatial and temporal features. The number of output channels is , represents the number of layers of the feature pyramid, Indicates the The learnable weight vector of the layer features, represents the bilinear interpolation upsampling operation, Indicates the Layer feature map, Indicates the target resolution, Indicates the total number of time frames, Indicates the Temporal attention weights of frame temporal features, Indicates the Frame time characteristics, represents the normalization function, represents a multilayer perceptron, Represents global average pooling.

[0031] As an optional embodiment of the present invention, optionally, obtaining the feature representation of the fused spatiotemporal-channel-modal information in step S3 includes: S301, generating a three-dimensional attention matrix based on the multi-scale feature map of the spatiotemporal features, and jointly weighting the spatial, channel, and temporal dimensions to obtain a weighted feature map; What needs to be explained in detail in step S301 is that the introduction of the three-dimensional attention matrix is ​​intended to enhance the representation ability of key features, and to achieve comprehensive capture of lesion features by comprehensively considering information in the three dimensions of space, channel and time. The matrix can adaptively learn the importance of features in different dimensions, thereby paying more attention in the feature fusion process. Through joint weighting, the present invention enables the weighted feature map to retain the key information of the lesion while reducing the interference of redundancy and noise, thereby improving the accuracy and robustness of feature expression. After obtaining the weighted feature map, the present invention further implements a cross-dimensional feature fusion strategy to deeply fuse features in different dimensions, thereby constructing a feature representation containing rich spatiotemporal-channel-modal information. This fusion strategy improves the comprehensiveness of feature expression.

[0032] S302: constructing a heterogeneous graph structure based on the weighted feature map, fusing multimodal features of the RGB image, the blood vessel segmentation map, and the lesion probability map to obtain an initial heterogeneous graph structure; What needs to be explained in detail in step S302 is that the construction of the heterogeneous graph structure is intended to achieve deep fusion of multimodal features. By constructing a heterogeneous graph, the present invention can perform correlation analysis on information of different modalities such as RGB images, vascular segmentation maps and lesion probability maps, and capture the intrinsic connection between different modalities. In the heterogeneous graph structure, each node represents a feature point, and the edge represents the relationship between feature points. By introducing a graph convolutional network, the present invention can carry out information propagation on a heterogeneous graph, so that each node can aggregate information from adjacent nodes, thereby achieving deep fusion of multimodal features. This fusion strategy not only improves the richness of feature expression, but also enhances the sensitivity of the model to lesion features. After obtaining the initial heterogeneous graph structure, the present invention further implements a dynamic graph convolutional network to transfer information between modalities, and by continuously updating the feature representation of the nodes, achieves deep fusion of multimodal features, and finally obtains a feature representation that integrates spatiotemporal-channel-modal information. This feature representation contains the key information of the lesion and reflects the relationship between different modalities.

[0033] S303, implementing multimodal feature embedding based on the initial heterogeneous graph structure, performing feature mapping on the RGB image features, the blood vessel segmentation map features, and the lesion probability map features, respectively, to obtain a multimodal feature representation of a unified dimension; What needs to be explained in detail in step S303 is that multimodal feature embedding aims to map features of different modalities to the same dimensional space for subsequent feature fusion and analysis. By performing feature mapping on RGB image features, vascular segmentation map features and lesion probability map features respectively, the present invention can eliminate the feature differences between different modalities and obtain a multimodal feature representation of unified dimension. This feature representation not only retains the key information of the lesion, but also integrates complementary information from different modalities, thereby improving the comprehensiveness and accuracy of feature expression. In the feature mapping process, the present invention adopts advanced feature transformation technology to ensure that the mapped features can retain the important information of the original features, while reducing the interference of redundancy and noise, and obtaining a multimodal feature representation of unified dimension.

[0034] S304: Based on the multimodal feature representation of the unified dimension, a dynamic graph convolutional network is used to transfer information between modalities, and the graph structure is dynamically adjusted according to feature similarity to obtain cross-modal fusion features; What needs to be explained in detail in step S304 is that the introduction of the dynamic graph convolutional network is intended to achieve flexible transmission of information between modalities. By dynamically adjusting the graph structure according to feature similarity, the present invention can ensure the effective transmission of information between modalities while reducing redundancy and noise interference. In the dynamic graph convolutional network, each node establishes a connection with other nodes based on its feature representation, and the weight of the edge is dynamically adjusted according to the feature similarity. This adaptive mechanism enables the model to respond flexibly in different situations and achieves effective fusion of information between modalities. Through the processing of step S304, the present invention obtains a cross-modal fusion feature, which not only contains the key information of the lesion, but also integrates complementary information from different modalities, further improving the comprehensiveness and accuracy of the feature expression.

[0035] S305: Implement multi-dimensional feature integration based on the cross-modal fusion features, fuse the spatiotemporal-channel-modal information through convolutional layers and perform dimensionality reduction and enhancement to obtain the final feature representation of the fused spatiotemporal-channel-modal information.

[0036] What needs to be explained in detail in step S305 is that the multi-dimensional feature integration is intended to deeply fuse the spatiotemporal, channel and modal information in the cross-modal fusion features, and reduce the feature dimension to improve classification efficiency and accuracy. By introducing the convolution layer, the present invention can further process the cross-modal fusion features to achieve the fusion and enhancement of features in different dimensions. In the convolution layer, the feature map is subjected to feature extraction and fusion through convolution operations, while the activation function is used to increase the nonlinear expression ability of the features. Through multiple convolution operations, the present invention can gradually integrate feature information in different dimensions to obtain a more compact and comprehensive feature representation. In the dimensionality reduction process, the present invention adopts advanced dimensionality reduction techniques, such as global average pooling, to reduce feature redundancy and improve the robustness of feature expression. Finally, through the processing of step S305, the present invention obtains a feature representation that fuses spatiotemporal-channel-modal information, which not only contains the key information of the lesion, but also integrates complementary information from different dimensions and modalities.

[0037] As an optional embodiment of the present invention, optionally, obtaining the final feature representation of the fused spatiotemporal-channel-modal information in step S305 includes: in, Represents the feature representation of the final fusion of spatiotemporal-channel-modal information, represents the activation function, Represents the three-dimensional convolutional layer weight matrix, which is used to fuse multimodal features. Represents the feature map after the weighting of the three-dimensional attention matrix, which contains the attention information of the spatiotemporal and channel dimensions. Feature concatenation operation along the channel dimension, Represents the cross-modal fusion features output by the dynamic graph convolutional network, represents the feature representation of RGB images, Represents the feature representation of the blood vessel segmentation map, represents the feature representation of the lesion probability map, represents the bias term of the 3D convolutional layer, represents the spatial attention map, represents element-wise multiplication, represents the channel attention map, represents the temporal attention map, represents the input feature map, represents the Sigmoid activation function, represents a 3×3 convolutional layer, represents the average pooling operation, represents the time dimension, and represents the weight matrix of the fully connected layer, represents global average pooling, represents the normalization operation, Indicates adjusting the feature map dimension.

[0038] As an optional embodiment of the present invention, optionally, obtaining the lesion classification result and credibility evaluation in step S4 includes: S401, constructing a dual-head classification structure based on the feature representation of the fused spatiotemporal-channel-modal information, generating a category probability distribution through a fully connected layer and a Softmax activation function, and outputting Dirichlet evidence parameters through an independent fully connected layer to obtain an initial classification result and uncertainty representation; What needs to be explained in detail in step S401 is that the introduction of the dual-head classification structure is intended to simultaneously achieve the classification of lesions and the uncertainty assessment of the classification results. By constructing a classification branch including a fully connected layer and a Softmax activation function, the present invention can generate a category probability distribution and achieve a preliminary judgment on the lesion category. At the same time, by introducing an independent fully connected layer to output the Dirichlet evidence parameter, the present invention can quantify the uncertainty of the classification results. As a flexible prior distribution, the Dirichlet distribution can capture the uncertainty in the classification results, so that the model can give more cautious and reliable predictions when faced with complex or ambiguous cases. After obtaining the initial classification results and uncertainty characterization, the present invention further implements a post-processing strategy, combines the prior knowledge of the lesions and the clinical experience of doctors, and refines the classification results, thereby improving the accuracy and credibility of the classification results.

[0039] S402, calculating the Kullback-Leibler divergence loss based on the initial classification result and the uncertainty representation, measuring the difference between the predicted probability distribution and the true label distribution, and obtaining a classification accuracy optimization target; What needs to be explained in detail in step S402 is that the introduction of the Kullback-Leibler divergence loss is intended to measure the difference between the predicted probability distribution and the true label distribution, thereby guiding model optimization to improve classification accuracy. By calculating the Kullback-Leibler divergence between the initial classification result and the true label, the present invention can quantify the uncertainty of the model prediction and use it as one of the optimization goals. During the training process, the model continuously adjusts its parameters by minimizing the Kullback-Leibler divergence loss, so that the predicted probability distribution gradually approaches the true label distribution. This optimization strategy not only improves the accuracy of classification, but also enhances the model's sensitivity to lesion characteristics, so that the model can give more reliable and robust predictions when faced with complex or ambiguous cases. At the same time, combined with uncertainty characterization, the present invention can also provide doctors with additional information about the credibility of the classification results, assisting doctors in making more informed diagnostic decisions.

[0040] S403, calculating the probability variance loss based on the Dirichlet evidence parameter, quantifying the uncertainty degree of the prediction result, and obtaining the reliability optimization target; What needs to be explained in detail in step S403 is that the introduction of probability variance loss is intended to quantify the degree of uncertainty in the prediction results, thereby guiding model optimization to improve the reliability of the classification results. By calculating the variance of the Dirichlet evidence parameter, the present invention can measure the uncertainty of the prediction results and use it as another optimization goal. During the training process, the model continuously reduces the uncertainty of the prediction results by minimizing the probability variance loss, thereby improving the stability and credibility of the classification results. This optimization strategy enables the model to give more conservative and reliable predictions when faced with uncertain or ambiguous cases, avoiding misdiagnosis due to overconfidence. At the same time, combined with the classification accuracy optimization goal, the present invention can ensure the reliability of the classification results while improving the classification accuracy, providing doctors with more accurate and reliable auxiliary diagnostic information.

[0041] S404: constructing a composite loss function based on the classification accuracy optimization objective and the reliability optimization objective, and balancing the influence of the two types of losses through weighted summation to obtain a final optimization objective; What needs to be explained in detail in step S404 is that the construction of the composite loss function is intended to comprehensively consider the optimization goals of classification accuracy and reliability in order to obtain a more comprehensive and robust model performance. Through the weighted summation method, the present invention combines the Kullback-Leibler divergence loss and the probability variance loss to form a composite loss function. In this composite loss function, the choice of weights is crucial, as it determines the degree of emphasis the model places on different losses during the optimization process. Through reasonable weight distribution, the present invention can ensure that the model does not neglect the reliability of the prediction results while pursuing classification accuracy. During the training process, the model continuously adjusts its parameters by minimizing the composite loss function to achieve dual optimization of classification accuracy and reliability. This optimization strategy enables the model to give more accurate and reliable classification results when faced with complex and changeable diabetic retinopathy images.

[0042] S405: Training a classification model based on the final optimization target, calculating an uncertainty index of a sample in a prediction phase, and triggering an active learning mechanism when the uncertainty index exceeds a preset threshold; What needs to be explained in detail in step S405 is that the introduction of the active learning mechanism is intended to further improve the generalization ability and diagnostic accuracy of the model. When the sample uncertainty index calculated by the model in the prediction stage exceeds the preset threshold, this means that the model lacks sufficient confidence in the classification result of the current sample, which may be due to the complexity, rarity or imbalance of the sample distribution. At this time, the active learning mechanism will be triggered, and the model will actively request doctors or experts to label the current sample, and add these newly labeled samples to the training set for further training and optimization of the model. Through this iterative learning method, the model can continuously learn from new samples and gradually improve its generalization ability and diagnostic accuracy. At the same time, the active learning mechanism can also effectively utilize limited medical resources and focus the doctor's attention on cases that are difficult for the model to judge, thereby improving overall diagnostic efficiency and accuracy.

[0043] S406 , performing manual review or dynamic weight adjustment on high-uncertainty samples based on the active learning mechanism, and obtaining the final lesion classification result and credibility assessment after updating the model parameters.

[0044] It is important to note in detail in step S406 that for high-uncertainty samples, the present invention employs two processing strategies: manual review or dynamic weight adjustment. In the manual review strategy, when a sample's uncertainty index exceeds a preset threshold, the model prompts a doctor or expert to manually review the current sample. The doctor or expert, based on their expertise and clinical experience, classifies and labels the sample and adds these newly labeled samples to the training set for further model training and optimization. This strategy ensures that the model delivers more accurate and reliable classification results for complex or ambiguous cases. In the dynamic weight adjustment strategy, the model dynamically adjusts the weights of different samples during training based on the sample's uncertainty index. High-uncertainty samples are given higher weights, allowing them to receive more attention and learning during training. Through this strategy, the model can gradually improve its ability to identify and handle complex cases, thereby enhancing overall classification performance and diagnostic accuracy. After updating the model parameters, the present invention retrains the model based on the new training set and again classifies and evaluates the samples to obtain the final lesion classification results and credibility assessment.

[0045] As an optional embodiment of the present invention, optionally, the expression for obtaining the final optimization target in step S404 is: in, Represents the composite loss function, the final optimization goal, represents the loss weight parameter, represents the KL divergence loss, represents the probability variance loss, represents the number of categories for lesion classification, represents the class probability of the true label, represents the category probability predicted by the model, represents the probability variance loss, represents the variance of the predicted probability calculated based on the evidence parameter, represents the probability variance of the true label.

[0046] As an optional embodiment of the present invention, optionally, the expression for obtaining the final lesion classification result and credibility evaluation in step S406 is: in, represents the final lesion classification result, Indicates taking The maximum value of Representation category The predicted probability of in, represents the credibility assessment score, represents the entropy of the predicted probability distribution and represents the number of categories for lesion classification.

[0047] Example 2 A deep learning-based diabetic retinopathy image classification system, the system comprising the deep learning-based diabetic retinopathy image classification method; the system further comprising: An acquisition module, used for acquiring diabetic retinopathy images; A preprocessing module, configured to preprocess the image acquired by the acquisition module to obtain preprocessed image data; A feature extraction module is used to extract, based on the preprocessed image data, a multi-scale feature map representation containing spatiotemporal features and a feature representation that integrates spatiotemporal-channel-modal information using the deep learning-based diabetic retinopathy image classification method; A classification module is used to construct a dual-head classification structure based on the feature representation of the fused spatiotemporal-channel-modal information to obtain lesion classification results and credibility assessment; The output module is used to output the lesion classification result and credibility assessment.

[0048] A deep learning-based diabetic retinopathy image classification system in this embodiment is used to implement the deep learning-based diabetic retinopathy image classification method described in Example 1. The principle of the deep learning-based diabetic retinopathy image classification system is as follows: first, the acquisition module acquires diabetic retinopathy images, and then the preprocessing module preprocesses the images to remove noise, enhance image contrast, and other factors, thereby obtaining preprocessed image data. Next, the feature extraction module, based on the preprocessed image data, employs the previously described deep learning-based diabetic retinopathy image classification method to extract a multi-scale feature graph representation containing spatiotemporal features and a feature representation that integrates spatiotemporal, channel, and modal information. These feature representations can comprehensively reflect the important information in diabetic retinopathy images. Based on these feature representations, the classification module constructs a two-headed classification structure, generates a class probability distribution through a fully connected layer and a Softmax activation function, and outputs Dirichlet evidence parameters through an independent fully connected layer to obtain a lesion classification result and a credibility assessment. Finally, the output module outputs the lesion classification result and credibility assessment to the user to assist in diagnostic decision-making. The entire system achieves automated classification and credibility assessment of diabetic retinopathy images, improving diagnostic accuracy and efficiency.

[0049] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and alterations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A deep learning-based diabetic retinopathy image classification method, characterized in that: The method comprises: S1. Using a no-reference image quality assessment algorithm to screen qualified original fundus images, and dynamically preprocessing the original fundus images to obtain dynamically preprocessed image data; S2. Constructing a dynamic adaptive feature extraction network based on the image data, the dynamic adaptive feature extraction network includes a dynamic convolution module based on an improved ConvNeXt-Lite backbone, integrating dynamic separable convolution and receptive field adaptation mechanisms, and configuring a lesion-sensitive path selector to dynamically adjust the computational path through a lightweight classifier array to obtain a multi-scale feature map containing spatiotemporal features; S3. Implementing cross-dimensional feature fusion based on the multi-scale feature map of the spatiotemporal features. The fusion process includes achieving joint weighting of the spatial, channel, and temporal dimensions through a three-dimensional attention matrix, constructing a heterogeneous graph structure that fuses multimodal features of the RGB image, the vascular segmentation map, and the lesion probability map, and using a dynamic graph convolutional network for inter-modal information transfer to obtain a feature representation that fuses the spatiotemporal, channel, and modal information. S4. Perform uncertainty-aware classification based on the feature representation of the fused spatiotemporal-channel-modal information. The classification process includes simultaneously outputting category probabilities and Dirichlet evidence parameters through a dual-headed classification structure, constructing a composite loss function based on Kullback-Leibler divergence and probability variance, and triggering an active learning mechanism when the prediction uncertainty exceeds a preset threshold to obtain lesion classification results and credibility assessment.

2. The deep learning-based diabetic retinopathy image classification method according to claim 1, wherein: In step S1, the no-reference image quality assessment algorithm includes: in, represents the quality assessment score, represents the natural base, 、 、 and represents the learnable parameters, Indicates the image clarity index, represents the maximum observed value of the clarity index on the training set, represents the contrast index, represents the maximum observed value of the contrast index on the training set, Indicates the noise level index, represents the maximum observed value of the noise index in the training set, Indicates the illumination uniformity index. represents the maximum observed value of the illumination uniformity index in the training set, Indicates the height of the image, Indicates the width of the image, represents the gradient response of the Sobel operator in the x direction, represents the gradient response of the Sobel operator in the y direction, represents a high-pass filter, represents the convolution operation, Indicates that the input fundus image is at coordinates The gray value at Represents the meshing parameters for illumination uniformity analysis, represents the local mean of the k-th grid, represents the global mean of the image, represents the local variance of the k-th grid.

3. The deep learning-based diabetic retinopathy image classification method according to claim 1, characterized in that: In step S2, a multi-scale feature map representation containing spatiotemporal features is obtained: S201, constructing an improved ConvNeXt-Lite backbone network based on the preprocessed image data, integrating a dynamic separable convolution module to obtain an initial feature map; S202, implementing a receptive field adaptation mechanism based on the initial feature map, embedding a deformable convolution module in a shallow network, inputting the initial feature map, and obtaining a feature map adapted to the lesion shape; S203, implementing a receptive field adaptation mechanism based on the feature map adapted to the lesion shape, embedding a dilated separable convolution module in the deep network, inputting the feature map adapted to the lesion shape, and obtaining a feature map with an expanded receptive field; S204: configuring a lesion-sensitive path selector based on the feature map after the receptive field is expanded, evaluating the quality of the feature map through a lightweight classifier array, inputting the feature map after the receptive field is expanded, and obtaining a dynamically adjusted calculation path; S205 , performing multi-scale feature fusion based on the dynamically adjusted calculation path, constructing a feature pyramid and fusing spatiotemporal information, inputting the dynamically adjusted calculation path, and obtaining a multi-scale feature map containing spatiotemporal features.

4. The deep learning-based diabetic retinopathy image classification method according to claim 3, wherein: The expression for obtaining the multi-scale feature map containing spatiotemporal features in step S205 is: in, Represents a multi-scale feature map containing spatiotemporal features, Represents a 2D convolutional layer, which is used to further integrate spatial and temporal features. The number of output channels is , represents the number of layers of the feature pyramid, Indicates the The learnable weight vector of the layer features, represents the bilinear interpolation upsampling operation, Indicates the Layer feature map, Indicates the target resolution, Indicates the total number of time frames, Indicates the Temporal attention weights of frame temporal features, Indicates the Frame time characteristics, represents the normalization function, represents a multilayer perceptron, Represents global average pooling.

5. The deep learning-based diabetic retinopathy image classification method according to claim 1, wherein: Obtaining the feature representation of the fused spatiotemporal-channel-modal information in step S3 includes: S301, generating a three-dimensional attention matrix based on the multi-scale feature map of the spatiotemporal features, and jointly weighting the spatial, channel, and temporal dimensions to obtain a weighted feature map; S302: constructing a heterogeneous graph structure based on the weighted feature map, fusing multimodal features of the RGB image, the blood vessel segmentation map, and the lesion probability map to obtain an initial heterogeneous graph structure; S303, implementing multimodal feature embedding based on the initial heterogeneous graph structure, performing feature mapping on the RGB image features, the blood vessel segmentation map features, and the lesion probability map features, respectively, to obtain a multimodal feature representation of a unified dimension; S304: Based on the multimodal feature representation of the unified dimension, a dynamic graph convolutional network is used to transfer information between modalities, and the graph structure is dynamically adjusted according to feature similarity to obtain cross-modal fusion features; S305: Implement multi-dimensional feature integration based on the cross-modal fusion features, fuse the spatiotemporal-channel-modal information through convolutional layers and perform dimensionality reduction and enhancement to obtain the final feature representation of the fused spatiotemporal-channel-modal information.

6. The deep learning-based diabetic retinopathy image classification method according to claim 5, characterized in that: The final feature representation of the fused spatiotemporal-channel-modal information obtained in step S305 includes: in, Represents the feature representation of the final fusion of spatiotemporal-channel-modal information, represents the activation function, Represents the three-dimensional convolutional layer weight matrix, which is used to fuse multimodal features. Represents the feature map after the weighting of the three-dimensional attention matrix, which contains the attention information of the spatiotemporal and channel dimensions. Feature concatenation operation along the channel dimension, represents the cross-modal fusion features output by the dynamic graph convolutional network, represents the feature representation of RGB images, Represents the feature representation of the blood vessel segmentation map, represents the feature representation of the lesion probability map, represents the 3D convolution layer bias term, represents the spatial attention map, represents element-wise multiplication, represents the channel attention map, represents the temporal attention map, represents the input feature map, represents the Sigmoid activation function, represents a 3×3 convolutional layer, represents the average pooling operation, represents the time dimension, and represents the weight matrix of the fully connected layer, represents global average pooling, represents the normalization operation, Indicates adjusting the feature map dimension.

7. The deep learning-based diabetic retinopathy image classification method according to claim 1, wherein: Obtaining the lesion classification result and credibility assessment in step S4 includes: S401, constructing a dual-head classification structure based on the feature representation of the fused spatiotemporal-channel-modal information, generating a category probability distribution through a fully connected layer and a Softmax activation function, and outputting Dirichlet evidence parameters through an independent fully connected layer to obtain an initial classification result and uncertainty representation; S402, calculating the Kullback-Leibler divergence loss based on the initial classification result and the uncertainty representation, measuring the difference between the predicted probability distribution and the true label distribution, and obtaining a classification accuracy optimization target; S403, calculating the probability variance loss based on the Dirichlet evidence parameter, quantifying the uncertainty degree of the prediction result, and obtaining the reliability optimization target; S404: constructing a composite loss function based on the classification accuracy optimization objective and the reliability optimization objective, and balancing the influence of the two types of losses through weighted summation to obtain a final optimization objective; S405: Training a classification model based on the final optimization target, calculating an uncertainty index of a sample in a prediction phase, and triggering an active learning mechanism when the uncertainty index exceeds a preset threshold; S406 , performing manual review or dynamic weight adjustment on high-uncertainty samples based on the active learning mechanism, and obtaining the final lesion classification result and credibility evaluation after updating the model parameters.

8. The deep learning-based diabetic retinopathy image classification method according to claim 7, characterized in that: The expression of the final optimization target obtained in step S404 is: in, Represents the composite loss function, the final optimization goal, represents the loss weight parameter, represents the KL divergence loss, represents the probability variance loss, represents the number of categories for lesion classification, represents the class probability of the true label, represents the category probability predicted by the model, represents the probability variance loss, represents the variance of the predicted probability calculated based on the evidence parameter, represents the probability variance of the true label.

9. The deep learning-based diabetic retinopathy image classification method according to claim 7, characterized in that: The final lesion classification result and the credibility evaluation expression obtained in step S406 are: in, represents the final lesion classification result, Indicates taking The maximum value of Representation category The predicted probability of in, represents the credibility assessment score, represents the entropy of the predicted probability distribution and represents the number of categories for lesion classification.

10. A deep learning-based diabetic retinopathy image classification system, characterized in that: The system includes the deep learning-based diabetic retinopathy image classification method according to any one of claims 1 to 9; the system further includes: An acquisition module, used for acquiring diabetic retinopathy images; A preprocessing module, configured to preprocess the image acquired by the acquisition module to obtain preprocessed image data; a feature extraction module for extracting, based on the preprocessed image data, a multi-scale feature map representation containing spatiotemporal features and a feature representation that integrates spatiotemporal-channel-modal information using the deep learning-based diabetic retinopathy image classification method according to any one of claims 1 to 9; A classification module is used to construct a dual-head classification structure based on the feature representation of the fused spatiotemporal-channel-modal information to obtain lesion classification results and credibility assessment; The output module is used to output the lesion classification result and credibility assessment.

Images