Non-invasive diabetic nephropathy grading and predicting method supporting convenient screening

Abstract

The invention relates to a non-invasive diabetic nephropathy grading and predicting method supporting convenient screening, which comprises the following steps: acquiring and preprocessing a color fundus image and non-invasive clinical features of a diabetic patient to obtain a multi-modal data set, and setting marks of various severity degrees of diabetic nephropathy; constructing a deep learning model A1, and obtaining diabetic nephropathy severity prediction probability distribution # imgabs0 # based on the color fundus image according to the multi-modal data set; a deep learning model A2 is constructed, and diabetic nephropathy severity prediction probability distribution # imgabs1 # based on clinical data is obtained according to the multi-modal data set; and constructing a fusion model, fusing the obtained diabetic nephropathy prediction probabilities by the fusion model through a self-adaptive weight distribution algorithm, dynamically adjusting the weight according to the prediction result output by each model, calculating a comprehensive diabetic nephropathy severity score, and outputting a final diabetic nephropathy prediction grading result according to the score.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing and artificial intelligence technology, and in particular to a refined prediction method for the severity of non-invasive diabetic nephropathy based on fundus images and non-invasive clinical characteristics. Background Art

[0002] Diabetic nephropathy is one of the most common and most threatening complications of type 2 diabetes and the main cause of end-stage diabetic kidney disease. Diabetic nephropathy significantly increases the morbidity and mortality risk of patients, and also imposes a heavy economic and social burden on global healthcare systems. Early detection and intervention of diabetic nephropathy are crucial to delaying disease progression and improving patient prognosis. However, the current screening coverage for diabetic nephropathy is low, mainly due to the lack of convenience of traditional screening methods and low patient compliance. For patients who have been diagnosed with diabetes for more than a certain number of years, urine albumin / creatinine ratio testing and estimated glomerular filtration rate assessment should be performed regularly to improve the early diagnosis rate of diabetic nephropathy and develop precise disease management strategies, thereby reducing the risk of renal failure and high mortality. Despite this, traditional diagnostic methods still have significant limitations.

[0003] Traditional diagnostic methods for diabetic nephropathy mainly include urine tests, blood tests, imaging tests, and renal biopsies. Among them, imaging tests are expensive, renal biopsies are invasive and carry potential risks of complications, and although urine and blood tests have a certain degree of universality as routine methods, they rely on specific examination conditions (such as fasting or premature discontinuation of medication), which limits their convenience. In addition, because patients with early-stage diabetic nephropathy usually have no obvious clinical symptoms, patients are less willing to actively undergo screening, resulting in a high rate of missed diagnosis and delayed diagnosis and treatment. Therefore, the development of a non-invasive, convenient, and low-cost method for diagnosing diabetic nephropathy has become an important need in current medical research.

[0004] With the development of medical imaging technology and deep learning methods, researchers have begun to explore methods for predicting diabetic nephropathy based on fundus images. However, existing research still has significant limitations. First, existing methods usually only achieve a binary classification (presence or absence of kidney disease) or a three-class classification (early, intermediate, and late stages) of diabetic nephropathy, which is difficult to meet the clinical needs of continuous grading and fine-grained management of disease progression. Second, existing methods mainly rely on traditional convolutional neural networks or general convolutional neural network architectures (such as ResNet and VGG). Convolutional neural network models are limited by local receptive fields and fixed weight mechanisms, making it difficult to accurately identify complex pathological changes in fundus images. In addition, current methods mostly rely on urine tests and blood tests as feature inputs. Although they have certain predictive capabilities, in actual applications, their invasiveness and preparation requirements seriously affect patient screening compliance. Summary of the Invention

[0005] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a non-invasive diabetic nephropathy grading and prediction method that supports convenient screening. It is a diabetic nephropathy severity prediction method based on fundus images and clinical characteristics to achieve non-invasive, efficient and accurate disease screening and severity assessment. This method uses the deep learning model A1 to extract fundus image lesion features, and combines it with a multi-layer perceptron to process the patient's non-invasive clinical features, and finally realizes disease severity prediction through adaptive weight allocation.

[0006] The present invention solves the technical problem by the following technical solutions: A non-invasive diabetic nephropathy grading and prediction method supporting convenient screening comprises the following steps: Step 1: Collect and preprocess color fundus images and non-invasive clinical features of diabetic patients to obtain a multimodal dataset and set annotations for various severity levels of diabetic nephropathy; Step 2: Build a deep learning model A1 and train it on the color fundus images in the multimodal dataset of step 1 to obtain the predicted probability distribution of diabetic nephropathy severity based on color fundus images. ; Step 3: Build a deep learning model A2 and train the non-invasive clinical features in the multimodal dataset in step 1 to obtain the predicted probability distribution of diabetic nephropathy severity based on clinical data. ; Step 4: Construct a fusion model. The fusion model fuses the predicted probabilities of diabetic nephropathy obtained in steps 2 and 3 through an adaptive weight allocation algorithm, dynamically adjusts the weights according to the prediction results output by each model, calculates the comprehensive diabetic nephropathy severity score, and outputs the final diabetic nephropathy prediction grading result based on the score.

[0007] Furthermore, the step 1 includes the following steps: Step 1.1: Collect sample data from diabetic patients and remove abnormal sample data from diabetic patients; Step 1.2, preprocessing the color fundus image in the sample data; Step 1.3: Preprocess the non-invasive clinical features in the sample data; Step 1.4: Classify the preprocessed sample data into 6 levels of diabetic nephropathy: Level 0: Normal; Level 1: Warning; Level 2: Mild diabetic nephropathy; Level 3: Moderate diabetic nephropathy; Level 4: Severe diabetic nephropathy; Level 5: End-stage diabetic nephropathy; Step 1.5: Construct a multimodal dataset, including color fundus images, non-invasive clinical features, and corresponding classification labels of diabetic patients. Non-invasive clinical features include: age at presentation, gender, duration of diabetes, and history of hypertension.

[0008] Moreover, the deep learning model A1 in step 2 includes a dual-branch Transformer, a multi-scale feature extraction module, a dual sparse attention module, and a hierarchical feature interaction module. The input ends of the dual-branch Transformer branch 1 and the dual-branch Transformer branch 2 are respectively connected to the multi-scale feature extraction module, the dual sparse attention module, the Transformer module, and the hierarchical feature interaction module in sequence. The specific working method of the deep learning model A1 is: Step 2.1: Input the color fundus images at different resolutions into the first and second branches of the dual-branch Transformer respectively; Step 2.2: The multi-scale feature extraction module extracts features from the image of the dual-branch Transformer branch 1 and the image of the dual-branch Transformer branch 2 to obtain two feature sequences X1 and X2. Step 2.3: The dual sparse attention module fuses and filters the results before and after the multi-scale feature extraction module to obtain the feature and , and the features and Input to the dual-branch Transformer module to obtain deep feature representation and ; Step 2.4: The hierarchical feature interaction module takes the features output by the dual sparse attention module and and obtained after the Transformer module and Do residual connection to get and , and complete the dual-branch information interaction and the fusion of different levels of features to obtain the final feature information .

[0009] Moreover, the multi-scale feature extraction module in the constructed deep learning model A1 includes: an initial convolution module, a multi-scale convolution enhancement module, a size unification module and an attention fusion output module, wherein the initial convolution module, the multi-scale convolution enhancement module, the size unification module and the attention fusion output module are connected in sequence; the initial convolution module is used to extract the initial features, the multi-scale convolution enhancement module extracts the multi-scale structural features based on the initial features, the size unification module unifies the size of the multi-scale structural features, and the attention fusion output module introduces a hybrid attention mechanism to unify the modeling of the multi-scale features. and .

[0010] Moreover, the specific working method of the dual sparse attention module in the deep learning model A1 is as follows: the extracted dual-branch multi-scale features X1 and X2 are respectively subjected to attention analysis with the original branch features Y1 and Y2 to obtain the attention analysis result M. Combined with the attention analysis result M, the dynamic threshold screening mechanism and the adaptive Top-K screening mechanism are used to perform sparse screening respectively, and the overlapping verification module is used to verify the results, and a fusion attention matrix is generated. The feature channel weights are adjusted accordingly, and the output features of the two branches are obtained as follows: and .

[0011] Moreover, the working method of the hierarchical feature interaction module in the deep learning model A1 network is to combine the image features from different resolution branches and After Transformer feature extraction and , divide out and Global characteristics of 、 and local features 、 ; The global features of different branches obtained by Transformer feature extraction 、 Local features of another branch 、 Splicing to get and ;calculate and 、 and The similarity weight matrix between them is obtained, and the enhanced global features are obtained. and ; Then, and 、 and The information interaction is completed through the cross attention mechanism, and finally the enhanced global features are and Add and fuse to obtain feature information and :

[0012]

[0013] To get and Extract global information separately and , and then the global information and obtained Splicing , and Splicing , and finally and Splice to get the final feature information ;Will Perform softmax function operation to obtain normalized probability distribution : .

[0014] Moreover, the deep learning model A2 in step 6 includes an input layer, a hidden layer, and an output layer, wherein the hidden layer uses the ReLU activation function and normalization to process the data, and the output layer calculates the prediction probability through the fully connected layer. .

[0015] Moreover, the adaptive weight allocation method adopted in step 4 is:

[0016] in, represents the predicted probability distribution based on fundus images calculated by deep learning model A1; represents the predicted probability distribution based on non-invasive clinical features calculated by the deep learning model A2; is the weight coefficient adaptively calculated based on the input sample features; By fusion probability distribution Perform the maximum index operation to obtain the final predicted category :

[0017] Prediction category The value range is , corresponding to six grading labels: normal, warning, mild diabetic nephropathy, moderate diabetic nephropathy, severe diabetic nephropathy and end-stage diabetic nephropathy.

[0018] The beneficial effects of the present invention are: 1. The present invention provides a non-invasive diabetic nephropathy grading and prediction method that supports convenient screening. The present invention uses a deep learning model A1 to extract image lesion features and combines it with a multi-layer perceptron to process non-invasive clinical features. Ultimately, disease severity grading is achieved through adaptive weight allocation, achieving completely non-invasive and accurate prediction of the severity of diabetic nephropathy.

[0019] 2. The present invention provides a non-invasive diabetic nephropathy grading and prediction method that supports convenient screening, with convenient screening and improved patient compliance: fundus image analysis combined with non-invasive clinical characteristics replaces traditional methods such as urine tests, blood tests and renal biopsies, providing patients with a more convenient, safe and comfortable screening method.

[0020] 3. The present invention provides a non-invasive diabetic nephropathy grading and prediction method that supports convenient screening and designs a model architecture for targeted tasks: This method innovatively combines convolutional neural networks and Transformer to construct a deep learning model A1, extracts multi-scale lesion features from fundus images, and combines a multi-layer perceptron to process non-invasive clinical features, thereby enhancing the comprehensive recognition ability of the disease. Through adaptive weight allocation, the color fundus images and non-invasive clinical features work together to improve prediction accuracy and generalization ability.

[0021] 4. The present invention provides a non-invasive diabetic nephropathy grading and prediction method that supports convenient screening, achieving fine-grained grading and precise personalized management. Traditional methods are mostly limited to the binary classification (presence or absence of nephropathy) or ternary classification (early, middle, and late stages) of diabetic nephropathy. The present invention can achieve a six-level detailed grading of the severity of diabetic nephropathy (0 to 5) for accurately describing the disease progression status. This grading system facilitates personalized treatment decision-making. Doctors can adjust intervention strategies based on the prediction results, thereby optimizing the patient's treatment plan and delaying disease progression.

[0022] 5. The present invention provides a non-invasive diabetic nephropathy grading and prediction method that supports convenient screening. The constructed deep learning model A1 is specially designed for the lesion pattern of fundus images of diabetic nephropathy. It can effectively identify typical pathological features such as tiny exudates and microaneurysms, and is significantly better than the adaptability and prediction effect of general image classification models in this task. The present invention combines deep learning technology with multimodal data, without the need for invasiveness and urine test indicators, and conveniently realizes early screening of diabetic nephropathy and prediction of level 6 severity. It is suitable for clinical auxiliary diagnosis and prediction of disease progression. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the overall framework of the present invention; Figure 2 This is a schematic diagram of the deep learning model A1 of the present invention identifying lesions in fundus images of diabetic nephropathy of different severity levels; Figure 3 This is a schematic diagram of different models of the present invention for identifying lesions in fundus images of diabetic nephropathy. Where a is the true label, b is a schematic diagram of the deep learning model A1 for identifying lesions in diabetic nephropathy fundus images, c is a schematic diagram of the model VIT for identifying lesions in diabetic nephropathy fundus images, and d is a schematic diagram of the model ResNet34 for identifying lesions in diabetic nephropathy fundus images; Figure 4 Schematic diagram of the deep learning model A1 of the present invention; Figure 5 Schematic diagram of the multi-scale feature extraction module of the present invention; Figure 6 Schematic diagram of the dual sparse attention module of the present invention; Figure 7 Schematic diagram of the hierarchical feature interaction module of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below through specific examples. The following examples are only illustrative and not restrictive, and the scope of protection of the present invention cannot be limited thereto.

[0025] A non-invasive method for grading and predicting diabetic nephropathy that supports convenient screening, such as Figure 1 As shown, the following steps are included: Step 1: Collect color fundus images and non-invasive clinical features of diabetic patients and preprocess them to obtain a multimodal dataset, and set annotations for various severity levels of diabetic nephropathy.

[0026] Step 1.1: Collect sample data from diabetic patients and remove abnormal data from diabetic patients. Sample data from patients with non-type 2 diabetic nephropathy, active malignant tumors, a history of kidney transplantation, or a follow-up interval of less than 3 months were removed.

[0027] Step 1.2: Preprocess the color fundus image in the sample data.

[0028] Step 1.3: Preprocess the non-invasive clinical features in the sample data. Non-invasive clinical features are obtained through the electronic medical record system: blood test indicators, urine test indicators, diabetes duration, history of hypertension, patient gender and age; Step 1.4: Classify the preprocessed sample data into 6 levels of diabetic nephropathy labels: Level 0: normal; Level 1: warning; Level 2: mild diabetic nephropathy; Level 3: moderate diabetic nephropathy; Level 4: severe diabetic nephropathy; Level 5: end-stage diabetic nephropathy.

[0029] Step 1.5: Construct a multimodal dataset, including color fundus images, non-invasive clinical features, and corresponding classification labels of diabetic patients. Non-invasive clinical features include: age at presentation, gender, duration of diabetes, and history of hypertension.

[0030] Step 2: Build a deep learning model A1 and train it on the color fundus images in the multimodal dataset of step 1 to obtain the predicted probability distribution of diabetic nephropathy severity based on color fundus images. .

[0031] like Figure 4 As shown, the deep learning model A1 includes a dual-branch Transformer, a multi-scale feature extraction module, a dual-sparse attention module, and a hierarchical feature interaction module. The input ends of the dual-branch Transformer branch 1 and the dual-branch Transformer branch 2 are respectively connected to the multi-scale feature extraction module, the dual-sparse attention module, the Transformer module, and the hierarchical feature interaction module in sequence. The specific working method of the deep learning model A1 is: Step 2.1: Input the color fundus images at different resolutions into the first and second branches of the dual-branch Transformer respectively; Step 2.2: The multi-scale feature extraction module extracts features from the image of the dual-branch Transformer branch 1 and the image of the dual-branch Transformer branch 2 to obtain two feature sequences X1 and X2. Step 2.3: The dual sparse attention module fuses and filters the results before and after the multi-scale feature extraction module to obtain the feature and , and the features and Input to the dual-branch Transformer module to obtain deep feature representation and ; Step 2.4: The hierarchical feature interaction module takes the features output by the dual sparse attention module and and obtained after the Transformer module and Do residual connection to get and , and complete the dual-branch information interaction and the fusion of different levels of features to obtain the final feature information .

[0032] like Figure 5 As shown, the multi-scale feature extraction module in the deep learning model A1 includes: an initial convolution module, a multi-scale convolution enhancement module, a size unification module and an attention fusion output module, wherein the initial convolution module, the multi-scale convolution enhancement module, the size unification module and the attention fusion output module are connected in sequence.

[0033] The initial convolution module first uses 7×7 depthwise convolution to capture the large receptive field intermediate feature map of obvious lesions, and strengthens the consistency and stability of channel dimension features through channel normalization. Then, point-by-point convolution and GELU activation are used for channel mapping and nonlinear transformation. Then, standard 3×3 convolution is used to extract the small receptive field information of blood vessels and microaneurysms. Finally, batch normalization and ReLU activation function are combined to obtain the final output:

[0034]

[0035] in, is the convolution operation, is layer normalization, yes activation function, is batch normalization, activation function, is the intermediate feature map, is the initial feature extracted by the initial convolution module; The multi-scale convolution enhancement module extracts the initial features Perform pooling operations and then extract multi-scale structural features at different spatial resolutions layer by layer to enhance the network's semantic perception of local lesion areas at different scales;

[0036] in, It is the maximum pooling layer, and the initial feature of the output is represented as , Indicates the number of channels;

[0037] in, is a multi-scale structural feature, , , , ; Size-unified module for multi-scale structural features Unify the size and pass 1×1 convolution step by step Mapping to a fixed dimension;

[0038] in, is a multi-scale feature with the same number of channels, , , , ; like Figure 6 As shown, the attention fusion output module introduces a hybrid attention mechanism for multi-scale features. Perform unified modeling to obtain feature information of three scales of two branches and , which is concatenated into two one-dimensional sequences to obtain and :

[0039]

[0040] in, is spatial attention, is the channel attention, represents element-wise product, are the features after spatial attention and channel attention, respectively and , It is a splicing operation.

[0041] like Figure 7 As shown, the dual sparse attention module in deep learning model A1 is responsible for feature screening and sparse modeling. Through joint modeling with dynamic thresholds and an adaptive Top-K mechanism, it effectively eliminates redundant background, highlights key regional features of diabetic nephropathy lesions, and improves expressiveness and discriminability. At the same time, the module possesses dynamic selection capabilities under heterogeneous structures, enhancing the model's adaptability to complex lesions and cross-regional recognition stability. The sparse attention matrix it outputs serves as a high-confidence guide, helping to improve the feature quality of Transformer's global modeling and the accuracy of hierarchical predictions. Its specific working method is as follows: Perform attention analysis on the extracted dual-branch multi-scale features X1 and X2 and the original branch features Y1 and Y2 respectively:

[0042] in, is a scaling parameter, multi-head attention is performed on n groups of Q, K and V, and an output with d=C / n channel dimension is obtained, Different scale features, It is the original branch feature that has not been processed by multiple scale feature extraction modules. ; Dynamic threshold screening mechanism generates adaptive thresholds through lightweight networks , performs masking operations on low response areas in the input features

[0043]

[0044]

[0045] in, represents the Sigmoid function, and is the learning parameter, It means taking the mean of the input matrix. As a nonlinear activation function, it is only activated when the response in the feature exceeds the dynamic threshold It will be retained only when It is a dynamic threshold screening mechanism; The adaptive Top-K screening mechanism dynamically determines the sparse retention factor based on the statistical characteristics of the feature response and selects the feature information with the top K response values:

[0046]

[0047] in, and The attention matrix The mean and standard deviation of each sequence ensure that the sparsity factor is moderate. is the length of the attention score vector, Round down to ensure the result of adaptive sparse dimension selection is an integer; The overlap verification module compares the consistency of the two sparse screening results, generates a fusion attention matrix, and adjusts the feature channel weights accordingly:

[0048]

[0049] in, It is an element-by-element multiplication, which is used to select the overlapping area of the two matrices; Softmax is normalized to ensure that the attention meets the probability distribution characteristics, and the output features of the two branches are and .

[0050] The working method of the hierarchical feature interaction module in the deep learning model A1 network is to combine the image features from different resolution branches and After Transformer feature extraction and , divide out and Global characteristics of 、 and local features 、 ; The global features of different branches obtained by Transformer feature extraction 、 Local features of another branch 、 Splicing to get and ;calculate and 、 and The similarity weight matrix between them is obtained, and the enhanced global features are obtained. and ; Then, and 、 and The information interaction is completed through the cross attention mechanism, and finally the enhanced global features are and Add and fuse to obtain feature information and :

[0051]

[0052] To get and Extract global information separately and , and then the global information and obtained Splicing , and Splicing , and finally and Splice to get the final feature information ;Will Perform softmax function operation to obtain normalized probability distribution : .

[0053] The fundus images used to train the deep learning model A1 in this paper were obtained from a multimodal dataset of diabetic patients' fundus photographs and clinical features obtained from partner hospitals. Patients with non-type 2 diabetic nephropathy, active malignant tumors, a history of renal transplantation, and follow-up intervals of less than three months were excluded. The corresponding color fundus photographs were clear and unobstructed. Subsequently, under the guidance of a physician, each sample was graded and assigned a label (0-5) based on the patient's detailed clinical features. The resulting dataset contained 3807 color fundus images and corresponding non-invasive clinical features. The image sizes ranged from 1024×1024 to 2992×2000. Of these, 1698 images were normal, 886 were warning, 335 were mild diabetic nephropathy, 481 were moderate diabetic nephropathy, 270 were severe diabetic nephropathy, and 137 were end-stage diabetic nephropathy. 3045 images were used for training, and 762 images were used for testing. The non-invasive clinical characteristics corresponding to each fundus photograph were: gender, age at presentation, duration of diabetes, and history of hypertension.

[0054] Step 3: Build a deep learning model A2 and train the non-invasive clinical features in the multimodal dataset in step 1 to obtain the predicted probability distribution of diabetic nephropathy severity based on clinical data. .

[0055] The deep learning model A2 consists of an input layer, a hidden layer, and an output layer. The hidden layer uses the ReLU activation function and normalization to process the data, and the output layer calculates the prediction probability through a fully connected layer. .

[0056] Step 4: Construct a fusion model. The fusion model fuses the predicted probabilities of diabetic nephropathy obtained in steps 2 and 3 through an adaptive weight allocation algorithm, dynamically adjusts the weights according to the prediction results output by each model, calculates the comprehensive diabetic nephropathy severity score, and outputs the final diabetic nephropathy prediction grading result based on the score.

[0057] The adaptive weight allocation method used is:

[0058] in, represents the predicted probability distribution based on fundus images calculated by deep learning model A1; represents the predicted probability distribution based on non-invasive clinical features calculated by the deep learning model A2; is the weight coefficient adaptively calculated based on the input sample features; By fusion probability distribution Perform the maximum index operation to obtain the final predicted category :

[0059] Prediction category The value range is , corresponding to six grading labels: normal, warning, mild diabetic nephropathy, moderate diabetic nephropathy, severe diabetic nephropathy and end-stage diabetic nephropathy.

[0060] Experimental comparison algorithm To verify the effectiveness of the deep learning model A1 proposed in this paper in the six-level prediction task of diabetic nephropathy severity, we compared it with current mainstream image classification neural network models, including: ResNet34: ResNet34 contains 34 convolutional layers, which can alleviate the gradient vanishing problem in deep neural networks and is widely used in tasks such as medical image analysis.

[0061] Inception-v2: Inception-v2 is a multi-path convolutional neural network that uses convolution kernels of different scales in the same layer to extract multi-scale features, improving the model's ability to model local and global image information.

[0062] VisionTransformer (ViT): ViT is an image classification model based on the self-attention mechanism, which is good at capturing long-range dependency information.

[0063] All of the above models were trained and tested using the same fundus image dataset, with consistent data partitioning (training set: 3045, test set: 762), and a uniform six-category labeling scheme. To ensure fairness, all compared models maintained consistent input image size, number of training rounds, and optimizer strategy; only the backbone feature extraction network architecture was modified. During testing, prediction accuracy, F1-score, and AUC were used to compare model performance.

[0064] The experimental results, shown in Table 1, show that the proposed deep learning model A1 outperformed existing mainstream models in the diabetic nephropathy grade prediction task, achieving superior performance in accuracy, F1-score, and AUC metrics. Compared to the ResNet, Inception, and ViT models, the deep learning model A1, with its modular structure specifically designed for diabetic nephropathy lesion modeling, achieved an approximately 2% improvement in accuracy and significant improvements in key metrics such as F1-score and AUC, validating its effectiveness and practical value in fine-grained medical image prediction tasks. Figure 3A schematic diagram of different models identifying lesions in diabetic nephropathy fundus images. The visualization clearly shows that deep learning model A1 more accurately focuses on key areas associated with diabetic nephropathy fundus lesions. Its focus areas closely match the true annotated lesions, demonstrating superior discrimination and interpretability. In contrast, the attention distribution of other models exhibits a degree of shift or redundant attention, making it difficult to accurately focus on high-risk areas.

[0065] Table 1: Comparison of experimental results of different models in diabetic nephropathy classification task

[0066] The above comparison results fully verify the effectiveness and superiority of the deep learning model A1 proposed in this invention in the task of grading the severity of diabetic nephropathy. Compared with existing models, this method has stronger feature expression and semantic differentiation capabilities, and can more accurately and meticulously reflect the pathological characteristics of diabetic nephropathy at different stages, providing stronger support for clinical early screening and personalized treatment. In addition, Figure 2 This is a schematic diagram of the deep learning model A1 of the present invention for identifying lesions in fundus images of diabetic nephropathy of different severity. It can be seen that as the severity of diabetic nephropathy increases, the area of fundus lesions increases.

[0067] Although the embodiments and drawings of the present invention are disclosed for illustrative purposes, those skilled in the art will understand that various substitutions, changes and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the scope of the present invention is not limited to the contents disclosed in the embodiments and drawings.

Claims

1. A non-invasive diabetic nephropathy grading and prediction method that supports convenient screening, characterized in that: The following steps are involved: Step 1: Collect and preprocess color fundus images and non-invasive clinical features of diabetic patients to obtain a multimodal dataset and set annotations for various severity levels of diabetic nephropathy; Step 2: Build a deep learning model A1 and train it on the color fundus images in the multimodal dataset of step 1 to obtain the predicted probability distribution of diabetic nephropathy severity based on color fundus images. ; Step 3: Build a deep learning model A2 and train the non-invasive clinical features in the multimodal dataset in step 1 to obtain the predicted probability distribution of diabetic nephropathy severity based on clinical data. ; Step 4: Construct a fusion model. The fusion model fuses the predicted probabilities of diabetic nephropathy obtained in steps 2 and 3 through an adaptive weight allocation algorithm, dynamically adjusts the weights according to the prediction results output by each model, calculates the comprehensive diabetic nephropathy severity score, and outputs the final diabetic nephropathy prediction grading result based on the score.

2. A non-invasive diabetic nephropathy grading and prediction method supporting convenient screening according to claim 1, characterized in that: The step 1 comprises the following steps: Step 1.1: Collect sample data from diabetic patients and remove abnormal sample data from diabetic patients; Step 1.2, preprocessing the color fundus image in the sample data; Step 1.3: Preprocess the non-invasive clinical features in the sample data; Step 1.4: Classify the preprocessed sample data into 6 levels of diabetic nephropathy: Level 0: Normal; Level 1: Warning; Level 2: Mild diabetic nephropathy; Level 3: Moderate diabetic nephropathy; Level 4: Severe diabetic nephropathy; Level 5: End-stage diabetic nephropathy; Step 1.5: Construct a multimodal dataset, including color fundus images, non-invasive clinical features, and corresponding classification labels of diabetic patients. Non-invasive clinical features include: age at presentation, gender, duration of diabetes, and history of hypertension.

3. The non-invasive diabetic nephropathy grading and prediction method supporting convenient screening according to claim 1, characterized in that: In step 2, the deep learning model A1 includes a dual-branch Transformer, a multi-scale feature extraction module, a dual-sparse attention module, and a hierarchical feature interaction module. The input ends of the dual-branch Transformer branch 1 and the dual-branch Transformer branch 2 are respectively connected to the multi-scale feature extraction module, the dual-sparse attention module, the Transformer module, and the hierarchical feature interaction module in sequence. The specific working method of the deep learning model A1 is as follows: Step 2.1: Input the color fundus images at different resolutions into the first and second branches of the dual-branch Transformer respectively; Step 2.2: The multi-scale feature extraction module extracts features from the image of the dual-branch Transformer branch 1 and the image of the dual-branch Transformer branch 2 to obtain two feature sequences X1 and X2. Step 2.3: The dual sparse attention module fuses and filters the results before and after the multi-scale feature extraction module to obtain the feature and , and the features and Input to the dual-branch Transformer module to obtain deep feature representation and ; Step 2.4: The hierarchical feature interaction module takes the features output by the dual sparse attention module and and obtained after the Transformer module and Do residual connection to get and , and complete the dual-branch information interaction and the fusion of different levels of features to obtain the final feature information .

4. A non-invasive diabetic nephropathy grading and prediction method supporting convenient screening according to claim 3, characterized in that: The multi-scale feature extraction module in the deep learning model A1 includes: an initial convolution module, a multi-scale convolution enhancement module, a size unification module and an attention fusion output module, wherein the initial convolution module, the multi-scale convolution enhancement module, the size unification module and the attention fusion output module are connected in sequence; the initial convolution module is used to extract the initial features, the multi-scale convolution enhancement module extracts the multi-scale structural features based on the initial features, the size unification module unifies the size of the multi-scale structural features, and the attention fusion output module introduces a hybrid attention mechanism to unify the modeling of the multi-scale features. and .

5. The non-invasive diabetic nephropathy grading and prediction method supporting convenient screening according to claim 3, characterized in that: The specific working method of the dual sparse attention module in the deep learning model A1 is as follows: the extracted dual-branch multi-scale features X1 and X2 are respectively subjected to attention analysis with the original branch features Y1 and Y2 to obtain the attention analysis result M. The dynamic threshold screening mechanism and the adaptive Top-K screening mechanism are used to perform sparse screening respectively, and the overlapping verification module is used to verify the results. The fused attention matrix is generated, and the feature channel weights are adjusted accordingly to obtain the output features of the two branches as follows: and .

6. The non-invasive diabetic nephropathy grading and prediction method supporting convenient screening according to claim 1, characterized in that: The working method of the hierarchical feature interaction module of the deep learning model A1 is to combine the image features from different resolution branches and After Transformer feature extraction and , divide out and Global characteristics of 、 and local features 、 ; The global features of different branches obtained by Transformer feature extraction 、 Local features of another branch 、 Splicing to get and ;calculate and 、 and The similarity weight matrix between them is obtained, and the enhanced global features are obtained. and ; Then, and 、 and The information interaction is completed through the cross attention mechanism, and finally the enhanced global features are and Add and fuse to obtain feature information and : ； ； To get and Extract global information separately and , and then the global information and obtained Splicing , and Splicing , and finally and Splice to get the final feature information ; Next, we will fuse the features Input to the classifier, after Operation to obtain the final probability distribution : ； in, is a fully connected layer; Used to output the predicted probability of each category.

7. The non-invasive diabetic nephropathy grading and prediction method supporting convenient screening according to claim 1, characterized in that: The deep learning model A2 in step 3 includes an input layer, a hidden layer, and an output layer, wherein the hidden layer uses the ReLU activation function and normalization to process the data, and the output layer calculates the prediction probability through the fully connected layer. .

8. The non-invasive diabetic nephropathy grading and prediction method supporting convenient screening according to claim 1, characterized in that: The adaptive weight allocation method used in step 4 is: ； in, represents the predicted probability distribution based on fundus images calculated by deep learning model A1; represents the predicted probability distribution based on non-invasive clinical features calculated by the deep learning model A2; is the weight coefficient adaptively calculated based on the input sample features; By fusion probability distribution Perform the maximum index operation to obtain the final predicted category : ； Prediction category The value range is , corresponding to six grading labels: normal, warning, mild diabetic nephropathy, moderate diabetic nephropathy, severe diabetic nephropathy and end-stage diabetic nephropathy.

Images