Methods, apparatus, electronic devices, and storage media of predicting diabetic nephropathy

Abstract

The present disclosure provides a method, device, electronic equipment and storage medium for predicting diabetic nephropathy; wherein the method comprises: obtaining survey data of a target diabetic patient, a first fundus image and a second fundus image of the same type; constructing a target Bayesian network according to the survey data; calculating a joint probability distribution of the target diabetic patient complicated with diabetic nephropathy based on the target Bayesian network to obtain a first probability value; performing difference feature extraction on the first fundus image and the second fundus image to obtain target difference feature data; performing probability prediction of the target difference feature data for the diabetic nephropathy to obtain a second probability value; and determining a target probability of the target diabetic patient complicated with the diabetic nephropathy according to the first probability value and the second probability value. The present disclosure aims to construct a simple, economical and low-trauma early screening system for diabetic nephropathy.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, and in particular, to a method and device for predicting diabetic nephropathy, an electronic device and a storage medium. BACKGROUND

[0002] Currently, the screening of diabetic nephropathy mainly relies on the determination of urine microalbumin / creatinine ratio. However, the level of urine microalbumin is affected by many factors, such as blood glucose control level, hypertension, kidney disease, body position change, inflammatory response, pregnancy, high-protein diet and drugs, etc. These factors can lead to instability of urine microalbumin test results, thereby affecting the accuracy and reliability of screening. Therefore, there are certain limitations in relying solely on urine microalbumin / creatinine ratio for screening of diabetic nephropathy.

[0003] In addition, the diagnosis and follow-up of diabetic nephropathy currently requires patients to undergo a series of cumbersome and high-cost examination methods, including blood and urine tests, kidney ultrasound, CT / MRI imaging examination and kidney biopsy, etc. Although these examinations can provide important clinical information, their complex procedures, high cost and potential trauma often lead to poor patient compliance, and some patients even refuse invasive examinations such as kidney biopsy due to fear of pain or trauma.

[0004] In view of this, there is an urgent need for a convenient, economical, low-trauma and efficient early screening method for diabetic nephropathy, which can perform accurate risk assessment based on the clinical data of patients, guide the necessity and direction of subsequent examinations, thereby significantly improving the early diagnosis rate of diabetic nephropathy and avoiding the high cost, complex procedures and patient compliance problems brought by traditional diagnostic methods. SUMMARY

[0005] In view of the above challenges and needs, the present application is dedicated to providing a diabetic nephropathy prediction method based on advanced artificial intelligence technology and its supporting equipment, aiming to build a simple, economical and low-trauma early screening system.

[0006] In a first aspect, the embodiments of the present disclosure provide a method for predicting diabetic nephropathy, the method comprising: obtaining survey data of a target diabetic patient, a first fundus image and a second fundus image of the same type; wherein the first fundus image is collected earlier than the second fundus image, and the survey data comprises answer values corresponding to a plurality of diabetic nephropathy-related factors respectively; constructing a target Bayesian network according to the survey data; wherein the target Bayesian network is used to indicate a correlation between the plurality of diabetic nephropathy-related factors and a diagnosis of concurrent diabetic nephropathy in the survey data; calculating a joint probability distribution of the concurrent diabetic nephropathy of the target diabetic patient based on the target Bayesian network, to obtain a first probability value; performing difference feature extraction on the first fundus image and the second fundus image to obtain target difference feature data; performing probability prediction of the concurrent diabetic nephropathy on the target difference feature data to obtain a second probability value; and determining a target probability of the concurrent diabetic nephropathy of the target diabetic patient according to the first probability value and the second probability value.

[0007] In a second aspect, the embodiments of the present disclosure provide a device for predicting diabetic nephropathy, the device comprising: an obtaining module configured to obtain survey data of a target diabetic patient, a first fundus image and a second fundus image of the same type; wherein the first fundus image is collected earlier than the second fundus image, and the survey data comprises answer values corresponding to a plurality of diabetic nephropathy-related factors respectively; a constructing module configured to construct a target Bayesian network according to the survey data; wherein the target Bayesian network is used to indicate a correlation between the plurality of diabetic nephropathy-related factors and a diagnosis of concurrent diabetic nephropathy in the survey data; a calculating module configured to calculate a joint probability distribution of the concurrent diabetic nephropathy of the target diabetic patient based on the target Bayesian network, to obtain a first probability value; an extracting module configured to perform difference feature extraction on the first fundus image and the second fundus image to obtain target difference feature data; a predicting module configured to perform probability prediction of the concurrent diabetic nephropathy on the target difference feature data to obtain a second probability value; and a determining module configured to determine a target probability of the concurrent diabetic nephropathy of the target diabetic patient according to the first probability value and the second probability value.

[0008] In a third aspect, the embodiments of the present disclosure provide an electronic device, comprising a processor and a memory, the memory storing machine executable instructions capable of being executed by the processor, and the processor executes the machine executable instructions to implement the method for predicting diabetic nephropathy.

[0009] In a fourth aspect, the embodiments of the present disclosure provide a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are invoked and executed by a processor, the computer-executable instructions cause the processor to implement the method for predicting diabetic nephropathy.

[0010] The embodiments of the present disclosure bring the following beneficial effects:

[0011] The method, device, electronic equipment and storage medium for predicting diabetic nephropathy compare and analyze fundus images collected by a diabetic patient at different examination periods, collect and analyze data of a questionnaire for the diabetic patient, and predict a probability of the diabetic patient having diabetic nephropathy. A simple, economical and low-trauma early screening system can be provided. The method aims to provide an unprecedented, convenient, economical and low-invasion early screening approach for diabetic nephropathy through efficient data mining and risk assessment strategies. The scheme carefully integrates multi-dimensional data of diabetic patients, performs deep learning and accurate prediction by means of advanced machine learning algorithms, aims to significantly improve the early identification rate of diabetic nephropathy, greatly reduce the diagnosis cost of patients, and provide a scientific basis for subsequent personalized medical decision-making, and finally realize the optimization of medical resources and the precision of disease management.

[0012] Other features and advantages of the present disclosure will be set forth in the descriptions below, and in part will become apparent to those skilled in the art from the descriptions, or can be learned by practice of the present disclosure. The purposes and other advantages of the present disclosure can be realized and achieved by the structures particularly pointed out in the descriptions, claims and drawings.

[0013] In order to make the above-mentioned purposes, features and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0015] Figure 1 An embodiment flowchart of the method for predicting diabetic nephropathy in the embodiments of the present disclosure;

[0016] Figure 2 A schematic diagram of the device for predicting diabetic nephropathy provided by the embodiments of the present disclosure;

[0017] Figure 3A schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the present disclosure will be clearly and completely described below with reference to the drawings. Obviously, the described embodiments are a part rather than all of the embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present disclosure.

[0019] The terms "first", "second", "third", "fourth" and the like (if any) in the description, claims and drawings of the present disclosure are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "comprise" or "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0020] For the convenience of understanding, the specific flow of the embodiments of the present disclosure is described below. Please refer to Figure 1 One embodiment of the method for predicting diabetic nephropathy in the embodiments of the present disclosure comprises:

[0021] In step S10, survey questionnaire data of a target diabetic patient, a first fundus image and a second fundus image of the same type are obtained, the first fundus image is collected earlier than the second fundus image, and the survey questionnaire data includes answer values corresponding to a plurality of diabetic nephropathy related factors respectively.

[0022] It can be understood that the target diabetic patient is a patient who has been diagnosed with diabetes. According to the classification of diabetes, the embodiments of the present disclosure can be used to predict diabetic nephropathy for patients with type 1 diabetes, type 2 diabetes, gestational diabetes and other types of diabetes. The survey questionnaire prepared for patients with different types of diabetes is different, and the obtained survey questionnaire data is also correspondingly different.

[0023] It should be noted that the target diabetic patient can periodically perform the diabetic nephropathy prediction method provided by the embodiments of the present disclosure, and at the end of each examination period, the questionnaire data collected in the current period and the second fundus image are provided, so that high-risk patients with concurrent diabetic nephropathy can be discovered in time through regular tracking and follow-up, and timely intervention treatment can be provided. For example, the target diabetic patient can regularly perform the diabetic nephropathy prediction examination every 3-6 months, and the specific period is not limited herein.

[0024] In the present embodiment, the questionnaire data is the data obtained after the target diabetic patient fills in the target questionnaire, and is the answer sheet of the target questionnaire, which includes answer values corresponding to various diabetic nephropathy-related factors. Each question in the questionnaire can include at least one diabetic nephropathy-related factor, and the diabetic nephropathy-related factor is a factor having a causal relationship with diabetic nephropathy, which can include physiological parameters, medical history parameters, family history parameters, lifestyle parameters, clinical detection parameters, case parameters, etc. The answer values of part of the diabetic nephropathy-related factors of the target diabetic patient can be selectively provided. The more answer values of the diabetic nephropathy-related factors are provided, the higher the accuracy of the diabetic nephropathy prediction is, and vice versa, the higher the convenience provided for the diabetic patient is.

[0025] By way of example and not limitation, the physiological parameters in the diabetic nephropathy-related factors can include age, gender, whether the target diabetic patient feels fatigue, whether the target diabetic patient has edema (such as eyelid swelling, foot swelling, etc.), whether the target diabetic patient has frequent and urgent urination, whether the target diabetic patient finds foam in urine, whether the target diabetic patient feels skin itching, whether the target diabetic patient has anorexia or nausea and vomiting, whether the target diabetic patient has high blood pressure, whether the target diabetic patient feels thirst and polydipsia, whether the target diabetic patient has muscle spasms, and whether the target diabetic patient feels difficulty in breathing, etc. It should be noted that the parameters corresponding to different types of diabetes can be different, and details are not described herein.

[0026] In an embodiment, the medical history parameters in the diabetic nephropathy-related factors can include personal medical history parameters of the target diabetic patient (such as whether the target diabetic patient has diseases such as hypertension, hyperlipidemia, heart disease, stroke, etc.), drug use history (such as whether the target diabetic patient takes drugs such as hypoglycemic drugs, antihypertensive drugs, and lipid-lowering drugs), etc. The medical history parameters having a causal relationship with diabetic nephropathy are not limited herein.

[0027] By way of example and not limitation, the family history parameters in the diabetic nephropathy-related factors can include whether the target diabetic patient has diabetes, whether the target diabetic patient has hypertension, whether the target diabetic patient has a kidney disease, etc. The family history parameters having a causal relationship with diabetic nephropathy are not limited herein.

[0028] In one embodiment, the lifestyle habit parameters in the diabetic nephropathy-associated factors may include the target diabetic patient's eating habit parameters, exercise habit parameters, work and rest habit parameters, whether he or she has a history of smoking, a history of long-term drinking, a habit of staying up late, and other lifestyle habit parameters that have a causal relationship with diabetic nephropathy, and the specifics are not limited here.

[0029] As an example and not a limitation, the clinical test parameters in the diabetic nephropathy-associated factors may include the target diabetic patient's blood pressure test results, urine protein test results, blood test results, blood lipid test results and other clinical indicator data that have a causal relationship with diabetic nephropathy, which are not specifically limited here. It should be noted that the clinical test parameters are historical inspection data, and can also be the inspection results of the current inspection cycle, or the inspection results of the historical inspection cycle and the current inspection cycle, which are not specifically limited here. In one embodiment, the clinical test parameters of the target diabetic patient can be automatically read from the case parameters of the target diabetic patient, or can be filled in and uploaded by themselves, which is highly flexible.

[0030] Furthermore, if the clinical test parameters include the test results of the target diabetic patients in the historical examination cycle and the current examination cycle, then the differences in the various indicators in the test results of the target diabetic patients can be calculated based on the clinical test parameters, and the differences can be used as the basis for the subsequent construction of the target Bayesian network. Specifically, the differences are used as the node values ​​of the corresponding indicators to construct the target Bayesian network, which can more accurately predict diabetic nephropathy.

[0031] In one embodiment, the case parameters in the diabetic nephropathy association factors may include case data uploaded by the target diabetic patient himself or herself, or may include case data of the target diabetic patient obtained through data mining technology. The case data may include the examination and treatment records of the target diabetic patient in different hospitals, and are obtained with the authorization of the target diabetic patient to ensure the security and legality of the data.

[0032] In this embodiment, the first fundus image and the second fundus image are fundus images of the same type, which include but are not limited to fundus images, fluorescence angiography images, and optical coherence tomography images, wherein the fluorescence angiography images include fluorescein fundus angiography (FFA) images and indocyanine green (ICG) angiography images, and the optical coherence tomography images include optical coherence tomography (OCT) images and optical coherence tomography angiography (OCTA) imaging images, and the specific types are not limited here.

[0033] It should be noted that the first fundus image and the second fundus image are fundus images of the target diabetic patient at different times, and the collection time of the first fundus image and the second fundus image is at least one examination cycle apart, and the examination cycle can be set as desired, usually 3-6 months, and the specific period is not limited here.

[0034] Step S20, constructing a target Bayesian network according to the questionnaire data; wherein the target Bayesian network is used to indicate the correlation between the multiple diabetic nephropathy associated factors in the questionnaire data and the diagnosis of concurrent diabetic nephropathy;

[0035] It should be noted that the Bayesian network (Bayesian Network) is also called Bayesian directed acyclic graph (Bayesian Directed Acyclic Graph, BDAG) or probabilistic dependence network (Probabilistic Dependence Network). The Bayesian network is a probabilistic graphical model that represents the probabilistic dependence relationship between variables through a directed acyclic graph. In the medical field, for the prediction of diabetic nephropathy in diabetic patients, the Bayesian network can effectively model the complex relationship between various influencing factors and give a more accurate prediction result.

[0036] It can be understood that the target Bayesian network is constructed by a pre-set Bayesian network model, and the pre-set Bayesian network model is generated based on a large amount of sample data of diabetic patients, wherein the sample data covers the multiple diabetic nephropathy associated factors described above, so that the prediction accuracy of diabetic nephropathy is improved.

[0037] In an embodiment, when constructing the target Bayesian network according to the questionnaire data, first, a plurality of diabetic nephropathy associated factors are extracted from the questionnaire data, and the diabetic nephropathy associated factors and the diagnosis of diabetic nephropathy are taken as nodes of the target Bayesian network. Based on prior knowledge, the directed edge connection structure of the target Bayesian network is constructed, and the target Bayesian network is obtained. Wherein, the two nodes with directed edge connection relationship are nodes with association / causal relationship, each node represents a diabetic nephropathy associated factor, and the end node represents the confirmed diagnosis of diabetic nephropathy.

[0038] In an embodiment, before step S10, a target Bayesian network model is also generated, that is, the above-mentioned preset Bayesian network model. Specifically, the process of generating the target Bayesian network model includes: obtaining sample data of a plurality of diabetic patients, cleaning and preprocessing the sample data, which includes processing missing values and abnormal values, and discretizing or standardizing continuous variables, and the like, to obtain standard sample data.

[0039] Then, the standard sample data is used to construct a basic Bayesian network. Specifically, the basic Bayesian network can be constructed by a pre-trained structure learning model / algorithm, or the different diabetic nephropathy associated factors and the association / causal relationship between the different diabetic nephropathy associated factors and diabetic nephropathy can be determined in combination with the opinions of medical experts, or a combination of the two (structure learning model / algorithm and medical expert opinions) can be used, so as to obtain the basic Bayesian network.

[0040] Further, the basic Bayesian network can be optimized. Specifically, a preset scoring function, such as Bayesian information criterion (BIC), Minimum Description Length (MDL), Akaike Information Criterion (AIC), cross-validation function, and scoring function based on likelihood ratio test, can be used to extract or adjust the basic Bayesian network to obtain a second Bayesian network with a more optimal causal relationship structure than the basic Bayesian network, so as to improve the prediction accuracy of diabetic nephropathy.

[0041] Next, according to the second Bayesian network, the conditional probability distribution of different diabetic nephropathy associated factors and diagnosed diabetic nephropathy can be calculated to obtain the conditional probability distribution data for the calculation of the first probability value. Specifically, the conditional probability distribution of different diabetic nephropathy associated factors and diagnosed diabetic nephropathy can be calculated by a Bayesian estimation algorithm, an expectation maximization (EM) algorithm, a maximum a posteriori (MAP) probability estimation algorithm, an empirical Bayes algorithm, a variational Bayes inference algorithm, etc. The specific algorithm is not limited here.

[0042] Step S30, based on the target Bayesian network, the joint probability distribution of the target diabetic patient complicated with diabetic nephropathy is calculated to obtain the first probability value.

[0043] It can be understood that the core of the Bayesian network is to decompose the complex joint probability distribution into the product of a series of conditional probability distributions. For a target Bayesian network containing n nodes, the joint probability distribution can be expressed as:

[0044] wherein, is the joint probability distribution of all variables, all variables including all diabetic nephropathy associated factors in the target Bayesian network, is the conditional probability distribution of node under the condition of its parent node.

[0045] As an example but not limited, it is assumed that a target Bayesian network contains A, B, C, D nodes, wherein A, B, C are nodes of diabetic nephropathy associated factors, and D is a node of diagnosed diabetic nephropathy. It is assumed that A represents urine protein, B represents serum creatinine, and C represents hypertension. The joint probability distribution of the target diabetic patient can be expressed as:

[0046] P(A, B, C, D) = P(A) * P(B) * P(C) * P(D|A, B)

[0047] Wherein, the conditional probability distribution can be queried from the conditional probability distribution data. Table 1 below is a schematic table of a conditional probability distribution:

[0048] Table 1

[0049] Urine protein (A) Serum creatinine (B) Hypertension (C) Probability of diabetic nephropathy (D=Yes) Low Low No 0.05 Low Low Yes 0.1 Low Medium No 0.15 Low Medium Yes 0.25 Low High No 0.3 Low High Yes 0.4 Medium Low No 0.2 Medium Low Yes 0.35 Medium Medium No 0.4 Medium Medium Yes 0.6 Medium High No 0.5 Medium High Yes 0.7 High Low No 0.35 High Low Yes 0.5 High Medium No 0.5 High Medium Yes 0.75 High High No 0.6 High High Yes 0.85

[0050] ​According to Table 1, the corresponding answer value of each diabetic nephropathy associated factor can be queried to obtain the final probability of diabetic nephropathy. For example, assuming that the urine protein of the target diabetic patient is "medium", the serum creatinine is "high", and the hypertension is "no", the first probability value of the target diabetic patient is 0.5, which is not limited here.

[0051] It should be noted that the conditional probability distribution in actual application is more complex than that of Table 1. The conditional probability distribution is precise data obtained by modeling calculation. The joint probability distribution of the target diabetic patient with concurrent diabetic nephropathy is calculated through the conditional probability distribution, so that the first probability value is accurate, and the prediction accuracy of diabetic nephropathy is improved.

[0052] Step S40, difference feature extraction is performed on the first eye fundus image and the second eye fundus image to obtain target difference feature data;

[0053] It can be understood that the microcirculation of the eye is one of the windows reflecting the health status of the blood vessels of the kidney, and the anatomical structure of the eye has many similarities with the kidney. Therefore, based on the difference between the first eye fundus image and the second eye fundus image, the change of the eye microcirculation of the target diabetic patient in different examination periods can be reflected, so as to reflect whether the change trend of the kidney has the risk of concurrent diabetic nephropathy, and the subtle difference that cannot be recognized by naked eyes can be identified, so that the accuracy of diabetic nephropathy prediction is higher than that of artificial naked eye identification.

[0054] In an embodiment, the difference feature extraction methods corresponding to different types of eye fundus images are also different. For different types of eye fundus images, the corresponding difference feature extraction model is pre-trained, so that the difference features of the eye fundus images collected in different examination periods are more accurate. It should be noted that the diabetic patients are allowed to provide different types of eye fundus images for the probability prediction of concurrent diabetic nephropathy, which brings great convenience to the diabetic patients. The patients can comprehensively consider the examination cost and difficulty, and self-select the type of eye fundus image for examination. No matter which type of eye fundus image, it is non-invasive, and the compliance of the patients is better.

[0055] It should be noted that the target difference feature data can indicate positive changes, negative changes and changes without direction of different fundus image indicators, and the fundus image indicators corresponding to different types of fundus images are different. The fundus image indicators are characteristics that can be reflected by the corresponding type of fundus image and have a strong correlation with diabetic nephropathy. As a rough example of a fundus image indicator, for the fundus image type of fundus image, blood vessels, optic disc and macula are characteristics that can be reflected by the fundus image and have a strong correlation with diabetic nephropathy, that is, the fundus image indicators corresponding to the fundus image type. For the fundus image type of fluorescence angiography image, the flow properties of blood vessels and choroidal blood vessels are characteristics that can be reflected by the fundus image and have a strong correlation with diabetic nephropathy, that is, the fundus image indicators corresponding to the fluorescence angiography image type. For the fundus image type of optical coherence tomography image, the retinal layer structure and microvessels are characteristics that can be reflected by the fundus image and have a strong correlation with diabetic nephropathy, that is, the fundus image indicators corresponding to the optical coherence tomography image type.

[0056] In practical applications, the fundus image indicators can be set more specifically. For example, for the fundus image type of optical coherence tomography image, retinal blood flow density, macular fovea avascular area, choroidal blood flow density, retinal thickness, choroidal thickness, etc. can be used as more specific fundus image indicators, which are not limited here.

[0057] It can be understood that the positive changes of the fundus image indicators include but are not limited to positive changes such as addition, diffusion, increase, enlargement, etc. For example, the addition and diffusion of blood vessels, the increase of the flow property of choroidal blood vessels, the increase of retinal blood flow density, etc. The negative changes include but are not limited to negative changes such as reduction, decrease and contraction, etc. For example, the disappearance and shrinkage of blood vessels, the decrease of the flow property of choroidal blood vessels, the decrease of retinal blood flow density, etc. Changes without direction can include structural changes such as optic disc and macula, changes of retinal layer structure, changes of microvessel structure, and changes of color, etc. which are not limited here.

[0058] Step S50, performing probability prediction of diabetic nephropathy on the target difference feature data to obtain a second probability value;

[0059] It can be understood that after obtaining the target difference feature data, the target difference feature data extracted can be mapped to the probability of diabetic nephropathy by a classifier, so as to obtain the second probability value. Specifically, a full connection layer and a softmax function can be used as a classifier to calculate the probability of diabetic nephropathy corresponding to the target difference feature data, and obtain the second probability value.

[0060] In an embodiment, the fourth confidence corresponding to the first eye fundus image and the second eye fundus image can be obtained from a preset confidence relationship table according to the types of the first eye fundus image and the second eye fundus image and the difference feature extraction model corresponding to the types of the first eye fundus image and the second eye fundus image, for calculation of the target probability. It should be noted that the greater the fourth confidence, the higher the reliability of the second probability value, and the closer to the target probability. The preset confidence relationship table is used to reflect the fourth confidence corresponding to different eye fundus image types and different difference feature extraction models, so that the probability value with higher accuracy occupies a higher proportion, and the accuracy of the prediction of diabetic nephropathy is higher.

[0061] In step S60, the target probability of the target diabetic patient with concurrent diabetic nephropathy is determined according to the first probability value and the second probability value.

[0062] In an embodiment, the first probability value and the second probability value can be weighted and superimposed according to the first preset weight value corresponding to the first probability value and the second preset weight value corresponding to the second probability value, so as to obtain the target probability of the target diabetic patient with concurrent diabetic nephropathy, wherein the first preset weight value can be 0.35, and the second preset weight value can be 0.65, which is not limited here.

[0063] In an embodiment, the confidence corresponding to the first probability value and the second probability value can also be calculated according to the target questionnaire and the eye fundus image (including the first eye fundus image and the second eye fundus image), and the target probability of the target diabetic patient with concurrent diabetic nephropathy can be calculated according to the confidence, the weight, and the first probability value and the second probability value, so that the prediction of diabetic nephropathy is more accurate.

[0064] In an embodiment, when the target probability of the target diabetic patient with concurrent diabetic nephropathy is determined according to the first probability value and the second probability value, it includes: calculating the first confidence of the first probability value according to the number of diabetic nephropathy related factors in the questionnaire data; calculating the second confidence of the first probability value according to the parameter number of the specified type of diabetic nephropathy related factor in the questionnaire data; determining the third confidence of the first probability value according to the first confidence and the second confidence; determining the fourth confidence of the second probability value according to the types of the first eye fundus image and the second eye fundus image; and weighting and superimposing the multiplication results of the third confidence and the first probability value and the multiplication results of the second probability value and the fourth confidence to obtain the target probability of the target diabetic patient with concurrent diabetic nephropathy.

[0065] It can be understood that the greater the number of diabetes nephropathy associated factors in the questionnaire data, the higher the first confidence of the first probability value, the greater the number of parameters of the specified type of diabetes nephropathy associated factors in the questionnaire data, the higher the second confidence of the first probability value, and the third confidence of the first probability value can be obtained by weighting and superimposing the first confidence and the second confidence according to the weights corresponding to the number and the parameter number of the diabetes nephropathy associated factors, so that the first probability value can have different confidences according to the different importance degrees corresponding to the number and the parameter number of the diabetes nephropathy associated factors in the questionnaire data, so that the prediction of diabetes nephropathy can be adjusted according to the comprehensiveness of the questionnaire filled by the patient, and a more accurate prediction effect can be achieved.

[0066] In the embodiment, different types of fundus images also have different degrees of influence on the confidence of the second probability value, and the type of fundus influence and the confidence of the second probability can be set through a preset corresponding relationship, which is not limited here.

[0067] Further, the third confidence is multiplied by the first probability value to obtain a first multiplication result, the fourth confidence is multiplied by the second probability value to obtain a second multiplication result, and the first multiplication result and the second multiplication result are weighted and superimposed according to the preset weights corresponding to the first multiplication result and the second multiplication result, so as to obtain the target probability of the target diabetic patient complicated with diabetes nephropathy.

[0068] The method for predicting diabetes nephropathy provided by the above embodiment compares and analyzes the fundus images collected at different examination cycles of the diabetic patient, collects and analyzes the data of the questionnaire filled by the diabetic patient, and predicts the probability of the diabetic patient complicated with diabetes nephropathy, so as to provide a simple, economical and low-trauma early screening system. The method aims to provide an unprecedented, convenient, economical and low-invasion early screening way for diabetes nephropathy through efficient data mining and risk assessment strategy. This scheme carefully integrates the multi-dimensional data of diabetic patients, uses advanced machine learning algorithms for deep learning and accurate prediction, aims to significantly improve the early identification rate of diabetes nephropathy, greatly reduce the diagnosis cost of patients, and provide a scientific basis for subsequent personalized medical decision-making, and finally realize the optimization of medical resources and the precision of disease management.

[0069] Next, the specific calculation method of the first probability value is described.

[0070] In an implementation, when the target difference feature data is obtained by performing difference feature extraction on the first eye fundus image and the second eye fundus image, the method comprises: preprocessing the first eye fundus image and the second eye fundus image by the first convolutional neural network to obtain the first eye fundus image and the second eye fundus image meeting the feature extraction requirement; performing feature extraction on the first eye fundus image and the second eye fundus image meeting the feature extraction requirement according to the types of the first eye fundus image and the second eye fundus image to obtain first feature data of the first eye fundus image and second feature data of the second eye fundus image; performing difference calculation on the first feature data and the second feature data to obtain target difference feature data; and performing significant feature capturing on the target difference feature data by the attention mechanism to obtain the target difference feature data.

[0071] In the implementation, the first eye fundus image and the second eye fundus image are eye fundus images collected in two different examination periods, and the two eye fundus images can be collected in different hospitals using different types of medical equipment. Therefore, the attributes of the two eye fundus images can be greatly different. The first convolutional neural network is used to preprocess the two eye fundus images, so that the influence attributes of the two eye fundus images are aligned, and the two eye fundus images meet the requirements of subsequent artificial intelligence model processing. Compared with preprocessing the two eye fundus images by image processing technology, the first convolutional neural network itself also belongs to the structure / module of the artificial intelligence model. Therefore, the preprocessing of the first convolutional neural network can more accurately make the two eye fundus images meet the requirements of the artificial intelligence model processing.

[0072] It should be noted that the first convolutional neural network is a pre-trained model structure, which can process different eye fundus images into images meeting the same feature extraction requirement. In an implementation, preprocessing the first eye fundus image and the second eye fundus image by the first convolutional neural network comprises: performing size alignment, noise reduction, contrast enhancement, and distortion correction on the first eye fundus image and the second eye fundus image by the first convolutional neural network, so as to obtain the first eye fundus image and the second eye fundus image meeting the feature extraction requirement in terms of size, noise amount, contrast, and distortion amount. Therefore, subsequent artificial intelligence-related processing can be more accurate and efficient.

[0073] In the implementation, different feature extraction algorithms / models are used to extract features for different types of eye fundus images to obtain the first feature data of the first eye fundus image and the second feature data of the second eye fundus image. In this way, the features of the eye fundus are more consistent with the features that the corresponding type of eye fundus image is good at presenting, so as to improve the accuracy of diabetic nephropathy prediction.

[0074] In an embodiment, before the first feature data and the second feature data are subjected to the differential calculation, the first feature data and the second feature data can be registered so that the feature points of the two are one-to-one corresponding, so that the subsequent differential calculation is more efficient and more accurate. In another embodiment, the first fundus image and the second fundus image can also be registered when they are preprocessed, or after the first fundus image and the second fundus image that meet the requirements of feature extraction are obtained. The first feature data and the second feature data obtained subsequently are already registered feature data, and the specific registration time is not limited here.

[0075] In the embodiment, the target differential feature data between the first feature data and the second feature data is obtained by differential calculation, and the target differential feature data is used to indicate the difference between the first feature data and the second feature data. By differential calculation, the growth, atrophy or morphological change of the fundus lesion area can be effectively detected, so that the subtle changes of the fundus can also be perceived. Compared with the traditional direct lesion area recognition of the first fundus image, the subtle changes of the patient's fundus can be better captured, so that the budding of the lesion can also be found, and then effective prevention and treatment medical means or further examination means can be taken, so that the diabetic nephropathy can be more effectively controlled.

[0076] In an embodiment, when the first feature data and the second feature data are subjected to the differential calculation, the first feature data and the second feature data can be first divided into feature points of the same size and the same number, and then the SURF (Speeded Up Robust Features) feature descriptors of each feature point are calculated. The corresponding feature points in the first feature data and the second feature data are matched by calculating the distance between each feature descriptor. According to the matched feature points, the first feature data and the second feature data are registered so that the feature points are aligned. Finally, the registered first feature data and second feature data are subtracted at the pixel level, and the target differential feature data is obtained. It should be noted that the above feature data is recorded by an image, so the above feature data is actually an image.

[0077] It can be understood that the target differential feature data objectively presents the difference between the first feature data and the second feature data, which includes the difference that has reference value for predicting diabetic nephropathy and the difference that has no reference value for predicting diabetic nephropathy. Therefore, in the embodiment, the attention mechanism is used to capture the significant features of the target differential feature data, so that the attention of feature capture is concentrated on the difference that has reference value for predicting diabetic nephropathy, and the target difference feature data is obtained, which plays a crucial role in predicting diabetic nephropathy.

[0078] It should be noted that according to different types of fundus images, the differences that need to be paid special attention to, that is, the differences that have reference value, are not the same, and therefore, in an embodiment, when the target difference feature data is captured by the attention mechanism, according to the types of the first fundus image and the second fundus image, the target difference feature data is captured by the corresponding attention mechanism, so that the extraction of the difference features is more accurate and more precise.

[0079] It can be understood that according to different types of fundus images, the feature extraction method adopted and the dimension of the extracted features are also different, and in an embodiment, according to the types of the first fundus image and the second fundus image, the first feature data of the first fundus image and the second feature data of the second fundus image are obtained by performing feature extraction on the first fundus image and the second fundus image that meet the feature extraction requirements, including: if the types of the first fundus image and the second fundus image are fundus images, the features of blood vessels, optic discs, and maculas of the first fundus image and the second fundus image are extracted respectively to obtain the first feature data of the first fundus image and the second feature data of the second fundus image; if the types of the first fundus image and the second fundus image are fluorescence angiography images, the features of blood vessels and / or choroidal blood vessels of the first fundus image and the second fundus image are extracted respectively to obtain the first feature data of the first fundus image and the second feature data of the second fundus image; if the types of the first fundus image and the second fundus image are optical coherence tomography images, the features of retinal layer structures and / or microvessels of the first fundus image and the second fundus image are extracted respectively to obtain the first feature data of the first fundus image and the second feature data of the second fundus image.

[0080] For fundus images, that is, fundus photos, the features of blood vessels, optic discs, and maculas in the fundus photos are all features that have reference value for predicting diabetic nephropathy, and therefore, when training the model structure for extracting the features of blood vessels, optic discs, and maculas from fundus images, the first sample data labeled with these features are used for training, and specifically, the feature labels of the first sample data include:

[0081] (1) Blood vessel features: blood vessel calcification (blood vessel wall calcification, manifested as bright white lines), blood vessel distortion (manifested as distortion of blood vessels), microaneurysm (manifested as small, round blood vessel dilation);

[0082] (2) Microvascular lesion features: microaneurysms (presenting as small, round, red or orange lesions), hard exudates (presenting as yellow, waxy deposits, often in the macula), cotton wool spots (presenting as white or grayish white avascular areas, indicating ischemia), neovascularization (presenting as newly grown abnormal blood vessels, often accompanied by hemorrhage), venous beading (presenting as focal venous dilation, diffuse venous dilation, and venous loop formation);

[0083] (3) Ganglion cell layer thickness features: the thickness of the ganglion cell layer reflects the number of retinal neurons, and is correlated with diabetic nephropathy;

[0084] (4) Macular morphological change features: the macula is the central area of the retina, and its morphological changes can reflect the damage of diabetes to the retina.

[0085] It should be noted that the above feature markers are all for fundus images, and the features that have reference value for predicting diabetic nephropathy are features that can be extracted by the feature extraction model trained by the first sample data with at least one of the above feature markers, so as to obtain a model capable of extracting the above features that have reference value for predicting diabetic nephropathy, for respectively extracting the features of blood vessels, optic disc, and macula of the first fundus image and the second fundus image.

[0086] In this embodiment, for the fluorescence angiography image, the features of the blood vessels and / or the flow properties of the choroidal blood vessels therein are all features that have reference value for predicting diabetic nephropathy, so when training the model structure for extracting the features of the blood vessels and / or the flow properties of the choroidal blood vessels of the fundus image, the second sample data with these feature markers is used for training, specifically, the feature markers of the second sample data include:

[0087] (1) Capillary leakage feature: leakage of fluorescein dye from blood vessels to surrounding tissues, presenting as fluorescent spots or plaques, indicating increased vascular wall permeability;

[0088] (2) Blood vessel occlusion feature: complete or partial occlusion of blood vessels, leading to local ischemia;

[0089] (3) Fluorescein filling duration feature: the time for fluorescein dye to reach the choroid from the posterior ciliary artery, reflecting the speed of choroidal blood flow;

[0090] (4) Fluorescein distribution feature: uniformity of fluorescein distribution in the choroid, reflecting the permeability of choroidal microvessels;

[0091] (5) Fluorescein decay time: the time for fluorescein to clear from the choroid, reflecting the speed of venous return.

[0092] It should be noted that the above feature markers are all for the fundus image of the type of fluorescence angiography image, and the features having reference value for predicting diabetic nephropathy are trained by the second sample data with at least one of the above feature markers, so as to obtain a model capable of extracting the features having reference value for predicting diabetic nephropathy, and used for extracting the features of blood vessels and / or choroidal blood vessels of the first fundus image and the second fundus image, respectively.

[0093] In this embodiment, for the optical coherence tomography image, the features of the retinal layer structure and / or microvessels therein are all features having reference value for predicting diabetic nephropathy, and therefore, when training the model structure for extracting the features of the retinal layer structure and / or microvessels of the fundus image, the third sample data with these feature markers are trained, and specifically, the feature markers of the third sample data include:

[0094] (1) Retinal thickness feature: including retinal nerve fiber layer thickness, inner nuclear layer thickness, outer nuclear layer thickness, etc.

[0095] (2) Internal limiting membrane fold feature: the internal limiting membrane is the innermost layer of the retina.

[0096] (3) Retinal microvessel feature: OCT blood vessel imaging (OCTA) can display information such as the density, distribution and blood flow velocity of retinal microvessels.

[0097] (4) Retinal inner layer cavity feature.

[0098] It should be noted that the above feature markers are all for the fundus image of the type of fluorescence angiography image, and the features having reference value for predicting diabetic nephropathy are trained by the second sample data with at least one of the above feature markers, so as to obtain a model capable of extracting the features having reference value for predicting diabetic nephropathy, and used for extracting the features of blood vessels and / or choroidal blood vessels of the first fundus image and the second fundus image, respectively.

[0099] Specifically, when training the feature extraction model by using the above sample data (including the first sample data, the second sample data, and the third sample data), a model containing a visual geometry group (VGG) and a residual neural network (ResNet) structure can be used as a basic model for training, the basic model is trained by inputting the above sample data into the basic model, and when the degree of decline of the value of the loss function of the basic model is less than a preset threshold, it is determined that the model training is completed, and the basic model after training is determined as the feature extraction model.

[0100] In an embodiment, in the significant feature capturing of the target difference feature data by the attention mechanism, the target difference feature data is obtained by: determining the feature dimensions related to the multiple concurrent diabetic nephropathies in the target difference feature data according to the types of the first fundus image and the second fundus image; for each feature dimension related to the concurrent diabetic nephropathy, as an attention feature channel, the following processing is performed: calculating the target difference feature data by three linear layers to obtain a query vector linear matrix, a key value vector linear matrix, and a value vector linear matrix; performing dot product calculation on the query vector linear matrix and the key value vector linear matrix to obtain a similarity matrix between the query vector linear matrix and the key value vector linear matrix; performing normalization processing on the similarity matrix to obtain the attention weight corresponding to each attention feature channel; multiplying the attention weight corresponding to each attention feature channel with the value vector linear matrix to obtain the first difference feature data corresponding to each attention feature channel; and based on the gating mechanism, fusing the first difference feature data corresponding to all attention feature channels to obtain the target difference feature data.

[0101] It should be noted that different types of fundus images have different feature dimensions related to concurrent diabetic nephropathy. Specifically, the feature dimensions related to concurrent diabetic nephropathy corresponding to different types of fundus images can be the same as the feature labels in the sample data of the corresponding type of fundus image. For example, the feature dimensions related to concurrent diabetic nephropathy corresponding to the fundus image of the fluorescence angiography type can include the feature labels in the second sample data described above. Similarly, the feature dimensions related to concurrent diabetic nephropathy corresponding to the fundus image of the optical coherence tomography type can include the feature labels in the third sample data described above. The specific embodiments are not limited here.

[0102] In the embodiment, an attention feature channel is created for each feature dimension related to concurrent diabetic nephropathy. The processing performed by each attention feature channel includes: calculating the target difference feature data by three linear layers to obtain a query vector linear matrix, a key value vector linear matrix, and a value vector linear matrix; performing dot product calculation on the query vector linear matrix and the key value vector linear matrix to obtain a similarity matrix between the query vector linear matrix and the key value vector linear matrix; performing normalization processing on the similarity matrix to obtain the attention weight corresponding to each attention feature channel; multiplying the attention weight corresponding to each attention feature channel with the value vector linear matrix to obtain the first difference feature data corresponding to each attention feature channel.

[0103] Specifically, the three linear layers are query (Query), key (Key), and value (Value) linear layers, and the target difference feature data is calculated through the three linear layers, so as to obtain the Query vector linear matrix, the Key vector linear matrix, and the Value vector linear matrix, i.e., the query vector linear matrix, the key vector linear matrix, and the value vector linear matrix.

[0104] Then, the similarity between the Query vector linear matrix and the Key vector linear matrix is calculated by dot product, a similarity matrix is obtained, the similarity matrix is normalized by Softmax, and the attention weight is obtained. The attention weight is multiplied by the Value vector linear matrix, and the weighted feature map, i.e., the first difference feature data corresponding to each feature dimension in each attention feature channel, is obtained.

[0105] It should be noted that after obtaining the first difference feature data corresponding to each feature dimension in each attention feature channel, all the first difference feature data in all the attention feature channels need to be fused. In this embodiment, the Gating Mechanism is used to fuse the first difference feature data corresponding to all the feature dimensions in all the attention feature channels, so as to obtain the target difference feature data.

[0106] In one embodiment, when the first difference feature data corresponding to all the feature dimensions in all the attention feature channels is fused based on the Gating Mechanism, the first difference feature data corresponding to all the feature dimensions in all the attention feature channels can be fused by a preset gating formula, wherein the preset gating formula includes a target gating variable, and the calculation method of the target gating variable includes: splicing the attention weights of the attention feature channels, inputting them into a fully connected layer to obtain an initial gating variable, and normalizing the initial gating variable by a Sigmoid function to obtain the target gating variable.

[0107] In one embodiment, when the first difference feature data corresponding to all the feature dimensions in all the attention feature channels is fused based on the Gating Mechanism, the first difference feature data corresponding to each attention feature channel is multiplied by the corresponding attention weight and the target gating variable, the obtained second difference feature data is added, and the final fused feature, i.e., the target difference feature data, is obtained.

[0108] It should be noted that the present embodiment fuses different channel features in the difference of the diabetic nephropathy fundus feature map based on the Gating Mechanism, which has the advantages of strong flexibility and strong interpretability, and can effectively improve the performance of the model and make the result of diabetic nephropathy prediction more interpretable.

[0109] In an embodiment, when the target difference feature data is used to predict the probability of concurrent diabetic nephropathy to obtain a second probability value, the following steps are included: dimensionality reduction of the target difference feature data is performed by a fully connected layer including a nonlinear transformation function to obtain reduced dimension difference feature data; and a probability distribution calculation of concurrent diabetic nephropathy and non-concurrent diabetic nephropathy is performed on the reduced dimension difference feature data, and the obtained probability of concurrent diabetic nephropathy is determined as the second probability value.

[0110] It can be understood that the embodiment is to map the target difference feature data to a low-dimensional space, that is, to map the extracted high-dimensional difference feature vector to a low-dimensional feature space through a fully connected layer. The dimension of the low-dimensional space is usually smaller than the dimension of the original feature vector, which can reduce the complexity of the model and improve the calculation efficiency. By introducing a nonlinear transformation function such as a rectified linear unit (ReLU) function in the fully connected layer, the expression ability of the model can be enhanced, and the reduced dimension difference feature data can better express the features of concurrent diabetic nephropathy.

[0111] Then, the embodiment also calculates the probability distribution, that is, the output of the fully connected layer is passed through a softmax function to obtain a probability distribution, which represents the probability of the sample belonging to each class. The classes include concurrent diabetic nephropathy and non-concurrent diabetic nephropathy. Finally, the output second probability value is the probability of belonging to the concurrent diabetic nephropathy class.

[0112] Next, the specific calculation method of the first probability value is described.

[0113] In an embodiment, before the steps of obtaining the survey data of the target diabetic patient, the first eye fundus image and the second eye fundus image of the same type, the following steps are further included: obtaining a plurality of first sample data of diabetic patients; wherein the first sample data includes patient parameters of a plurality of diabetic nephropathy associated factors; constructing a causal relationship of the first sample data by a pre-set structural learning algorithm to obtain a plurality of first Bayesian networks; determining a second Bayesian network with the highest score from the plurality of first Bayesian networks according to the Bayesian information criterion; calculating a conditional probability distribution of the second Bayesian network by maximizing a likelihood function to obtain conditional probability distribution data of the plurality of diabetic nephropathy associated factors; and based on the target Bayesian network, calculating a joint probability distribution of concurrent diabetic nephropathy of the target diabetic patient to obtain the first probability value, including: obtaining a target conditional probability of each node in the target Bayesian network from the conditional probability distribution data; and calculating a posterior probability of concurrent diabetic nephropathy of the target diabetic patient according to the target conditional probability of each node in the target Bayesian network to obtain the first probability value.

[0114] It can be understood that the first sample data contains patient parameters of multiple diabetic nephropathy associated factors of multiple voluntary data-providing diabetic patients, and the patient parameters correspond to the answer values of the multiple diabetic nephropathy associated factors in the target questionnaire. The patient parameters are the parameter values of the corresponding diabetic nephropathy associated factors of the diabetic patients. The number of diabetic nephropathy associated factors in the first sample data can be more than the number of diabetic nephropathy associated factors in the target questionnaire, so that the first sample data can cover a more comprehensive questionnaire and improve the prediction accuracy of diabetic nephropathy.

[0115] In the embodiment, when determining the second Bayesian network, a structure learning algorithm such as a greedy search algorithm K2 or a hill climbing algorithm can be used to construct the causal relationship of the first sample data, and the second Bayesian network is determined by BIC score. Hereinafter, the K2 algorithm and the BIC score are taken as examples for description:

[0116] It can be understood that the K2 algorithm, as a structure learning algorithm, plays an important role in constructing the Bayesian network model for predicting diabetic nephropathy. It can automatically learn the causal relationship between variables (i.e., diabetic nephropathy associated factors) and confirmed diabetic nephropathy from data, thereby helping us better understand the pathogenesis of diabetic nephropathy and establish a more accurate prediction model.

[0117] First, according to the domain knowledge (such as the pathogenesis of diabetic nephropathy) or the results of data analysis, the variables are sorted, preferably, the diabetic nephropathy associated factors having greater impact on diabetic nephropathy, such as blood glucose, blood pressure, etc. are sorted first. Then, each variable is taken as a node, and there is no connection between the nodes at the beginning. The edges are added through the following iterative process:

[0118] (1) For each node (e.g., urine protein), traverse all the nodes in front of it (e.g., blood glucose, blood pressure);

[0119] (2) Calculate the BIC score increment brought by connecting the current node with each preceding node;

[0120] (3) Select the edge that brings the maximum score increment and add it to the network. For example, if it is found that the connection between blood glucose and urine protein brings the maximum model score increment, an edge is added between them;

[0121] (4) Repeat steps 1-3 until the parent nodes of all nodes are determined, and the second Bayesian network is obtained.

[0122] It should be noted that the K2 algorithm has the following advantages in the prediction of diabetic nephropathy:

[0123] Automatic learning structure: No need to define the network structure in advance, can automatically learn the relationship between variables from data;

[0124] Strong interpretability: The learned network structure can intuitively show the causal relationship between variables, which helps us understand the pathogenesis of diabetic nephropathy;

[0125] High prediction accuracy: Through the learned network structure, the risk of kidney disease in diabetic patients can be accurately predicted.

[0126] It can be understood that maximizing the likelihood function is a common parameter estimation method, the core idea of which is: given a data set, find a set of parameter values that maximize the probability of the data set under that parameter, in other words, we think the most likely parameter to produce the observed data is the best parameter.

[0127] It should be noted that in the second Bayesian network, the maximum likelihood function is used to estimate the conditional probability table of each node. For a node X, its conditional probability table represents the probability of X taking various values given the values of its parent nodes, i.e., the conditional probability distribution data, where each node corresponds to a diabetic nephropathy associated factor.

[0128] In this embodiment, after obtaining the conditional probability distribution data of multiple diabetic nephropathy associated factors, the joint probability distribution of the target diabetic patient with concurrent diabetic nephropathy can be calculated according to the conditional probability distribution data, and the first probability value is obtained.

[0129] Specifically, the nodes in the target Bayesian network can be substituted into the preset Bayesian network model, the target conditional probability of each node in the target Bayesian network can be obtained through the conditional probability distribution data in the Bayesian network model, and the posterior probability of the terminal node (i.e., the diagnosed diabetic nephropathy) can be calculated through the Bayesian formula, thereby obtaining the first probability value.

[0130] It should be noted that the Bayesian network has the following advantages in diabetic nephropathy prediction:

[0131] (1) Causal relationship modeling, Bayesian network can intuitively show the causal relationship between variables;

[0132] (2) Uncertainty processing, Bayesian network uses probability to represent the relationship between variables, which can well handle the uncertainty in medical data, making the model more robust;

[0133] (3) Multivariate analysis, Bayesian network can consider the interaction between multiple variables at the same time, unlike traditional statistical methods that often only consider the relationship between two variables, which can comprehensively evaluate various influencing factors;

[0134] (4) Incremental learning, when new data or knowledge appears, the Bayesian network can be updated incrementally, thereby improving the accuracy of the model, so that the model can adapt to new data;

[0135] (5) Predictive ability, according to the individual characteristics of the patient, such as age, gender, blood glucose control, etc., the occurrence and development of the disease can be personalized predicted, and personalized prediction is provided;

[0136] (6) Strong interpretability, the structure and parameters of the Bayesian network have clear meanings, so that the model has strong interpretability, which assists clinical decision-making;

[0137] (7) Handling missing data, Bayesian networks can use existing information to infer missing data, thereby improving data utilization and data utilization.

[0138] Corresponding to the above method embodiments, see Figure 2 A schematic diagram of a device for predicting diabetic nephropathy is shown, and the device comprises: an acquisition module 21, configured to acquire survey questionnaire data of a target diabetic patient, a first eye fundus image and a second eye fundus image of the same type; wherein the first eye fundus image is collected earlier than the second eye fundus image, and the survey questionnaire data comprises answer values corresponding to a plurality of diabetic nephropathy associated factors respectively; a construction module 22, configured to construct a target Bayesian network according to the survey questionnaire data; wherein the target Bayesian network is used to indicate the association between the plurality of diabetic nephropathy associated factors in the survey questionnaire data and the diagnosis of concurrent diabetic nephropathy; a calculation module 23, configured to calculate the joint probability distribution of the concurrent diabetic nephropathy of the target diabetic patient based on the target Bayesian network, and obtain a first probability value; an extraction module 24, configured to extract difference features from the first eye fundus image and the second eye fundus image, and obtain target difference feature data; a prediction module 25, configured to perform probability prediction of the concurrent diabetic nephropathy on the target difference feature data, and obtain a second probability value; and a determination module 26, configured to determine the target probability of the concurrent diabetic nephropathy of the target diabetic patient according to the first probability value and the second probability value.

[0139] The device for predicting diabetic nephropathy compares and analyzes fundus images collected from the diabetic patient at different examination periods, and collects and analyzes the data of the questionnaire of the diabetic patient, so as to predict the probability of the diabetic patient complicated with diabetic nephropathy, and to provide a simple, economical and low-trauma early screening system. The method aims to provide an unprecedented, convenient, economical and low-invasive early screening way for diabetic nephropathy through efficient data mining and risk assessment strategy. The scheme carefully integrates multi-dimensional data of the diabetic patient, and performs deep learning and accurate prediction by means of advanced machine learning algorithm, so as to significantly improve the early identification rate of diabetic nephropathy, greatly reduce the diagnosis cost of the patient, and provide a scientific basis for subsequent personalized medical decision-making, and finally realize the optimization of medical resources and the precision of disease management.

[0140] Optionally, the extraction module 24 comprises: a preprocessing unit configured to preprocess the first fundus image and the second fundus image by using a first convolutional neural network to obtain the first fundus image and the second fundus image meeting the feature extraction requirement; an extraction unit configured to extract features from the first fundus image and the second fundus image meeting the feature extraction requirement according to the types of the first fundus image and the second fundus image to obtain first feature data of the first fundus image and second feature data of the second fundus image; a calculation unit configured to perform differential calculation on the first feature data and the second feature data to obtain target differential feature data; and a capturing unit configured to capture significant features from the target differential feature data by using an attention mechanism to obtain target difference feature data.

[0141] Optionally, the extraction unit is specifically configured to: if the types of the first fundus image and the second fundus image are fundus images, extract features of blood vessels, optic discs and maculas from the first fundus image and the second fundus image respectively to obtain the first feature data of the first fundus image and the second feature data of the second fundus image; if the types of the first fundus image and the second fundus image are fluorescence angiography images, extract features of blood vessels and / or flow properties of choroidal vessels from the first fundus image and the second fundus image respectively to obtain the first feature data of the first fundus image and the second feature data of the second fundus image; and if the types of the first fundus image and the second fundus image are optical coherence tomography images, extract features of retinal layer structures and / or microvessels from the first fundus image and the second fundus image respectively to obtain the first feature data of the first fundus image and the second feature data of the second fundus image.

[0142] Optionally, the capturing unit is configured to: determine feature dimensions related to the plurality of concurrent diabetic nephropathies in the target differential feature data according to types of the first fundus image and the second fundus image; for each feature dimension related to the concurrent diabetic nephropathies, as an attention feature channel, perform the following processing: calculate the target differential feature data through three linear layers to obtain a query vector linear matrix, a key value vector linear matrix, and a value vector linear matrix; perform dot product calculation on the query vector linear matrix and the key value vector linear matrix to obtain a similarity matrix between the query vector linear matrix and the key value vector linear matrix; perform normalization processing on the similarity matrix to obtain an attention weight corresponding to each attention feature channel; multiply the attention weight corresponding to each attention feature channel with the value vector linear matrix to obtain first differential feature data corresponding to each attention feature channel; and fuse the first differential feature data corresponding to all the attention feature channels based on a gating mechanism to obtain the target differential feature data.

[0143] Optionally, the prediction module 25 is specifically configured to: perform feature dimension reduction on the target differential feature data through a fully connected layer containing a nonlinear transformation function to obtain reduced dimension differential feature data; and perform probability distribution calculation of the concurrent diabetic nephropathy and the non-concurrent diabetic nephropathy on the reduced dimension differential feature data, and determine a concurrent diabetic nephropathy probability obtained as the second probability value.

[0144] Optionally, the device further includes: an obtaining module configured to obtain first sample data of a plurality of diabetic patients; wherein the first sample data includes patient parameters of a plurality of diabetic nephropathy associated factors; a constructing module configured to perform causal relationship construction on the first sample data through a preset structure learning algorithm to obtain a first Bayesian network; a scoring module configured to determine a second Bayesian network with the highest score from the first Bayesian network according to a Bayesian information criterion; and an algorithm module configured to perform conditional probability distribution calculation on the second Bayesian network through a maximum likelihood function to obtain conditional probability distribution data of the plurality of diabetic nephropathy associated factors; the calculation module 23 is specifically configured to: obtain a target conditional probability of each node in a target Bayesian network from the conditional probability distribution data; and a posterior module configured to calculate a posterior probability of concurrent diabetic nephropathy of a target diabetic patient according to the target conditional probability of each node in the target Bayesian network to obtain the first probability value.

[0145] Optionally, the determining module 26 is specifically configured to: calculate a first confidence degree of the first probability value according to a number of the diabetes nephropathy related factors in the questionnaire data; calculate a second confidence degree of the first probability value according to a number of parameters of the diabetes nephropathy related factors of the specified type in the questionnaire data; determine a third confidence degree of the first probability value according to the first confidence degree and the second confidence degree; determine a fourth confidence degree of the second probability value according to the types of the first fundus image and the second fundus image; and obtain the target probability of the target diabetic patient complicated with the diabetes nephropathy by weighted superposition of a multiplication result of the third confidence degree and the first probability value and a multiplication result of the second probability value and the fourth confidence degree.

[0146] The embodiment also provides an electronic device including a processor and a memory, the memory storing machine executable instructions capable of being executed by the processor, and the processor executes the machine executable instructions to implement the method for predicting the diabetes nephropathy. The electronic device can be a server or a terminal device.

[0147] Referring to Figure 3 The electronic device includes a processor 100 and a memory 101, the memory 101 storing machine executable instructions capable of being executed by the processor 100, and the processor 100 executes the machine executable instructions to implement the method for predicting the diabetes nephropathy.

[0148] Further, Figure 3 The electronic device also includes a bus 102 and a communication interface 103, and the processor 100, the communication interface 103 and the memory 101 are connected through the bus 102.

[0149] The memory 101 can include a high-speed random access memory (RAM) and can also include a non-volatile memory such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used. The bus 102 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the figure to represent only one bus or one type of bus.

[0150] The processor 100 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 100 or the instruction in the form of software. The processor 100 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiment of the present disclosure can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiment of the present disclosure can be directly embodied as a hardware code processor to execute, or be executed by a combination of hardware and software modules in the code processor. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 101, and the processor 100 reads the information in the memory 101, and combines the hardware to complete the steps of the method of the above embodiment.

[0151] The embodiment also provides a computer readable storage medium, the computer readable storage medium stores computer executable instructions, when the computer executable instructions are called and executed by the processor, the computer executable instructions cause the processor to implement the method for predicting diabetic nephropathy.

[0152] The method, device, electronic equipment and storage medium computer program product for predicting diabetic nephropathy provided by the embodiment of the present disclosure include a computer readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the method described in the foregoing method embodiment. For specific implementation, please refer to the method embodiment, which will not be described here.

[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0154] In addition, in the description of the embodiments of the present disclosure, unless explicitly defined and limited otherwise, the terms "mount", "connect", "connection" should be understood in a broad sense, for example, can be fixed connection, can also be detachable connection, or integrally connected; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through an intermediate medium, can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present disclosure can be understood according to the specific circumstances.

[0155] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present disclosure essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0156] In the description of the present disclosure, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present disclosure. In addition, the terms "first", "second", "third" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance.

[0157] Finally, it should be noted that: the above embodiments are only specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and not to limit them, the protection scope of the present disclosure is not limited thereto, although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art within the technical scope disclosed by the present disclosure can modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for predicting diabetic nephropathy, characterized in that: The method comprises: Obtaining questionnaire data, a first fundus image and a second fundus image of the same type from a target diabetic patient; wherein the first fundus image is collected earlier than the second fundus image; the questionnaire data includes answer values ​​corresponding to a plurality of diabetic nephropathy-associated factors, wherein diabetic nephropathy-associated factors are factors having a causal relationship with diabetic nephropathy, including physiological parameters, medical history parameters, family genetic history parameters, lifestyle parameters, clinical examination parameters, and case parameters; and the types of images include: fundus images, fluorescein angiography images, and optical coherence tomography images; Constructing a target Bayesian network based on the questionnaire data; wherein the target Bayesian network is used to indicate the correlation between multiple diabetic nephropathy-related factors and the diagnosis of concurrent diabetic nephropathy in the questionnaire data; Calculating the joint probability distribution of the target diabetic patient complicated with diabetic nephropathy based on the target Bayesian network to obtain a first probability value; performing difference feature extraction on the first fundus image and the second fundus image to obtain target difference feature data; Performing a probability prediction of concurrent diabetic nephropathy on the target difference feature data to obtain a second probability value; Determining a target probability of the target diabetic patient having diabetic nephropathy according to the first probability value and the second probability value; Preprocessing the first fundus image and the second fundus image by a first convolutional neural network to obtain the first fundus image and the second fundus image that meet the feature extraction requirements; According to the types of the first fundus image and the second fundus image, feature extraction is performed on the first fundus image and the second fundus image that meet the feature extraction requirements to obtain first feature data of the first fundus image and second feature data of the second fundus image, including: If the first fundus image and the second fundus image are fundus images, extracting features of blood vessels, optic discs, and macula from the first fundus image and the second fundus image, respectively, to obtain first feature data of the first fundus image and second feature data of the second fundus image; If the first fundus image and the second fundus image are fluorescein angiography images, extracting features of blood vessels and / or choroidal blood vessels from the first fundus image and the second fundus image to obtain first feature data of the first fundus image and second feature data of the second fundus image; If the types of the first fundus image and the second fundus image are optical coherence tomography images, the features of the retinal layer structures and / or microvessels are extracted from the first fundus image and the second fundus image respectively to obtain the first feature data of the first fundus image and the second feature data of the second fundus image.

2. The method according to claim 1, characterized in that The step of extracting difference features from the first fundus image and the second fundus image to obtain target difference feature data includes: performing a differential calculation on the first feature data and the second feature data to obtain target differential feature data; The target differential feature data is captured by an attention mechanism to obtain the target differential feature data.

3. The method according to claim 2, characterized in that The step of capturing significant features of the target differential feature data through an attention mechanism to obtain target differential feature data includes: determining, according to the types of the first fundus image and the second fundus image, feature dimensions associated with multiple concurrent diabetic nephropathy in the target differential feature data; For each feature dimension related to concurrent diabetic nephropathy, as an attention feature channel, the following processing is performed: Calculating the target differential feature data through three linear layers to obtain a query vector linear matrix, a key-value vector linear matrix, and a value vector linear matrix; Performing a dot product calculation on the query vector linear matrix and the key-value vector linear matrix to obtain a similarity matrix between the query vector linear matrix and the key-value vector linear matrix; Normalizing the similarity matrix to obtain the attention weight corresponding to each attention feature channel; Multiplying the attention weights corresponding to the respective attention feature channels by the linear matrix of the value vector to obtain first difference feature data corresponding to each attention feature channel; Based on the gating mechanism, the first difference feature data corresponding to all the attention feature channels are fused to obtain the target difference feature data.

4. The method according to claim 1, wherein The step of performing probability prediction of concurrent diabetic nephropathy on the target difference feature data to obtain a second probability value includes: Performing feature dimensionality reduction on the target difference feature data through a fully connected layer including a nonlinear transformation function to obtain reduced-dimensionality difference feature data; The probability distribution of concurrent diabetic nephropathy and non-concurrent diabetic nephropathy is calculated for the dimension-reduced difference feature data, and the obtained probability of concurrent diabetic nephropathy is determined as a second probability value.

5. The method according to claim 1, wherein Before the step of obtaining questionnaire data, a first fundus image and a second fundus image of the same type from the target diabetic patient, the method further includes: Acquire first sample data of a plurality of diabetic patients; wherein the first sample data includes patient parameters of a plurality of diabetic nephropathy-related factors; Constructing a causal relationship for the first sample data using a preset structure learning algorithm to obtain a first Bayesian network; determining a second Bayesian network with the highest score from the first Bayesian network according to the Bayesian Information Criterion; Performing conditional probability distribution calculation on the second Bayesian network by maximizing the likelihood function to obtain conditional probability distribution data of the multiple diabetic nephropathy-associated factors; The step of calculating the joint probability distribution of the target diabetic patient complicated with diabetic nephropathy based on the target Bayesian network to obtain a first probability value includes: Obtaining a target conditional probability of each node in the target Bayesian network from the conditional probability distribution data; The posterior probability of the target diabetic patient having diabetic nephropathy is calculated according to the target conditional probability of each node in the target Bayesian network to obtain a first probability value.

6. The method according to claim 1, characterized in that The step of determining the target probability of the target diabetic patient having diabetic nephropathy based on the first probability value and the second probability value includes: Calculating a first confidence level of the first probability value according to the number of diabetic nephropathy-associated factors in the questionnaire data; Calculating a second confidence level of the first probability value according to the number of parameters of the diabetic nephropathy-associated factors of the specified type in the questionnaire data; Determining a third confidence level of the first probability value based on the first confidence level and the second confidence level; determining a fourth confidence level of the second probability value according to types of the first fundus image and the second fundus image; A target probability of diabetic nephropathy in the target diabetic patient is obtained by weightedly superimposing a product of the third confidence level and the first probability value and a product of the second probability value and the fourth confidence level.

7. A device for predicting diabetic nephropathy, the device being used to execute the method for predicting diabetic nephropathy according to claim 1, characterized in that: The device comprises: an acquisition module, configured to acquire questionnaire data, a first fundus image, and a second fundus image of the same type from a target diabetic patient; wherein the first fundus image is acquired earlier than the second fundus image; and wherein the questionnaire data includes response values ​​corresponding to a plurality of diabetic nephropathy-associated factors, wherein diabetic nephropathy-associated factors are factors having a causal relationship with diabetic nephropathy, including physiological parameters, medical history parameters, family genetic history parameters, lifestyle parameters, clinical examination parameters, and case parameters; and wherein the types include fundus images, fluorescein angiography images, and optical coherence tomography images; A construction module is used to construct a target Bayesian network based on the questionnaire data; wherein the target Bayesian network is used to indicate the correlation between multiple diabetic nephropathy-related factors and the diagnosis of concurrent diabetic nephropathy in the questionnaire data; a calculation module, configured to calculate, based on the target Bayesian network, a joint probability distribution of the target diabetic patient complicated with diabetic nephropathy to obtain a first probability value; an extraction module, configured to extract difference features from the first fundus image and the second fundus image to obtain target difference feature data; A prediction module, configured to predict the probability of concurrent diabetic nephropathy based on the target difference feature data to obtain a second probability value; A determination module is used to determine a target probability of the target diabetic patient having diabetic nephropathy based on the first probability value and the second probability value.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the method for predicting diabetic nephropathy according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when called and executed by a processor, prompt the processor to implement the method for predicting diabetic nephropathy according to any one of claims 1 to 6.

Images