Method for predicting risk of retinopathy in diabetic patients and system therefor

Abstract

Provided are a method for predicting retinopathy in diabetic patients and a system therefor. The method for predicting retinopathy according to some embodiments may comprise the steps of: obtaining one or more machine-learning models prepared to predict risk information related to retinopathy in diabetic patients; obtaining status information of a target diabetic patient; and predicting risk information related to retinopathy of the target diabetic patient from the status information obtained through the one or more machine-learning models. According to the method, the risk of developing retinopathy among various complications that can occur in diabetic patients can be accurately predicted at an early stage.

Description

Method and System for Predicting Risk of Retinopathy in Diabetic Patients

[0001] The present disclosure relates to a technology for predicting the risk of developing retinopathy (i.e., diabetic retinopathy) among various complications that may occur in diabetic patients.

[0002]

[0003] Diabetic retinopathy is the most common complication in diabetic patients. This disease is accompanied by symptoms such as vision loss, but in many cases, the disease has already progressed significantly by the time patients notice symptoms. Accordingly, the U.S. Centers for Disease Control and Prevention recommends that even asymptomatic diabetic patients undergo regular screenings for retinopathy.

[0004] Since diabetic retinopathy is one of the preventable causes of blindness, preventive interventions for diabetic patients can significantly lower the incidence of the disease and substantially reduce the social costs associated with it. However, while it is known that the risk of developing the disease increases with longer duration of diabetes or poor blood sugar control, no method has yet been proposed to accurately predict the risk of retinopathy in diabetic patients at an early stage.

[0005]

[0006] The technical problem to be solved through some embodiments of the present disclosure is to provide a method and system capable of accurately predicting the risk of developing retinopathy (i.e., diabetic retinopathy) in diabetic patients.

[0007] Another technical problem to be solved through some embodiments of the present disclosure is to provide information on key variables (features) that influence the development of retinopathy in diabetic patients.

[0008] The technical problems of the present disclosure are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by a person skilled in the art of the present disclosure from the description below.

[0009]

[0010] A method for predicting the risk of retinopathy in a diabetic patient according to some embodiments of the present disclosure for solving the above-described technical problem may include a method performed by at least one processor, comprising the steps of: acquiring one or more machine-learning models prepared to predict risk information related to retinopathy in a diabetic patient; acquiring state information of a target diabetic patient; and predicting risk information related to retinopathy in the target diabetic patient from the state information through the one or more machine-learning models.

[0011] In some embodiments, the one or more machine learning models may include at least one of XGBoost (eXtreme Gradient Boosting), Random Forest, and LightGBM (Light Gradient Boosting Machine).

[0012] In some embodiments, the status information may include demographic characteristics, disease history, medication history, blood test results, and physical examination results.

[0013] In some embodiments, the condition information may include a history of diseases related to dyslipidemia, cancer, hypertension, chronic kidney disease, neuropathy, and cardiovascular disease.

[0014] In some embodiments, the status information may include test results regarding glycated hemoglobin (HbA1c), blood glucose, LDL (low-density lipoprotein) cholesterol, HDL (high-density lipoprotein) cholesterol and triglycerides, and drug history regarding cilostazol and statins.

[0015] In some embodiments, the preparation process of the one or more machine learning models may include: a step of constructing a training set based on the records of diabetic patients of a first cohort; a step of constructing a test set based on the records of diabetic patients of a second cohort, wherein the second cohort is a cohort independent of the first cohort and the test set does not include the records of diabetic patients of the first cohort; a step of training a specific machine learning model using the training set; and a step of evaluating the performance of the specific machine learning model using the test set.

[0016] In some embodiments, the preparation process of the one or more machine learning models may include: a step of preparing a training set and a test set based on the records of diabetic patients of a first cohort and the records of diabetic patients of a second cohort, wherein the second cohort is a cohort independent of the first cohort; a step of training a plurality of candidate machine learning models using the training set; a step of evaluating the performance of each of the plurality of candidate machine learning models using the test set; and a step of selecting a model among the plurality of candidate machine learning models whose evaluated performance is above a threshold as the one or more machine learning models.

[0017] In some embodiments, the preparation process of the one or more machine learning models comprises: a step of training a first machine learning model using a training set, wherein each sample constituting the training set contains condition information of an individual diabetic patient, and the condition information of the individual diabetic patient relates to multiple variables; a step of deriving the importance of each of the multiple variables using the training result of the first machine learning model, wherein the multiple variables are classified into core variables, intermediate variables, and non-core variables based on the importance; a step of strengthening the training set based on the importance; and a step of training a second machine learning model using the strengthened training set, wherein the step of strengthening the training set includes: a step of removing the non-core variables from the training set; and a step of generating multiple synthetic samples by sampling the value of the core variable within a preset range while fixing the value of the intermediate variable, and adding the generated synthetic samples to the training set, and the step of predicting the risk information of the target diabetic patient may include a step of determining the risk information of the target diabetic patient by combining the prediction result of the first machine learning model and the prediction result of the second machine learning model.

[0018] In some embodiments, a specific machine learning model among the one or more machine learning models comprises: an embeddinger that generates one or more state embeddings by embedding state information of the diabetic patient; an integration encoder that generates an integration embedding by encoding the one or more state embeddings; a first predictor that performs a first task of predicting risk information related to the retinopathy based on the integration embedding; and a second predictor that performs a second task of predicting risk information of the retinopathy and other diabetic complications based on the integration embedding, wherein the specific machine learning model is trained by performing the first task and the second task, and the step of predicting risk information of the target diabetic patient may include the step of predicting risk information of the target diabetic patient related to the retinopathy through the first predictor.

[0019] In some embodiments, the specific machine learning model further includes an image encoder that encodes an image of the diabetic patient associated with the retinopathy to generate an image embedding, and the integration encoder may further encode the image embedding to generate the integration embedding.

[0020] In some embodiments, a specific machine learning model among the one or more machine learning models comprises: an embeddinger that generates one or more state embeddings by embedding state information of the diabetic patient; an integration encoder that generates an integration embedding by encoding the one or more state embeddings; a first predictor that predicts the risk of the retinopathy occurring within a first period based on the integration embedding; and a second predictor that predicts the risk of the retinopathy occurring within a second period different from the first period based on the integration embedding, and the step of predicting the risk information of the target diabetic patient may include: a step of predicting a first risk of the retinopathy occurring in the target diabetic patient within the first period through the first predictor; a step of predicting a second risk of the retinopathy occurring in the target diabetic patient within the second period through the second predictor; and a step of deriving a risk increase trend for the retinopathy based on the difference between the first risk and the second risk.

[0021] In some embodiments, the one or more machine learning models include a first machine learning model that outputs a risk of the retinopathy occurring within a specific period and a second machine learning model that outputs a time of occurrence of the retinopathy, and the step of predicting risk information for the target diabetic patient may include: a step of predicting the risk of the retinopathy occurring in the target diabetic patient through the first machine learning model; and a step of predicting the time of the retinopathy occurring in the target diabetic patient through the second machine learning model when the predicted risk is greater than or equal to a reference value.

[0022] A system for predicting the risk of retinopathy in a diabetic patient according to some embodiments of the present disclosure for solving the technical problem described above comprises: one or more processors; and a memory for storing a computer program executed by said one or more processors, wherein the computer program may include instructions for: acquiring one or more machine-learning models prepared to predict risk information related to retinopathy in a diabetic patient; acquiring state information of a target diabetic patient; and predicting risk information related to retinopathy in said target diabetic patient from said state information through said one or more machine-learning models.

[0023] A computer program according to some embodiments of the present disclosure for solving the above-described technical problem may be stored on a computer-readable recording medium to execute the steps of: acquiring one or more machine-learning models prepared to predict risk information related to retinopathy of a diabetic patient, coupled with a processor of a computer; acquiring state information of a target diabetic patient; and predicting risk information related to retinopathy of the target diabetic patient from the state information through the one or more machine-learning models.

[0024]

[0025] According to some embodiments of the present disclosure, by comprehensively analyzing various status information of diabetic patients (e.g., demographic characteristics, disease history, medication history, blood test results, physical examination results, etc.) through a machine-learning model, risk information regarding retinopathy (i.e., diabetic retinopathy) in the diabetic patient can be accurately predicted at an early stage. For example, the risk of retinopathy developing in the diabetic patient within a specific period can be accurately predicted. Furthermore, by implementing preventive interventions based on the accurate prediction of the risk of retinopathy development, the incidence of retinopathy in diabetic patients and the resulting social costs can be significantly reduced.

[0026] In addition, a training set may be constructed based on the records of diabetic patients from a first cohort, and a test set may be constructed based on the records of diabetic patients from a second cohort independent of the first cohort. In this case, the performance of the machine learning model can be objectively and thoroughly evaluated (verified).

[0027] The effects according to the technical concept of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0028]

[0029] FIG. 1 is an exemplary drawing for explaining the operation of a system for predicting the risk of retinopathy in diabetic patients according to some embodiments of the present disclosure at the system level.

[0030] FIG. 2 is an exemplary drawing for further explaining the operation of a diabetic patient retinopathy risk prediction system according to some embodiments of the present disclosure.

[0031] FIG. 3 is an exemplary drawing showing input and output information of a machine-learning model according to some embodiments of the present disclosure.

[0032] FIG. 4 illustrates a case in which a machine learning model is implemented based on XGBoost (eXtreme Gradient Boosting) according to some embodiments of the present disclosure.

[0033] FIG. 5 illustrates a case in which a machine learning model is implemented based on a neural network according to some other embodiments of the present disclosure.

[0034] FIG. 6 illustrates a case in which a machine learning model is implemented based on a neural network according to some other embodiments of the present disclosure.

[0035] FIG. 7 is an exemplary flowchart schematically illustrating a method for predicting the risk of retinopathy in diabetic patients according to some embodiments of the present disclosure.

[0036] Figure 8 is an exemplary flowchart illustrating an example of the detailed process of the machine learning model preparation step shown in Figure 7.

[0037] FIGS. 9 and FIGS. 10 are exemplary drawings for explaining detailed embodiments related to the training set and test set preparation steps illustrated in FIG. 8.

[0038] FIG. 11 is an exemplary diagram to further explain the candidate machine learning model training and performance evaluation steps illustrated in FIG. 8.

[0039] FIG. 12 is an exemplary drawing for illustrating a method of building an additional machine learning model using variable importance according to some embodiments of the present disclosure.

[0040] FIGS. 13 to 15 are exemplary drawings for explaining performance tests conducted by the inventors of the present disclosure.

[0041] FIG. 16 illustrates an exemplary computing device capable of implementing a system for predicting the risk of retinopathy in diabetic patients according to some embodiments of the present disclosure.

[0042]

[0043] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings. The advantages and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the attached drawings. However, the technical concept of the present disclosure is not limited to the following embodiments but can be implemented in various different forms. The following embodiments are provided merely to complete the technical concept of the present disclosure and to fully inform those skilled in the art of the scope of the present disclosure, and the technical concept of the present disclosure is defined only by the scope of the claims.

[0044] In describing the various embodiments of the present disclosure, if it is determined that a detailed description of related known configurations or functions could obscure the essence of the present disclosure, such detailed description is omitted.

[0045] Unless otherwise defined, terms used in the following embodiments (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which this disclosure pertains, but this may vary depending on the intent of those skilled in the art, case law, the emergence of new technology, etc. The terms used in this disclosure are for describing the embodiments and are not intended to limit the scope of this disclosure.

[0046] In the following embodiments, singular expressions include plural concepts unless the context clearly specifies them as singular. Additionally, plural expressions include singular concepts unless the context clearly specifies them as plural.

[0047] In addition, terms such as first, second, A, B, (a), (b), etc. used in the following embodiments are used merely to distinguish one component from another, and the essence, order, or sequence of the said component is not limited by such terms.

[0048] The components described by reference to terms such as part or unit, module, block, ~or, ~er, etc. used in the following embodiments, and the functional blocks illustrated in the drawings may be implemented in the form of software, hardware, or a combination thereof. Software may be, for example, machine code, firmware, embedded code, and application software. Additionally, hardware may include, for example, electrical circuits, electronic circuits, processors, computers, integrated circuits, integrated circuit cores, passive components, or a combination thereof.

[0049] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0050] FIG. 1 is an exemplary drawing for explaining the operation of a retinopathy risk prediction system (10) for diabetic patients according to some embodiments of the present disclosure at the system level. In FIG. 1 and others, the name of the retinopathy risk prediction system (10) is indicated as 'prediction system', and for convenience of explanation, the retinopathy risk prediction system (10) will also be abbreviated as 'prediction system' below.

[0051] As illustrated in FIG. 1, the prediction system (10) according to the embodiments is a computing device / system that predicts the risk (possibility) of developing retinopathy (i.e., diabetic retinopathy) among complications that may occur in a diabetic patient (e.g., a patient with type 2 diabetes). For instance, the prediction system (10) can predict various risk information (13) related to retinopathy from the diabetic patient's condition information (12).

[0052] The condition information (12) of a diabetic patient may include, for example, demographic characteristics, disease history, medication history, blood test results, and physical examination results, but the scope of the present disclosure is not limited thereto. For example, the condition information (12) may further include various images related to the diabetic patient's retinopathy. Examples of such images may include various retinal / fundus images (e.g., fundus photography, OCT (Optical Coherence Tomography) images, OCT-A (Optical Coherence Tomography Angiography) images, etc.), but the scope of the present disclosure is not limited thereto.

[0053] The risk information (13) related to retinopathy may include, for example, the type of retinopathy, risk level (e.g., probability of retinopathy occurring within a specific period), whether it has occurred, stage of progression, and time of occurrence (i.e., expected time of occurrence). However, the scope of the present disclosure is not limited thereto.

[0054] Specifically, the prediction system (10) can prepare (train) a machine-learning model (11) using a training set composed of status information of a group of diabetic patients and retinopathy information (i.e., label information). Then, the prediction system (10) can predict risk information (13) related to retinopathy from the status information (12) of the target diabetic patient through the machine-learning model (11). The structure and preparation process of this machine-learning model (11) will be explained in detail shortly with reference to the drawings from Fig. 3 onwards. The machine-learning model (11) may refer to a single model or a set of multiple models.

[0055] The above-described prediction system (10) may be implemented in at least one computing device. For example, all functions of the prediction system (10) may be implemented in a single computing device, or the first function of the prediction system (10) may be implemented in a first computing device and the second function may be implemented in a second computing device. Alternatively, specific functions of the prediction system (10) may be implemented in multiple computing devices.

[0056] A computing device may include any device equipped with computing functions, and for an example of such a device, refer to FIG. 16. Since a computing device is a collection of various components (e.g., memory, processor, etc.) that interact, it may be referred to as a 'computing system' depending on the case. Of course, the term computing system may also encompass the concept of a collection of multiple computing devices that interact.

[0057] Meanwhile, in some embodiments, the prediction system (10) may provide a service for predicting risk information related to retinopathy. For example, as illustrated in FIG. 2, the prediction system (10) may provide such a prediction service through communication with a diabetic patient terminal (21) and / or a medical institution terminal (22). Specifically, the prediction system (10) may receive a request including status information from the diabetic patient terminal (21) and, in response, predict and provide risk information related to retinopathy of the diabetic patient. Alternatively, the prediction system (10) may receive a request including status information of a specific diabetic patient (e.g., a diabetic patient managed by a medical institution) from the medical institution terminal (22) and, in response, predict and provide risk information related to retinopathy of the diabetic patient. As illustrated, the patient terminal (21), the medical institution terminal (22), and the prediction system (10) may communicate through a network. Here, the network can be implemented as any type of wired or wireless network, such as a Local Area Network (LAN), Wide Area Network (WAN), mobile radio communication network, and Wibro (wireless broadband internet).

[0058] Up to now, the operation of a prediction system (10) according to some embodiments of the present disclosure has been described schematically with reference to FIGS. 1 and 2. Below, the input / output information (data), structure, and training method of a machine learning model (11) will be described with reference to FIGS. 3 to 6.

[0059] FIG. 3 is an exemplary drawing for explaining input / output information of a machine learning model (11) according to some embodiments of the present disclosure.

[0060] As illustrated in FIG. 3, the machine learning model (11) may be configured to receive status information including demographic characteristics, disease history, medication history, blood test results and / or physical examination results of a diabetic patient, and to predict and output risk information related to retinopathy of the diabetic patient. In some cases, the machine learning model (11) may be configured to receive additional input, such as images of the diabetic patient related to retinopathy.

[0061] Demographic characteristics, disease history, medication history, blood test results, and physical examination results can each consist of various variables, specific examples of which will be briefly explained shortly with reference to Tables 1 through 5. For reference, the term variable may also be referred to as feature, attribute, element, or factor depending on the context.

[0062] Table 1 below shows examples of variables belonging to demographic characteristics. For reference, the range of variables such as age may include derived variables such as age groups (i.e., these derived variables may also be used as input variables for the machine learning model (11)).

[0063] Distinguishing Variables (Features) Demographic Characteristics Age (age) Sex (sex)

[0064] Table 2 below shows examples of variables included in the disease history. For reference, the scope of disease variables such as hypertension may include, in addition to the presence or absence of the disease (i.e., onset), the specific type of the disease, the stage of progression, disease-related values ​​(e.g., blood pressure levels), the time of onset, the number of onsets, whether the disease was cured, the time of cured, the time of diagnosis, the duration of the disease, whether the disease recurred, the time of recurrence, the number of recurrences, the severity of symptoms, whether surgery was performed, the time of surgery, the number of surgeries, etc.

[0065] Distinguishing Variables (Features) Disease History Comorbidities Hypertension Dyslipidemia Macrovascular Complications Cardiovascular Disease Dementia Parkinson's Disease Peripheral Vascular Disease Microvascular Complications Chronic Kidney Disease End-stage Renal Disease Neuropathy Cancer

[0066] Table 3 below shows examples of variables included in the drug history. For reference, the scope of drug variables such as metformin may include, in addition to whether the drug was taken, derived variables such as dosage (e.g., recent dosage, average, median, mode, range, standard deviation, dosage increase / decrease trend / pattern, etc.), number of doses (frequency), time of administration, duration of administration, time of start, time of end, effect of administration, presence of side effects, and severity of side effects.

[0067] Differentiating Variables (Features) Drug History Antidiabetic drug Metformin Sulfonylurea Dipeptidyl peptidase-4 inhibitor Meglitinide Thiazolidinedione α-glucosidase inhibitor Insulin GLP-1 receptor agonist Sodium-glucose co-transporter 2 inhibitor Antihypertensive drug Angiotensin II receptor blocker Angiotensin-converting enzyme inhibitor Calcium channel blocker (CCB) Diuretics Beta-blocker Dyslipidemia drug (dyslipidemia drug) Statin Fibrate Ezetimibe Omega-3 Other dyslipidemia drugs Antiplatelet agents Aspirin Clopidogrel Cilostazol Glycoprotein IIb / IIIa antagonist Other antiplatelet agents

[0068] Table 4 below shows examples of variables included in blood test results. For reference, the scope of variables related to test items, such as glycated hemoglobin, may include derived variables such as the time of the test and the number of tests, in addition to the numerical values ​​of the test item (e.g., recent value, numerical range, average value over a certain period, median, mode, numerical range, standard deviation, trend / pattern of increase / decrease in value, etc.). Furthermore, the scope of variables related to test items, such as blood glucose, may include derived variables such as numerical values ​​of fasting blood glucose, postprandial blood glucose, preprandial blood glucose, and average blood glucose (e.g., recent value, numerical range, average value over a certain period, median, mode, numerical range, standard deviation, trend / pattern of increase / decrease in value, etc.).

[0069] Distinguishing Variables (Features) Blood Test Results Glycated Hemoglobin (HbA1c) Blood Glucose Total Cholesterol Triglycerides High-Density Lipoprotein (HDL) Cholesterol Low-Density Lipoprotein (LDL) Cholesterol Serum Creatinine Aspartate Aminotransferase (AST) Alanine Aminotransferase (ALT) Gamma-Glutamyl Transferase (GGT) Alkaline Phosphate (ALP)

[0070] Table 5 below shows examples of variables included in the physical examination results. For reference, the range of variables related to examination items, such as body mass index, may include not only the numerical values ​​of the examination item (e.g., recent values, numerical range, average values ​​over a certain period, median, mode, numerical range, standard deviation, numerical increase / decrease trend / pattern, etc.) but also derived variables such as the time of examination and the number of examinations.

[0071] Distinguishing Variables (Features) Physical Examination Results Body Mass Index (BMI) Systolic Blood Pressure Diastolic Blood Pressure Pulse Rate

[0072] The risk information related to retinopathy output (predicted) by the machine learning model (11) may include information such as the type of retinopathy (e.g., types of retinopathy with a high risk of occurrence), risk level (e.g., probability of retinopathy occurring within a specific period), occurrence (onset), stage of progression, and time of occurrence (i.e., expected time of occurrence). However, the scope of the present disclosure is not limited thereto. The machine learning model (11) described above may be designed and implemented in various ways. For example, the machine learning model (11) may be designed and implemented based on various types of models / algorithms such as decision tree, random forest, XGBoost (eXtreme Gradient Boosting), LightGBM (Light Gradient Boosting Machine), CatBoost (Categorical Boosting), SVM (Support Vector Machine), logistic regression, AdaBoost (Adaptive Boost), and neural network. If you are skilled in the relevant technical field, you will already be familiar with the structure and training methods of the exemplified models, so an explanation thereof will be omitted. FIGS. 4 to 6 illustrate detailed embodiments of a machine learning model (11). FIGS. 4 to 6 distinguishes the reference numbers of the machine learning model (11) as '11A', '11B', and '11C' according to the embodiments for clarity of the present disclosure.

[0073] First, FIG. 4 illustrates a case in which a machine learning model (11A) is implemented based on XGBoost according to some embodiments of the present disclosure. FIG. 4 assumes a case in which a decision tree is used as a weak learner (or weak classifier).

[0074] As illustrated in FIG. 4, in these embodiments, the machine learning model (11A) is configured to include N weak learners (41-1 to 41-N) and may be configured to output a final prediction result by combining the prediction results of the weak learners (41-1 to 41-N) based on weights (e.g., weights are assigned based on the performance of the weak learners (41-1 to 41-N)). Each of the weak learners (41-1 to 41-N) is trained sequentially, and each weak learner (e.g., 41-2) may be trained in a direction that reduces the residual (i.e., the difference between the predicted value and the actual value) of the previous weak learner (e.g., 41-1). As those skilled in the art are likely already familiar with the operating principles and training methods of XGBoost, further explanation regarding this will be omitted.

[0075] For reference, 'N', the number of weak learners, is a hyperparameter, and its value can be set through hyperparameter tuning techniques such as grid search, random search, and Bayesian optimization. As professionals in the relevant technical field are likely already familiar with the exemplified hyperparameter tuning techniques, further explanation regarding them will be omitted.

[0076] FIG. 5 illustrates a case in which a machine learning model (11B) is implemented based on a neural network according to some other embodiments of the present disclosure.

[0077] As illustrated in FIG. 5, in these embodiments, the machine learning model (11B) may be configured to include an embedder (51), an image encoder (52), an integration encoder (53), and a predictor (54).

[0078] The embedder (51) is a neural network module that embeds the diabetic patient's state information (e.g., demographic characteristics, disease history, medication history, blood test results, physical examination results, etc.). The embedder (51) may be configured to receive the diabetic patient's state information as input, encode it, and output one or more state embeddings (e.g., embedding vectors). For instance, the embedder (51) may be configured to output state embeddings corresponding to each of the input demographic characteristics information, disease history, medication history, blood test results, and physical examination results.

[0079] The embedder (51) may be implemented as any type of neural network. For example, the embedder (51) may be implemented based on a Multi-Layer Perceptron (MLP) (or a fully-connected layer) or an attention-based neural network such as a transformer.

[0080] The embedder (51) may consist of a single embedder or a set of multiple embedders. For example, the embedder (51) may be composed of a first embedder that embeds demographic characteristic information of a diabetic patient, a second embedder that embeds disease history, a third embedder that embeds medication history, a fourth embedder that embeds blood test results, and a fifth embedder that embeds physical examination results. In this case, if the disease history consists of a sequence of individual disease information (records) according to the order of onset (time), the second embedder may be implemented as a neural network that handles sequences, such as a Recurrent Neural Network (RNN) or a Transformer, to embed the disease history while considering the order of onset. Similarly, if the medication history consists of a sequence of individual medication information (records) according to the order of medication intake (time), the third embedder may also be implemented as a neural network that handles sequences.

[0081] Next, the image encoder (52) is a neural network module that encodes (embeds) images of diabetic patients associated with retinopathy. The image encoder (52) may be configured to receive an image of a diabetic patient (e.g., fundus image, etc.) as input, encode it, and output one or more image embeddings (e.g., embedding vectors). For example, a pretrained image encoder (or feature extractor) (e.g., VGG-16, ViT (Vision Transformer), etc.) may be used as the image encoder (52), but the scope of the present disclosure is not limited thereto. The image encoder (52) may, in some cases, be named as an 'image embedding' or a 'feature extractor'.

[0082] For reference, the image embedding generated from the fundus image can serve to further improve the prediction accuracy of the machine learning model (11B) by capturing information regarding lesions (or visual features) in the fundus area related to retinopathy and transmitting it to the integrated encoder (53) and predictor (54).

[0083] Next, the integration encoder (53) is a neural network module that encodes the state embedding output from the embedding (51) and the image embedding output from the image encoder (52) together. For example, the integration encoder (53) may be configured to receive the state embedding and the image embedding as inputs, aggregate them, and output an integrated embedding (e.g., embedding vector).

[0084] The integrated encoder (53) may be implemented as any type of neural network. For example, the integrated encoder (53) may be implemented based on an MLP (or fully connected layer) or as an attention-based neural network such as a Transformer.

[0085] Next, the predictor (54) is a neural network module that predicts risk information related to retinopathy in diabetic patients. The predictor (54) may be configured to receive an integrated embedding output from an integrated encoder (53) and to predict and output such risk information. That is, the predictor (54) may be configured to predict risk information related to retinopathy by comprehensively analyzing various information contained in the integrated embedding (e.g., demographic characteristics, disease history, medication history, blood test results, physical examination results, images related to retinopathy, etc.). Such a predictor (54) may be implemented, for example, based on an MLP (or a fully connected layer), but the scope of the present disclosure is not limited thereto.

[0086] The predictor (54) may be configured to output the risk (i.e., probability) of retinopathy occurring in a diabetic patient within (or thereafter) a specific period (e.g., 6 months, 1 year, 3 years, etc.) (e.g., configured to output a confidence score regarding whether retinopathy will occur), or configured to output the stage of progression of retinopathy (e.g., configured to output a confidence score for the stage of progression of retinopathy). Alternatively, the predictor (54) may be configured to output the time of occurrence of retinopathy (i.e., the expected time of occurrence).

[0087] The machine learning model (11B) described above can be trained using a supervised learning technique. That is, the parameters of the machine learning model (11B) are updated based on a loss representing the difference between the output (i.e., prediction result) of the predictor (54) and the label information, and this process can be performed repeatedly on various samples of the training set. By doing so, the machine learning model (11B) can accurately predict risk information related to retinopathy by comprehensively considering various condition information of diabetic patients (e.g., demographic characteristics, disease history, medication history, blood test results, physical examination results, images related to retinopathy, etc.). For reference, each sample of the training set may represent condition information of an individual diabetic patient, and the training set may consist of these samples and their label information (i.e., retinopathy information). Additionally, the details of the label information may be determined according to the task of the predictor (54).

[0088] FIG. 6 illustrates a case in which a machine learning model (11C) is implemented based on a neural network according to some other embodiments of the present disclosure. Hereinafter, for the clarity of the present disclosure, names such as 'predictor 1' and 'predictor 2' will be named as 'first predictor' and 'second predictor', and this naming rule will be applied in the same way to other names used in the present disclosure.

[0089] As illustrated in FIG. 6, the overall structure of the machine learning model (11C) is similar to that of the machine learning model (11B) in FIG. 5. However, the machine learning model (11C) differs from the machine learning model (11B) in that it includes a plurality of predictors (e.g., 61-1, 61-2). FIG. 6 illustrates a case where the machine learning model (11C) is equipped with two predictors (61-1, 61-2), but the scope of the present disclosure is not limited thereto. For instance, the machine learning model (11C) may further include a third predictor (not shown), etc.

[0090] The first predictor (61-1) is a neural network module that predicts first risk information related to retinopathy, and the second predictor (61-2) is a neural network module that predicts second risk information different from the first risk information (see below for examples of each risk information). That is, the first predictor (61-1) and the second predictor (61-2) are configured to perform different tasks (i.e., a task to predict first risk information and a task to predict second risk information) based on an integrated embedding.

[0091] The machine learning model (11-C) described above can be trained using a multi-task learning technique, and the number and types of tasks can be defined and designed in various ways. That is, the number and types of predictors connected to the integrated encoder (53) can be designed in various ways. FIG. 6 assumes a case where multi-task learning is performed using two tasks (or predictors (61-1, 61-2)).

[0092] For example, a machine learning model (11C) can be trained through a first task that predicts the risk of retinopathy occurring within a specific period and a second task that predicts the stage of progression of retinopathy (i.e., the first risk information is the risk of retinopathy occurring within a specific period, and the second risk information is the stage of progression of retinopathy). That is, based on the prediction loss of the first task, the parameters of the first predictor (61-1), the integrated encoder (53), the embedder (51), and / or the image encoder (52) can be updated, and based on the prediction loss of the second task, the parameters of the second predictor (61-2), the integrated encoder (53), the embedder (51), and / or the image encoder (52) can be updated.

[0093] As another example, the machine learning model (11C) can be trained through a first task that predicts the risk of retinopathy occurring within a first period and a second task that predicts the risk of retinopathy occurring within a second period (i.e., the first risk information is the risk of retinopathy occurring within the first period and the second risk information is the risk of retinopathy occurring within the second period). As a more specific example, the machine learning model (11C) can be trained by performing a first task that predicts the risk of retinopathy occurring within one year through a first predictor (61-1) and a second task that predicts the risk of retinopathy occurring within three years through a second predictor (61-2).

[0094] As another example, the machine learning model (11C) can be trained through a first task that predicts the risk of retinopathy occurring within a specific period and a second task that predicts the timing of the occurrence of the retinopathy (i.e., the first risk information is the risk of retinopathy occurring within a specific period, and the second risk information is the timing of the occurrence of the retinopathy). As a more specific example, the machine learning model (11C) can be trained by performing a first task that predicts the risk of retinopathy occurring within 3 years through a first predictor (61-1), and a second task that predicts the timing of the occurrence of the retinopathy through a second predictor (61-2) corresponding to the regression head.

[0095] As another example, the machine learning model (11C) can be trained through a first task that predicts first risk information related to retinopathy and a second task that predicts second risk information related to retinopathy and other diabetes complications (e.g., cardiovascular disease, neurodegenerative disease, neuropathy, chronic kidney disease, etc.). As a more specific example, the machine learning model (11C) can be trained by performing a first task that predicts the risk (i.e., whether or not) of retinopathy occurring within a specific period through a first predictor (61-1), and by performing a second task that predicts the risk of cardiovascular disease (or chronic kidney disease, etc.) occurring within a specific period through a second predictor (61-2).

[0096] As another example, a machine learning model (11C) may be trained based on various combinations of the examples described above.

[0097] Up to now, the input / output information / data, structure, and training method of the machine learning model (11) have been described with reference to FIGS. 3 to 6. Below, various methods that can be performed in the prediction system (10) described above will be described with reference to FIGS. 7 and subsequent figures.

[0098] For convenience of understanding, the following description will continue by assuming that all steps / operations of the methods described below are performed in a prediction system (10, e.g., at least one processor). Therefore, if the subject of a specific step / operation is omitted, it can be understood that the step / operation is performed by the prediction system (10). However, in an actual environment, some steps / operations of the methods described below may be performed on a different computing device. For example, the preparation process of the machine learning model (11) (e.g., training, performance evaluation, etc.) may be performed on a different computing device.

[0099] FIG. 7 is an exemplary flowchart illustrating a method for predicting the risk of retinopathy in diabetic patients according to some embodiments of the present disclosure. However, this is merely an exemplary embodiment for achieving the purpose of the present disclosure, and it is understood that some steps may be added or deleted as necessary.

[0100] As illustrated in FIG. 7, the method for predicting the risk of retinopathy according to the embodiments may begin with step S71 of preparing a machine learning model (11) that predicts risk information related to retinopathy in diabetic patients. An example of the detailed process of step S71 is illustrated in FIG. 8.

[0101] FIG. 8 is an exemplary flowchart showing the detailed process of the machine learning model (11) preparation step S71.

[0102] As illustrated in FIG. 8, a training set and a test set are prepared first (S81). Here, the training set may include a validation set used to improve (evaluate) the performance of the machine learning model (11) or to tune hyperparameters. The test set may be understood as a dataset for evaluating the final performance of the machine learning model (11). Each of the training set and the test set may consist of multiple samples and label information (i.e., retinopathy information) for each sample, and each sample may include condition information of an individual diabetic patient.

[0103] The specific method of preparing the training set and test set may vary depending on the example.

[0104] In some embodiments, as illustrated in FIG. 9, a training set and a test set may be prepared based on the records of diabetic patients (e.g., status information of a diabetic patient group) of two independent cohorts (or three or more cohorts). Specifically, the prediction system (10) may construct (create) a training set (95) based only on the records of diabetic patients of the first cohort (91) (i.e., the training set (95) does not include the records of diabetic patients of the second cohort (92)), and construct (create) a test set (96) based only on the records of diabetic patients of the second cohort (92) (i.e., the test set (96) does not include the records of diabetic patients of the first cohort (91)). Alternatively, the training set (95) may be constructed based on the records of diabetic patients of the first cohort (91), and the test set (96) may be constructed based on the records of diabetic patients of the first cohort (91) and the second cohort (92). Each of the training set (95) and the test set (96) may include, for example, status information (i.e., samples) of a group of diabetic patients who have developed retinopathy and a group of diabetic patients who have not. FIG. 9 illustrates, as an example, a case where the first cohort (91) consists of a group of diabetic patients from a first medical institution (93, e.g., one or more medical institutions) and the second cohort (92) consists of a group of diabetic patients from a second medical institution (94, e.g., one or more medical institutions) (i.e., the two cohorts (91, 92) are distinguished by the difference in medical institutions), but the scope of the present disclosure is not limited thereto. According to this embodiment, by constructing the test set (96) based on the records of diabetic patients from an external cohort (92) that were not used for training the machine learning model (11), the performance of the machine learning model (11) can be evaluated objectively and thoroughly.

[0105] In the preceding embodiments, the average age of the diabetic patients constituting the test set (96) may be designed to be lower than that of the training set (95). In this case, the performance of the machine learning model (11) can be evaluated more strictly (certainly) (because, since the risk of developing retinopathy in young diabetic patients is relatively low, samples with high prediction difficulty are included in the test set (96)). Alternatively, the duration of diabetes of the patients constituting the test set (96) may be designed to be shorter than that of the training set (95), or the average blood glucose level of the diabetic patients constituting the test set (96) may be designed to be lower than that of the training set (95).

[0106] In some other embodiments, as illustrated in FIG. 10, an over-sampling technique that increases the number of samples of a minority class may be applied to the training set and / or test set to address the data imbalance problem. For example, the prediction system (10) may generate synthetic samples (e.g., 101, 102) belonging to a minority class (i.e., a group of diabetic patients with retinopathy) using an over-sampling technique such as SMOTE (Synthetic Minority Over-sampling Technique), but the scope of the present disclosure is not limited thereto. As another example, the prediction system (10) may calculate the variance of each variable value constituting the status information of the diabetic patient group over all samples. Then, the prediction system (10) can generate synthetic samples of the minority class (e.g., 101, 102) by fixing the value of the variable with a relatively small variance and sampling the value of the variable with a relatively large variance from a preset range (e.g., a range of values ​​that are a certain distance away from the mean of the minority class samples of the variable). In this case, the diversity of samples belonging to the minority class can be easily increased.

[0107] In some other embodiments, training sets and test sets may be prepared based on various combinations of the embodiments described above.

[0108] Referring again to Fig. 8, the explanation will be provided.

[0109] In step S82, multiple candidate machine learning models are trained (built) using the training set. For example, as illustrated in FIG. 11, the prediction system (10) can train (build) a first candidate machine learning model (111-1), a second candidate machine learning model (111-2), etc. using the training set (112). At this time, the prediction system (10) may use a portion of the training set (112) as a validation set to tune the hyperparameter values ​​of each of the candidate machine learning models (e.g., 111-1, 111-2). Here, the second candidate machine learning model (111-2) may be a model designed such that the type of model / algorithm, structure, and hyperparameter values ​​are at least partially different from the first candidate machine learning model (111-1).

[0110] In step S83, the performance of each candidate machine learning model is evaluated using a test set. For example, referring again to FIG. 11, the prediction system (10) can evaluate the performance of each trained candidate machine learning model (e.g., 111-1, 111-2) using a test set (113). At this time, the metrics used for performance evaluation may include accuracy, precision, sensitivity, specificity, balanced accuracy, F1 score, AUROC (Area Under ROC Curve), etc., but the scope of the present disclosure is not limited thereto.

[0111] In step S84, a machine learning model (11) for prediction (inference) is selected from among a plurality of candidate machine learning models based on the evaluation results. For example, the prediction system (10) may select one or more candidate machine learning models with performance above a threshold as the machine learning model (11) for prediction.

[0112] Meanwhile, in some embodiments, the prediction system (10) may build an additional machine learning model using the importance of variables. For example, as illustrated in FIG. 12, let us assume that the prediction system (10) trains a first machine learning model (11-1, e.g., a machine learning model selected for prediction) using a training set (121), and the importance (122) of each of a plurality of variables (i.e., variables constituting the condition information of a diabetic patient) is derived. Here, the variable importance (122) refers to the degree of influence that a specific variable (feature) has on the prediction result of the first machine learning model (11-1). Furthermore, let us assume that the plurality of variables are classified into core (key) variables, intermediate variables, and non-core variables based on their importance (wherein a core variable is a variable whose importance is greater than or equal to a first threshold, a non-core variable is a variable whose importance is lower than or equal to a second threshold, and an intermediate variable is a variable whose importance is less than or equal to a first threshold and greater than or equal to a second threshold). In this case, the prediction system (10) can enhance the training set (121) based on variable importance (122). For example, the prediction system (10) can remove non-core variables from the training set (121) (i.e., remove all values ​​of non-core variables). Then, the prediction system (10) can fix the values ​​of intermediate variables and generate multiple synthetic samples by sampling the values ​​of core variables within a preset range, and add the generated synthetic samples to the training set. For instance, the prediction system (10) can generate synthetic samples of a first group of diabetic patients with retinopathy by sampling the values ​​of the corresponding core variable within a range of values ​​that are more than a certain distance away from the mean of the specific core variable of the first group of diabetic patients with retinopathy (i.e., minority class).Similarly, the prediction system (10) can generate a synthetic sample of a group of second-diabetic patients by sampling the value of a specific key variable from a range of values ​​that are more than a certain distance away from the mean of the second-diabetic patient group (i.e., multiple classes) that has not developed retinopathy. Next, the prediction system (10) can train (build) a second machine learning model (11-2) using an enhanced training set (123). In the inference step, the prediction system (10) can determine risk information for the target diabetic patient related to retinopathy by combining the prediction results of the first machine learning model (11-1) and the prediction results of the second machine learning model (11-2). Any specific method may be used to combine the two prediction results.

[0113] Additionally, in some embodiments, the prediction system (10) may each construct a first machine learning model that predicts first risk information and a second machine learning model that predicts second risk information. For example, the prediction system (10) may each construct a first machine learning model that predicts the risk of retinopathy occurring in a diabetic patient within a specific period and a second machine learning model that predicts the timing of the occurrence of retinopathy. In this case, the first machine learning model and the second machine learning model may be designed to share at least some parameters (see FIG. 6). In some cases, the prediction system (10) may further construct a third machine learning model, etc.

[0114] Referring again to Fig. 7, the explanation will be provided.

[0115] In step S72, condition information of the target diabetic patient is obtained. As described above, the condition information may consist of the values ​​of a number of variables (i.e., input variables of the machine learning model (11)), and specific examples of these variables are to be referred to in Tables 1 through 5.

[0116] In step S73, risk information regarding retinopathy in the target diabetic patient is predicted from the corresponding state information through the machine learning model (11). For example, the prediction system (10) can input the corresponding state information into the machine learning model (11) to predict the risk (i.e., whether or not) that retinopathy will occur in the target diabetic patient within a specific period. Alternatively, the prediction system (10) may predict the stage of progression of retinopathy, or predict the time when retinopathy will occur in the target diabetic patient (i.e., the expected time of occurrence). As a more specific example, let us assume that the first machine learning model is configured to output (predict) the risk of retinopathy occurring within a specific period, and the second machine learning model is configured to output (predict) the time of occurrence of retinopathy. In this case, the prediction system (10) can predict the risk of retinopathy occurring in the target diabetic patient within a specific period through the first machine learning model. And, if the predicted risk is greater than or equal to a threshold value, the prediction system (10) can predict the time when retinopathy will occur in the target diabetic patient through the second machine learning model.

[0117] In some embodiments, the prediction system (10) may derive and provide an increasing trend of risk for retinopathy (i.e., an increasing trend of risk over time). For example, let us assume that a machine learning model (11) is configured as illustrated in FIG. 6. And let us assume that a first predictor (61-1) is configured to output (predict) the risk of retinopathy occurring within a first period (e.g., 1 year), and a second predictor (61-2) is configured to output (predict) the risk of retinopathy occurring within a second period (e.g., 3 years) that is different from the first period. In this case, the prediction system (10) can predict the first risk of retinopathy occurring in the target diabetic patient within the first period through the first predictor (61-1), and predict the second risk of retinopathy occurring in the target diabetic patient within the second period through the second predictor (61-2). Next, the prediction system (10) can derive an increasing trend of risk for retinopathy in the target diabetic patient based on the difference between the first risk level and the second risk level. For example, if the difference between the first risk level and the second risk level is significantly large (provided that the second period is longer than the first period), the prediction system (10) can determine that the risk of developing retinopathy increases relatively steeply over time.

[0118] The prediction system (10) can provide the prediction results to the target diabetic patient (e.g., by transmitting them to the target diabetic patient's terminal (21), etc.). In addition, if the risk of developing retinopathy is high, the prediction system (10) may provide additional preventive information (e.g., information regarding dietary habits, lifestyle habits, exercise methods, etc.), medical institution information, etc., to the target diabetic patient. Alternatively, the prediction system (10) may automatically schedule a medical appointment for the target diabetic patient in conjunction with the medical institution terminal (22).

[0119] Up to this point, a method for predicting the risk of retinopathy in diabetic patients according to several embodiments of the present disclosure has been described with reference to FIGS. 8 to 12. As described above, by comprehensively analyzing various condition information of diabetic patients (e.g., demographic characteristics, disease history, medication history, blood test results, physical examination results, etc.) through a machine learning model (11), the risk information of the diabetic patient regarding retinopathy can be accurately predicted at an early stage. For example, the risk of retinopathy occurring in the diabetic patient within a specific period can be accurately predicted. Furthermore, by implementing preventive interventions based on the accurate prediction of the risk of retinopathy occurrence, the incidence rate of retinopathy in diabetic patients and the resulting social costs can be significantly reduced.

[0120] Hereinafter, performance tests conducted by the inventors of the present disclosure will be briefly introduced with reference to FIGS. 13 to 15.

[0121] The inventors conducted experiments to evaluate the predictive performance regarding the risk of developing retinopathy using various machine learning models.

[0122] Specifically, as illustrated in FIG. 13, the inventors constructed a training set based on the records of diabetic patients of a first longitudinal cohort and a test set based on the records of diabetic patients of a second longitudinal cohort (i.e., a cohort independent of the first longitudinal cohort). Here, the first longitudinal cohort consisted of a group of diabetic patients from one hospital, and the second longitudinal cohort consisted of a group of diabetic patients from two hospitals different from the first longitudinal cohort. After purification processing of the diabetic patient records (e.g., removal of records of patients with type 1 diabetes, removal of records of patients with missing data, etc.), the training set consisted of status information (refer to the variables listed in Tables 1 to 5) of a group of 13,178 diabetic patients who did not develop retinopathy and a group of 348 diabetic patients who developed retinopathy within 3 years (i.e., within 3 years from the time of diagnosis of diabetes). In addition, the test set consisted of status information from a group of 1,719 diabetic patients who did not develop retinopathy and a group of 137 diabetic patients who developed retinopathy within 3 years.

[0123] Next, the inventors conducted performance experiments on XGBoost (i.e., the decision tree-based AdaBoost shown in Fig. 4), CatBoost, LightGBM, Random Forest, and AdaBoost. The inventors performed model training, hyperparameter tuning, and performance evaluation (i.e., first-order performance evaluation) by applying a 10-fold cross-validation technique to the training set, and evaluated the final performance of each machine learning model using the test set. Performance metrics used included AUROC, accuracy, sensitivity, specificity, and balance accuracy, and the results of the performance evaluation are presented in Tables 6 and 7 and Fig. 14 below. Specifically, Table 6 lists the performance evaluation results for the validation set (i.e., the dataset split from the training), and Table 7 lists the performance evaluation results for the test set. Fig. 14 shows the AUROC measurement results of XGBoost for the validation set (i.e., the dataset split from the training). In Tables 6 and 7, 'RF' stands for Random Forest.

[0124] Model AUROC Accuracy(%) Sensitivity(%) Specificity(%) Balance Accuracy(%) XGBoost8 2.36 73.76 73.88 73.76 73.82 CatBoost8 0.10 72.17 72.13 72.17 72.15 LightGBM7 9.97 71.92 72.13 71.92 72.02 RF7 8.07 70.12 70.40 70.12 70.26 AdaBoost7 8.81 69.71 70.10 69.70 69.90

[0125] Model AUROC Accuracy(%) Sensitivity(%) Specificity(%) Balance Accuracy(%) XGBoost7 1.6765.1464.9665.1565.06CatBoost7 1.6065.8465.6965.8565.77LightGBM7 2.7066.7066.4266.7266.57RF7 2.6665.1464.9665.1565.06AdaBoost7 1.7866.6566.4266.6766.55

[0126] Referring to Tables 6 and 7 and Fig. 14, it can be seen that the performance of each machine learning model is generally excellent. This is attributed to the fact that the machine learning models were designed to predict the occurrence of retinopathy (i.e., risk) by comprehensively considering various variables related to diabetic patients (refer to Tables 1 through 5). According to the results of the inventors' performance experiments, XGBoost, LightGBM, and Random Forest were found to have relatively superior performance among the various machine learning models. Fig. 15 illustrates the top 15 variables of high importance obtained from the training results of XGBoost. In Fig. 15, 'CKD', 'SD', 'CVD', and 'TG' represent chronic kidney disease, standard deviation, cardiovascular disease, and triglycerides, respectively.

[0127] As shown in Figure 15, a history of dyslipidemia was identified as the variable that had the greatest impact on the performance of the machine learning model, followed by a history of cancer, hypertension, and chronic kidney disease, as well as glycated hemoglobin, blood glucose, LDL and HDL cholesterol levels, which were found to be of high importance. In addition, a history of medication, such as cilostazol and statins, was also found to be of high importance.

[0128] Up to now, performance experiments conducted by the inventors have been briefly introduced with reference to FIGS. 13 to 15. Below, with reference to FIG. 16, an exemplary computing device (160) capable of implementing a prediction system (10) according to some embodiments of the present disclosure will be described.

[0129] FIG. 16 is an exemplary hardware configuration diagram showing a computing device (160).

[0130] As illustrated in FIG. 16, a computing device (160) may include one or more processors (161), a bus (163), a communication interface (164), a memory (162) for loading a computer program (166) executed by the processor (161), and a storage (165) for storing the computer program (166). However, FIG. 16 illustrates only the components related to the embodiments of the present disclosure. Therefore, a person skilled in the art to which the present disclosure belongs will understand that other general-purpose components may be included in addition to the components (161 to 166) illustrated in FIG. 16. That is, the computing device (160) may include various additional components in addition to the components (161 to 166) illustrated in FIG. 16. Furthermore, depending on the case, the computing device (160) may be configured in a form in which some of the components (161 to 166) illustrated in FIG. 16 are omitted. Below, each component of the computing device (160) is described.

[0131] The processor (161) can control the overall operation of each component of the computing device (160). The processor (161) may be configured to include at least one of a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), or any form of processor well known in the art of the present disclosure. Additionally, the processor (161) may perform operations on at least one application or program to execute specific steps / operations / methods. The computing device (160) may have one or more processors.

[0132] Next, the memory (162) may store various data, commands and / or information. The memory (162) may load a computer program (166) from storage (165) to execute specific steps / operations / methods. The memory (162) may be implemented as volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0133] Next, the bus (163) can provide communication functions between components of the computing device (160). The bus (163) can be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

[0134] Next, the communication interface (164) may support wired and wireless internet communication of the computing device (160). Additionally, the communication interface (164) may support various communication methods other than internet communication. To this end, the communication interface (164) may be configured to include a communication module well known in the art of the present disclosure.

[0135] Next, the storage (165) may store one or more computer programs (166) non-temporarily. The storage (165) may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which this disclosure belongs.

[0136] Next, the computer program (166) may include instructions that cause the processor (161) to perform specific steps / actions / methods when loaded into memory (162). That is, the processor (161) can perform specific steps / actions / methods by executing the instructions loaded into memory (162).

[0137] For example, a computer program (166) may include instructions for the operation of acquiring one or more machine learning models (11) prepared to predict risk information related to retinopathy in diabetic patients, the operation of acquiring condition information of a target diabetic patient, and the operation of predicting risk information related to retinopathy in a target diabetic patient from the condition information through one or more machine learning models (11).

[0138] As another example, a computer program (166) may include instructions to perform at least some of the steps / actions / methods described with reference to FIGS. 1 through 15.

[0139] As illustrated, a prediction system (10) according to some embodiments of the present disclosure can be implemented through a computing device (160).

[0140] Meanwhile, in some embodiments, the computing device (160) illustrated in FIG. 16 may refer to a virtual machine implemented based on cloud technology. For example, the computing device (160) may be a virtual machine running on one or more physical servers included in a server farm. In this case, at least some of the processor (161), memory (162), and storage (165) illustrated in FIG. 16 may be virtual hardware, and the communication interface (164) may also be implemented as a virtualized networking element such as a virtual switch.

[0141] Up to now, with reference to FIG. 16, an exemplary computing device (160) capable of implementing a prediction system (10) according to some embodiments of the present disclosure has been described.

[0142] Various embodiments of the present disclosure and effects according to those embodiments have been described with reference to FIGS. 1 to 16. The effects according to the technical concept of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0143] Furthermore, just because the above embodiments describe a plurality of components being combined into one or operating in combination, the technical concept of the present disclosure is not necessarily limited to these embodiments. That is, within the scope of the purpose of the technical concept of the present disclosure, all such components may be selectively combined into one or more combinations to operate.

[0144] The technical concept of the present disclosure described above may be implemented as computer-readable code on a computer-readable recording medium. A computer program stored on a computer-readable recording medium may be transmitted to another computing device via a network such as the Internet and installed on said computing device, thereby being used on said computing device.

[0145] Although operations are depicted in a specific order in the drawings, it should not be understood that the operations must necessarily be executed in the specific order depicted or in a sequential order, or that all depicted operations must be executed to obtain the desired result. In certain situations, multitasking and parallel processing may be advantageous. Although various embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will understand that the technical concept of the present disclosure may be implemented in other specific forms without altering the technical concept or essential features thereof. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of protection of the present disclosure shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the technical concept defined by the present disclosure.

Claims

1. A method performed by at least one processor, A step of acquiring one or more machine-learning models prepared to predict risk information related to retinopathy in diabetic patients; A step of obtaining condition information of a target diabetic patient; and A method comprising the step of predicting risk information of the target diabetic patient related to the retinopathy from the state information through the above one or more machine learning models. Method for predicting the risk of retinopathy in diabetic patients.

2. In Paragraph 1, The above one or more machine learning models include at least one of XGBoost (eXtreme Gradient Boosting), Random Forest, and LightGBM (Light Gradient Boosting Machine). Method for predicting the risk of retinopathy in diabetic patients.

3. In Paragraph 1, The above status information includes demographic characteristics, disease history, medication history, blood test results, and physical examination results. Method for predicting the risk of retinopathy in diabetic patients.

4. In Paragraph 1, The above status information includes a history of diseases related to dyslipidemia, cancer, hypertension, chronic kidney disease, neuropathy, and cardiovascular disease, Method for predicting the risk of retinopathy in diabetic patients.

5. In Paragraph 1, The above status information includes test results regarding glycated hemoglobin (HbA1c), blood glucose, LDL (low-density lipoprotein) cholesterol, HDL (high-density lipoprotein) cholesterol, and triglycerides, as well as drug history regarding cilostazol and statins. Method for predicting the risk of retinopathy in diabetic patients.

6. In Paragraph 1, The preparation process for the above one or more machine learning models is: A step of constructing a training set based on the records of diabetic patients of the first cohort; Step of constructing a test set based on the records of diabetic patients of a second cohort - the second cohort is a cohort independent of the first cohort, and the test set does not include the records of diabetic patients of the first cohort -; A step of training a specific machine learning model using the above training set; and A method comprising the step of evaluating the performance of the specific machine learning model using the above test set, Method for predicting the risk of retinopathy in diabetic patients.

7. In Paragraph 1, The preparation process for the above one or more machine learning models is: A step of preparing a training set and a test set based on the records of diabetic patients of the first cohort and the records of diabetic patients of the second cohort - the second cohort is an independent cohort from the first cohort -; A step of training a plurality of candidate machine learning models using the above training set; A step of evaluating the performance of each of the plurality of candidate machine learning models using the above test set; and A method comprising the step of selecting, among the plurality of candidate machine learning models, a model whose evaluated performance is above a threshold as one or more machine learning models. Method for predicting the risk of retinopathy in diabetic patients.

8. In Paragraph 1, The preparation process for the above one or more machine learning models is: A step of training a first machine learning model using a training set - each sample constituting the training set includes condition information of an individual diabetic patient, and the condition information of the individual diabetic patient relates to multiple variables -; A step of deriving the importance of each of the plurality of variables using the training results of the first machine learning model - the plurality of variables are classified into core variables, intermediate variables, and non-core variables based on the importance -; A step of reinforcing the training set based on the above importance; and The method includes the step of training a second machine learning model using the above-mentioned enhanced training set, and The step of strengthening the above training set is, A step of removing the non-core variables from the training set; and The method includes the step of generating a plurality of synthetic samples by sampling the value of the core variable within a preset range while fixing the value of the intermediate variable, and adding the generated synthetic samples to the training set. The step of predicting risk information for the above-mentioned target diabetes patient is, A method comprising the step of determining risk information for the target diabetic patient by combining the prediction result of the first machine learning model and the prediction result of the second machine learning model. Method for predicting the risk of retinopathy in diabetic patients.

9. In Paragraph 1, Among the one or more machine learning models mentioned above, a specific machine learning model is: An embedder that generates one or more state embeddings by embedding state information of the above-mentioned diabetic patient; An integrated encoder that generates an integrated embedding by encoding one or more of the above state embeddings; A first predictor that performs a first task of predicting risk information related to the retinopathy based on the above integrated embedding; and It includes a second predictor that performs a second task of predicting risk information of the retinopathy and other diabetes complications based on the integrated embedding above, and The above specific machine learning model is trained by performing the above first task and the above second task, and The step of predicting risk information for the above-mentioned target diabetes patient is, A step comprising predicting risk information of the target diabetic patient related to the retinopathy through the first predictor, Method for predicting the risk of retinopathy in diabetic patients.

10. In Paragraph 9, The specific machine learning model mentioned above is: It further includes an image encoder that generates image embeddings by encoding an image of the diabetic patient associated with the retinopathy mentioned above, and The above-mentioned integrated encoder further encodes the image embedding to generate the above-mentioned integrated embedding, Method for predicting the risk of retinopathy in diabetic patients.

11. In Paragraph 1, Among the one or more machine learning models mentioned above, a specific machine learning model is: An embedder that generates one or more state embeddings by embedding state information of the above-mentioned diabetic patient; An integrated encoder that generates an integrated embedding by encoding one or more of the above state embeddings; A first predictor that predicts the risk of the retinopathy occurring within a first period based on the above integrated embedding; and It includes a second predictor that predicts the risk of the retinopathy occurring within a second period different from the first period based on the above integrated embedding, and The step of predicting risk information for the above-mentioned target diabetes patient is, A step of predicting a first risk of the retinopathy occurring in the target diabetic patient within the first period through the first predictor; A step of predicting a second risk of the retinopathy occurring in the target diabetic patient within the second period through the second predictor; and A step comprising deriving an increasing risk trend for the retinopathy based on the difference between the first risk level and the second risk level. Method for predicting the risk of retinopathy in diabetic patients.

12. In Paragraph 1, The above one or more machine learning models include a first machine learning model that outputs a risk of the retinopathy occurring within a specific period and a second machine learning model that outputs a time of occurrence of the retinopathy. The step of predicting risk information for the above-mentioned target diabetes patient is, A step of predicting the risk of the retinopathy occurring in the target diabetic patient through the first machine learning model; and If the predicted risk level is greater than or equal to a threshold, the method includes the step of predicting the time when the retinopathy occurs in the target diabetic patient through the second machine learning model. Method for predicting the risk of retinopathy in diabetic patients.

13. One or more processors; and It includes memory for storing computer programs executed by one or more of the above processors, and The above computer program is: The operation of acquiring one or more machine-learning models prepared to predict risk information related to retinopathy in diabetic patients; An action of acquiring status information of a target diabetic patient; and Instructions for an operation to predict risk information of the target diabetic patient related to the retinopathy from the state information through the above one or more machine learning models, Prediction system for the risk of retinopathy in diabetic patients.

14. In Paragraph 13, The preparation process for the above one or more machine learning models is: Action to construct a training set based on the records of diabetic patients in the first cohort; Operation of constructing a test set based on the records of diabetic patients of a second cohort - the second cohort is a cohort independent of the first cohort, and the test set does not include the records of diabetic patients of the first cohort -; The operation of training a specific machine learning model using the above training set; and A method including an operation to evaluate the performance of the specific machine learning model using the above test set, Prediction system for the risk of retinopathy in diabetic patients.

15. Combined with the computer processor, A step of acquiring one or more machine-learning models prepared to predict risk information related to retinopathy in diabetic patients; A step of obtaining condition information of a target diabetic patient; and A computer-readable recording medium stored therein to execute the step of predicting risk information of the subject diabetic patient related to the retinopathy from the state information through the above one or more machine learning models. Computer program.

Images