Method for diagnosing cardiovascular diseases and device using same

Abstract

Disclosed are a method for diagnosing cardiovascular diseases and a device using same. A method for controlling a diagnostic device according to an embodiment may comprise the steps of: acquiring a retinal image of a subject; and acquiring cardiovascular disease diagnostic information about the subject by using a machine-learning model on the basis of the retinal image, wherein the machine-learning model includes a first model and a second model, the first model being a neural network model and the second model being a regression-based machine-learning model.

Description

Method for diagnosing cardiovascular disease and device using the same

[0001] This application relates to a method for diagnosing cardiovascular disease and a device using the same.

[0002]

[0003] Retinal examinations can detect abnormalities in the retina, optic nerve, and macula, and are frequently used in ophthalmology due to their relatively simple imaging. Meanwhile, recent rapid advancements in artificial intelligence (AI) technology have led to active development of diagnostic AI in the medical diagnostic field, particularly in image-based diagnostics. Global companies are also investing heavily in AI development for analyzing various imaging medical data, including through collaborations with the medical community and the input of large-scale data. Some companies have even successfully developed AI diagnostic tools that produce excellent diagnostic results.

[0004] Retinal images can be used to noninvasively observe blood vessels in the body, and there is a growing demand for expanded diagnostic applications using retinal images not only for eye diseases but also for cardiovascular diseases.

[0005]

[0006] The technical challenge of the present application is to provide a method for diagnosing cardiovascular disease by obtaining information on cardiovascular disease with high accuracy using retinal images and a machine learning model.

[0007] The technical problems of the present application are not limited to the problems described above, and problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present application belongs from this specification and the attached drawings.

[0008]

[0009] According to one aspect, a method for controlling a diagnostic device according to one embodiment includes the steps of: obtaining a retinal image of a subject; and obtaining cardiovascular disease diagnosis information for the subject using a machine learning model based on the retinal image, wherein the machine learning model includes a first model and a second model, and the first model may be a neural network model and the second model may be a regression-based machine learning model.

[0010] Technical solutions are not limited to the above-described solutions, and technical solutions not mentioned will be clearly understood by a person skilled in the art to which the present application pertains from this specification and the attached drawings.

[0011]

[0012] According to the present application, information on cardiovascular disease can be obtained with high accuracy using retinal images and machine learning models.

[0013] The effects of the invention of the present application are not limited to the effects described above, and effects not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present application belongs from this specification and the attached drawings.

[0014]

[0015] Figure 1 illustrates a diagnostic system according to one embodiment.

[0016] Figure 2 is a block diagram illustrating a learning device according to one embodiment.

[0017] FIG. 3 is a block diagram illustrating a diagnostic device according to one embodiment.

[0018] Figure 4 illustrates a diagnostic system according to one embodiment.

[0019] FIG. 5 is a block diagram illustrating a client device according to one embodiment.

[0020] Figure 6 is a diagram for explaining a diagnostic process according to one embodiment.

[0021] Fig. 7 is a drawing for explaining the configuration of a learning unit according to one embodiment.

[0022] Figure 8 is a conceptual diagram illustrating an image data set according to one embodiment.

[0023] Figure 9 is a block diagram illustrating the learning process of a diagnostic model according to one embodiment.

[0024] Fig. 10 is a drawing for explaining the configuration of a diagnostic unit according to one embodiment.

[0025] Fig. 11 is a diagram for explaining a diagnostic process according to one embodiment.

[0026] Fig. 12 is a block diagram illustrating a diagnostic unit according to one embodiment.

[0027] Fig. 13 is a diagram for explaining a diagnostic process according to one embodiment.

[0028] Fig. 14 is a drawing for explaining a diagnostic system according to one embodiment.

[0029] Fig. 15 is a diagram for explaining a serial diagnostic model according to one embodiment.

[0030] Fig. 16 is a drawing for explaining a serial diagnostic model according to another embodiment.

[0031] Figure 17 is a diagram for explaining a serial diagnostic model according to another embodiment.

[0032] Fig. 18 is a drawing for explaining a diagnostic method using a diagnostic model according to one embodiment.

[0033] FIG. 19 is a diagram for explaining a method for diagnosing cardiovascular disease according to one embodiment.

[0034] FIG. 20 illustrates a diagnostic model for obtaining cardiovascular disease diagnostic information according to one embodiment.

[0035] Figure 21 shows the clinical characteristics of subjects according to cardiovascular disease diagnosis information according to Example 1.

[0036] Figures 22a and 22b are diagrams for explaining cardiovascular disease diagnosis information according to Example 1 and the incidence of cardiovascular disease-related events according to QRISK3.

[0037] Figure 23 is a diagram for explaining the performance of predicting the occurrence of cardiovascular disease-related events within 10 years using cardiovascular disease diagnosis information according to Example 1.

[0038] Figures 24a to 24c are diagrams for explaining the results of applying cardiovascular disease diagnosis information to a group with a QRISK3 score of 7.5 to 10% according to Example 1.

[0039] Figure 25 is a diagram for explaining in detail the performance when applying cardiovascular disease diagnosis information and QRISK3 score together according to Example 1.

[0040] Figure 26 shows the clinical characteristics of subjects according to cardiovascular disease diagnosis information according to Example 2.

[0041] Figure 27 is a diagram for explaining the performance of cardiovascular disease diagnosis information according to Example 2.

[0042] Figure 28 is a diagram illustrating the performance of cardiovascular disease diagnosis information for various ethnicities according to Example 2.

[0043] Fig. 29 is a drawing for explaining a method for diagnosing cardiovascular disease according to another embodiment.

[0044] FIG. 30 is a diagram illustrating a method for providing guide information for cardiovascular disease diagnosis information according to one embodiment.

[0045] FIG. 31 is a diagram illustrating a method for providing guide information using results based on existing biomarkers and cardiovascular disease diagnosis information according to one embodiment.

[0046] FIG. 32 is a diagram for explaining a method for predicting the rate of progression of cardiovascular disease using cardiovascular disease diagnosis information according to one embodiment.

[0047]

[0048] A method for controlling a diagnostic device according to one embodiment includes the steps of: obtaining a retinal image of a subject; and obtaining cardiovascular disease diagnosis information for the subject using a machine learning model based on the retinal image, wherein the machine learning model includes a first model and a second model, wherein the first model may be a neural network model, and the second model may be a regression-based machine learning model.

[0049]

[0050] The above-described purposes, features, and advantages of the present application will become more apparent through the following detailed description, taken in conjunction with the accompanying drawings. However, as the present application is susceptible to various modifications and various embodiments, specific embodiments will be illustrated in the drawings and described in detail below.

[0051] In the drawings, the thicknesses of layers and regions are exaggerated for clarity, and when an element or layer is referred to as "on" or "on" another element or layer, this includes not only the case where the element or layer is directly on top of the other element or layer, but also the case where another layer or other element is interposed. In principle, the same reference numerals represent the same elements throughout the specification. In addition, elements that have the same function within the scope of the same idea that appear in the drawings of each embodiment are described using the same reference numerals.

[0052] If a detailed description of a known function or configuration related to this application is deemed to unnecessarily obscure the gist of this application, such detailed description will be omitted. Furthermore, numbers (e.g., "first," "second," etc.) used throughout the description of this specification are merely identifiers used to distinguish one component from another.

[0053] In addition, the suffixes "module" and "part" used for components in the following description are given or used interchangeably only for the convenience of writing the specification, and do not have distinct meanings or roles in themselves.

[0054] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

[0055]

[0056] 1. Diagnosis using retinal images

[0057] 1.1. Diagnostic Systems and Processes

[0058] 1.1.1. Purpose and Definition

[0059] Below, we describe diagnostic systems and methods for assisting medical professionals in determining the presence or absence of a disease or abnormalities that serve as the basis for such determination based on ocular images. In this specification, the term "diagnosis" may refer to a diagnostic aid intended to assist in the diagnosis of a disease, rather than directly diagnosing the disease. For convenience of explanation, the term "diagnosis" is used below, but it can also refer to a diagnostic aid.

[0060] In particular, a machine learning model for diagnosing a disease using deep learning techniques is constructed, and a diagnostic method for assisting in detecting the presence or absence of a disease or abnormal findings using the constructed model is described. In this specification, an ocular image is an image including an eye in a subject, and may include various images such as a retinal image and / or a fundus image. In this specification, for the convenience of explanation, the technology of this specification is described with a focus on retinal images, but the present invention is not limited thereto, and the description of this specification can of course be applied to other ocular images.

[0061] The machine learning model described herein can be designed based on various machine learning libraries. For example, the machine learning model can refer to various types of models designed based on supervised, unsupervised, semi-supervised, or reinforcement learning artificial intelligence algorithms such as decision trees, random forest algorithms, stochastic gradient descent algorithms, neural network algorithms, k-nearest neighbor algorithms, linear regression, logistic regression, Cox proportional hazards model (survival model) (regression-based), support vector machines, k-means, hierarchical cluster analysis (HCA), expectation maximization, principal component analysis (PCA), kernel PCA, locally linear embedding (LLE), t-distributed stochastic neighbor embedding (t-SNE), Apriori, and Eclat.

[0062] In the following, unless otherwise specified, the machine learning model is mainly described as a neural network model for convenience, but this does not necessarily mean only a model form based on a neural network algorithm, and it is obvious that the model may be replaced with a model based on another algorithm within the scope of the function and purpose of the invention described in this specification.

[0063] In one embodiment, a diagnostic system or method may be provided that assists in diagnosing at least one of an eye disease, a cardiovascular disease (and / or a cerebrovascular disease), a kidney disease, and other systemic diseases based on a retinal image.

[0064] For example, in the present specification, the ocular disease may include at least one of cataract, glaucoma, macular degeneration, diabetic retinopathy, epiretinal membrane, macular hole, high / degenerative myopia, melanoma, retinal detachment, dry eye syndrome, presbyopia, and astigmatism.

[0065] Additionally, cardiovascular disease may include at least one of coronary artery disease (CAD), aortic valve stenosis, hypertension, arterial hypertension, heart failure, arrhythmia, atrial fibrillation, valvular heart disease, cardiomyopathy, peripheral vascular disease, peripheral artery disease (PAD), heart attack, aneurysm, aortic aneurysm (e.g., abdominal aortic aneurysm, thoracic aortic aneurysm), thrombotic disease (e.g., deep vein thrombosis, pulmonary embolism), myocardial infarction, and cerebrovascular disease. Additionally, cardiovascular disease may include at least one of stroke, ischemic stroke, cerebral infarction, cerebral hemorrhage, subspinal hemorrhage, transient ischemic attack, and death due to cardiovascular disease. Cardiovascular disease may include complications. Furthermore, complications may include aspiration pneumonia, dysphagia, motor dysfunction, speech dysfunction, cognitive dysfunction, sleep disturbance, emotional disturbance, neuropathic pain, urinary tract infection, malnutrition, deep vein thrombosis, bedsores, falls, pain, seizures, and depression.

[0066] Additionally, kidney disease may include at least one of chronic kidney disease (CKD), acute kidney injury (AKI), kidney stones, nephrotic syndrome, glomerulonephritis, polycystic kidney disease (PKD), kidney cancer, and kidney infection (pyelonephritis). Additionally, kidney disease may include complications. Additionally, complications may include side effects from dialysis, such as low blood pressure, muscle cramps, nausea and vomiting, headache, dialysis imbalance syndrome, itching, and arrhythmia.

[0067] Additionally, other systemic diseases may include at least one of diabetes, hypertension, hypotension, Alzheimer's, cytomegalovirus, and arteriosclerosis.

[0068]

[0069] Additionally, according to another embodiment, various parameters of a subject can be predicted based on a retinal image. For example, the parameters may include at least one parameter representing physical information of the subject, such as biological age, gender, height, weight, BMI, body mass index, body fat percentage, and body muscle mass. In addition, the above parameters may include at least one of diagnostic numerical parameters such as hematocrit level, red blood cell count, white blood cell count, hemoglobin level, platelet count, total iron binding capacity (TIBC), iron level, ferritin (storage iron protein) level, total protein level, albumin level, asparagine transferase level (AST), aminotransferase level, γ-GTP, γ-GT, alkaline phosphatase (ALP) level, globulin level, hepatitis antigen level, hepatitis antibody level, glycated hemoglobin (HbA1C), blood urea nitrogen (BUN) level, creatinine level, uric acid level, total cholesterol level, high-density lipoprotein (HDL) cholesterol level, low-density lipoprotein (LDL) cholesterol level, triglyceride (TG) level, bicarbonate level, systolic blood pressure (SBP), and diastolic blood pressure (DBP).

[0070]

[0071] In another embodiment, a diagnostic system or method for detecting abnormal retinal findings that can be used in the diagnosis of eye diseases or other diseases may be provided. For example, color abnormalities throughout the retina, lens opacity, abnormalities in the cup to disc ratio (C / D ratio), macular abnormalities (e.g., macular hole), abnormalities in the diameter and course of blood vessels, abnormalities in the diameter of retinal arteries, retinal hemorrhages, microaneurysms, hard exudates, epiretinal membranes, myelinated nerve fibers, chorioretinal atrophy, RNFL defects, the occurrence of exudates, drusen, cataracts, glaucoma, diabetic retinopathy, tessellated fundus, large optic cup, retinal vein occlusion (RVO), branch retinal vein occlusion (BRVO), central retinal vein occlusion (CRVO), retinal artery occlusion (RAO), rhegmatogenous retinal detachment (RD), and posterior hydrocephalus. (Posterior serous / exudative RD), Central Cystic Retinopathy (CSCR), VKH disease, Maculopathy, Retinopathy of Prematurity (ERM), Macular Hole (MH), Pathological myopia, Optic nerve degeneration, Optic atrophy, Severe hypertensive retinopathy, Disc swelling and elevation, Dragged disc, Pigmentary degeneration, Congenital disc abnormality,Retinitis pigmentosa, Bietti crystalline dystrophy, Peripheral retinal degeneration and break, Myelinated nerve fiber, Vitreous particles, Fundus neoplasm, Massive hard exudates, Yellow-white spots / flecks, Cotton-wool spots, Vessel tortuosity, Chorioretinal atrophy / coloboma, Preretinal hemorrhage, Fibrosis, Laser spots, Silicone oil in eye, Blur fundus, Blur fundus without PDR, Blur suspected of PDR A diagnostic system or method for obtaining information on findings such as fundus with suspected PDR may be provided.

[0072] In this specification, diagnostic information may be understood to encompass diagnostic information based on the determination of the presence or absence of a disease or information on findings that serve as the basis therefor.

[0073]

[0074] 1.1.2. Diagnostic System Configuration

[0075] According to one embodiment, a diagnostic system may be provided.

[0076] FIG. 1 illustrates a diagnostic system according to one embodiment. Referring to FIG. 1, the diagnostic system (1) may include a learning device (10) that trains a diagnostic model, a diagnostic device (20) that performs a diagnosis using the diagnostic model, and a client device (30) that obtains a diagnostic request. The diagnostic system (1) may include a plurality of learning devices, a plurality of diagnostic devices, or a plurality of client devices.

[0077] The learning device (10) may include a learning unit (100). The learning unit (100) may perform training of a diagnostic model. For example, the learning unit (100) may acquire a retinal image data set and perform training of a diagnostic model that detects diseases or abnormal findings from the retinal image. The learning unit (100) may be included in a processor of the learning device (10) described below, and may be meant to functionally express the processing of the processor for learning.

[0078] The diagnostic device (20) may include a diagnostic unit (200). The diagnostic unit (200) may use a diagnostic model to diagnose a disease or acquire auxiliary information used for diagnosis. For example, the diagnostic unit (200) may use a diagnostic model trained by a learning unit to acquire diagnostic information. The diagnostic unit (200) may be included in a processor of the diagnostic device (20) described below, and may mean a functional representation of the processor's processing for diagnosis.

[0079] The client device (30) may include an imaging unit (300). The imaging unit (300) may capture retinal images. The client device may be an ophthalmic retinal imaging device. Alternatively, the client device (30) may be a handheld device such as a smartphone or tablet PC.

[0080] In the diagnosis system (1) according to the present embodiment, the learning device (10) acquires a data set and performs learning of a diagnosis model model to determine a diagnosis model to be used for diagnosis, and when an information request is obtained from a client, the diagnosis device acquires diagnosis information according to a diagnosis target image using the determined diagnosis model model, and the client device can request information from the diagnosis device and acquire the diagnosis information transmitted in response thereto.

[0081] A diagnostic system according to another embodiment may include a diagnostic device and a client device that learn a diagnostic model and perform a diagnosis using the same. A diagnostic system according to another embodiment may include a diagnostic device that learns a diagnostic model, acquires a diagnostic request, and performs a diagnosis. A diagnostic system according to another embodiment may include a learning device that learns a diagnostic model and a diagnostic device that acquires a diagnostic request and performs a diagnosis.

[0082] The diagnostic system disclosed in this specification is not limited to the embodiments described above, and may be implemented in any form including a learning unit that performs model learning, a diagnostic unit that obtains diagnostic information according to the learned model, and an imaging unit that obtains a diagnostic target image.

[0083] Below, several embodiments of each device constituting the system are described.

[0084]

[0085] 1.1.2.1. Learning Device

[0086] A learning device according to one embodiment can perform training of a diagnostic model model that assists in diagnosis.

[0087] FIG. 2 is a block diagram illustrating a learning device according to one embodiment. Referring to FIG. 2, the learning device (10) may include a processor (12) and a storage module (11).

[0088] The learning device (10) may include a processor (12). The processor (12) may control the operation of the learning device (10).

[0089] The processor (12) may include one or more of a CPU (Central Processing Unit), a RAM (Random Access Memory), a GPU (Graphics Processing Unit), one or more microprocessors, and other electronic components capable of processing input data according to predetermined logic. In addition, there may be at least one processor (12).

[0090] The processor (12) can read the system program and various processing programs stored in the storage module (11). For example, the processor (12) can deploy processes, methods, etc. for performing the diagnosis described below on RAM and perform various processing according to the deployed program. The processor (12) can perform learning of the diagnosis model described below.

[0091] The learning device (10) may include a storage module (11). The storage module (11) may store data and learning models required for learning.

[0092] The storage module (11) can be implemented as a nonvolatile semiconductor memory, hard disk, flash memory, RAM, ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), or other types of tangible nonvolatile recording media.

[0093] The storage module (11) can store various processing programs, parameters for performing program processing, or data resulting from such processing. For example, the storage module (11) can store data processing process programs for performing the diagnosis described below, diagnostic process programs, parameters for performing each program, and data (e.g., processed data or diagnostic result values) obtained by performing such programs. In addition, the storage module (11) can store various diagnostic models described below.

[0094] The learning device (10) may include a separate learning unit. The learning unit may perform learning of a diagnostic model.

[0095] The learning unit may be included in the aforementioned processor (12). The learning unit may be stored in the aforementioned storage module (11). The learning unit may be implemented by some components of the aforementioned processor (12) and storage module (11). For example, the learning unit may be stored in the storage module (11) and driven by the processor (12).

[0096] The learning device (10) may further include a communication module (13). The communication module (13) may communicate with an external device. For example, the communication module (13) may communicate with a diagnostic device, a server device, or a client device, which will be described later. The communication module (13) may perform wired or wireless communication. The communication module (13) may perform bidirectional or unidirectional communication.

[0097]

[0098] 1.1.2.2. Diagnostic Device

[0099] The diagnostic device can obtain diagnostic information using the diagnostic model.

[0100] Fig. 3 is a block diagram illustrating a diagnostic device according to one embodiment. Referring to Fig. 3, the diagnostic device (20) may include a processor (22) and a storage module (21).

[0101] The processor (22) may include one or more of a CPU (Central Processing Unit), a RAM (Random Access Memory), a GPU (Graphics Processing Unit), one or more microprocessors, and other electronic components capable of processing input data according to predetermined logic. In addition, there may be at least one processor (22).

[0102] The processor (22) can read the system program and various processing programs stored in the storage module (21). For example, the processor (22) can deploy processes, methods, etc. for performing the diagnosis described below on RAM and perform various processing according to the deployed program. The processor (22) can generate diagnostic information using a diagnostic model. The processor (22) can obtain diagnostic data for diagnosis (e.g., retinal data of the subject) and obtain diagnostic information predicted by the diagnostic data using a learned diagnostic model.

[0103] The storage module (21) can store a diagnostic model. The storage module (21) can store parameters, variables, etc. of the diagnostic model.

[0104] The storage module (21) can be implemented as a nonvolatile semiconductor memory, a hard disk, a flash memory, RAM, ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), or other types of tangible nonvolatile recording media.

[0105] The storage module (21) can store various processing programs, parameters for performing program processing, or data resulting from such processing. For example, the storage module (21) can store a data processing process program for performing the diagnosis described below, a diagnostic process program, parameters for performing each program, and data (e.g., processed data or diagnostic result values) obtained by performing such programs. In addition, the storage module (21) can store various diagnostic models described below.

[0106] Although not shown, the diagnostic device (20) may further include an input module. The input module may obtain user input. For example, the input module may obtain user input requesting diagnostic information. In addition, the input module may obtain physical information of the subject (at least one of height, weight, age, gender, race, smoking status, blood pressure (e.g., blood pressure level, high blood pressure), diabetes (or blood sugar level), and cholesterol level). The processor (22) may obtain information input through the input module.

[0107] The diagnostic device (20) may further include a communication module (23). The communication module (23) may communicate with a learning device and / or a client device. For example, the diagnostic device (20) may be provided in the form of a server that communicates with a client device. This will be described in more detail below.

[0108]

[0109] 1.1.2.3. Server Device

[0110] According to one embodiment, the diagnostic system may include a server device. According to one embodiment, the diagnostic system may include a plurality of server devices.

[0111] The server device can store and / or run the diagnostic model. The server device can store the weight values ​​that constitute the learned diagnostic model. The server device can collect or store data used for diagnosis.

[0112] The server device can output the results of the diagnostic process using the diagnostic model to the client device. The server device can obtain feedback from the client device. The server device can operate similarly to the diagnostic device described above.

[0113] FIG. 4 illustrates a diagnostic system according to one embodiment. Referring to FIG. 4, the diagnostic system (20) according to one embodiment may include a diagnostic server (40), a learning device, and a client device.

[0114] The diagnostic server (40), i.e., the server device, can communicate with a plurality of learning devices or a plurality of diagnostic devices. Referring to FIG. 6, the diagnostic server (40) can communicate with a first learning device (10a) and a second learning device (10b). Referring to FIG. 4, the diagnostic server (40) can communicate with a first client device (30a) and a second client device (30b).

[0115] For example, the diagnostic server (40) can communicate with a first learning device (10a) that trains a first diagnostic model that obtains first diagnostic information and a second learning device (10b) that trains a second diagnostic model that obtains second diagnostic information.

[0116] The diagnostic server (40) stores a first diagnostic model for acquiring first diagnostic information and a second diagnostic model for acquiring second diagnostic information, and can acquire diagnostic information in response to a request for acquiring diagnostic information from a first client device (30a) or a second client device (30b), and transmit the acquired diagnostic information to the first client device (30a) or the second client device (30b).

[0117] Alternatively, the diagnostic server (40) may communicate with a first client device (30a) requesting first diagnostic information and a second client device (30b) requesting second diagnostic information.

[0118]

[0119] 1.1.2.4. Client Device

[0120] A client device can request diagnostic information from a diagnostic device or server device. The client device can obtain data necessary for diagnosis and transmit the acquired data to the diagnostic device.

[0121]

[0122] FIG. 5 is a block diagram illustrating a client device according to one embodiment. Referring to FIG. 5, a client device (30) according to one embodiment may include an imaging module (31), a processor (32), and a communication module (33).

[0123] The imaging module (31) can acquire images or video data. The imaging module (31) can acquire retinal images. However, the client device (30) may be replaced with a data acquisition unit of a different type than the imaging module (31).

[0124] The communication module (33) can communicate with an external device, such as a diagnostic device or a server device. The communication module (33) can perform wired or wireless communication.

[0125] The processor (32) can control the imaging module (31) to acquire an image or data. The processor (32) can control the imaging module (31) to acquire a retinal image. The processor (32) can transmit the acquired retinal image to a diagnostic device. The processor (32) can transmit the image acquired through the imaging module (31) to a server device through the communication module (33), and acquire diagnostic information generated based on the image.

[0126] Although not shown, the client device may further include an output module. The output module may include a display for outputting video or images or a speaker for outputting audio. The output module may output video or image data acquired by the acquired imaging unit. The output module may output diagnostic information acquired from the diagnostic device.

[0127] Although not shown, the client device may further include an input module. The input module may obtain user input. For example, the input module may obtain user input requesting diagnostic information. The input module may obtain user information for evaluating the diagnostic information obtained from the diagnostic device. In addition, the input module may obtain physical information of the subject (at least one of height, weight, age, gender, race, smoking status, blood pressure (e.g., blood pressure level, high blood pressure), diabetes (or blood sugar level), and cholesterol level).

[0128] Additionally, although not shown, the client device may further include a storage module. The storage module may store images acquired by the imaging unit.

[0129]

[0130] 1.1.3. Diagnostic Process Overview

[0131] A diagnostic process can be performed by the diagnostic system or device disclosed herein. The diagnostic process can be broadly divided into a learning process for learning a diagnostic model used for diagnosis, and a diagnostic process utilizing the diagnostic model.

[0132] FIG. 6 is a diagram illustrating a diagnostic process according to one embodiment. Referring to FIG. 6, the diagnostic process according to one embodiment may include a learning process of acquiring and processing data (S11), learning a diagnostic model (S12), and acquiring parameters of the learned diagnostic model (S13), and a diagnostic process of acquiring diagnostic target data (S21) and using a diagnostic model (S22) learned based on the diagnostic target data to acquire diagnostic information (S23).

[0133] More specifically, the learning process may include a data processing process for processing input learning image data into a state that can be used for model training, and a learning process for training a model using the processed data. The learning process may be performed by the aforementioned learning device.

[0134] The diagnostic process may include a data processing process for processing input inspection target image data into a state in which a diagnosis can be performed using a diagnostic model, and a diagnostic process for performing a diagnosis using the processed data. The diagnostic process may be performed by the aforementioned diagnostic device or server device.

[0135] Below, each process is described.

[0136]

[0137] 1.2. Learning Process

[0138] In one embodiment, a process for training a diagnostic model may be provided. As a specific example, a process for training a diagnostic model that performs or assists in diagnosis based on a retinal image may be disclosed.

[0139] The learning process described below can be performed by the learning device described above.

[0140]

[0141] 1.2.1. Learning Department

[0142] In one embodiment, the learning process may be performed by a learning unit. The learning unit may be provided within the aforementioned learning device.

[0143] FIG. 7 is a diagram illustrating the configuration of a learning unit according to one embodiment. Referring to FIG. 7, the learning unit (100) may include a data processing module (110), a queue module (130), a learning module (150), and a learning result acquisition module (170). Each module may perform individual steps of the data processing process and the learning process, as described below. However, not all of the components and functions performed by each component described in FIG. 7 are essential, and depending on the learning type, some components may be added or omitted.

[0144]

[0145] 1.2.2. Data processing process

[0146] 1.2.2.1. Image data acquisition

[0147] In one embodiment, a data set can be obtained. In one embodiment, a data processing module can obtain the data set.

[0148] The dataset can be an image dataset.

[0149] For example, it may be a retinal image data set. The retinal image data set can be acquired using a general non-mydriatic retinal camera, etc. The retinal image can be a panoramic retinal image or a wide retinal image. The retinal image can be a red-free image. The retinal image can be an infrared photographed image. The retinal image can be an autofluorescence photographed image. The image data can be acquired in any of the following formats: JPG, PNG, DCM (DICOM), BMP, GIF, and TIFF.

[0150] As another example, the dataset may be an image dataset including any one of an Optical Coherence Tomography (OCT) image, an OCT angiography image, or a retinal angiography image. In this case, a diagnostic model trained using a dataset including an OCT image, an OCT angiography image, or a retinal angiography image can predict or output diagnostic information (or a label) based on the target OCT image, the target OCT angiography image, or the target retinal angiography image.

[0151] A data set may include a training data set. A data set may include a test data set. A data set may include a validation data set. In other words, a data set may be assigned to at least one of a training data set, a test data set, and a validation data set.

[0152] The dataset can be determined based on the diagnostic information desired to be obtained using the diagnostic model trained through the dataset. For example, if a diagnostic model for obtaining diagnostic information related to cataracts is to be trained, the acquired dataset may be determined to be an infrared retinal image dataset. Alternatively, if a diagnostic model for obtaining diagnostic information related to macular degeneration is to be trained, the acquired dataset may be an autofluorescence retinal image dataset.

[0153]

[0154] Individual data included in a dataset may include a label. There may be multiple labels. In other words, individual data included in a dataset may be labeled with respect to at least one feature. For example, the dataset may be a retinal image dataset including multiple retinal image data, and each retinal image data may include a label of diagnostic information (e.g., presence or absence of a specific disease) and / or finding information (e.g., presence or absence of an abnormality in a specific area) according to the image.

[0155] As another example, the data set may be a retinal image data set, and each retinal image data may include a peripheral information label for the image. For example, each retinal image data may include peripheral information labels including left and right eye information, such as whether the retinal image is an image of the left eye or the right eye, gender information, such as whether the retinal image is a female or male retinal image, and age information, such as the age of the subject who took the retinal image.

[0156] FIG. 8 is a conceptual diagram illustrating an image data set according to one embodiment. Referring to FIG. 8, an image data set (DS) according to one embodiment may include a plurality of image data (ID). Each image data (ID) may include an image (I) and a label (L) assigned to the image. Referring to FIG. 10, an image data set (DS) may include first image data (ID1) and second image data (ID2). The first image data (ID1) may include a first image (I1) and a first label (L1) corresponding to the first image.

[0157] In Fig. 8, the description is based on the case where one image data includes one label, but as described above, one image data may include multiple labels.

[0158]

[0159] 1.2.2.2. Image Preprocessing

[0160] In one embodiment, image preprocessing may be performed. If images are used for learning as they are, overfitting may occur in learning results for unnecessary features, and learning efficiency may also be reduced.

[0161] To prevent this, the data processing module can improve learning efficiency and performance by appropriately preprocessing and using image data to suit the purpose of learning.

[0162] In one embodiment, the data processing module may perform image preprocessing using various techniques, such as image resizing, grayscale conversion, histogram flattening, normalization, feature enhancement, image augmentation, noise removal, edge detection, segmentation, morphological operations, and color space conversion, or using a suitable combination thereof. Below, image preprocessing using several techniques is described.

[0163]

[0164] 1.2.2.2.1. Image Resizing

[0165] In one embodiment, the size of the acquired image data may be adjusted, i.e., the images may be resized. In one embodiment, the image resizing may be performed by the data processing module of the learning unit described above.

[0166] The size or aspect ratio of an image can be adjusted. The acquired multiple images can be resized to have a consistent size. Alternatively, the images can be resized to have a consistent aspect ratio. Resizing an image can involve applying an image transformation filter to the image.

[0167] If the size or capacity of the acquired individual images is excessively large or small, the image size or capacity can be adjusted to convert them to an appropriate size. Alternatively, if the individual images have different sizes or capacities, the sizes or capacities can be unified through resizing.

[0168] In one embodiment, the image size can be adjusted. For example, if the image size exceeds an appropriate range, the image can be reduced through downsampling. Alternatively, if the image size falls below an appropriate range, the image can be enlarged through upsampling or interpolation.

[0169] In another embodiment, the image size or aspect ratio can be adjusted by cropping the image or adding pixels to the acquired image. For example, if an image contains parts unnecessary for learning, a portion of the image can be cropped to remove them. Alternatively, if a portion of the image is cropped and the aspect ratio is not correct, the image aspect ratio can be adjusted by adding columns or rows. In other words, the aspect ratio can be adjusted by adding margins or padding to the image.

[0170] In another embodiment, the image's capacity and size or aspect ratio can be adjusted together. For example, if the image is large, the image can be downsampled to reduce its capacity, and unnecessary portions of the reduced image can be cropped to convert it into appropriate image data.

[0171] Additionally, according to another embodiment, the orientation of the image data may be changed.

[0172] As a specific example, when a retinal image data set is used as a data set, the capacity or size of each retinal image can be adjusted. Cropping can be performed to remove white space excluding the retinal portion of the retinal image, or padding can be performed to adjust the aspect ratio by supplementing the cropped portion of the retinal image.

[0173]

[0174] 1.2.2.2.2. Feature Highlighting

[0175] In one embodiment, the data processing module may perform preprocessing to emphasize features of the retinal image. For example, the data processing module may perform preprocessing to facilitate the detection of abnormalities in the retinal image due to ocular diseases, or to emphasize changes in retinal blood vessels or blood flow.

[0176] For example, image preprocessing may be performed on an image that has undergone the aforementioned resizing process. However, the invention disclosed herein is not limited to this, and preprocessing may be performed on the image without resizing. Preprocessing an image may involve applying a preprocessing filter to the image.

[0177] In one embodiment, a blur filter may be applied to the image. A Gaussian filter may be applied to the image. A Gaussian blur filter may also be applied to the image. Alternatively, a deblur filter may be applied to the image to sharpen the image.

[0178] In another embodiment, a filter may be applied that adjusts or modulates the color of an image. For example, a filter may be applied that changes the values ​​of some components of the RGB values ​​that make up the image, or a filter that binarizes the image may be applied.

[0179] In another embodiment, a filter may be applied to emphasize specific elements in an image. For example, preprocessing may be performed on retinal image data to emphasize blood vessel elements in each image. The preprocessing to emphasize blood vessel elements may involve applying one or more filters sequentially or in combination.

[0180] Additionally, according to one embodiment, image preprocessing may be performed considering the characteristics of the diagnostic information to be obtained. For example, when obtaining diagnostic information related to findings such as retinal hemorrhages, drusen, microangiopathy, and exudates, preprocessing may be performed to convert the acquired retinal image into a red-free retinal image format.

[0181]

[0182] 1.2.2.2.3. Image Augmentation

[0183] In one embodiment, an image may be augmented or expanded. Image augmentation may be performed by the data processing module of the learning unit described above.

[0184] Augmented images can be used to improve the training performance of diagnostic models. For example, when the amount of training data for a diagnostic model is insufficient, existing training image data can be modified to increase the training data set. The modified (or altered) images can then be used alongside the original images to increase the training image data set. This can suppress overfitting, deepen the model layers, and improve prediction accuracy.

[0185] For example, image data expansion can be performed by reversing the left and right sides of the image, cropping a portion of the image, correcting the color values ​​of the image, or adding artificial noise. Specifically, cropping a portion of the image can be performed by cropping a portion of an element that constitutes the image or randomly cropping a portion of the image. More examples include reversing the image horizontally, vertically, resizing it at a certain ratio, cropping, padding, adjusting the color, or adjusting the brightness of the image.

[0186] Additionally, in one embodiment, the data processing module can rotate the retinal image to augment the image. Since the retina is circular in shape, there may be no data loss even if the retina is rotated. Accordingly, when performing image augmentation by rotating the retinal image, multiple retinal images can be acquired without data loss. For example, the data processing module can acquire multiple retinal images by rotating the retinal image based on a predetermined angle (e.g., 15 degrees, 30 degrees, 60 degrees, etc.).

[0187] Additionally, in one embodiment, the augmentation or expansion of image data described above can generally be applied to a training data set. However, it can also be applied to other data sets, such as a test data set, i.e., a data set for testing a model that has been trained using training data and validated using validation data.

[0188] As a specific example, when a retinal image data set is used as a data set, an augmented retinal image data set can be obtained by randomly applying one or more of the following processing to increase the number of data: inverting the image, cropping it, adding noise, or changing its color.

[0189]

[0190] 1.2.2.3. Image Serialization

[0191] In one embodiment, image data can be serialized. The image can be serialized by the data processing module of the learning unit described above. The serialization module can serialize the preprocessed image data and pass it to the queue module.

[0192] If image data is used for training as it is, it requires decoding because it is in the form of an image file such as JPG, PNB, or DCM. However, if training is performed by decoding every time, the performance of the model training may deteriorate. Therefore, instead of using the image file as it is for training, training can be performed by serializing it. Therefore, serialization of image data can be performed to improve training performance and speed. The image data to be serialized may be image data to which one or more of the aforementioned image resizing and image preprocessing steps have been applied, or may be image data to which neither of the steps has been processed.

[0193] Each image data contained in the image dataset can be converted into a string format. The image data can be converted into a binarized data format. Specifically, the image data can be converted into a data format suitable for use in training a diagnostic model. For example, the image data can be converted into a TFRecord format for use in training a diagnostic model using TensorFlow.

[0194] As a specific example, when a set of retinal images is used as a data set, the acquired set of retinal images can be converted into TFRecord format and used for learning a diagnostic model.

[0195]

[0196] 1.2.2.4. Queue

[0197] Queues can be used to resolve data bottlenecks. The queue module of the aforementioned learning unit can store image data in a queue and transmit it to the learning model module.

[0198] In particular, when conducting a learning process using a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit) together, using a queue can minimize bottlenecks between the CPU and GPU, facilitate access to the database, and improve memory usage efficiency.

[0199] A queue can store data used to train a diagnostic model. A queue can store image data. The image data stored in the queue can be image data that has undergone at least one of the aforementioned data processing processes (i.e., resizing, preprocessing, and augmentation) or can be images in their original state.

[0200] A queue can store image data, preferably serialized image data as described above. The queue can store the image data and supply it to a diagnostic model. The queue can transmit image data to the diagnostic model in batch sizes.

[0201] A queue can provide image data. The queue can provide data to a learning module, as described below. As the learning module extracts data, the amount of data accumulated in the queue can decrease.

[0202] As the diagnostic model's training progresses, if the number of data stored in the queue drops below a certain threshold, the queue can request additional data. The queue can request additional data of a specific type. When requested, the learning unit can add data to the queue.

[0203] Queues can be created in the system memory of a learning device. For example, queues can be created in the random access memory (RAM) of a central processing unit (CPU). In this case, the queue size, or capacity, can be determined based on the CPU's RAM capacity. A first-in, first-out (FIFO) queue, a priority queue, or a random queue can be used.

[0204]

[0205] 1.2.3. Learning Process

[0206] In one embodiment, a learning process of a diagnostic model can be initiated.

[0207] According to one embodiment, the learning of the diagnostic model may be performed by the aforementioned learning device. The learning process may be performed by the processor of the aforementioned learning device. The learning process may be performed by the learning module of the aforementioned learning unit.

[0208] Fig. 9 is a block diagram illustrating a learning process of a diagnostic model according to one embodiment. Referring to Fig. 9, the learning process of a diagnostic model according to one embodiment can be performed by acquiring data (S31), learning a diagnostic model (S32), verifying the learned model (S33), and acquiring variables of the learned model (S34).

[0209]

[0210] 1.2.3.1. Data Entry

[0211] A data set can be obtained for training the diagnostic model.

[0212] The acquired data may be an image data set processed by the aforementioned data processing process. For example, the data set may include serialized retinal image data after being resized, preprocessed, and augmented.

[0213] During the training phase of the diagnostic model, a training data set can be acquired and utilized. During the validation phase of the diagnostic model, a validation data set can be acquired and utilized. During the testing phase of the diagnostic model, a test data set can be acquired and utilized. Each data set can include retinal images and labels.

[0214] Data sets can be retrieved from a queue. Data sets can be retrieved from the queue in batch sizes. For example, if a batch size of 60 is specified, data sets can be retrieved from the queue in batches of 60. The batch size may be limited by the RAM capacity of the GPU.

[0215] Data sets can be randomly obtained from the queue by the learning module. Data sets can also be obtained in the order in which they were accumulated in the queue.

[0216] The learning module can extract data by specifying the structure of the data set obtained from the queue. For example, the learning module can extract retinal image data with a left eye label and retinal image data with a right eye label of a specific subject to be used together for learning.

[0217] The learning module can obtain a data set with a specific label from the queue. For example, the learning module can obtain a retinal image data set with abnormal diagnostic information labels from the queue. The learning module can obtain the data set by specifying a ratio of the number of data according to the label from the queue. For example, the learning module can obtain a retinal image data set such that the number of retinal image data with abnormal diagnostic information labels is 1:1 and the number of retinal image data with normal diagnostic information labels is 1:1.

[0218]

[0219] 1.2.3.2. Model Design

[0220] The diagnostic model can be designed using various models, such as a neural network model or a machine learning model. In one embodiment, if the diagnostic model includes a neural network model, the neural network model may include multiple layers or layers.

[0221] A neural network model can be implemented as a classifier that generates diagnostic information. The classifier can perform dual or multi-classification. For example, the neural network model may be a binary classification model that classifies input data into normal or abnormal classes based on target diagnostic information, such as a specific disease or abnormality. Alternatively, the neural network model may be a multi-classification model that classifies input data into multiple graded classes based on a specific characteristic (e.g., the degree of disease progression). Alternatively, the neural network model may be implemented as a regression model that outputs a specific numerical value associated with a specific disease.

[0222] The neural network model may include a convolutional neural network (CNN). As a CNN structure, at least one of AlexNet, LENET, NIN, VGGNet, ResNet, WideResnet, GoogleNet, FractaNet, DenseNet, FitNet, RitResNet, HighwayNet, MobileNet, and DeeplySupervisedNet may be used. The neural network model may be implemented using multiple CNN structures.

[0223] For example, a neural network model can be implemented to include multiple VGGNet blocks. More specifically, the neural network model can be prepared by combining a first structure in which a CNN layer having 64 filters of 3x3 size, a BN (Batch Normalization) layer, and a ReLu layer are sequentially combined, and a second block in which a CNN layer having 128 filters of 3x3 size, a ReLu layer, and a BN layer are sequentially combined.

[0224] The neural network model may include a max pooling layer following each CNN block, and may include a global average pooling (GAP) layer, a fully connected (FC) layer, and an activation layer (e.g., sigmoid, softmax, etc.) at the end.

[0225] Additionally, in another embodiment, when the diagnostic model includes a machine learning model, the machine learning model may include a linear regression model, a Cox proportional hazards model, or the like.

[0226]

[0227] 1.2.3.3. Model Training

[0228] The diagnostic model can be trained using a training data set.

[0229] Diagnostic models can be trained using labeled data sets. However, the diagnostic model training process described herein is not limited to this, and the diagnostic model can also be trained unsupervised using unlabeled data.

[0230] The learning of the diagnostic model can be accomplished by obtaining a result value using a diagnostic model with arbitrary weight values ​​based on training image data, comparing the obtained result value with the label values ​​of the training data, and performing backpropagation based on the error to optimize the weight values. Furthermore, the learning of the diagnostic model can be influenced by the model verification results, test results, and / or feedback from the diagnostic step described below.

[0231] The training of the above-described diagnostic model can be performed using TensorFlow. However, the present application is not limited to this, and frameworks such as Theano, Keras, Caffe, Torch, and CNTK (Microsoft Cognitive Toolkit) can also be used to train the diagnostic model.

[0232]

[0233] 1.2.3.4. Model Validation

[0234] A diagnostic model can be validated using a validation dataset. Validation of the diagnostic model can be performed by obtaining output values ​​for the validation dataset from the trained diagnostic model and comparing the output values ​​with the labels in the validation dataset. Validation can be performed by measuring the accuracy of the output values. Based on the validation results, parameters (e.g., weights and / or bias) or hyperparameters (e.g., learning rate) of the diagnostic model can be adjusted.

[0235] For example, a learning device according to one embodiment can train a diagnostic model that predicts diagnostic information based on a retinal image, and perform verification of the diagnostic model by comparing diagnostic information for a verification retinal image of the trained model with a verification label corresponding to the verification retinal image.

[0236] To validate a diagnostic model, a separate validation set (external data set) can be used, i.e., a data set containing distinct factors not included in the training data set. For example, an external validation set could be a data set that differs from the training data set in factors such as race, environment, age, and gender.

[0237]

[0238] 1.2.3.5. Testing the Model

[0239] The diagnostic model can be tested using a test data set.

[0240] According to one embodiment, a learning process may be used to test a diagnostic model using a test data set distinct from the training data set and the validation data set. Based on the test results, parameters (e.g., weights and / or biases) or hyperparameters (e.g., learning rate) of the diagnostic model may be adjusted.

[0241] For example, a learning device according to one embodiment can obtain a result value from a diagnostic model learned to predict diagnostic information based on a retinal image, using test retinal image data that was not used for training and verification as input, and perform a test of the learned and verified diagnostic model.

[0242] For testing the diagnostic model, a separate validation set (external data set), i.e. a data set having factors distinct from the training and / or validation data, may be used.

[0243]

[0244] 1.2.3.6. Output of results

[0245] As a result of the diagnostic model's training, optimized model parameter values ​​can be obtained. As described above, by repeatedly training the model using the test data set, more appropriate parameter (or variable) values ​​can be obtained. Once training has progressed sufficiently, optimized weight and / or bias values ​​can be obtained.

[0246] According to one embodiment, the learned diagnostic model and / or the parameters or variables of the learned diagnostic model may be stored in the learning device and / or the diagnostic device (or server). The learned diagnostic model may be used to predict diagnostic information by the diagnostic device and / or a client device. Furthermore, the parameters or variables of the learned diagnostic model may be updated based on feedback obtained from the diagnostic device or the client device.

[0247]

[0248] 1.2.3.7. Model ensemble

[0249] In one embodiment, during the training of a diagnostic model, multiple sub-models may be trained simultaneously. The multiple sub-models may have different hierarchical structures.

[0250] In this case, a diagnostic model according to one embodiment can be implemented by combining multiple sub-diagnostic models. In other words, the diagnostic model can be trained using an ensemble technique that combines multiple sub-diagnostic models.

[0251] When a diagnostic model is constructed by forming an ensemble, predictions can be made by synthesizing the predicted results from various types of sub-diagnostic models, thereby improving the accuracy of the result prediction.

[0252]

[0253] 1.3. Diagnostic Process

[0254] In one embodiment, a diagnostic process (or diagnostic process) may be provided that obtains diagnostic information using a diagnostic model. As a specific example, the diagnostic process may use a retinal image to predict diagnostic information (e.g., diagnostic information or opinion information) through a learned diagnostic model.

[0255] The diagnostic process described below can be performed by a diagnostic device.

[0256]

[0257] 1.3.1. Diagnostic Department

[0258] According to one embodiment, the diagnostic process may be performed by a processor of the aforementioned diagnostic device. The processor may be provided within the aforementioned diagnostic device.

[0259] FIG. 10 is a diagram illustrating the configuration of a diagnostic unit according to one embodiment. Referring to FIG. 10, the diagnostic unit (200) may include a diagnostic request acquisition module (210), a data processing module (230), a diagnostic module (250), and an output module (270).

[0260] Each module can perform individual steps of the data processing and learning processes, as described below. However, not all components and functions performed by each component described in Figure 10 are essential, and some elements may be added or omitted depending on the diagnostic aspect.

[0261]

[0262] 1.3.2. Data Acquisition and Diagnostic Requests

[0263] According to one embodiment, a diagnostic device can acquire diagnostic target data and obtain diagnostic information based on the data. The diagnostic target data may be image data. Data acquisition and acquisition of a diagnostic request can be performed by the diagnostic request acquisition module of the aforementioned diagnostic unit.

[0264] For example, the diagnostic target data (TD) may include a diagnostic target image (TI) and diagnostic target object information (PI; patient information).

[0265] A diagnostic target image (TI) may be an image for obtaining diagnostic information about a diagnostic target object. For example, the diagnostic target image may be a retinal image and / or a retinal image. The diagnostic target (TI) may have any of the following formats: JPG, PNG, DCM (DICOM), BMP, GIF, or TIFF.

[0266] Diagnostic object information (PI) may be information for identifying the diagnostic target object. Alternatively, the diagnostic object information (PI) may be characteristic information of the diagnostic target object or image. For example, the diagnostic object information (PI) may include information such as the capture date and time of the diagnostic target image, the shooting equipment, the identification number, ID, name, gender, age, weight, race, smoking status, blood pressure (whether hypertension or hypertension), and diabetes status of the diagnostic target subject. If the diagnostic target image is a retinal image, the diagnostic object information (PI) may further include ocular-related information such as binocular information indicating whether the image is a left or right eye.

[0267] The diagnostic device can obtain a diagnostic request. The diagnostic device can obtain diagnostic target data along with the diagnostic request. Once the diagnostic request is obtained, the diagnostic device can obtain diagnostic information using a learned diagnostic model. The diagnostic device can obtain the diagnostic request from a client device. Alternatively, the diagnostic device can obtain the diagnostic request from a user through a separately provided input means.

[0268]

[0269] 1.3.3. Data processing process

[0270] The acquired data can be processed. Data processing can be performed by the data processing module of the diagnostic unit described above.

[0271] The data processing process can generally be performed similarly to the data processing process in the aforementioned learning process. Below, the data processing process in the diagnostic process will be described, focusing on the differences between it and the data processing process in the learning process.

[0272] In the diagnostic process, the diagnostic device can acquire data, similar to the learning process. The acquired data may be in the same format as the data acquired in the learning process. For example, if the learning device trains a diagnostic model using image data in DCM format during the learning process, the diagnostic device can acquire DCM images and use the trained diagnostic model to obtain diagnostic information.

[0273] In the diagnostic process, the acquired diagnostic target image may be resized similarly to the image data used in the learning process. The diagnostic target image may be resized to have an appropriate capacity, size, and / or aspect ratio to efficiently perform diagnostic information prediction using the learned diagnostic model.

[0274] For example, if the image to be diagnosed is a retinal image, resizing, such as cropping unnecessary parts of the image or reducing its size, may be performed to predict diagnostic information based on the retinal image.

[0275] In the diagnostic process, a preprocessing filter may be applied to the acquired diagnostic target image, similar to the image data used in the learning process. Appropriate filters may be applied to the diagnostic target image to further improve the accuracy of diagnostic information prediction using the learned diagnostic model.

[0276] For example, if the image to be diagnosed is a retinal image, preprocessing that facilitates prediction of diagnostic information, such as image preprocessing that emphasizes blood vessels or image preprocessing that emphasizes or weakens specific colors, can be applied to the image to be diagnosed.

[0277] In the diagnostic process, the acquired diagnostic target image can be serialized, similar to the image data used in the learning process. The diagnostic target image can be converted or serialized into a form that facilitates the operation of a diagnostic model in a specific framework.

[0278] Serialization of the target image can be omitted. This may be because, unlike the training phase, the processor processes only a small amount of data at a time during the diagnosis phase, thus placing relatively less strain on data processing speed.

[0279] In the diagnostic process, the acquired diagnostic target images can be stored in a queue, similar to the image data used in the learning process. However, since the diagnostic process requires less data to process than the learning process, the step of storing the data in a queue may be omitted.

[0280] Meanwhile, in the diagnostic process, since an increase in the amount of data is not required, unlike the learning process, data augmentation or image augmentation procedures may not be used to obtain accurate diagnostic information.

[0281]

[0282] 1.3.4. Diagnostic Process

[0283] According to one embodiment, a diagnostic process using a learned diagnostic model may be initiated. The diagnostic process may be performed on the aforementioned diagnostic device. The diagnostic process may be performed on the aforementioned diagnostic server. The diagnostic process may be performed on the control unit of the aforementioned diagnostic device. The diagnostic process may be performed by the diagnostic module of the aforementioned diagnostic unit.

[0284] Figure 11 is a diagram illustrating a diagnostic process according to one embodiment. Referring to Figure 11, the diagnostic process may be performed by acquiring diagnostic target data (S31), utilizing a learned diagnostic model (S42), and obtaining results corresponding to the acquired diagnostic target data (S43). However, data processing may be optionally performed.

[0285] Below, each step of the diagnostic process is described with reference to Fig. 11.

[0286]

[0287] 1.3.4.1. Data Entry

[0288] In one embodiment, the diagnostic module may acquire diagnostic target data. The acquired data may be processed data as described above. For example, the acquired data may be retinal image data of a subject, preprocessed to adjust the size and emphasize blood vessels. In one embodiment, the left and right eye images of a single subject may be input together as diagnostic target data.

[0289]

[0290] 1.3.4.2. Data Classification

[0291] A diagnostic model prepared in the form of a classifier can classify an input diagnostic target image into a positive or negative class with respect to a given label.

[0292] A trained diagnostic model can input diagnostic target data and output a predicted label. The trained diagnostic model can then output a predicted value for the diagnostic information. Diagnostic information can be obtained using the trained diagnostic model. The diagnostic information can be determined based on the predicted label.

[0293] For example, a diagnostic model can predict diagnostic information (i.e., information about the presence or absence of a disease) or finding information (i.e., information about the presence or absence of an abnormality) regarding an ocular or systemic disease of a subject. At this time, the diagnostic information or finding information can be output in the form of a probability. For example, the probability that a subject has a specific disease or the probability that a subject has a specific abnormality in a retinal image can be output. When using a diagnostic model prepared in the form of a classifier, the predicted label can be determined by considering whether the output probability value (or prediction score) exceeds a threshold.

[0294] As a specific example, a diagnostic model can output a probability value for whether or not the subject has diabetic retinopathy by using a retinal image of a subject as a diagnostic target image. When using a diagnostic model in the form of a classifier that sets 1 as normal, a retinal image of the subject can be input into the diagnostic model, and a probability value of normal:abnormal can be obtained in the form of 0.74:0.26 for whether or not the subject has diabetic retinopathy.

[0295] Here, the description is based on the case of classifying data using a diagnostic model in the form of a classifier, but it is not limited to this, and a diagnostic model implemented in the form of a regression model can also be used to predict specific diagnostic values ​​(e.g., blood pressure, etc.).

[0296]

[0297] In another embodiment, image suitability information may be obtained. The suitability information may indicate whether the diagnosis target image is suitable for obtaining diagnostic information using a diagnostic model.

[0298] Image suitability information may be image quality information. Quality information or suitability information may indicate whether the image being diagnosed meets a reference level.

[0299] For example, if a diagnostic target image has a defect due to a defect in the photographing equipment or the influence of lighting during the photographing, an unsuitable result may be output as suitability information for the diagnostic target image. If a diagnostic target image contains noise above a certain level, the diagnostic target image may be judged as unsuitable.

[0300] The suitability information may be a predicted value using a diagnostic model. Alternatively, the suitability information may be information obtained through a separate image analysis process.

[0301] According to one embodiment, even if the image is classified as unsuitable, diagnostic information can be obtained based on the unsuitable image.

[0302] In one embodiment, images classified as inappropriate can be re-examined by the diagnostic model.

[0303] At this time, the diagnostic model performing the re-examination may differ from the diagnostic model performing the initial review. For example, the diagnostic device may store a first diagnostic model and a second diagnostic model, and images classified as unsuitable by the first diagnostic model may be reviewed by the second diagnostic model.

[0304]

[0305] According to another embodiment, a map can be obtained from a learned diagnostic model. The diagnostic information can include the map. The map can be obtained together with other diagnostic information. For example, the map can include a saliency map, a class activation map (CAM), a heat map, etc. In addition, a CAM can be selectively obtained. For example, in the case of a CAM, a CAM can be extracted and / or output when the diagnostic information or finding information obtained by the diagnostic model is classified into an abnormal class.

[0306]

[0307] 1.3.5. Outputting diagnostic information

[0308] Diagnostic information can be determined based on the results output from the diagnostic model. The output of the diagnostic information can be performed by the output module of the aforementioned diagnostic unit. The diagnostic information can be output from the diagnostic device to a client device. The diagnostic information can be output from the diagnostic device to a server device. The diagnostic information can be stored in the diagnostic device or the diagnostic server. The diagnostic information can be stored in a separately provided server device, etc.

[0309] Diagnostic information can be managed in a database. For example, acquired diagnostic information can be stored and managed along with the diagnostic images of the subject, based on the subject's identification number. In this case, the subject's diagnostic images and diagnostic information can be managed chronologically. By managing diagnostic information and diagnostic images in a chronological order, tracking and history management of individual diagnostic information can be facilitated.

[0310] Diagnostic information may be provided to the user. The diagnostic information may be provided to the user via an output means of the diagnostic device or client device. The diagnostic information may be output so that the user can perceive it via a visual or auditory output means provided on the diagnostic device or client device.

[0311] According to one embodiment, an interface may be provided for effectively providing diagnostic information to a user.

[0312] Additionally, if an image is classified as unsuitable, information regarding the image's suitability may be provided. For example, if an image is classified as unsuitable, diagnostic information obtained based on the image and information regarding the unsuitability judgment may be provided together.

[0313] A diagnostic target image determined to be unsuitable may be classified as a re-capture target image. In this case, re-capture guidance for the target object in the image classified as a re-capture target may be provided along with suitability information. Meanwhile, in response to providing the diagnostic information acquired through the diagnostic model, feedback related to the learning of the diagnostic model may be obtained. For example, feedback for adjusting parameters or hyperparameters related to the learning of the diagnostic model may be obtained. The feedback may be obtained through an input module provided in the diagnostic device or client device.

[0314]

[0315] According to one embodiment, the diagnostic information corresponding to the diagnosis target image may include grade information. The grade information may be selected from a plurality of grades. The grade information may be determined based on the diagnostic information and / or finding information obtained through the diagnosis model. The grade information may be determined by considering the suitability information or quality information of the diagnosis target image. If the diagnosis model is a classifier model that performs multiple classifications, the grade information may be determined by considering the classes into which the diagnosis target image is classified by the diagnosis model. If the diagnosis model is a regression model that outputs a numerical value related to a specific disease, the grade information may be determined by considering the output numerical value. Additionally, the grade information may be determined by applying a predetermined cutoff value to a score according to the diagnosis information.

[0316] For example, the diagnostic information acquired in response to the diagnosis target image may include any one of the first grade information and the second grade information. The grade information may be selected as the first grade information if abnormal finding information or abnormal diagnosis information is acquired through the diagnosis model. The grade information may be selected as the second grade information if abnormal finding information or abnormal diagnosis information is not acquired through the diagnosis model. Alternatively, the grade information may be selected as the first grade information if the value acquired through the diagnosis model exceeds a reference value, and may be selected as the second grade information if the value acquired is below the reference value. The first grade information may indicate that there is strong abnormality information in the diagnosis target image compared to the second grade information.

[0317] Meanwhile, grade information may be selected as third-grade information when the quality of the image to be diagnosed is determined to be below standard using image analysis or a diagnostic model. Alternatively, the diagnostic information may include third-grade information along with first- or second-grade information.

[0318]

[0319] 1.4. Diagnostic system using multiple diagnostic models

[0320] According to one embodiment, diagnostic information can be output using the diagnostic model described above. The aforementioned diagnostic model may consist of a single diagnostic model or multiple diagnostic models. Furthermore, when the diagnostic model consists of multiple diagnostic models, the multiple diagnostic models may be configured in parallel or serially. Parallel and serial diagnostic models are described below.

[0321]

[0322] 1.4.1.1. Configuring the Parallel Diagnostic System

[0323] According to one embodiment, a parallel diagnostic system for acquiring multiple diagnostic information may be provided. The parallel diagnostic system may train multiple diagnostic models for acquiring multiple diagnostic information, and may acquire multiple diagnostic information using the trained multiple diagnostic models.

[0324] For example, a parallel diagnostic system can train a first diagnostic model that obtains first diagnostic information related to the presence or absence of an eye disease in a subject based on a retinal image and a second diagnostic model that obtains second diagnostic information related to the presence or absence of a systemic disease in the subject, and output diagnostic information related to the presence or absence of an eye disease and a systemic disease in the subject using the trained first diagnostic model and second diagnostic model.

[0325] Multiple diagnostic models can be trained in parallel and / or independently. By training models to predict different labels using multiple diagnostic models, prediction accuracy for each label can be improved and the efficiency of prediction operations can be enhanced. Since the previously described principles for training multiple diagnostic models apply, a detailed description is omitted.

[0326]

[0327] 1.4.1.2. Parallel Diagnostic Process

[0328] In one embodiment, a diagnostic process for obtaining multiple diagnostic information may be provided. The diagnostic process for obtaining multiple diagnostic information may be implemented in the form of a parallel diagnostic process comprising multiple independent diagnostic processes.

[0329]

[0330] In one embodiment, the diagnostic process may be performed by multiple diagnostic modules. Each diagnostic process may be performed independently.

[0331] Fig. 12 is a block diagram illustrating a diagnostic unit according to one embodiment.

[0332] Referring to FIG. 12, a diagnostic unit (200) according to one embodiment may include a diagnostic request acquisition module (211), a data processing module (231), a first diagnostic module (251), a second diagnostic module (253), and an output module (271). Unless otherwise specified, each module of the diagnostic unit (200) may operate similarly to the diagnostic module of the diagnostic unit illustrated in FIG. 10.

[0333] In Fig. 12, even when the diagnostic unit (200) includes multiple diagnostic modules, the diagnostic request acquisition module (211), the data processing module (231), and the output module (271) are illustrated as being common, but the configuration is not limited to this, and the diagnostic request acquisition module, the data processing module, and / or the output module may also be provided in multiple numbers. The multiple diagnostic request acquisition modules, the data processing modules, and / or the output modules may also operate in parallel.

[0334] For example, the diagnosis unit (200) includes a first data processing module that performs a first processing on an input diagnosis target image and a second processing module that performs a second data processing on the diagnosis target image, and the first diagnosis module can obtain first diagnosis information based on the first processed diagnosis target image, and the second diagnosis module can obtain second diagnosis information based on the second processed diagnosis target image. The first processing and / or the second processing may be any one selected from among image resizing, image color modulation, blur filter application, blood vessel emphasis processing, red-free conversion, cropping of a portion of the area, and extraction of a portion of the element.

[0335] Multiple diagnostic modules can obtain different diagnostic information. Multiple diagnostic modules can obtain diagnostic information using different diagnostic models. For example, a first diagnostic module can obtain first diagnostic information related to whether the subject has an ocular disease using a first diagnostic model that predicts whether the subject has an ocular disease, and a second diagnostic module can obtain second diagnostic information related to whether the subject has a systemic disease using a second diagnostic model that predicts whether the subject has a systemic disease.

[0336] As a more specific example, the first diagnostic module may obtain first diagnostic information regarding whether the subject has diabetic retinopathy using a first diagnostic model that predicts whether the subject has diabetic retinopathy based on a retinal image, and the second diagnostic module may obtain second diagnostic information regarding whether the subject has hypertension using a second diagnostic model that predicts whether the subject has hypertension based on a retinal image.

[0337]

[0338] Additionally, a diagnostic process according to one embodiment may include multiple sub-diagnostic processes. Each sub-diagnostic process may be performed using a different diagnostic model. Each sub-diagnostic process may be performed in a different diagnostic process. For example, a first diagnostic module may perform a first sub-diagnostic process that obtains first diagnostic information through a first diagnostic model. Alternatively, a second diagnostic module may perform a second sub-diagnostic process that obtains second diagnostic information through a second diagnostic model.

[0339] Multiple trained diagnostic models can input diagnostic target data and output predicted labels or probabilities. Each diagnostic model is implemented as a classifier and can classify the input diagnostic target data based on a given label. In this case, the multiple diagnostic models can be implemented as classifiers trained on different characteristics.

[0340] Meanwhile, a map can be obtained from each diagnostic model. The map can be obtained selectively. The map can be extracted when a predetermined condition is satisfied. For example, if the first diagnostic information indicates that the subject is abnormal for the first characteristic, the first map can be obtained from the first diagnostic model.

[0341]

[0342] Fig. 13 is a diagram for explaining a diagnostic process according to one embodiment.

[0343] Referring to FIG. 13, a diagnostic process according to one embodiment may include acquiring diagnostic target data (S51), and using a first diagnostic model and a second diagnostic model (S51a, S51b) to acquire diagnostic information according to the diagnostic target data (S53). The diagnostic target data may be processed data.

[0344] A diagnostic process according to one embodiment may include obtaining first diagnostic information through a learned first diagnostic model and obtaining second diagnostic information through a learned second diagnostic model. The first diagnostic model and the second diagnostic model may obtain first diagnostic information and second diagnostic information, respectively, based on the same diagnostic target data.

[0345] For example, the first diagnostic model and the second diagnostic model can obtain first diagnostic information regarding an eye disease of the subject and second diagnostic information regarding whether the subject has a cardiovascular disease, respectively, based on the retinal image to be diagnosed.

[0346] In addition, unless otherwise stated, the diagnostic process described with respect to FIG. 13 can be implemented similarly to the diagnostic process described above with respect to FIG. 11.

[0347]

[0348] 1.4.1.3. Output of diagnostic information

[0349] According to one embodiment, diagnostic information may be acquired through a parallel diagnostic process. The acquired diagnostic information may be stored in a diagnostic device, a server device, and / or a client device. The acquired diagnostic information may be transmitted to an external device.

[0350] The plurality of diagnostic information may each indicate multiple labels predicted by multiple diagnostic models. The plurality of diagnostic information may each correspond to multiple labels predicted by multiple diagnostic models. Alternatively, the diagnostic information may be information determined based on multiple labels predicted by multiple diagnostic models. The diagnostic information may correspond to multiple labels predicted by multiple diagnostic models.

[0351] In other words, the first diagnostic information may be diagnostic information corresponding to the first label predicted through the first diagnostic model. Alternatively, the first diagnostic information may be diagnostic information determined by considering both the first label predicted through the first diagnostic model and the second label predicted through the second diagnostic model.

[0352] Meanwhile, map images obtained from multiple diagnostic models may be output. The map images may be output when a predetermined condition is satisfied. For example, in either the case where the first diagnostic information indicates that the subject is abnormal for the first characteristic or the case where the second diagnostic information indicates that the subject is abnormal for the second characteristic, a map image obtained from a diagnostic model that outputs diagnostic information indicating that the subject is abnormal may be output.

[0353] Multiple diagnostic information and / or map images may be provided to the user. The multiple diagnostic information and / or the like may be provided to the user via an output means of the diagnostic device or client device.

[0354]

[0355] According to one embodiment, the diagnostic information corresponding to the diagnosis target image may include grade information. The grade information may be selected from a plurality of grades. The grade information may be determined based on a plurality of diagnostic information and / or finding information obtained through a diagnosis model. The grade information may be determined by considering suitability information or quality information of the diagnosis target image. The grade information may be determined by considering the classes into which the diagnosis target image is classified by a plurality of diagnosis models. The grade information may be determined by considering numerical values ​​output from a plurality of diagnosis models.

[0356] For example, the diagnostic information acquired in response to the diagnostic target image may include any one of the first grade information and the second grade information. The grade information may be selected as the first grade information when at least one abnormal finding information or abnormal diagnosis information is acquired from among the diagnostic information acquired through multiple diagnostic models. The grade information may be selected as the second grade information when no abnormal finding information or abnormal diagnosis information is acquired from among the diagnostic information acquired through the diagnostic model.

[0357] Grade information may be selected as first-grade information if at least one value obtained through the diagnostic model exceeds a reference value, or as second-grade information if all values ​​obtained fall below the reference value. Compared to second-grade information, first-grade information may indicate the presence of strong abnormality information in the diagnostic target image.

[0358] Grade information may be selected as third-grade information when the quality of the image to be diagnosed is determined to be below standard using image analysis or a diagnostic model. Alternatively, the diagnostic information may include third-grade information along with first- or second-grade information.

[0359]

[0360] Additionally, a diagnostic system according to one embodiment may include a retinal image acquisition unit, a first processing unit, a second processing unit, a third processing unit, and a diagnostic information output unit.

[0361] According to one embodiment, the diagnostic system may include a diagnostic device. The diagnostic device may include a retinal image acquisition unit, a first processing unit, a second processing unit, a third processing unit, and / or a diagnostic information output unit. However, the present invention is not limited thereto, and each unit included in the diagnostic system may be located at an appropriate location on the learning device, the diagnostic device, the learning diagnostic server, and / or the client device. For convenience, the following description will be made based on the case where the diagnostic device of the diagnostic system includes a retinal image acquisition unit, a first processing unit, a second processing unit, a third processing unit, and a diagnostic information output unit.

[0362] Fig. 14 is a diagram illustrating a diagnostic system according to one embodiment. Referring to Fig. 14, the diagnostic system includes a diagnostic device, and the diagnostic device may include a retinal image acquisition unit, a first processing unit, a second processing unit, a third processing unit, and a diagnostic information output unit.

[0363] According to one embodiment, a diagnostic system for assisting diagnosis of a plurality of diseases based on retinal images may include a retinal image acquisition unit for acquiring a target retinal image that serves as a basis for acquiring diagnostic information about a subject, a first processing unit for acquiring a first result about the subject using a first diagnostic model for the target retinal image, the first diagnostic model being machine-learned based on a first retinal image set, a second processing unit for acquiring a second result about the subject using a second diagnostic model for the target retinal image, the second diagnostic model being machine-learned based on a second retinal image set, at least a portion of which is different from the first retinal image set, a third processing unit for determining diagnostic information about the subject based on the first result and the second result, and a diagnostic information output unit for providing the determined diagnostic information to a user.

[0364] The third processing unit can consider the first result and the second result together to determine whether the diagnostic information according to the target retinal image is normal information or abnormal information.

[0365] The third processing unit can determine diagnostic information for the subject by giving priority to abnormal results to improve diagnostic accuracy.

[0366] The third processing unit can determine the diagnostic information as normal if the first result is normal and the second result is normal, and can determine the diagnostic information as abnormal if the first result is not normal or the second result is not normal.

[0367] The first and second results may be for the same disease or for different diseases.

[0368] At least one map related to the first / second result is obtained through the first processing unit and / or the first / second diagnostic model, and the diagnostic information output unit can output an image of at least one map.

[0369] The diagnostic information output unit can output an image of at least one map when the diagnostic information obtained by the third processing unit is abnormal diagnostic information.

[0370] The diagnostic system further includes a fourth processing unit that obtains quality information of a target retinal image, and the diagnostic information output unit can output quality information of the target retinal image obtained by the fourth processing unit.

[0371] If the fourth processing unit determines that the quality information of the target retinal image is below a predetermined quality level, the diagnostic information output unit can provide the user with information indicating that the quality information of the target retinal image is below a predetermined quality level together with the determined diagnostic information.

[0372]

[0373] 1.4.2.1. Configuration of the serial diagnostic system

[0374] According to one embodiment, a serial diagnostic system may be provided in which multiple diagnostic models are serially connected. Below, several examples of serial-type diagnostic models are described.

[0375]

[0376] FIG. 15 is a diagram illustrating a serial diagnostic model according to one embodiment. Referring to FIG. 15, the diagnostic model (1000) may include a first serial model (1100) and a second serial model (1200).

[0377] The first diagnostic model (1100) can obtain input data including a retinal image and obtain a first output (or intermediate output).

[0378] The first diagnostic model (1100) can obtain input data including retinal images and / or other medical diagnostic images and / or non-visual diagnostic data. The input data can include retinal images, OCT images, iris images, ophthalmoscopic images, lung CT images, lung CT images, cardiac CT images, lung X-ray images, cardiac X-ray images, kidney X-ray images, other tomography images, MRI images, or X-ray images. The input data can include data indicating the subject's age, height, gender, smoking status, family history, etc.

[0379] The first output may be a value obtained by the output layer of the first diagnostic model (1100). For example, the first diagnostic model (1100) may be a classifier model and the first output may include output values ​​from multiple nodes of the output layer of the first diagnostic model (1100). Also, for example, the first diagnostic model (1100) may be a regression model and the first output may include a numerical value obtained by the first diagnostic model (1100).

[0380] The first output may be a value provided by some layer of the first diagnostic model (1100). For example, the first output may be a value obtained based on a value of the output layer of the first diagnostic model (1100). Alternatively, the first output may be a value obtained based on a value of the hidden layer of the first diagnostic model (1100).

[0381] According to one embodiment, the first output may be a value obtained by an activation function in the output layer of the first diagnostic model (1100). When the output layer of the first diagnostic model (1100) includes a plurality of nodes (or neurons), the first output may include an output value according to each of the plurality of nodes or a value obtained through a predetermined function (e.g., summation) based on each output value.

[0382] The activation function can be any one of the following: sigmoid function, hyperbolic tangent function, ReLu (Rectified Linear Unit) function, PReLu, Leaky ReLU function, identity function, ELU (Exponential Linear Unit) function, and Maxout function.

[0383] The first output may be a feature map or feature value associated with the target disease. The first output may be a probability map, saliency map, heat map, etc. associated with the target disease. The second diagnostic model (1200) may be configured to obtain diagnostic information based on the feature map or feature value associated with the target disease.

[0384] The first output may be a probability phenotype associated with the target disease. For example, if the target disease is coronary artery disease and the diagnostic information is numerical information related to the target coronary artery disease, the first output may be the probability of the subject having the target coronary artery disease, obtained based on the eye image. The second diagnostic model (1200) may be configured to obtain diagnostic information about the target disease based on the probability phenotype associated with the target disease.

[0385] The second diagnostic model (1200) can obtain a second output (or diagnostic information) based on the first output. The second diagnostic model (1200) may be a diagnostic model (1000) trained to obtain a second output using the first output as input. The second output may be various forms of diagnostic information described herein.

[0386]

[0387] FIG. 16 is a diagram illustrating a serial diagnostic model according to another embodiment. Referring to FIG. 16, the diagnostic model (1000) may include a first diagnostic model (1100) and a second diagnostic model (1200).

[0388] The first diagnostic model (1100) can acquire input data including an eye image and obtain a first output (intermediate output). The second diagnostic model (1200) can obtain diagnostic information based on the first output and the second input.

[0389] The second input may be the same input data as the first input. For example, the first diagnostic model (1100) may obtain a first output based on an eye image, and the second diagnostic model (1200) may obtain diagnostic information based on the first output and the eye image.

[0390] The second input may be input data obtained based on the first input. For example, the second input may be input data obtained by performing image processing on an eye image. For example, the second input may be a black-and-white processed eye image, an eye image with blood vessels highlighted, an image of blood vessels extracted from an eye image, or an eye image with blood vessels removed.

[0391] The second input may be input data that is at least partially different from the first input.

[0392] The second input may be image data different from the first input. The second input may include a retinal image, an OCT image, an iris image, an ophthalmoscopic image, a lung CT image, a lung CT image, a cardiac CT image, a lung X-ray image, a cardiac X-ray image, a kidney X-ray image, other tomography images, an MRI image, or an X-ray image.

[0393] The second input may include non-visual information about the subject. For example, the first diagnostic model (1100) may obtain a first output about the target disease based on an eye image, and the second diagnostic model (1200) may obtain diagnostic information based on the first output and the second input (for example, at least one of the subject's physical information (height, weight, age, gender, race, smoking status, blood pressure (e.g., blood pressure level, high blood pressure), diabetes (or blood sugar level), and cholesterol level).

[0394]

[0395] FIG. 17 is a diagram illustrating a serial diagnostic model according to another embodiment. Referring to FIG. 17, the diagnostic model (1000) may include a first diagnostic model (1100) and a second diagnostic model (1200).

[0396] Compared to FIG. 16, the first diagnostic information obtained by the first diagnostic model (1100) can be further acquired by the diagnostic model (1000). The diagnostic model (1000) can acquire intermediate diagnostic information and secondary diagnostic information acquired based on the intermediate diagnostic information.

[0397] The diagnostic model (1000) can obtain first diagnostic information and second diagnostic information. The first diagnostic model (1100) can obtain input data including an eye image and obtain a first output (first diagnostic information or an intermediate output). The diagnostic model (1000) can obtain the first diagnostic information based on the first output. The second diagnostic model (1200) can obtain the second diagnostic information based at least in part on the first output. The diagnostic model (1000) can obtain the second diagnostic information based at least in part on the first output and by taking into consideration other information extracted from the eye image.

[0398] The first diagnostic information and the second diagnostic information may be diagnostic information for the same target disease. The first diagnostic information may be diagnostic information obtained (clinically or through a machine-learned model) based on an eye image, such as a probability expression for the presence or absence of an ocular disease, vascular abnormality, or cardiovascular disease.

[0399] The second diagnostic information may provide more detailed diagnostic information than the first diagnostic information. For example, the second diagnostic information may include, for the same target disease as the first diagnostic information, grade information indicating the risk level for the target disease or score information indicating a score associated with the target disease.

[0400] The second diagnostic information may be associated with the first diagnostic information, but may include diagnostic information that can be obtained by considering information other than the image, such as the rate of disease progression, guide information related to the diagnostic information, etc.

[0401] Meanwhile, the first diagnostic information and the second diagnostic information may be diagnostic information for different diseases. The first diagnostic information and the second diagnostic information may be diagnostic information for different diseases belonging to the same group. For example, the first diagnostic information may be diagnostic information related to glaucoma, which belongs to the eye disease group, and the second diagnostic information may be diagnostic information related to macular degeneration, which belongs to the eye disease group. For example, the first diagnostic information may be diagnostic information related to drusen, which belongs to the eye disease group, and the second diagnostic information may be diagnostic information related to diabetic retinopathy, which belongs to the eye disease group.

[0402] The first and second diagnostic information may be diagnostic information for different diseases belonging to different groups. For example, the first diagnostic information may be diagnostic information related to macular degeneration or drusen, which fall within the eye disease group, while the second diagnostic information may be diagnostic information related to hyperlipidemia, which falls within the cardiovascular disease group.

[0403]

[0404] In the above embodiments, the diagnosis model (1000) is described based on the case where the diagnosis model (1000) has a first diagnosis model (1100) and a second diagnosis model (1200), but the diagnosis model (1000) may include a greater number of diagnosis models. In addition, each diagnosis model may be connected through the aforementioned parallel connection or serial connection.

[0405]

[0406] 1.4.2.2. Diagnosis through serial diagnostic model

[0407] According to one embodiment of the invention described herein, a diagnostic method using a diagnostic model including serially connected sub-models can be provided.

[0408] Fig. 18 is a drawing for explaining a diagnostic method using a diagnostic model according to one embodiment.

[0409] Referring to FIG. 18, a diagnostic method according to one embodiment may include a step of acquiring input data (S61), a step of acquiring first diagnostic information (S62), and a step of acquiring second diagnostic information (S63).

[0410] The step of acquiring input data (S61) may include acquiring a retinal image of the subject. The step of acquiring input data (S61) may further include acquiring a medical image of a body part other than the eye of the subject. The step of acquiring input data (S61) may further include acquiring non-visual information about the subject (e.g., body information of the subject). The step of acquiring input data (S61) may further include performing preprocessing necessary for acquiring diagnostic information on the eye image of the subject.

[0411] Step (S62) of obtaining first diagnostic information may include obtaining first diagnostic information about the subject based on the eye image and through the first diagnostic model. Obtaining the first diagnostic information may include obtaining diagnostic information related to the first disease. For example, obtaining the first diagnostic information may include obtaining first diagnostic information indicating a probability that the subject's coronary artery calcium score is 0 or greater based on the eye image.

[0412] The step of obtaining second diagnostic information (S63) may include obtaining second diagnostic information about the subject based on the first diagnostic information and through the second diagnostic model.

[0413] Obtaining the second diagnostic information may include obtaining diagnostic information related to the first disease and different from the first diagnostic information. For example, obtaining the first diagnostic information may include obtaining first diagnostic information indicating a probability that the subject's coronary artery calcium score is 0 or greater, and obtaining the second diagnostic information may include obtaining second diagnostic information indicating a score related to whether the subject has the target cardiovascular disease (e.g., a probability of occurrence of a cardiovascular disease-related event within 10 years) based on the first diagnostic information of the subject.

[0414] Alternatively, obtaining the second diagnostic information may include obtaining diagnostic information related to a second disease that is different from the first disease. For example, obtaining the first diagnostic information may include obtaining first diagnostic information indicating whether the subject has the target ocular disease, and obtaining the second diagnostic information may include obtaining second diagnostic information indicating whether the subject has a cerebrovascular disease.

[0415]

[0416] The diagnosis using the diagnostic model described above will be described in detail below with specific examples.

[0417]

[0418] 2. Diagnosis of cardiovascular disease

[0419] 2.1.1. Biomarkers for Cardiovascular Disease

[0420] In one embodiment, a cardiovascular disease biomarker may be used in a cardiovascular disease diagnosis method to predict, measure, or confirm the presence or progression of cardiovascular disease. In the present disclosure, the cardiovascular disease biomarker may include a cardiovascular disease risk assessment tool.

[0421] For example, cardiovascular disease biomarkers may include the Coronary Artery Calcium (CAC) score, the Pooled Cohort Equation (PCE) score, the QRISK score, the modified Framingham Score (FRS), the Carotid Intima-Media Thickness (CIMT) score, and the Brachial-Ankle Pulse Wave Velocity (baPWV) score.

[0422] The coronary artery calcium score (or cardiac calcification index) can be used as an indicator of coronary artery calcification. As plaque accumulates within the blood vessels, the coronary arteries become calcified, narrowing the walls of the heart vessels and causing various heart diseases such as coronary artery disease, myocardial infarction, angina, and ischemic heart disease. Therefore, the coronary artery calcium index can be used as the basis for assessing the risk of various heart diseases. For example, a high coronary artery calcium score may be judged to indicate a high risk of coronary artery disease. In particular, the coronary artery calcium score is directly related to heart disease, particularly coronary artery disease (cardiac calcification), compared to factors indirectly related to heart disease, such as smoking status, age, and gender. Therefore, it can be used as a powerful biomarker for heart health. Based on the coronary artery calcium score, the risk of cardiovascular disease can be classified as very low risk (coronary artery calcium score 0), somewhat increased risk (1–99), intermediate risk (100–299), and high risk (300 or more).

[0423] The PCE score is an indicator used to estimate an individual's 10-year risk of developing atherosclerotic cardiovascular disease (ASCVD) and is used in guidelines as a basis for prescribing antihypertensive medications such as statins. The PCE score can be primarily utilized as a cardiovascular disease biomarker in the United States. For example, based on the PCE score, cardiovascular disease risk can be categorized as low risk (PCE score 0%-5%), borderline risk (5%-7.4%), intermediate risk (7.5%-19.9%), and high risk (20% or higher).

[0424] The QRISK score can be used as an indicator to estimate an individual's 10-year risk of developing cardiovascular disease. QRISK is a cardiovascular disease biomarker primarily used in the UK and has several versions. For convenience, this document describes QRISK 3. QRISK 3 is a formal risk assessment tool developed using data from 1.28 million individuals. Current UK guidelines recommend administering stage 1 antihypertensive medication for a QRISK 3 score of 10%. For example, using the QRISK 3 score, the risk of cardiovascular disease can be categorized into low risk (QRISK 3 score 0-10%), intermediate risk (10-20%), and high risk (20% or more).

[0425] The modified Framingham Risk Score (RRS) can be used to estimate an individual's 10-year risk of developing cardiovascular disease. The RRS is primarily used in Singapore. For example, the RRS can be used to categorize cardiovascular disease risk into low risk (QRISK3 score 0-10%), intermediate risk (10-20%), and high risk (20% or more). These risk groups guide various treatment guidelines in Singapore, particularly in prescribing recommended LDL cholesterol-lowering medications.

[0426] The carotid intima-media thickness (CIMT) score is an ultrasound measurement of the thickness between the inner and middle layers of the carotid artery and can be used as a biomarker for cardiovascular disease. For example, the CIMT score can be used to categorize cardiovascular disease risk into low risk (carotid intima-media thickness score <0.5 mm), borderline risk (0.5 to 0.75 mm), and high risk (≥0.75 mm).

[0427] The brachial-ankle pulse wave velocity (baPWV) score measures the speed at which pulse waves travel between the brachial and ankle joints and can be used as a biomarker for cardiovascular disease. For example, a baPWV score of 1,400 cm / s or higher may indicate a high risk of cardiovascular disease.

[0428] In addition, various cardiovascular disease biomarkers can be used in cardiovascular disease diagnosis methods.

[0429]

[0430] 2.1.2. Diagnosis methods for cardiovascular disease

[0431] A cardiovascular disease diagnosis method according to one embodiment of the present disclosure may be performed using at least one of the aforementioned diagnostic models, parallel diagnostic models, or serial diagnostic models. For convenience of explanation, the following description focuses on performing the cardiovascular disease diagnosis method using a serial diagnostic model.

[0432]

[0433] FIG. 19 is a diagram for explaining a method for diagnosing cardiovascular disease according to one embodiment.

[0434] Referring to FIG. 19, the processor of the diagnostic device may include a step of acquiring a retinal image (S100) and a step of acquiring cardiovascular disease diagnosis information (S200).

[0435] In step S100, the processor of the diagnostic device may acquire a retinal image. Furthermore, depending on the embodiment, the processor of the diagnostic device may perform preprocessing, augmentation, serialization, etc. on the acquired retinal image. Since the above description applies to this, a detailed description will be omitted.

[0436] Additionally, in step S200, the processor of the diagnostic device can obtain cardiovascular disease diagnosis information. In this specification, the cardiovascular disease diagnosis information can be expressed as Reti-CVD. A method for diagnosing cardiovascular disease is described using FIG. 20.

[0437]

[0438] FIG. 20 illustrates a diagnostic model for obtaining cardiovascular disease diagnostic information according to one embodiment.

[0439] Referring to FIG. 20, the diagnostic model (1000) may be included in the processor and / or storage module of the diagnostic device. Furthermore, the description of the diagnostic models of FIGS. 15 to 18 may be applied to the diagnostic model (1000).

[0440] The diagnostic model (1000) may include a first diagnostic model (1100) and a second diagnostic model (1200). The first diagnostic model (1100) and the second diagnostic model (1200) may be machine learning models. Furthermore, the first diagnostic model (1100) and the second diagnostic model (1200) may be models based on the same algorithm, or may be models based on different algorithms. For example, the first diagnostic model (1100) and the second diagnostic model (1200) may be neural network models. Furthermore, the first diagnostic model (1100) may be a neural network model, and the second diagnostic model (1200) may be a machine learning model other than a neural network model. In the following, for convenience of explanation, the first diagnostic model (1100) is described as a neural network model, and the second diagnostic model (1200) is described as a machine learning model rather than a neural network model. However, the explanation in this specification is not limited thereto.

[0441]

[0442] The processor of the diagnostic device can input a retinal image (or a preprocessed retinal image, a serialized retinal image) into the first diagnostic model (1100). Then, the processor of the diagnostic device can obtain a probability value and / or a grade for the probability that the subject of the retinal image has a coronary artery calcium score of 0 or higher from the first diagnostic model (1100). Here, the probability value can be a number between 0 and 1.

[0443] Specifically, the first diagnostic model (1100) may be a CNN-structured neural network model. In one embodiment, the first diagnostic model (1100) may consider more areas in judgment than a general CNN-structured neural network model by using a 7x7-sized kernel. In addition, the first diagnostic model (1100) may calculate probability values ​​for three cutoff values ​​(coronary artery calcium score of 0 or more, coronary artery calcium score of 100 or more, coronary artery calcium score of 300 or more) and output a probability value for the probability that the coronary artery calcium score is 0 or more based on the calculated probability values.

[0444] In addition, the first diagnostic model (1100) can be trained using retinal images labeled with coronary artery calcium scores. The first diagnostic model (1100) can be trained based on not only retinal images labeled with a coronary artery calcium score of 0, but also all retinal images labeled with a coronary artery calcium score greater than 0. For example, the first diagnostic model (1100) can be trained on retinal images remaining in the retinal image data set labeled with coronary artery calcium scores, excluding duplicate retinal images, retinal images without a retinal image of one eye, and retinal images with low quality. In addition, according to an embodiment, retinal images of subjects in a specific age group (e.g., less than 40 years old and over 70 years old) can be excluded from the training of the first diagnostic model (1100).

[0445] Additionally, in one embodiment, the training target retinal images may be divided into a development set and an internal test set. For example, the development set and the internal test set may be randomly distributed in an 8:2 ratio. Furthermore, to prevent overfitting, the training target retinal images may be classified into the development set or the internal test set for each subject, such that retinal images from the same subject are not overlapped in the development set and the internal test set.

[0446] In addition, in one embodiment, the first diagnostic model (1100) can be trained using the Adam Optimizer with a learning rate of 2e-4, a cosine learning rate, and a schedule of 25 epochs. In addition, at least one of mixup, cutmix, randomization, an enhancing contrast module, and random cropping can be used to augment data of the training target retinal image. In addition, focal loss and exponential moving average can be used for training the first diagnostic model (1100), and the size of the training target retinal image can be set to 384x384. Additionally, to address the difficult labeling problem, for example, a sigmoid function may be used in the training of the first diagnostic model (1100) so that a coronary artery calcium score of 299 is not classified as 300, and soft labels may be used in the training of the coronary artery calcium score cutoffs of 100 and 300.

[0447] In addition, the processor of the diagnostic device can input the output value of the first diagnostic model (1100) and the subject's physical information (at least one of age, gender, race, smoking status, blood pressure (e.g., blood pressure level, high blood pressure), diabetes (or blood sugar level), and cholesterol level) into the second diagnostic model (1200). In addition, the processor of the diagnostic device can obtain a probability value and / or a grade for the probability of occurrence of an event related to cardiovascular disease of the subject within 10 years from the second diagnostic model (1200). Here, the event related to cardiovascular disease within 10 years may include death due to cardiovascular disease, occurrence of cardiovascular disease, and occurrence of various events (hospitalization, procedure, surgery, death, etc.) caused by cardiovascular disease within 10 years from the time of capturing the retinal image. Additionally, in some cases, the processor of the diagnostic device may obtain a probability value for the probability of occurrence of a cardiovascular disease-related event of the subject within 10 years from the second diagnostic model (1200), and apply a predetermined cutoff value to the obtained probability value to obtain a grade corresponding to the obtained probability value.

[0448] The processor of the diagnostic device can output, as cardiovascular disease diagnostic information, a probability value (score) and / or a grade for the probability of occurrence of a cardiovascular disease-related event of the subject within 10 years.

[0449] In addition, according to an embodiment, the processor of the diagnostic device may obtain a probability value and / or a grade for the current risk of cardiovascular disease from the second diagnostic model (1200). In addition, according to an embodiment, the processor of the diagnostic device may obtain a probability value and / or a grade for the probability of occurrence of a cardiovascular disease-related event of the subject within 10 years from the second diagnostic model (1200), and obtain a probability value and / or a grade for the current risk of cardiovascular disease based on the obtained probability value and / or grade for the probability of occurrence of a cardiovascular disease-related event of the subject within 10 years. For example, the processor of the diagnostic device may compare the probability value and / or grade for the probability of occurrence of a cardiovascular disease-related event of the subject within 10 years obtained from the second diagnostic model (1200) with at least one or more predetermined reference probability values ​​and / or grades, and obtain a probability value and / or a grade for the current risk of cardiovascular disease based on the comparison result.

[0450] In addition, although the present specification focuses on cardiovascular disease-related events within 10 years, it is not limited thereto, and it is obvious that, depending on the embodiment, cardiovascular disease-related events of various periods, such as within 3 years or within 5 years, can be applied to the description of the present specification.

[0451]

[0452] Additionally, in one embodiment, the second diagnostic model (1200) may be trained based on training target data. Here, training of the second diagnostic model (1200) may include the meaning of fitting the second diagnostic model (1200). Additionally, the second diagnostic model (1200) may be constructed using a regression-based Cox proportional hazards model.

[0453] For example, the training target data may include physical information of the subjects of the training target retinal image of the first diagnostic model (1100), the follow-up observation results of cardiovascular disease events of the subjects for 10 years, and the probability value of the first diagnostic model (1100) for the training target retinal image. In addition, according to an embodiment, the subjects of the training target retinal image of the first diagnostic model (1100) and the subjects of the training target data of the second diagnostic model (1200) may be the same or at least partially different. For example, among the training target data of the subjects of the training target retinal image of the first diagnostic model (1100), the training target data of the subjects who have already had a cardiovascular disease event or are suffering from cardiovascular disease at a reference point in time may be excluded from the training of the second diagnostic model (1200). This means that the accuracy of the probability of occurrence of a cardiovascular disease event within 10 years output from the second diagnostic model (1200) can be increased by training the second diagnostic model (1200) using training target data on normal people who do not have cardiovascular disease and do not suffer from cardiovascular disease.

[0454] And, through training of the second diagnostic model (1200), HR = - exp{β'} representing the hazard ratio can be estimated. Here, β' can represent a regression coefficient. And, through exponential transformation of the regression coefficient (β') and the reference cumulative hazard function A0(t), the following mathematical expression 1 is derived, and the second diagnostic model (1200) can obtain a probability value for the occurrence probability of a cardiovascular disease-related event of the subject within 10 years based on the following mathematical expression 1. For example, the Cox proportional hazards model of the second diagnostic model (1200) can be configured based on the mathematical expression 1.

[0455]

[0456] [Mathematical Formula 1]

[0457] CVD risk(10|S) = 1 - exp{-A0(10)exp{β' S x S + β' Age x Age + β' Gender x I(Male)}}

[0458]

[0459] Here, CVD risk(10|S) represents a probability value for the occurrence probability of a cardiovascular disease-related event of the subject within 10 years, A0(10) represents a standard cumulative function based on 10 years, S represents a probability value output from the first diagnostic model (1100) for the retinal image of the subject when the retinal image of the subject is input to the first diagnostic model (1100), AGE represents the age of the subject, and I(Male) represents the gender of the subject. For example, I(Male) may be an indicator function that reflects a value of 1 when the subject is male and a value of 0 when the subject is female.

[0460] Also, β' S represents the regression coefficient for the output value output from the first diagnostic model (1100), and β' Age represents the regression coefficient for age, and β' gender can represent regression coefficients for gender. Through training of the second diagnostic model (1200), the values ​​of the regression coefficients can be adjusted.

[0461] In addition, the processor of the diagnostic device can obtain a probability value for the occurrence probability of a cardiovascular disease-related event within 10 years from the second diagnostic model (1200), and obtain a grade corresponding to the probability value by applying a predetermined cutoff value to the probability value. Of course, in some cases, a predetermined cutoff value may be applied in advance to the second diagnostic model (1200) to obtain a grade corresponding to the probability value from the second diagnostic model (1200). At this time, the cutoff value may be one or more. For example, when the cutoff value is one, there may be two grades corresponding to the probability value, and when the cutoff value is two, there may be three grades corresponding to the probability value. The description in this specification can be applied to various numbers of cutoff values ​​and various cutoff values.

[0462] In addition, the processor of the diagnostic device can present the risk of cardiovascular disease in the 10th or 5th year based on the above mathematical formula 1 (or by applying the mathematical formula 1), and can present the risk of cardiovascular disease for various groups such as different genders and different races based on the mathematical formula 1 (or by applying the mathematical formula 1).

[0463] In one embodiment, the processor of the diagnostic device may set a predetermined cutoff value based on the population distribution (i.e., the incidence rate in each group) of each group divided into multiple grades according to the follow-up observation results of a predetermined population. For example, the cutoff value may be determined based on the results of follow-up observation for a predetermined population. For example, if the follow-up observation results of the entire follow-up subjects are obtained, 0 to n1% are low-risk group, n1 to n2% are intermediate-risk group, and n2 to 100% are high-risk group, n1% and n2% may be set as cutoff values, respectively. In addition, even if n1%, n2%, and n3% are not cutoff values, the cutoff values ​​may be set so that the population distribution in each risk group of cardiovascular disease diagnosis information becomes similar to the follow-up observation results of the predetermined population.

[0464] In addition, the processor of the diagnostic device can set a cutoff value based on the incidence rate in the cardiovascular disease biomarker. For example, in the case of PCE, the risk of cardiovascular disease can be classified into low risk (PCE score 0% to 5%), borderline risk (5% to 7.4%), intermediate risk (7.5% to 19.9%), and high risk (20% or more) based on the PCE score. In addition, if the incidence rate in PCE, that is, people corresponding to low risk, borderline risk, intermediate risk, and high risk, is x1%, x2%, x3%, and x4%, respectively, the cutoff value can be set so that the proportion of people included in each grade of the cardiovascular disease diagnosis information output from the processor of the diagnostic device is adjusted to x1%, x2%, x3%, and x4%. This can be applied not only to PCE but also to biomarkers such as QRISK3 and the modified Framingham risk score. Additionally, the cutoff value may be an optimized value to best detect the group corresponding to high risk for each biomarker.

[0465] As a specific example, the processor of the diagnostic device can set the cutoff value using the ratio of high-risk group and low-risk group classified by the 10-year cardiovascular risk calculator according to the 10-year cardiovascular risk calculation method of the applied clinical guideline or race. Accordingly, the performance of predicting the high-risk group of cardiovascular disease diagnostic information can be improved. For example, in order to maximize the performance of cardiovascular disease diagnostic information that diagnoses the high-risk group (intermediate & high) in PCE, the cutoff value of the cardiovascular disease diagnostic information can be set so that the ratio of subjects corresponding to the low-risk group of PCE (e.g., 53.1%) and the ratio of subjects corresponding to the high-risk group (e.g., 46.9%) of the total examinees corresponding to the low-risk group of the cardiovascular disease diagnostic information and the ratio of the total examinees corresponding to the high-risk group are similar. For example, the processor of the diagnostic device may set the cutoff value of the cardiovascular disease diagnostic information so that the proportion of all examinees in the low-risk group and the proportion of all examinees in the high-risk group of the cardiovascular disease diagnostic information fall within a predetermined range of the proportion of subjects in the low-risk group of PCE (e.g., 53.1%) and the proportion of subjects in the high-risk group (e.g., 46.9%). In addition, the processor of the diagnostic device may set the cutoff value of the cardiovascular disease diagnostic information so that the proportion of all examinees in the low-risk group and the proportion of all examinees in the high-risk group of the cardiovascular disease diagnostic information fall within a predetermined range of the proportion of subjects whose QRISK3 score is less than 10% (e.g., 75.3%) and the proportion of subjects whose QRISK3 score is 10% or more (e.g., 24.7%). In this case, the processor of the diagnostic device may accurately determine subjects whose QRISK3 score is 10% or more.

[0466]

[0467] In this way, while the biomarker of the coronary artery calcium score is used for training the first diagnostic model (1100), a biomarker different from the coronary artery calcium score, such as PCE, QRISK3, or modified Framingham risk score, may be used for setting the cutoff value of the second diagnostic model (1200).

[0468] However, the present invention is not limited thereto, and the coronary artery calcium score may be used to set the cutoff value of the second diagnostic model (1200). For example, in the case of the coronary artery calcium score, the risk of cardiovascular disease may be classified into low risk (coronary artery calcium score 0), intermediate risk (coronary artery calcium score greater than 0 and less than or equal to 100), and high risk (coronary artery calcium score 100 or more) based on the coronary artery calcium score. In addition, when the incidence rate in the coronary artery calcium score, that is, the number of people corresponding to low risk, intermediate risk, and high risk, is y1%, y2%, and y3%, respectively, the cutoff value may be set so that the proportion of people included in each grade of the cardiovascular disease diagnosis information output from the processor of the diagnostic device is adjusted to y1%, y2%, and y3%. For example, in the above case, the processor of the diagnostic device may set the cutoff value of the cardiovascular disease diagnostic information so that the proportion of all subjects corresponding to the low-risk group, the proportion of all subjects corresponding to the intermediate-risk group, and the proportion of all subjects corresponding to the high-risk group of the cardiovascular disease diagnostic information are within a predetermined range of the proportion of subjects corresponding to the low-risk group of the coronary artery calcium score (e.g., y1%), the proportion of subjects corresponding to the intermediate-risk group (e.g., y2%), and the proportion of subjects corresponding to the high-risk group (e.g., y3%).

[0469] Furthermore, in one embodiment, the processor of the diagnostic device may set a cutoff value of the cardiovascular disease diagnosis information based on a specific proportion of subjects according to the coronary artery calcium score in a given population. For example, the processor of the diagnostic device may output a first grade and a second grade as the cardiovascular disease diagnosis information, and according to a specific proportion (e.g., a1%) of a distribution of subjects from 0 to 100% according to the coronary artery calcium score, the processor of the diagnostic device may set the cutoff value of the cardiovascular disease diagnosis information such that the proportion of all subjects corresponding to the first grade of the cardiovascular disease diagnosis information and the proportion of all subjects corresponding to the second grade are aligned with a% and 100%-a% (e.g., such that the proportion of all subjects corresponding to the first grade of the cardiovascular disease diagnosis information and the proportion of all subjects corresponding to the second grade are within a specific range of the ratio a% to 100%-a%).

[0470]

[0471] In addition, when a cutoff value is set based on each biomarker, the grade output as cardiovascular disease diagnosis information from the processor of the diagnostic device may be similar to the actual result of the biomarker for which the cutoff value is set. For example, when the cutoff value according to the biomarker of PCE is applied to the probability value for the occurrence probability of a cardiovascular disease-related event of the subject within 10 years output from the second diagnostic model (1200), and the grade of the subject is output from the processor of the diagnostic device, the grade of the subject output from the processor of the diagnostic device may match the grade determined when the subject actually performs a test according to PCE. When an actual test is performed according to PCE, QRISK3, or the modified Framingham risk score, inconveniences such as blood draws may occur for the subject. However, in the case of the cardiovascular disease diagnosis method according to the present specification, highly accurate cardiovascular disease diagnosis information is obtained using only the retinal image of the subject, so that user convenience can be improved.

[0472] In addition, in one embodiment, when cardiovascular disease diagnosis information is applied to the score and / or grade of an existing biomarker, the incidence of cardiovascular disease-related events according to the score and / or grade of the existing biomarker may be categorized or stratified according to each group (e.g., low-risk group, intermediate-risk group, and high-risk group) of the cardiovascular disease diagnosis information. This will be described in detail in the description of FIG. 24. In addition, the cutoff of the cardiovascular disease diagnosis information may be set so that, when cardiovascular disease diagnosis information is applied to the score and / or grade of an existing biomarker, the incidence of cardiovascular disease-related events according to the score and / or grade of the existing biomarker is clearly stratified according to each group of the cardiovascular disease diagnosis information.

[0473]

[0474] Hereinafter, an embodiment of a method for diagnosing cardiovascular disease according to the present specification is described in detail.

[0475]

[0476] 2.1.3.1. Example 1

[0477] Example 1 describes the experimental results of a method for diagnosing cardiovascular disease according to the present disclosure using clinical data and retinal images from the UK Biobank. The UK Biobank is a prospective cohort in the United Kingdom.

[0478] In the experimental subject data, clinical data and retinal images of the UK Biobank that are overlapping, retinal images of low quality, patients with type 1 diabetes, those who already had cardiovascular disease at baseline (specifically, those who had heart disease, other heart diseases, stroke, transient ischemic attack, peripheral arterial disease, and those who underwent cardiovascular surgery, and those who underwent cardiovascular procedures), and those who were under 40 years of age may be excluded from the experiment of Example 1. Accordingly, the experimental subject data may be retinal images and data of 48,260 subjects.

[0479] Additionally, the subjects of the experimental data can be divided into three groups. The first group is those not taking statins (45,473 people), the second group is stage 1 hypertensive patients not taking antihypertensive medication (11,966 people), and the third group is middle-aged people aged 40-64 years at baseline (38,941 people).

[0480] One of the goals of cardiovascular disease diagnostic methods is to achieve primary prevention of cardiovascular disease, and the experimental data may consist of data from normal individuals (those who do not have cardiovascular disease) who are likely to be unaware of risk factors related to cardiovascular disease.

[0481]

[0482] In one embodiment, a QRISK3 score can be obtained for each subject in the experimental data. Furthermore, based on their QRISK3 scores, subjects can be categorized into five groups (0-5%, 5-10%, 10-15%, 15-20%, and ≥20%). Furthermore, since the recommended threshold for statins and antihypertensive medications according to guidelines is a QRISK3 score of 10%, a group with a QRISK3 score of 7.5-10% can be additionally designated as a borderline risk group.

[0483] In addition, the processor of the control device can input the experimental subject data into the diagnostic model (1000). That is, the processor of the control device can input the retinal data of the subjects into the first diagnostic model (1100) to obtain a probability value for the probability that the subject's coronary artery calcium score is 0 or higher from the first diagnostic model (1100), and input the probability value and the subject's physical information (e.g., age, gender) into the second diagnostic model (1200) to obtain a probability value for the probability of occurrence of a cardiovascular disease-related event for each subject within 10 years. In addition, the processor of the control device can classify the probability value using a predetermined cutoff value to classify the subjects into a predetermined grade. In Example 1, the cutoff value can be set based on 40% and 95% based on the population distribution of the score of the cardiovascular disease diagnostic information so as to maximize the classification of the risk of occurrence of cardiovascular disease and to enable application of the cardiovascular disease diagnostic information compared to existing guidelines. As a result, a probability value of 40% for the probability of occurrence of a cardiovascular disease-related event within 10 years may be set as a first cutoff value so that the proportion of people corresponding to the low-risk grade of cardiovascular disease diagnosis information is similar to the proportion of people corresponding to the QRISK3 score of 0 to 5% (i.e., the incidence rate corresponding to the QRISK3 score of 0 to 5%). In addition, a probability value of 95% for the probability of occurrence of a cardiovascular disease-related event within 10 years may be set as a second cutoff value so that the proportion of people corresponding to the medium-risk grade of cardiovascular disease diagnosis information is similar to the proportion of people corresponding to the QRISK3 score of 5 to 10% (i.e., the incidence rate corresponding to the QRISK3 score of 5 to 10%).

[0484]

[0485] Below, the results according to Example 1 are described. Below, the grade of cardiovascular disease diagnosis information may represent a grade corresponding to the score of cardiovascular disease diagnosis information.

[0486] Figure 21 shows the clinical characteristics of subjects according to cardiovascular disease diagnosis information according to Example 1.

[0487] Referring to Figure 21, among the 48,260 subjects, the median 10-year cardiovascular disease risk of QRISK3 was 4.5% (IQR 2.2-8.0%, SD, 4.2%), and cardiovascular disease-related events occurred in 2,766 subjects (5.7%) during the follow-up period (up to 11.4 years). The Spearman's rank correlation coefficient between the probability value of cardiovascular disease diagnosis information and the QRISK3 score may be 0.50 (p < 0.001).

[0488] The incidence of cardiovascular disease-related events was 2.8% (545 / 19,304) in the low-risk group of cardiovascular disease diagnosis information, 7.2% (1,900 / 26,543) in the medium-risk group, and 13.3% (321 / 2413) in the high-risk group. Among the low-risk group of cardiovascular disease diagnosis information, 84.3% had a QRISK3 score of 0 to less than 5%, and among the high-risk group of cardiovascular disease diagnosis information, 8.6% had a QRISK3 score of 0 to less than 5%, 40.1% had a QRISK3 score of 5% to less than 10%, and 36.3% had a QRISK3 score of 10% to less than 15%.

[0489]

[0490] Figures 22a and 22b are diagrams for explaining cardiovascular disease diagnosis information according to Example 1 and the incidence of cardiovascular disease-related events according to QRISK3.

[0491] Referring to FIGS. 22a and 22b, in the graphs of FIGS. 22a and 22b, the x-axis represents time (years), and the y-axis represents the incidence of cardiovascular disease-related events (expressed as the incidence of cardiovascular disease events in FIGS. 22a and 22b).

[0492] And, the graph of Fig. 22a shows the incidence rate of cardiovascular disease-related events according to the QRISK3 score, and the graph of Fig. 22b shows the incidence rate of cardiovascular disease-related events according to cardiovascular disease diagnosis information. Specifically, the graph of Fig. 22a shows a Kaplan-Meier curve showing the results of performing Kaplan-Meier survival analysis on five groups of subjects according to five grades of QRISK3 scores, and the graph of Fig. 22b shows a Kaplan-Meier curve showing the results of performing Kaplan-Meier survival analysis on three groups of subjects according to three grades of cardiovascular disease diagnosis information.

[0493] As shown in the graph in Figure 22a, QRISK3 can well distinguish the risk of cardiovascular disease in the general population of the UK Biobank. Furthermore, as shown in the graph in Figure 22b, cardiovascular disease diagnosis information can also clearly distinguish the risk of cardiovascular disease based on three groups within the general population of the UK Biobank. Based on cardiovascular disease diagnosis information, the probability of occurrence of a cardiovascular disease-related event per 1,000 person-years was 2.6 (95% confidence interval, 2.4-2.8) in the low-risk group, 6.8 (95% confidence interval, 6.5-7.1) in the medium-risk group, and 13.1 (95% confidence interval, 11.7-14.6) in the high-risk group, indicating that the probability of occurrence of a cardiovascular disease-related event within 10 years in the high-risk group of cardiovascular disease diagnosis information is 13.1%. That is, since the cutoff values ​​are set at 40% and 95% based on the population distribution of the score for cardiovascular disease diagnosis information, the incidence rates of each group of cardiovascular disease diagnosis information and each group in QRISK3 may become similar. In other words, the difference between the incidence rates of each group of cardiovascular disease diagnosis information and each group in QRISK3 may fall below a predetermined threshold. In addition, a similar tendency may appear in people who are not taking statins and in stage 1 hypertensive patients who are not taking antihypertensive medication.

[0494]

[0495] Figure 23 is a diagram for explaining the performance of predicting the occurrence of cardiovascular disease-related events within 10 years using cardiovascular disease diagnosis information according to Example 1.

[0496] Referring to Figure 23, the table in Figure 23 may represent the results of applying cardiovascular disease diagnosis information to a subgroup with a high BMI (Body Mass Index) (BMI of 25 kg / m2 or higher), a subgroup with hypertension (subjects taking antihypertensive medication), and a subgroup with pre-diabetes or diabetes. In the table, CI represents a confidence interval, and N represents a parameter for each group.

[0497] The results in Table 23 indicate that cardiovascular disease diagnostic information, with three grades, can further differentiate the risk of cardiovascular disease occurrence within each of the above subgroups. Specifically, in the subgroup of hypertensives (those taking antihypertensive medication), the incidence rate of cardiovascular disease-related events within 10 years in the high-risk group was 17.7 (95% confidence interval, 15.0-20.8), which is very high. Referring to FIGS. 22a, 22b and 23, the processor of the diagnostic device can predict the risk of cardiovascular disease (in particular, the risk of having a 10-year or greater probability of occurrence of a cardiovascular disease-related event) with high accuracy by providing cardiovascular disease diagnosis information based on a diagnostic model through input of a retinal image and the subject's physical information, for a normal person who does not suffer from cardiovascular disease, a person who does not take a statin, a stage 1 hypertensive patient who does not take antihypertensive medication, a person with a high BMI, a person who is taking antihypertensive medication, and a pre-diabetic / diabetic patient.

[0498]

[0499] Figures 24a to 24c are diagrams for explaining the results of applying cardiovascular disease diagnosis information to a group with a QRISK3 score of 7.5 to 10% according to Example 1.

[0500] Referring to FIG. 24, in the graphs of FIGS. 24a to 24c, the x-axis represents time (years), and the y-axis represents the incidence rate of cardiovascular disease-related events (expressed as the incidence rate of cardiovascular disease events in FIGS. 24a to 24c).

[0501] And, the graph in Fig. 24a is for the group not taking statins, the graph in Fig. 24b is for the group with stage 1 hypertension, and the graph in Fig. 24c is for the middle-aged (40-60 years old) group, and each graph shows the results of performing Kaplan-Meier survival analysis.

[0502] In addition, in FIGS. 24a to 24c, a1, b1, and c1 represent the incidence of cardiovascular disease events in a group having a QRISK3 score of 5 to 7.5%, a2, b2, and c2 represent the incidence of cardiovascular disease events in a group having a QRISK3 score of 7.5 to 10% and a low-risk grade of cardiovascular disease diagnosis information, a3, b3, and c3 represent the incidence of cardiovascular disease events in a group having a QRISK3 score of 7.5 to 10% and a medium-risk grade of cardiovascular disease diagnosis information, a4, b4, and c4 represent the incidence of cardiovascular disease events in a group having a QRISK3 score of 7.5 to 10% and a high-risk grade of cardiovascular disease diagnosis information, and a5, b5, and c5 represent the incidence of cardiovascular disease events in a group having a QRISK3 score of 10 to 12.5%.

[0503] In general, the incidence of cardiovascular disease-related events may be higher in a group with a QRISK3 score of 10-12.5% ​​than in a group with a QRISK3 score of 7.5-10%. However, as shown in FIGS. 24a to 24c, when cardiovascular disease diagnosis information (e.g., low-risk group, intermediate-risk group, and high-risk group) is applied to the QRISK3 score, the incidence of cardiovascular disease-related events in a group according to the QRISK3 score may be classified or stratified according to each group (e.g., low-risk group, intermediate-risk group, and high-risk group) of cardiovascular disease diagnosis information. For example, a group with a QRISK3 score of 7.5-10% and a high-risk cardiovascular disease diagnosis information may tend to have a higher incidence of cardiovascular disease-related events than a group with a QRISK3 score of 10-12.5%. This may mean that a high-risk group that cannot be predicted by the QRISK3 score, i.e., existing biomarkers, can be accurately predicted using cardiovascular disease diagnosis information.

[0504] In addition, the incidence of cardiovascular disease-related events should be higher in the group with a QRISK3 score of 7.5~10% than in the group with a QRISK3 score of 5~7.5%. However, as the incidence of cardiovascular disease-related events in the group with a QRISK3 score of 7.5~10% is stratified by applying the cardiovascular disease diagnosis information to the group, as shown in the graph in Fig. 24c, the incidence of cardiovascular disease events in the group with a QRISK3 score of 5~7.5% and the group with a QRISK3 score of 7.5~10% and a low-risk cardiovascular disease diagnosis information may show a similar tendency. This may mean that even for the borderline risk group, which is still classified as a low-risk group for which drug treatment is not recommended by the existing biomarker, the cardiovascular disease diagnosis information can be used to further stratify and accurately predict whether they belong to a lower or higher risk group.

[0505] Therefore, the cardiovascular disease diagnostic information in this specification can predict the risk of cardiovascular disease with higher accuracy than QRISK3, especially in the borderline risk group with a QRISK3 score of 7.5-10%. Considering that the guidelines recommend initiating statin and antihypertensive drug treatment for a QRISK3 score of 10% or higher, the cardiovascular disease diagnostic information in this specification can serve as a risk enhancer that can identify hidden risks in the borderline risk group with a QRISK3 score of 7.5-10% and guide the initiation of statin and antihypertensive drug treatment.

[0506]

[0507] Furthermore, when the cardiovascular disease diagnostic information in this specification is applied together with the QRISK3 score, the risk of cardiovascular disease can be predicted with high accuracy. This is described in detail using Figure 25.

[0508] Figure 25 is a diagram for explaining in detail the performance when applying cardiovascular disease diagnosis information and QRISK3 score together according to Example 1.

[0509] Referring to FIG. 25, in the table of FIG. 25, cardiovascular disease diagnosis information + age, gender represents a case where age and gender are added to cardiovascular disease diagnosis information, cardiovascular disease diagnosis information + QRISK3 represents a case where cardiovascular disease diagnosis information and QRISK3 are applied together, △Cardiovascular disease diagnosis information + QRISK3 vs QRISK3 represents the difference between a case where cardiovascular disease diagnosis information and QRISK3 are applied together and a case where only QRISK3 is applied, and NRI (Net reclassification Index) can represent an index measuring how much better a new model is than a past model. As shown in Table 25, when cardiovascular disease diagnosis information and QRISK3 were applied together, the C statistic for predicting the occurrence of cardiovascular disease-related events increased by 0.014 (95% confidence interval, 0.010-0.017) in the group not taking statins, by 0.013 (95% confidence interval, 0.007-0.019) in the stage 1 hypertension group, and by 0.023 (95% confidence interval, 0.018-0.029) in the middle-aged cohort. Furthermore, when cardiovascular disease diagnostic information and QRISK3 were applied together, the continuous NRI could be 0.133 (95% confidence interval, 0.088–0.173) in the non-statin cohort, 0.094 (0.008–0.174) in the stage 1 hypertension cohort, and 0.248 (0.190–0.301) in the middle-aged cohort. These results indicate that applying the cardiovascular disease diagnostic information and QRISK3 score in this specification together can predict the risk of cardiovascular disease with high accuracy.

[0510] Additionally, in the table of Figure 25, the C statistic for predicting the occurrence of cardiovascular disease-related events when age and gender are added to the cardiovascular diagnosis information can be shown to be higher than the C statistic for predicting the occurrence of cardiovascular disease-related events when only QRISK3 is used. Accordingly, it can be confirmed that the accuracy of predicting cardiovascular disease risk is higher when age and gender are added to the cardiovascular diagnosis information than when only QRISK3 is used.

[0511]

[0512] Additionally, utilizing the cardiovascular disease diagnostic information in this specification, or applying it in conjunction with QRISK3, can provide more appropriate guidance information to the subject. This is explained in detail below.

[0513]

[0514] 2.1.3.2. Example 2

[0515] Example 2 describes experimental results of the ability of the present disclosure to identify individuals at intermediate and high risk for cardiovascular disease using various biomarkers.

[0516] As mentioned above, the PCE, QRISK3, and modified Framingham Risk Score may be official risk assessment tools and biomarkers used as guidelines for the primary prevention of cardiovascular disease in the United States, the United Kingdom, and Singapore, respectively. However, these biomarkers do not necessarily require blood tests and are not noninvasive means of distinguishing between intermediate and high-risk groups for cardiovascular disease. In contrast, the cardiovascular disease diagnosis method according to the present disclosure may be a retinal image-based biomarker that can noninvasively identify intermediate and high-risk groups for cardiovascular disease. Below, by comparing existing biomarkers with the cardiovascular disease diagnosis method according to the present disclosure, it is explained that the cardiovascular disease diagnosis method according to the present disclosure can function as a noninvasive screening tool that can identify individuals based on their risk of cardiovascular disease.

[0517] In Example 2, data from the UK Biobank and SEED (Singapore Epidemiology of Eye Diseases) may be used as experimental subject data. In addition, clinical data and retinal images of duplicated retinal images, retinal images of low quality, patients with type 1 diabetes, patients with cardiovascular disease at baseline (specifically, patients with coronary artery disease, other heart diseases, stroke, transient ischemic attack, peripheral arterial disease, and patients who underwent cardiovascular surgery or cardiovascular procedures), and patients under the age of 40 may be excluded from the experiment of Example 2. Accordingly, clinical data and retinal images of 48,260 patients without a history of cardiovascular disease may be included in the experiment of Example 2.

[0518] Additionally, in the entire data of SEED, clinical data and retinal images of individuals with missing data and individuals with cardiovascular disease at baseline may be excluded from the experiment of Example 2. In addition, clinical data and retinal images of 6,810 individuals (2,548 Chinese, 1,976 Indian, and 2,286 Malay) representing the general population without a history of cardiovascular disease in SEED may be included in the experiment of Example 2.

[0519]

[0520] Additionally, in Example 2, a PCE score, a QRISK3 score, and a modified Framingham risk score can be obtained for each subject of the experimental data. Based on biomarkers such as the PCE, QRISK3, and modified Framingham risk score, individuals who may be prioritized for cardiovascular disease risk management can be defined as intermediate and high-risk groups. Furthermore, based on biomarkers such as the PCE, QRISK3, and modified Framingham risk score, individuals who may be prioritized for cardiovascular disease risk management can be defined as intermediate and high-risk groups. For example, for the PCE score, a score of less than 7.5% can be defined as low-risk, and a score of 7.5% or more can be set as intermediate and high-risk groups. Furthermore, for the QRISK3 score and modified Framingham risk score, a score of less than 10% can be defined as low-risk, and a score of 10% or more can be set as intermediate and high-risk groups.

[0521]

[0522] In addition, in one embodiment, experimental subject data may be input into the diagnostic model (1000). That is, the processor of the control device may input the retinal data of the subjects into the first diagnostic model (1100) to obtain a probability value for the probability that the subject has a coronary artery calcium score of 0 or higher from the first diagnostic model (1100), and input the probability value and the subject's physical information (e.g., age, gender) into the second diagnostic model (1200) to obtain a probability value for the probability of occurrence of a cardiovascular disease-related event for each subject within 10 years. In addition, the processor of the control device may classify the probability value using a predetermined cutoff value to classify the subjects into a predetermined grade.

[0523] In Example 2, the probability value can be classified using a single cutoff value to output a binary grade of an intermediate-high risk group and a low-risk group. For example, the cutoff value can be set so that the proportion of people corresponding to the intermediate-high risk group of cardiovascular disease diagnosis information is similar to the proportion of people corresponding to 7.5% or more of the standard for the intermediate-high risk group of PCE score (i.e., the incidence rate corresponding to a PCE score of 7.5% or more). In addition, the cutoff value can be set so that the proportion of people corresponding to the intermediate-high risk group of cardiovascular disease diagnosis information is similar to the proportion of people corresponding to 10% or more of the standard for the intermediate-high risk group of QIRKS3 score / modified Framingham risk score (i.e., the incidence rate corresponding to 10% or more of QIRKS3 score / modified Framingham risk score).

[0524] Additionally, as an example, a preliminary analysis can be performed to determine the cutoff for detection of intermediate-high risk groups to achieve maximum performance using receiver operating characteristic (ROC) curves and the Youden Index.

[0525] Specifically, a logistic regression can be performed using a set of probability values ​​of continuous cardiovascular disease diagnosis information as independent variables, and 1) binary categories of low-borderline risk group vs. intermediate-high risk group according to the PCE score of the UK Biobank, and 2) binary categories of low-risk group vs. high-risk group according to the QRISK3 score of the UK Biobank and the modified Framingham risk score of SEED as dependent variables. Based on this logistic regression, a receiver operating characteristic (ROC) curve can be generated according to the PCE score of the UK Biobank, the QRISK3 score of the UK Biobank, and the modified Framingham risk score of SEED, respectively, and a cutoff value can be determined by applying the Yoden index to the ROC curve. Based on the determined cutoff value, a cardiovascular disease diagnosis method can identify the intermediate-high risk group in each of the PCE, QRISK3, and the modified Framingham risk score with high accuracy.

[0526]

[0527] Additionally, covariate-adjusted ROC analysis can be performed to adjust for potential covariates that influence cardiovascular disease risk. Specifically, in traditional ROC analysis, the ROC curve can be defined as shown in Equation 2.

[0528]

[0529] [Equation 2]

[0530] ROC(u) = P(1-F0(Y)≤u|D=1)

[0531]

[0532] Here, u represents the false positive rate, and Y can represent a predictor variable. F0(Y) represents the percentile value based on the marginal distribution among the control group, and ROC(u) can represent the sensitivity at that level.

[0533] In covariate-adjusted ROC analysis, instead of using the marginal distribution in traditional ROC analysis, percentile values ​​can be estimated based on the conditional distribution among the control group. To estimate percentiles under the conditional distribution, a linear model for age and gender, which are covariates for Y, can be regressed. According to this regression, the AUC (Area under covariate-adjusted ROC Curve) of the ROC curves for the UK Biobank PCE score, the UK Biobank QRISK3 score, and the SEED modified Framingham risk score could be improved respectively (UK Biobank-PCE: 0.853 (95% CI: 0.849~0.859), UK Biobank-QRISK3: 0.820 (95% CI: 0.813~0.827)), SEED-modified Framingham risk score_Chinese / Indian / Malaysian: 0.858 (95% CI: 0.832~0.879) / 0.874 (95% CI: 0.848~0.892) / 0.838 (95% CI: 0.794~0.87)). Accordingly, variables of the second diagnostic model can be fitted (trained) and / or a cutoff can be determined to have higher accuracy.

[0534]

[0535] Below, the results according to Example 2 are described.

[0536] Figure 26 shows the clinical characteristics of subjects according to cardiovascular disease diagnosis information according to Example 2.

[0537] Referring to Fig. 26, the table in Fig. 26 shows the clinical characteristics of the subjects of the UK Biobank and SEED, and each row represents the subject, the low-risk group and the high-risk group according to the cardiovascular diagnosis information, age, sex, hypertension, diabetes, smoking status, and overweight. In addition, each column can represent the low-borderline risk group and the intermediate-high risk group (10-year cardiovascular disease risk rate of 7.5% or more) according to the PCE score of the UK Biobank, and the low-risk group and the intermediate-high risk group (10-year cardiovascular disease risk rate of 10% or more) according to the QRISK3 score of the UK Biobank and the modified Framingham risk score of SEED.

[0538] As shown in Table 26, among those classified as intermediate-to-high risk based on each biomarker, the proportion of those classified as high-risk based on cardiovascular diagnostic information may be greater than 80%. These results demonstrate that the cardiovascular diagnostic information provided herein can accurately predict individuals classified as intermediate-to-high risk based on each biomarker.

[0539] Also, for reference, as shown in the table in Figure 26, cardiovascular disease-causing factors such as age, gender, hypertension, diabetes, smoking, and overweight may have a greater impact on the intermediate-high risk group than on the low risk group.

[0540]

[0541] Figure 27 is a diagram for explaining the performance of cardiovascular disease diagnosis information according to Example 2.

[0542] Referring to Figure 27, in the table of Figure 27, each row represents the prevalence, which is the total proportion of subjects classified into the intermediate-high risk group according to the PCE score of the UK Biobank, and the sensitivity, specificity, PPV (positive predictive value), and NPV (negative predictive value) of cardiovascular disease diagnostic information for the intermediate-high risk group according to the PCE score of the UK Biobank. In addition, the prevalence, sensitivity, specificity, PPV, and NPV are also described for the QRISK3 score of the UK Biobank and the modified Framingham risk score of SEED. In addition, each column represents the total subjects, females, and males.

[0543] As shown in Table 27, the cardiovascular disease diagnostic information can predict the intermediate-high risk group according to the PCE score of the UK Biobank with a sensitivity of 82.7%, a specificity of 87.6%, a PPV of 86.5%, and an NPV of 84.0%. In addition, the cardiovascular disease diagnostic information can predict the intermediate-high risk group according to the QRISK3 score of the UK Biobank with a sensitivity of 82.6%, a specificity of 85.5%, a PPV of 49.9%, and an NPV of 96.6%, and can indicate the intermediate-high risk group according to the modified Framingham risk score of SEED with a sensitivity of 82.1%, a specificity of 80.6%, a PPV of 76.4%, and an NPV of 85.5%.

[0544] In this way, cardiovascular disease diagnostic information can predict intermediate- to high-risk groups with high accuracy based on various biomarkers.

[0545]

[0546] Figure 28 is a diagram illustrating the performance of cardiovascular disease diagnosis information for various ethnicities according to Example 2.

[0547] Referring to Figure 28, the table in Figure 28 shows the performance of cardiovascular disease diagnostic information for intermediate-high risk groups according to the modified Framingham risk score of SEED. Each row can represent the prevalence, sensitivity, specificity, PPV, and NPV for Chinese / Indian / Malaysian individuals. In addition, each column represents the total subjects, females, and males.

[0548] As shown in Table 28, cardiovascular disease diagnostic information can predict intermediate-to-high risk groups based on the modified Framingham risk score of SEED for Chinese, Indian, and Malaysian individuals with high sensitivities of 79.7%, 83.2%, and 82.1%, respectively. Furthermore, cardiovascular disease diagnostic information can predict intermediate-to-high risk groups based on the modified Framingham risk score of SEED for Chinese women with a very high NPV of 99.3%.

[0549] In this way, cardiovascular disease diagnostic information can predict intermediate- to high-risk groups with high accuracy based on different biomarkers for various ethnicities.

[0550]

[0551] 2.1.3.3. Example 3

[0552] Example 3 is intended to verify the performance of cardiovascular disease diagnostic information according to the present disclosure on the US population of the Age-Related Eye Disease Studies (AREDS).

[0553] In Example 3, data from AREDS subjects can be used as experimental subject data. Furthermore, the predictive performance of cardiovascular disease diagnostic information according to the present specification for atherosclerotic cardiovascular disease (ASCVD) can be evaluated using a Cox proportional hazards model. In this case, the cardiovascular disease diagnostic information according to the present specification can be classified into three grades based on a predetermined cutoff value, with a probability of occurrence of a cardiovascular disease-related event within 10 years.

[0554] In Example 3, 282 of 3,555 subjects (7.9%) experienced nonfatal and fatal ASCVD events, and 84 (2.4%) experienced fatal ASCVD events during the 13-year follow-up period. In Example 3, we evaluated whether adding cardiovascular diagnostic information to these subjects improved risk prediction. Cardiovascular diagnostic information was significantly associated with an increased risk of ASCVD, with an adjusted hazard ratio (HR) trend of 1.13 (95% CI, 1.03-1.24) for nonfatal and fatal ASCVD events and an adjusted HR trend of 1.27 (1.07-1.52) for fatal ASCVD events. Accordingly, the cardiovascular diagnostic information significantly improved the overall predictive performance of traditional risk models, with continuous NRIs of 0.247 (0.106-0.364) for nonfatal and fatal ASCVD events and 0.232 (0.107-0.359) for fatal ASCVD events. Furthermore, the processor of the diagnostic device was capable of acquiring a map of the cardiovascular diagnostic information, and analysis of the map confirmed that traditional risk features such as arteriovenous nicking and arterial narrowing were well detected by the cardiovascular diagnostic information. Accordingly, the cardiovascular diagnostic information of the present disclosure may have significant potential as a biomarker or risk classification tool for cardiovascular disease applicable to the general US population.

[0555]

[0556] 2.1.3.4. Example 4

[0557] Example 4 is intended to verify the future cardiovascular risk prediction performance of the cardiovascular diagnostic method of the present disclosure based on a retrospective analysis of a prior prospective cohort study.

[0558] In Example 4, the main point may be to validate a three-step cardiovascular risk classification system using retinal images evaluated by cardiovascular disease diagnostic information. In Example 4, the cardiovascular disease diagnostic information may be based on the results output from the first diagnostic model described above and / or the results output from the second diagnostic results. Additionally, coronary artery calcium score (CAC 0, >0-100, >100), carotid intima-media thickness score (CIMT, <90% and ≥90%), and brachiocephalic artery pulse wave velocity score (baPWV, <1800 and ≥1800 cm / s) measured by cardiac CT may also be measured as independent variables for future cardiovascular disease diagnostic information. In addition, in Example 4, the cumulative incidence of non-fatal and / or fatal cardiovascular disease-related events was evaluated, and the hazard ratio (HR) trend was estimated using a Cox proportional hazards model.

[0559] In Example 4, in a given clinical cohort (n=1106), 33 subjects (3.0%) experienced nonfatal or fatal cardiovascular events during the 5-year period, and cardiovascular diagnosis information was significantly associated with increased CVD risk (HR for trend=2.02, 95% CI, 1.26-3.24). Furthermore, in a multivariable Cox model including cardiovascular diagnosis information, coronary calcium score, carotid intima-media thickness score, brachiocephalic pulse wave velocity score, and other traditional risk factors, the odds ratio of cardiovascular diagnosis information in the high-risk group was significantly associated with increased CVD risk (HR=3.56 [1.34-9.51] in the high-risk group, with reference to low risk), whereas other biomarkers may show lesser associations than cardiovascular diagnosis information. For example, the atherosclerotic calcium score has a HR of 2.45 [0.88-6.84] when the atherosclerotic calcium score is greater than 100, compared to an atherosclerotic calcium score of 0; the carotid intima-media thickness score has a HR of 1.50 (0.64-3.51) when the atherosclerotic calcium score is greater than 90% compared to less than 90%; and the brachiocephalic pulse wave velocity score has a HR of 1.27 (0.53-3.03) when the atherosclerotic calcium score is greater than 1800 cm / s compared to less than 1800 cm / s.

[0560] Accordingly, the cardiovascular disease diagnostic information of this specification can predict cardiovascular disease risk with higher accuracy than the coronary artery calcium score, carotid intima-media thickness score, and brachiocephalic artery pulse wave velocity score.

[0561]

[0562] 2.1.4. Obtaining information on cardiovascular disease risk based on existing biomarkers

[0563] Fig. 29 is a drawing for explaining a method for diagnosing cardiovascular disease according to another embodiment.

[0564] Referring to FIG. 29, a method for diagnosing cardiovascular disease according to another embodiment may include a step of obtaining a retinal image (S300) and a step of obtaining information on cardiovascular disease risk according to an existing biomarker as cardiovascular disease diagnosis information (S400).

[0565] In step S300, the processor of the diagnostic device can acquire a retinal image. Since the descriptions of FIG. 19 and above apply to this, a detailed description is omitted.

[0566] Additionally, in step S400, the processor of the diagnostic device can obtain information on cardiovascular disease risk based on existing biomarkers as cardiovascular disease diagnostic information. The details described in the aforementioned diagnostic model can be applied to step S400.

[0567] Here, the information on the risk of cardiovascular disease according to the existing biomarker means the risk of cardiovascular disease calculated according to the existing biomarker, and the processor of the diagnostic device can obtain information on the risk of cardiovascular disease calculated according to the existing biomarker using the retinal image. For example, according to step S400, the processor of the diagnostic device can obtain information on the risk of cardiovascular disease calculated according to the existing biomarker using the retinal image, using the diagnostic model based on the retinal image, on the coronary calcium score and / or the grade according to the coronary calcium score, the PCE score and / or the grade according to the PCE score, the QRISK3 score and / or the grade according to the QRISK3 score, the modified Framingham risk score and / or the grade according to the modified Framingham risk score, the carotid intima-media thickness score and / or the grade according to the carotid intima-media thickness score, the brachial-ankle pulse wave velocity score and / or the grade according to the brachial-ankle pulse wave velocity score, and output the obtained information, or output information on the final score and / or the grade according to the final score based on the obtained information.

[0568]

[0569] In one embodiment, the processor of the diagnostic device can obtain information on cardiovascular disease risk according to existing biomarkers as cardiovascular disease diagnostic information using the diagnostic model.

[0570] For example, the diagnostic model can be trained using retinal images and information on cardiovascular disease risk according to existing biomarkers labeled in the retinal images. For example, each retinal image can be labeled with at least one of a coronary artery calcium score (and / or a grade thereof), a PCE score (and / or a grade thereof), a QRISK3 score (and / or a grade thereof), a modified Framingham risk score (and / or a grade thereof), a carotid intima-media thickness score (and / or a grade thereof), or a brachial-ankle pulse wave velocity score (and / or a grade thereof), and the diagnostic model can be trained using the retinal images and their corresponding labels. In addition, the labels labeled in the retinal images can also include information on at least one of the age, sex, race, smoking status, blood pressure (whether hypertension or diabetes) of the subject of the retinal image.

[0571]

[0572] The processor of the diagnostic device can input a retinal image of a subject into a learned single diagnostic model to obtain information on the risk of cardiovascular disease based on existing biomarkers. For example, if the diagnostic model is learned based on the PCE score, the processor of the diagnostic device can input the retinal image into the diagnostic model and obtain the PCE score and / or information on other grades of the PCE score from the diagnostic model. In addition, according to an embodiment, the processor of the diagnostic device can input information on at least one of the subject's age, gender, race, smoking status, blood pressure (whether hypertension or hypertension), and diabetes status together with the retinal image into the diagnostic fake model to obtain the PCE score and / or information on other grades of the PCE score from the diagnostic model.

[0573]

[0574] Additionally, in one embodiment, the diagnostic models may be constructed in parallel. For example, the descriptions of the diagnostic models in 1.4.1.1 to 1.4.1.3 may apply to the diagnostic models.

[0575] Specifically, the diagnostic model may include multiple diagnostic models. For example, the diagnostic model may include a first diagnostic model and a second diagnostic model. The first diagnostic model and the second diagnostic model may be trained using information on cardiovascular disease risk according to different biomarkers. For example, the first diagnostic model may be trained using a retinal image labeled with a PCE score (and / or a grade thereof), and the second diagnostic model may be trained using a retinal image labeled with a QRISK3 score (and / or a grade thereof). Of course, the present invention is not limited thereto, and the diagnostic model may include additional diagnostic models, such as a third / fourth diagnostic model, in addition to the first / second diagnostic models. In addition, the labels labeled in the retinal image may include information on at least one of the age, sex, race, smoking status, blood pressure (whether hypertension or hypertension), and diabetes of the subject of the retinal image.

[0576] In addition, the processor of the diagnostic device can input the retinal image of the subject into the first diagnostic model and the second diagnostic model to obtain information on the risk of cardiovascular disease according to different biomarkers. For example, the processor of the diagnostic device can obtain the PCE score (and / or grade thereof) of the subject from the first diagnostic model, obtain the QRISK3 score (and / or grade thereof) of the subject from the second diagnostic model, and output the obtained information. In addition, the processor of the diagnostic device can output information on the final score and / or grade according to the final score based on the information obtained from each diagnostic model. For example, the processor can obtain and output information on the final score and / or grade according to the final score based on the PCE score (and / or grade thereof) of the subject obtained from the first diagnostic model and the QRISK3 score (and / or grade thereof) of the subject obtained from the second diagnostic model. Accordingly, the processor of the diagnostic device can obtain information on the risk of cardiovascular disease according to existing biomarkers as non-invasive cardiovascular disease diagnosis information using retinal images without using invasive methods such as blood tests.

[0577]

[0578] Additionally, in one embodiment, the diagnostic models may be constructed serially. For example, the descriptions of the diagnostic models in 1.4.2.1, 1.4.2.2, and 2.1.2 may apply.

[0579] For example, the diagnostic model may include a first diagnostic model and a second diagnostic model, as shown in FIG. 20. For example, the processor of the diagnostic device may input a retinal image of a subject into the first diagnostic model, and obtain a probability value for the probability that the subject's coronary artery calcium score is 0 or higher from the first diagnostic model. In addition, the processor of the diagnostic device may input an output value of the first diagnostic model and the subject's physical information (at least one of height, weight, age, gender, race, smoking status, blood pressure (e.g., blood pressure level, high blood pressure), diabetes (or blood sugar level), and cholesterol level). At this time, the second diagnostic model may set a cutoff value based on various biomarkers such as PCE, QRISK3, and modified Framingham risk score, similar to Example 2. For example, as the cutoff value of the second diagnostic model, a cutoff value may be set similar to the proportion of people corresponding to 7.5% or more, which is the standard for the intermediate-high risk group of the PCE score, and / or a cutoff value may be set similar to the proportion of people corresponding to 10% or more, which is the standard for the intermediate-high risk group of the QRISK3 / modified Framingham risk score. Accordingly, cardiovascular disease diagnostic information can predict the intermediate-high risk group according to various biomarkers with high accuracy. When the diagnostic models are configured in series, the content described in Example 2 described above can be applied to obtaining information on the risk of cardiovascular disease according to existing biomarkers.

[0580]

[0581] 2.1.5. Providing guidance information on cardiovascular disease diagnosis.

[0582] FIG. 30 is a diagram illustrating a method for providing guide information for cardiovascular disease diagnosis information according to one embodiment.

[0583] Referring to FIG. 30, a method for providing guide information according to one embodiment may include a step of obtaining cardiovascular disease diagnosis information (S500) and a step of providing guide information for the diagnosis information (S600).

[0584] In step S500, the processor of the diagnostic device can obtain diagnostic information. Since the above description may apply to step S500, a detailed description thereof will be omitted.

[0585] Additionally, in step S600, the processor of the diagnostic device may provide guidance information regarding the diagnostic information. For convenience of explanation, steps S500 and S600 focus on cardiovascular disease, but the description is not limited thereto, and the guidance information in this specification can of course be applied to various diseases, including ocular disease, cardiovascular disease, and kidney disease. Accordingly, steps S500 and S600 focus on guidance information based on cardiovascular diagnostic information.

[0586] In one embodiment, the guidance information may refer to information about medical and non-medical treatments recommended for the subject based on cardiovascular disease diagnosis information. For example, the guidance information may include prescription information, action information, and management information.

[0587] Prescription information may refer to information on medications (e.g., prescription drugs) recommended to a subject to maintain or improve the risk of cardiovascular disease based on cardiovascular disease diagnosis information. In addition, the prescription information may include information on the prescribed medication, timing of administration, and dosage. For example, the prescription information may include information on prescriptions for one or more of a statin (including various agents such as simvastatin, atorvastatin, and rosuvastatin), an HMG-CoA reductase inhibitor, a PCSK9 inhibitor, a fibric acid derivative combination therapy, aspirin, a bile acid sequestrant, and nicotinic acid, an omega-3 fatty acid, ezetimibe, and a fibrate.

[0588] Additionally, action information may refer to information about future recommended actions for the subject to maintain or improve cardiovascular disease risk based on cardiovascular disease diagnosis information. For example, additional screening information may include information about secondary diagnoses or medical treatments. For example, additional screening information may include information about additional required tests, hospitals / medical professionals who can perform the additional tests, and recommended procedures / surgeries.

[0589] Additionally, management information may include information on non-medical treatments recommended for the subject to maintain or improve cardiovascular disease risk based on cardiovascular disease diagnosis information. For example, management information may include lifestyle information, dietary information, exercise information, and information on non-prescription medications such as nutritional supplements to reduce cardiovascular disease risk.

[0590]

[0591] Additionally, in one embodiment, the diagnostic device may be linked to an external monitoring device. Here, the monitoring device may refer to a device that monitors the subject's lifestyle or behavior. For example, the monitoring device may include a portable device, a wearable device, a wellness measurement device, etc. Furthermore, the monitoring device may be the client device described above. For example, the monitoring device may include an imaging unit that can capture images of the inside and outside of the eye through the imaging unit.

[0592] In addition, the monitoring device can monitor various information such as the subject's activity level, exercise method, exercise time, food intake, intake amount, health supplement intake information, sleep time, sleep habits, heart rate, blood pressure, blood sugar level, body water content, oxygen content, body temperature, oxygen saturation, pulse wave, whether the subject has visited a hospital, whether the subject has undergone an examination, whether the subject has undergone a procedure / surgery, and eye images.

[0593] The processor of the diagnostic device can communicate with the monitoring device via a communication module, either wired or wirelessly.

[0594] The processor of the diagnostic device can provide guidance information to the monitoring device. The monitoring device can then provide various information to the subject based on the guidance information and the monitored information. For example, the monitoring device can obtain management information (e.g., lifestyle information, dietary information, exercise information) from the diagnostic device as guidance information, compare the management information with the monitored information, determine whether the monitored information matches the management information, and provide the determination result and / or additional information based on the result.

[0595] For example, if the monitored exercise time is less than the exercise time indicated in the management information, the monitoring device may provide the subject with information to exercise according to the management information. Furthermore, if the food intake in the monitored information corresponds to the food intake indicated in the management information, the monitoring device may provide the subject with information indicating that the food intake is appropriately consuming according to the management information.

[0596] Additionally, the processor of the diagnostic device can obtain monitored information from the monitoring device. The processor of the diagnostic device can provide various information to the subject based on the guide information and the monitored information. For example, the processor of the diagnostic device can compare the monitored information with the guide information, determine whether the monitored information matches the management information, and provide the determination result and / or additional information based on the comparison. The examples of the monitoring device described above can be applied to the operation of the processor of the diagnostic device.

[0597] Additionally, the processor of the diagnostic device can generate guidance information by reflecting monitoring information received from the monitoring device. For example, the processor of the diagnostic device can obtain status information (e.g., exercise status, lifestyle, eating habits) of the subject based on the monitoring information, and can modify guidance information determined as cardiovascular disease diagnosis information based on the subject's status information to suit the subject.

[0598]

[0599] In one embodiment, the processor of the diagnostic device may provide guidance information using a predetermined database. For example, the diagnostic device may include a database that matches the score and / or grade of cardiovascular disease diagnosis information with the guide information. For example, if the cardiovascular disease diagnosis information is expressed in three grades, the database may include guide information matching a low-risk grade (e.g., no prescription information, action information - information about the timing of the next treatment, management information - providing dietary information, providing exercise information), guide information matching a medium-risk grade (e.g., no prescription information, action information - providing additional test information, management information - providing dietary information, providing exercise information, providing non-prescription drug information), and guide information matching a high-risk grade (e.g., prescription information - providing statin prescription information, action information - providing additional test information, providing recommended procedure / surgery information, management information - providing dietary information, providing exercise information, providing non-prescription drug information). The processor of the diagnostic device may provide guide information matching the cardiovascular disease diagnosis information based on the database.

[0600] In another embodiment, the processor of the diagnostic device may provide guide information using a machine learning model. For example, the diagnostic device may include a guide information model based on a machine learning model or a neural network model. The guide information model may be learned based on the score and / or grade of the cardiovascular disease diagnostic information and the guide information. Additionally, the guide information model may also be learned together with the subject's physical information (at least one of height, weight, age, gender, race, smoking status, blood pressure (e.g., blood pressure level, high blood pressure), diabetes (or blood sugar level), and cholesterol level). Accordingly, the processor of the diagnostic device may input the score and / or grade of the cardiovascular disease diagnostic information and the subject's physical information into the guide information model to obtain guide information about the subject. In addition, depending on the embodiment, the guide information model may be included in the diagnostic model or may be configured independently of the diagnostic model.

[0601]

[0602] 2.1.6. Providing guidance information using existing biomarker results and cardiovascular disease diagnosis information.

[0603] FIG. 31 is a diagram illustrating a method for providing guide information using results based on existing biomarkers and cardiovascular disease diagnosis information according to one embodiment.

[0604] Referring to FIG. 31, a method for providing guide information according to one embodiment may include a step of obtaining a result value according to a biomarker (S710), a step of obtaining cardiovascular disease diagnosis information (S720), and a step of providing guide information using a comparison result between the result value according to the biomarker and the cardiovascular disease diagnosis information (S730).

[0605] According to step S710, the processor of the diagnostic device can obtain a result value according to the biomarker. For example, the processor of the diagnostic device can obtain a score and / or grade of the biomarker from an external device or an input module of the diagnostic device. Here, the biomarker may include the aforementioned PCE, QRISK, modified Framingham risk score, carotid intima-media thickness, brachial-ankle pulse wave velocity, etc. In addition, the biomarker may include other cardiovascular disease biomarkers or cardiovascular disease risk assessment tools. For example, the processor of the diagnostic device may also obtain information on the rate of blood pressure increase, the rate of LDL increase, the rate of HbA1c increase, and the frequency of uncontrolled diabetes / hypertension / hyperlipidemia (e.g., blood pressure of 160 mmHg or higher observed four times in the past six months, etc.).

[0606]

[0607] Additionally, in step S720, the processor of the diagnostic device may acquire cardiovascular disease diagnostic information. Here, the cardiovascular disease diagnostic information may refer to cardiovascular disease diagnostic information based on a retinal image of the same subject as the subject whose biomarker-based results were acquired in step S710. The above description may apply to step S720, so a detailed description thereof will be omitted.

[0608] Additionally, in step S730, guide information can be provided using the comparison results between the biomarker-based result value and the cardiovascular disease diagnosis information.

[0609] Generally, medical guidelines defining cardiovascular disease risk and corresponding medical treatment information are used in the medical community. Furthermore, biomarkers used to determine cardiovascular disease risk may vary across countries. For example, the UK uses QRISK3, the US uses PCE, and Singapore uses the modified Framingham Risk Score.

[0610]

[0611] Below, an embodiment of providing guide information using the results of comparing the results according to biomarkers included in medical guidelines with the results of cardiovascular disease diagnosis information is described.

[0612] In the above embodiment, if the biomarker included in the medical guideline is QRISK3, the processor of the diagnostic device can compare the score or grade of QRISK3 with the score or grade according to the cardiovascular disease diagnosis information.

[0613] For example, if the result according to QRISK3 is a high-risk group and the result according to the cardiovascular disease diagnosis information is a high-risk grade, the processor of the diagnostic device may determine that the risk of cardiovascular disease of the subject is high-risk and provide treatment information for the high-risk group in the medical guideline. In addition, according to an embodiment, the processor of the diagnostic device may provide guide information (e.g., prescription information - statin prescription information, action information - additional test information, recommended procedure / surgery information, management information - dietary habit information, exercise information, non-prescription drug information) provided when the cardiovascular disease diagnosis information described in FIG. 30 is determined to be a high-risk group.

[0614] Additionally, if the QRISK3 result indicates a high-risk group and the cardiovascular disease diagnosis information indicates a non-high-risk group (e.g., low-risk group or intermediate / borderline risk group), the processor of the diagnostic device may determine that the subject is at high risk for cardiovascular disease and provide treatment information for the high-risk group from medical guidelines. Furthermore, according to an embodiment, the processor of the diagnostic device may also provide guidance information provided when the cardiovascular disease diagnosis information is determined to be at high risk.

[0615] In addition, if the result according to QRISK3 is a non-high risk group (e.g., low risk group or intermediate / borderline risk level) and the result according to the cardiovascular disease diagnosis information is a high risk group, the processor of the diagnostic device can determine that the risk of cardiovascular disease of the subject is a high risk group even if the result according to QRISK3 is a non-high risk group. This may be based on the experimental result showing that, as described in Example 1 above, even if the subject is determined to be a non-high risk group by QRISK3, a high risk group that was not predicted by QRISK3 can be accurately predicted using the cardiovascular disease diagnosis information. Accordingly, the processor of the diagnostic device can provide guide information that is provided when the cardiovascular disease diagnosis information is determined to be a high risk group.

[0616] Additionally, if the QRISK3 results indicate a non-high risk group and the cardiovascular disease diagnosis information indicates a non-high risk grade, the processor of the diagnostic device may determine that the subject's cardiovascular disease risk is non-high risk. Accordingly, the processor of the diagnostic device may provide guidance information (e.g., no prescription information, action information regarding the timing of the next treatment, management information regarding dietary habits, and exercise information) provided when the cardiovascular disease diagnosis information is determined to be non-high risk.

[0617]

[0618] In addition, the following describes an embodiment that provides guide information by using the results of comparing cardiovascular disease diagnosis information with results according to biomarkers not included in medical guidelines.

[0619] In the above embodiment, if the biomarker used for reference but not included in the medical guideline is QRISK3, the processor of the diagnostic device can compare the score or grade of QRISK3 with the score or grade according to the cardiovascular disease diagnosis information.

[0620] For example, if the result according to QRISK3 is a high-risk group and the result according to the cardiovascular disease diagnosis information is a high-risk grade, the processor of the diagnostic device can determine that the subject's risk of cardiovascular disease is high-risk and provide guide information that is provided when the cardiovascular disease diagnosis information is determined to be a high-risk group.

[0621] In addition, if the result according to QRISK3 is a high-risk group and the result according to the cardiovascular disease diagnosis information is a non-high-risk grade, the processor of the diagnostic device may determine that the risk of cardiovascular disease of the subject is a non-high-risk group. This may be based on the experimental results (particularly, the example of FIG. 24c) showing that even if a subject is determined to be a somewhat high-risk group by QRISK3, it is possible to accurately predict that they belong to a lower-risk group using the cardiovascular disease diagnosis information, as described in Example 1 above. Accordingly, the processor of the diagnostic device may provide guide information provided when the cardiovascular disease diagnosis information is determined to be a non-high-risk group. In addition, according to an embodiment, if the result according to QRISK3 is a high-risk group and the result according to the cardiovascular disease diagnosis information is a non-high-risk grade, the processor of the diagnostic device may determine that the risk of cardiovascular disease of the subject is a high-risk group as a preventive measure and provide guide information provided when the cardiovascular disease diagnosis information is determined to be a high-risk group.

[0622] In addition, if the result according to QRISK3 is a non-high risk group (e.g., low risk group or intermediate / borderline risk level) and the result according to the cardiovascular disease diagnosis information is a high risk level, the processor of the diagnostic device can determine that the risk of cardiovascular disease of the subject is a high risk group even if the result according to QRISK3 is a non-high risk group. This may be based on the experimental results (particularly, the embodiment of FIG. 24) showing that even if QRISK3 is determined to be a non-high risk group, a high risk group that was not predicted by QRISK3 can be accurately predicted using the cardiovascular disease diagnosis information, as described in Example 1 above. Accordingly, the processor of the diagnostic device can provide guide information that is provided when the cardiovascular disease diagnosis information is determined to be a high risk group.

[0623] In addition, if the result according to QRISK3 is a non-high risk group and the result according to the cardiovascular disease diagnosis information is a non-high risk grade, the processor of the diagnostic device can determine that the risk of the subject's cardiovascular disease is a non-high risk group and provide guide information that is provided when the cardiovascular disease diagnosis information is determined to be a non-high risk group.

[0624]

[0625] In the above examples, the results based on biomarkers and the results based on cardiovascular disease diagnosis information were explained as two grades, high risk and non-high risk, but this is not limited to this, and the above explanation may be applied even when the results based on biomarkers and the results based on cardiovascular disease diagnosis information are in three or more grades.

[0626]

[0627] 2.1.7. Predicting the Progression Rate of Cardiovascular Disease Using Cardiovascular Disease Diagnostic Information

[0628] FIG. 32 is a diagram for explaining a method for predicting the rate of progression of cardiovascular disease using cardiovascular disease diagnosis information according to one embodiment.

[0629] Referring to FIG. 32, a method for predicting the progression rate of cardiovascular disease according to one embodiment may include a step of obtaining a result value according to a biomarker (S810), a step of obtaining cardiovascular disease diagnosis information (S820), and a step of determining the progression rate of cardiovascular disease using the result value according to the biomarker and the cardiovascular disease diagnosis information (S830).

[0630] As for steps S810 and S820, the details described in steps S710 and S720 described above may be applied, so a detailed description thereof is omitted.

[0631] Additionally, in step S830, the progression rate of cardiovascular disease can be determined using the result value according to the biomarker and the cardiovascular disease diagnosis information.

[0632] Specifically, even if biomarker results are identical, the rate of progression of cardiovascular disease can vary. For example, even if a person's QRISK3 score is 8%, indicating a low-risk group, their cardiovascular disease diagnostic scores may differ. However, even with identical QRISK3 scores, individuals with higher scores may experience a faster progression of cardiovascular disease than individuals with lower scores.

[0633] In one embodiment, the processor of the diagnostic device can determine whether the score of the cardiovascular disease diagnostic information is above a predetermined threshold. If the score of the cardiovascular disease diagnostic information is above the predetermined threshold, the progression of the cardiovascular disease is determined to be rapid. If the score of the cardiovascular disease diagnostic information is below the predetermined threshold, the progression of the cardiovascular disease is determined to be slow.

[0634] Additionally, in one embodiment, the processor of the diagnostic device can predict the rate of progression of cardiovascular disease based on the amount of change in cardiovascular disease diagnostic information.

[0635] For example, the processor of the diagnostic device may obtain first cardiovascular disease diagnostic information and second cardiovascular disease diagnostic information of the subject. The first and second cardiovascular disease diagnostic information may be based on retinal images taken at different times of the same subject. For example, the first cardiovascular disease diagnostic information may be based on a retinal image taken at an earlier time than the second cardiovascular disease diagnostic information.

[0636] Additionally, the processor of the diagnostic device can obtain the amount of change in the acquired cardiovascular disease diagnostic information. For example, the processor of the diagnostic device can obtain the amount of change in the cardiovascular disease diagnostic information based on the time difference between the time the retinal image of the first cardiovascular diagnostic information was captured and the time the retinal image of the second cardiovascular diagnostic information was captured, and the score difference between the score of the first cardiovascular diagnostic information and the score of the second cardiovascular diagnostic information.

[0637] Additionally, the processor of the diagnostic device can predict the progression rate of cardiovascular disease in the subject based on the amount of change in cardiovascular disease diagnostic information. For example, if the amount of change in cardiovascular disease diagnostic information is greater than a predetermined threshold, the processor of the diagnostic device can determine that the progression rate of cardiovascular disease is rapid, and if the amount of change in cardiovascular disease diagnostic information is less than the predetermined threshold, the processor can determine that the progression rate of cardiovascular disease is slow. These thresholds can be set to one or more. Additionally, depending on the embodiment, the thresholds can be set based on various criteria. For example, the thresholds can be set so that the incidence rate of cardiovascular disease occurrence events is clearly stratified.

[0638]

[0639] Additionally, the processor of the diagnostic device may provide information on the progression rate of cardiovascular disease and / or guidance information based on the progression rate. In this case, as the progression rates of cardiovascular disease in the first and second subjects differ, the processor of the diagnostic device may provide different guidance information to the first and second subjects.

[0640] For example, even if the first and second subjects belong to the same risk group based on existing biomarkers and / or cardiovascular disease diagnosis information, different guidance information may be provided depending on the progression rate of the cardiovascular disease of the first and second subjects. For example, if the progression rate of the cardiovascular disease of the first subject is predicted to be rapid and the first subject is in the low-risk group, the processor of the diagnostic device may provide guidance information to the effect that blood pressure, diabetes, hypertension, etc. should be thoroughly managed. As an example, the processor of the diagnostic device may provide, as guidance information, action information including information on additional tests (e.g., description of the additional test, date of the additional test, information on hospitals / medical staff that can perform the additional test, etc.) and / or management information including recommended lifestyle correction goals, recommended dietary information, etc.

[0641]

[0642] Additionally, if the progression of cardiovascular disease in the first subject is predicted to be rapid and the first subject is in a high-risk group, the processor of the diagnostic device may increase the prescription amount for the first subject as prescription information.

[0643]

[0644] In addition, the processor of the diagnostic device can communicate with the aforementioned monitoring device. The processor of the diagnostic device can provide guidance information determined based on the rate of progression of cardiovascular disease to the monitoring device. For example, if the processor of the diagnostic device determines that the rate of progression of cardiovascular disease in a first subject in the low-risk group of cardiovascular disease diagnosis information who has no symptoms and no cardiovascular risk factors is rapid, the processor of the diagnostic device can determine corresponding action information (e.g., additional test information, etc.) and / or management information (e.g., adjusting food intake, target exercise amount, etc.) as guidance information and provide the guidance information to the monitoring device. The monitoring device can provide the guide information obtained from the diagnostic device to the subject and determine whether the monitored information and the guide information match. If the determination result is a match, the monitoring device can provide information indicating that the subject is properly complying with the guide information. If the determination result is a mismatch, the monitoring device can provide a warning to the subject to comply with the guide information.

[0645]

[0646] Additionally, the processor of the diagnostic device may provide guide information using a predetermined database and / or guide information model, as described in 2.1.5. At this time, cardiovascular disease diagnosis information and guide information according to the rate of progression of the cardiovascular disease may be matched and stored in the database, or the guide information model may be trained based on the cardiovascular disease diagnosis information and the guide information according to the rate of progression of the cardiovascular disease. Accordingly, the processor of the diagnostic device may obtain guide information from the database and / or guide information model using the cardiovascular disease diagnosis information and the rate of progression of the cardiovascular disease.

[0647]

[0648] For example, if the QRISK3 score is 8%, which is low risk, and the score of the cardiovascular disease diagnosis information is above a predetermined threshold, the processor of the diagnostic device may determine that the subject's cardiovascular risk is low, but the subject's cardiovascular disease progresses at a faster rate than other low-risk groups. Accordingly, the processor of the diagnostic device may provide information indicating that the cardiovascular disease progresses at a fast rate and / or guidance information accordingly (e.g., guidance information provided when the cardiovascular disease diagnosis information is determined to be medium risk).

[0649] Additionally, if the QRISK3 score is 8%, which is a low-risk group, and the score of the cardiovascular disease diagnosis information is below a predetermined threshold, the processor of the diagnostic device may determine that the subject's cardiovascular risk is low and that the subject's cardiovascular disease progression rate is slow compared to other low-risk groups. Accordingly, the processor of the diagnostic device may provide information indicating that the cardiovascular disease progression rate is slow and / or guidance information accordingly (e.g., guidance information provided when the cardiovascular disease diagnosis information is determined to be low-risk).

[0650] Additionally, depending on the embodiment, different thresholds may be applied to cardiovascular disease diagnostic information based on biomarker-specific results. For example, thresholds for QRISK3 in the low-risk group, intermediate-risk group, and high-risk group may be set differently in the diagnostic device.

[0651]

[0652] Various embodiments of the present specification may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The machine is a device that can call instructions stored in the storage medium and operate according to the called instructions, and may include an electronic device according to the disclosed embodiments. When the instructions are executed by a processor, the processor can perform a function corresponding to the instructions directly or under the control of the processor using other components. The instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" does not mean that it does not include a signal and is tangible, and does not distinguish whether data is stored semi-permanently or temporarily in the storage medium. For example, the "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0653] According to one embodiment, the method according to the various embodiments disclosed in the present specification may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a device-readable storage medium (e.g., Compact Disc Read Only Memory, CD-ROM) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0654] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0655] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In the control method of the diagnostic device, A step of acquiring a retinal image of a subject; and A step of obtaining cardiovascular disease diagnosis information for the subject using a machine learning model based on the retinal image. Including, The above machine learning model includes a first model and a second model, The above first model is a neural network model, The above second model is a regression-based machine learning model. A method for controlling a diagnostic device.

2. In paragraph 1, The first model is trained based on first training data, wherein the first training data includes a plurality of retinal images, and a result value according to a first biomarker corresponding to the first training data, The step of obtaining cardiovascular disease diagnosis information for the above subject is: The retinal image is input into the first model to obtain a first score from the first model, By inputting the first score and the physical information of the subject into the second model, a second score is obtained from the second model, Obtaining the cardiovascular disease diagnosis information based on the second score, A method for controlling a diagnostic device.

3. In paragraph 2, The first biomarker is the coronary artery calcium (CAC) score, The above first score represents the probability that the subject's coronary artery calcium score is 0 or greater. A method for controlling a diagnostic device.

4. In paragraph 2, The step of obtaining cardiovascular disease diagnosis information for the above subject is: By applying a predetermined cutoff value to the second score, one of the multiple grades corresponding to the subject is obtained as the cardiovascular disease diagnosis information. A method for controlling a diagnostic device.

5. In paragraph 4, The above cutoff value is, Based on information about a second biomarker that is at least partially different from the first biomarker, A method for controlling a diagnostic device.

6. In paragraph 5, The above cutoff value is, Based on the population distribution in groups according to multiple grades of the above second biomarker, A method for controlling a diagnostic device.

7. In paragraph 6, The second biomarker is, Including at least one of the PCE score, QRISK3 score, or modified Framingham risk score; A method for controlling a diagnostic device.

8. In paragraph 4, The above cutoff value is, Based on the population distribution in groups according to multiple grades distinguished according to the results of follow-up observation of a given population, A method for controlling a diagnostic device.

9. In paragraph 1, A step of providing guide information for the subject based on the above cardiovascular disease diagnosis information. including more, A method for controlling a diagnostic device.

10. In paragraph 9, The step of providing guide information for the subject based on the above cardiovascular disease diagnosis information is: A control method of a diagnostic device that provides guide information corresponding to the cardiovascular disease diagnosis information using a pre-stored database or guide information model.

11. In paragraph 9, The step of providing guide information for the subject based on the above cardiovascular disease diagnosis information is: Obtain the result value according to the biomarker of the above subject, Providing guide information for the subject using the result value according to the above biomarker and the above cardiovascular disease diagnosis information. A method for controlling a diagnostic device.

12. In paragraph 11, Even if the above cardiovascular disease diagnosis information is the same, if the result value according to the biomarker of the subject is different, the guide information for the subject is different depending on the result value according to the biomarker of the subject. A method for controlling a diagnostic device.

13. In paragraph 11, If the result value according to the biomarker of the above-mentioned subject is a non-high-risk group and the cardiovascular disease diagnosis information is a high-risk level, the guide information provided when the cardiovascular disease diagnosis information is a high-risk level is provided. A method for controlling a diagnostic device.

14. In paragraph 11, When the result value according to the biomarker of the above-mentioned subject is a high-risk group and the cardiovascular disease diagnosis information is a non-high risk grade, the guide information provided when the cardiovascular disease diagnosis information is a non-high risk grade is provided. A method for controlling a diagnostic device.

15. In paragraph 11, Further comprising a step of determining the progression rate of cardiovascular disease of the subject by using at least one of the result value according to the biomarker of the subject or the past cardiovascular disease diagnosis information of the subject. A method for controlling a diagnostic device.

16. In paragraph 15, The step of providing guide information for the subject based on the above cardiovascular disease diagnosis information is: Providing guidance information for the subject using the progression rate of the subject's cardiovascular disease and the cardiovascular disease diagnosis information. A method for controlling a diagnostic device.

17. A recording medium having recorded thereon a program for performing the method of any one of claims 1 to 16.

18. In the diagnostic device, storage module; and Contains at least one processor, At least one processor, Obtaining a retinal image of the subject, Obtaining cardiovascular disease diagnosis information for the subject using the machine learning model stored in the storage module based on the retinal image, The above machine learning model includes a first model and a second model, The above first model is a neural network model, The above second model is a machine learning model based on linear regression. Diagnostic device.

Images