Method and apparatus for evaluating retinal image-based condition using artificial intelligence

Abstract

The embodiments disclose a retinal image-based condition assessment method. The method for assessing the risk of cognitive impairment may include: receiving an input image of a subject's retina and demographic data about the subject by means of an electronic device; preprocessing the input image by means of an electronic device; calculating a rule-based feature vector representing quantitative features of the subject's retinal structure for the preprocessed input image by means of an electronic device; obtaining a latent feature vector of the subject's retina from the input image through a deep learning-based feature extraction model by means of an electronic device; obtaining an integrated feature vector by combining the rule-based feature vector and the latent feature vector by means of an electronic device; obtaining a final class probability vector and model-specific uncertainty statistics for the subject's cardiometabolic and neurological disease states by means of inputting the integrated feature vector into a plurality of classification models learned in different ways and applying dropout masking by means of an electronic device; and outputting output data including risk levels by disease type based on the final class probability vector by means of an electronic device.

Description

Technology Field

[0001] The embodiments of the present disclosure relate to the field of medical diagnosis and artificial intelligence technology, and to a technology for non-invasively predicting the state of cardiometabolic and neurological diseases by analyzing retinal images of a subject acquired from various imaging devices. Background Technology

[0002] Due to the global trend of population aging, the number of patients with Alzheimer's disease (AD), dementia, and the pre-disease stage known as Mild Cognitive Impairment (MCI) is rapidly increasing, causing serious social costs and public health issues. In particular, as MCI serves as an intermediate stage between the normal aging process and early-stage dementia, early screening is of paramount importance, as early diagnosis and appropriate therapeutic intervention during this period can significantly slow the progression of the disease.

[0003] Recently, the medical community has been focusing on the fact that the retina serves as an embryological, structural, and functional extension of the brain's central nervous system. As it has been clinically proven that degenerative changes in the brain are accompanied by atrophy of the retinal ganglion cell layer, decreased microvascular density, changes in vascular torsion, and deformation of vascular diameter, non-invasive diagnostic technologies for cognitive impairment using retinal imaging are gaining attention as next-generation screening tools.

[0004] However, existing retinal analysis technologies have several limitations. First, they often rely on expensive specialized equipment such as optical coherence tomography (OCT), making it difficult to distribute them in low-spec environments such as primary care clinics or local health centers. Second, it has been pointed out that when simple deep learning models are used, it is difficult to clinically explain the basis for the model's judgment, and the reliability or uncertainty of the results produced by the model cannot be quantitatively presented, resulting in a lack of reliability as an auxiliary tool for medical professionals to make a final diagnosis.

[0005] Furthermore, since the quality of retinal images varies significantly depending on the type of imaging equipment or lighting environment, there is an urgent need to develop advanced algorithms capable of consistently standardizing them and analyzing them by harmoniously combining explicit features based on anatomical structures (Handcrafted Features) with abstract pattern information captured by deep learning. Prior Art: U.S. Patent Application Publication US2025 / 0166829 (May 22, 2025) The problem to be solved

[0006] Embodiments of the present disclosure can provide a method for early and non-invasively evaluating the risk of mild cognitive impairment and dementia without separate expensive equipment by utilizing a single retinal image obtained from a standard digital fundus camera or a smartphone-linked imaging device widely distributed in primary care institutions.

[0007] The embodiments of the present disclosure can provide a method that can simultaneously improve the clinical explainability and predictive accuracy of diagnostic results by extracting and combining explicit quantitative features based on anatomical structures such as density, curvature, and diameter of retinal blood vessels and high-dimensional nonlinear latent features captured by a deep learning model in parallel.

[0008] Embodiments of the present disclosure can provide a method for quantitatively calculating model-specific uncertainty statistics representing the confidence level of risk prediction values ​​produced by artificial intelligence, beyond simple class classification, by combining an ensemble learning system using multiple classification models learned in different ways with a dropout masking technique.

[0009] Embodiments of the present disclosure may provide a method for medical professionals to recognize and respond to the possibility of misdiagnosis by artificial intelligence in advance by providing warning information recommending re-imaging or additional examination to the user when the calculated uncertainty score exceeds a threshold, and by visually highlighting retinal regions that contribute to judgment or have high uncertainty.

[0010] The embodiments of the present disclosure can provide a method to help medical staff and subjects intuitively understand professional analysis results and maximize convenience of use in clinical settings by automatically generating and outputting quantified risk scores and uncertainty information as summary text in natural language form through a large language model (LLM).

[0011] The technical problems to be solved in the embodiments are not limited to those mentioned above, and other unmentioned technical problems may be considered by those skilled in the art from the various embodiments described below. means of solving the problem

[0012] According to one embodiment, a retinal image-based condition evaluation method comprises: receiving an input image of a subject's retina and demographic data regarding the subject; preprocessing the input image with respect to the input image by the electronic device; calculating a rule-based feature vector representing quantitative features of the subject's retinal structure with respect to the preprocessed input image by the electronic device; obtaining a latent feature vector of the subject's retina from the input image through a deep learning-based feature extraction model by the electronic device; obtaining an integrated feature vector by combining the rule-based feature vector and the latent feature vector by the electronic device; obtaining a final class probability vector and model-specific uncertainty statistics for the subject's cardiometabolic and neurological disease states by inputting the integrated feature vector into a plurality of classification models learned in different ways and applying dropout masking by the electronic device; and outputting output data including risk levels by disease type based on the final class probability vector by the electronic device.

[0013] The operation of obtaining the final class probability vector and model-specific uncertainty statistics may include: an operation of calculating a model-specific probability distribution by performing multiple inferences on the same input with a portion of the feature dimension probabilistically deactivated through dropout masking for each classification model by the electronic device; an operation of calculating a model-specific average probability vector and model-specific uncertainty statistics based on the model-specific probability distribution of each classification model by the electronic device; and an operation of calculating the final class probability vector based on the model-specific average probability vector by the electronic device.

[0014] The operation of calculating the probability distribution for each model may include applying dropout masking to each classification model that takes the integrated feature vector as input by the electronic device, randomly selecting and deactivating some of the constituent dimensions of the integrated feature vector or some of the internal operation nodes of the classification model according to a pre-set masking probability, and performing multiple forward propagation operations on the same integrated feature vector based on multiple masking patterns in which the dropout masking is applied differently from each other, thereby calculating the probability distribution for each model that includes multiple class probability vectors for each classification model.

[0015] The operation of calculating the average probability vector and model-specific uncertainty statistics may include generating the average probability vector by model by aggregating a plurality of class probability vectors calculated for each classification model by the electronic device and calculating the average value for each class, and calculating the model-specific uncertainty statistics representing the prediction uncertainty of the classification model by calculating at least one of the variance, standard deviation, entropy, or probability interval of the plurality of class probability vectors.

[0016] The operation of calculating the final class probability vector may include the operation of calculating the final class probability vector by applying weights that are pre-set or determined during the learning process to the average probability vector of each classification model by the electronic device, or by performing an operation of combining the average probability vectors of each classification model in a stacking manner.

[0017] The operation of preprocessing the input image may include an operation of performing image standardization processing on the input image by an electronic device, the image standardization process including at least one of resolution and color space normalization, noise removal, and contrast correction.

[0018] The operation of generating the rule-based feature vector may include: an operation of identifying, by means of an electronic device, a retinal structure including a retinal vascular structure and an optic disc region from the input image by applying a deep learning-based segmentation model to the input image in pixel units; an operation of generating, by means of an electronic device, a plurality of quantitative vascular features including at least one of a vascular diameter, vascular curvature, fractal dimension of a vascular network, arteriovenous diameter ratio (AVR), and vascular density, and a plurality of quantitative optic disc features including at least one of a cup-to-disc ratio, optic disc area, optic disc dislocation area, optic disc rim area, or texture feature based on the retinal structure; and an operation of generating the rule-based feature vector by means of an electronic device by combining the plurality of quantitative vascular features, the plurality of quantitative optic disc features, and the demographic data.

[0019] The operation of calculating the plurality of quantitative optic disc features described above may include, by means of an electronic device, calculating a plurality of quantitative vascular features including at least one of a blood vessel centerline length, blood vessel diameter, blood vessel curvature, blood vessel branching angle, fractal dimension of a blood vessel network, arteriovenous diameter ratio (AVR), and blood vessel density according to a predefined mathematical, statistical, or geometric calculation formula based on the retinal structure, and calculating a plurality of quantitative optic disc features including at least one of an optic disc cup-to-disc ratio, optic disc area, optic disc cup-to-disc area, optic disc rim area, circularity index, color distribution statistics, or grayscale co-occurrence matrix-based texture statistics.

[0020] The operation of acquiring the integrated feature vector may include: an operation of acquiring an integrated feature vector containing feature information having different generation principles by combining a rule-based feature vector calculated by mathematical or geometric calculation from the retinal structure and a latent feature vector acquired by a deep learning-based feature extraction network from the input image by the electronic device; an operation of acquiring the integrated feature vector by combining the rule-based feature vector and the latent feature vector in a vector concatenation manner, or by combining them after mapping them to the same feature space through weighted combination or dimensionality reduction transformation by the electronic device; and an operation of acquiring an integrated feature vector containing interpretable information and non-linear pattern information simultaneously by combining the rule-based feature vector containing explicit quantitative information about the anatomical structure of the retina and the latent feature vector containing a high-dimensional pattern representation of the input image by the electronic device.

[0021] The operation of outputting the above output data may include, by the electronic device, calculating at least one of entropy, standard deviation, or variance based on the final class probability vector and the model-specific uncertainty statistics to calculate an uncertainty score, and outputting output data including the uncertainty score together with the disease-specific risk level.

[0022] The operation of outputting the above output data may further include the operation of outputting the above output data, which includes warning information recommending re-imaging or additional examination along with the risk level by disease type when the uncertainty score exceeds the above threshold by the electronic device.

[0023] The operation of outputting the above output data may include, by the electronic device, calculating a probability interval for the probability of each disease type based on the uncertainty statistics for each model, and outputting output data including the probability interval along with the risk level for each disease type.

[0024] The above probability interval can be calculated as at least one of a percentile-based interval or an interval obtained by adding or subtracting a multiple of the standard deviation from the mean value.

[0025] The operation of outputting the above output data may include the operation of outputting output data that includes, together with the risk level for each disease type, a distribution graph or probability interval graph that visually represents the probability distribution for each disease type based on the average probability vector for each model and the final uncertainty statistics by the electronic device.

[0026] The operation of outputting the above output data may include, by the electronic device, identifying the retinal region that contributed to the uncertainty based on the final class probability vector and the model-specific uncertainty statistics, and outputting the output data including an uncertainty-based activation map that visually highlights the retinal region along with the disease-specific risk.

[0027] The operation of outputting the above output data may include the operation of generating text output data by the electronic device, which summarizes the risk level by disease type included in the final class probability vector in a natural language form through an external Large Language Model (LLM). Effects of the invention

[0028] According to the embodiments, the risk of mild cognitive impairment and dementia can be screened early in a non-invasive and low-cost manner by utilizing a single retinal image obtained with a standard fundus camera or a smartphone-linked imaging device distributed to primary care institutions or public health centers without expensive optical coherence tomography (OCT) equipment.

[0029] According to the embodiments, by extracting and combining explicit quantitative indicators based on anatomical structures such as the density, curvature, and fractal dimension of retinal blood vessels and high-dimensional nonlinear latent features captured by deep learning in parallel, it is possible to secure higher diagnostic accuracy than simple image analysis while providing reliable clinical judgment grounds to medical professionals.

[0030] According to the embodiments, by quantitatively calculating not only the model's predicted values ​​but also the uncertainty statistics based thereon through an ensemble learning system and a dropout masking technique, the stability and reliability of the diagnosis can be dramatically improved, such as by recommending re-photography or additional detailed examinations for ambiguous cases that are difficult for artificial intelligence to judge.

[0031] According to the embodiments, by visually highlighting retinal regions that have a decisive influence on risk determination or show high uncertainty through Grad-CAM or uncertainty-based activation maps, medical professionals can intuitively understand the analysis process of artificial intelligence and review it by comparing it with the actual presence or absence of retinal lesions.

[0032] According to the embodiments, by automatically generating and providing summary text in natural language form through a large language model (LLM) for complex numerical data and analysis results, diagnostic content can be easily explained even to subjects lacking specialized knowledge, and the efficiency of writing result reports in the medical field can be significantly improved.

[0033] The effects obtainable from the embodiments are not limited to those mentioned above, and other unmentioned effects can be clearly derived and understood by a person skilled in the art based on the detailed description below. Brief explanation of the drawing

[0034] The accompanying drawings, included as part of the detailed description to aid in understanding the embodiments, provide various embodiments and explain the technical features of the various embodiments together with the detailed description. FIG. 1 is a diagram showing the configuration of an electronic device according to one embodiment. FIG. 2 is a diagram showing the configuration of a program according to one embodiment. FIG. 3 is a conceptual diagram illustrating the overall configuration of a system according to one embodiment. FIG. 4 is a flowchart illustrating the operation of a retinal image-based condition evaluation method according to one embodiment. FIG. 5 is a diagram illustrating an example of preprocessing an image of a retina to calculate a rule-based vector by an electronic device according to one embodiment. FIG. 6 is a diagram illustrating the structure of a software layer of an electronic device that performs a retinal image-based condition evaluation method according to one embodiment. FIG. 7 is a diagram illustrating the structure of an inference unit according to one embodiment. Specific details for implementing the invention

[0035] The following embodiments are combinations of the components and features of the embodiments in a specific form. Each component or feature may be considered optional unless otherwise explicitly stated. Each component or feature may be implemented in a form not combined with other components or features. Additionally, various embodiments may be constructed by combining some components and / or features. The order of operations described in various embodiments may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment.

[0036] In the description of the drawings, procedures or steps that could obscure the essence of the various embodiments were not described, nor were procedures or steps that could be understood by a person of ordinary knowledge in the relevant technical field described.

[0037] Throughout the specification, when a part is described as "comprising" or "including" a component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part," "...unit," and "module" as used in the specification refer to a unit that performs at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software. Additionally, "one (a or an)," "one," "the," and similar related terms may be used in the context describing various embodiments (particularly in the context of the following claims) in both singular and plural forms, unless otherwise indicated in the specification or clearly contradicted by the context.

[0038] Hereinafter, embodiments according to various examples will be described in detail with reference to the accompanying drawings. The detailed description disclosed below, together with the accompanying drawings, is intended to describe exemplary embodiments of various examples and is not intended to represent the only embodiment.

[0039] In addition, specific terms used in various embodiments are provided to aid in understanding the various embodiments, and the use of such specific terms may be modified in other forms within the scope of not departing from the technical concept of the various embodiments.

[0040] FIG. 1 is a diagram showing the configuration of an electronic device according to one embodiment.

[0041] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)). The electronic device (101) may be referred to as a client, terminal, or peer.

[0042] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0043] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model.

[0044] An artificial intelligence model can be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, or a combination of two or more of the above, but is not limited to the examples described above. Artificial intelligence models may include software structures, either additionally or as a substitute, in addition to hardware structures.

[0045] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0046] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0047] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0048] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0049] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0050] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0051] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a motion sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, a biosensor, an acoustic sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0052] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0053] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0054] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0055] The camera module (180) can capture still images and images. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0056] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0057] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0058] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0059] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0060] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0061] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0062] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0063] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0064] The server (108) is connected to an electronic device (101) and can provide services to the connected electronic device (101). Additionally, the server (108) may proceed with the membership registration process, store and manage various information of users who have registered as members accordingly, and provide various purchase and payment functions related to the service. Furthermore, the server (108) may share execution data of service applications running on each of multiple electronic devices (101) in real time so that services can be shared among users. In terms of hardware, this server (108) may have the same configuration as a conventional web server or service server. However, in terms of software, it may include program modules that perform various functions and are implemented through any language such as C, C++, Java, Python, Golang, Kotlin, etc. Additionally, the server (108) generally refers to a computer system that is connected to an unspecified number of clients and / or other servers through an open computer network such as the Internet, receives requests for task execution from clients or other servers, and derives and provides the results of the task, as well as computer software (server program) installed for this purpose. Furthermore, the server (108) should be understood as a broad concept that includes, in addition to the aforementioned server program, a series of application programs running on the server (108) and, in some cases, various databases (DB: Database, hereinafter referred to as "DB") built internally or externally. Accordingly, the server (108) classifies membership registration information and various information and data regarding games, stores them in the DB, and manages them; such DB can be implemented internally or externally of the server (108).Additionally, the server (108) can be implemented using various server programs provided for general server hardware according to operating systems such as Windows, Linux, UNIX, and Macintosh. Representative examples include IIS (Internet Information Server) used in Windows environments and CERN, NCSA, APPACH, and TOMCAT used in UNIX environments to implement web services. Additionally, the server (108) may be linked with an authentication system and a payment system for user authentication of the service or purchase payment related to the service.

[0065] The first network (198) and the second network (199) refer to a connection structure capable of exchanging information between each node, such as terminals and servers, or a network connecting a server (108) and electronic devices (101, 104). The first network (198) and the second network (199) include, but are not limited to, the Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), 3G, 4G, LTE, 5G, Wi-Fi, etc. The first network (198) and the second network (199) may be closed types such as LAN, WAN, etc., but it is preferable that they be open types such as the Internet. The Internet refers to a global open computer first network (198) and second network (199) structure that provides protocols such as TCP / IP protocol, TCP, and UDP (user datagram protocol), and various services existing in the upper layer, namely HTTP (HyperText Transfer Protocol), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), SMTP (Simple Mail Transfer Protocol), SNMP (Simple Network Management Protocol), NFS (Network File Service), and NIS (Network Information Service).

[0066] A database may have a general data structure implemented in the storage space (hard disk or memory) of a computer system using a database management program (DBMS). A database may have a data storage form that allows for the free retrieval (extraction), deletion, editing, and addition of data. A database may be implemented to suit the purpose of an embodiment of the present disclosure using a relational database management system (RDBMS) such as Oracle, Informix, Sybase, and DB2, an object-oriented database management system (OODBMS) such as Gemston, Orion, and O2, and an XML native database such as Excelon, Tamino, and Sekaiju, and may have appropriate fields or elements to achieve its functions.

[0067] FIG. 2 is a diagram showing the configuration of a program according to one embodiment.

[0068] FIG. 2 is a block diagram (200) illustrating a program (140) according to various embodiments. According to one embodiment, the program (140) may include an operating system (142), middleware (144), or an application (146) executable on the operating system (142) for controlling one or more resources of an electronic device (101). The operating system (142) may include, for example, Android™, iOS™, Windows™, Symbian™, Tizen™, or Bada™. At least some of the programs (140) may be preloaded into the electronic device (101) at manufacturing time, for example, or downloaded or updated from an external electronic device (e.g., electronic device (102 or 104), or server (108)) when used by a user. All or part of the program (140) may include a neural network.

[0069] The operating system (142) can control the management (e.g., allocation or reclamation) of one or more system resources (e.g., processes, memory, or power) of the electronic device (101). The operating system (142) may additionally or substantially include one or more driver programs for driving other hardware devices of the electronic device (101), e.g., an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197).

[0070] Middleware (144) may provide various functions to an application (146) so that functions or information provided from one or more resources of an electronic device (101) can be used by the application (146). Middleware (144) may include, for example, an application manager (201), a window manager (203), a multimedia manager (205), a resource manager (207), a power manager (209), a database manager (211), a package manager (213), a connectivity manager (215), a notification manager (217), a location manager (219), a graphics manager (221), a security manager (223), a call manager (225), or a voice recognition manager (227).

[0071] The application manager (201) can, for example, manage the life cycle of the application (146). The window manager (203) can, for example, manage one or more GUI resources used on the screen. The multimedia manager (205) can, for example, identify one or more formats required for the playback of media files and perform encoding or decoding of the corresponding media files among the media files using a codec that matches the selected corresponding format. The resource manager (207) can, for example, manage the source code of the application (146) or the memory space of the memory (130). The power manager (209) can, for example, manage the capacity, temperature, or power of the battery (189) and, using the relevant information, determine or provide relevant information required for the operation of the electronic device (101). According to one embodiment, the power manager (209) can interact with the BIOS (basic input / output system) (not shown) of the electronic device (101).

[0072] The database manager (211) can, for example, create, search, or modify a database to be used by the application (146). The package manager (213) can, for example, manage the installation or update of the application distributed in the form of a package file. The connectivity manager (215) can, for example, manage a wireless or direct connection between the electronic device (101) and an external electronic device. The notification manager (217) can, for example, provide a function to notify the user of the occurrence of a specified event (e.g., an incoming call, a message, or an alarm). The location manager (219) can, for example, manage location information of the electronic device (101). The graphics manager (221) can, for example, manage one or more graphic effects or related user interfaces to be provided to the user.

[0073] The security manager (223) may, for example, provide system security or user authentication. The telephony manager (225) may, for example, manage voice call functions or video call functions provided by the electronic device (101). The voice recognition manager (227) may, for example, transmit user voice data to the server (108) and receive from the server (108) a command corresponding to a function to be performed on the electronic device (101) based on at least part of the voice data, or text data converted based on at least part of the voice data. According to one embodiment, the middleware (244) may dynamically delete some existing components or add new components. According to one embodiment, at least part of the middleware (144) may be included as part of the operating system (142) or implemented as separate software different from the operating system (142).

[0074] The application (146) may include, for example, a home (251), a dialer (253), an SMS / MMS (255), an IM (instant message) (257), a browser (259), a camera (261), an alarm (263), a contact (265), a voice recognition (267), an email (269), a calendar (271), a media player (273), an album (275), a watch (277), a health (279) (e.g., measuring biometric information such as exercise volume or blood sugar), or an environmental information (281) (e.g., measuring atmospheric pressure, humidity, or temperature information). According to one embodiment, the application (146) may further include an information exchange application (not shown) capable of supporting information exchange between the electronic device (101) and an external electronic device. The information exchange application may include, for example, a notification relay application configured to transmit information (e.g., a call, a message, or an alarm) designated to an external electronic device, or a device management application configured to manage the external electronic device. The notification relay application may transmit notification information corresponding to a designated event (e.g., receiving mail) generated in another application of the electronic device (101) (e.g., an email application (269)) to the external electronic device. Additionally or alternatively, the notification relay application may receive notification information from the external electronic device and provide it to the user of the electronic device (101).

[0075] A device management application can control the power (e.g., turn-on or turn-off) or function (e.g., brightness, resolution, or focus) of an external electronic device or a part of its components (e.g., a display module or camera module of the external electronic device) that communicates with the electronic device (101). The device management application can additionally or substantially support the installation, deletion, or updating of applications running on the external electronic device.

[0076] Throughout this specification, the terms neural network, neural network, and network function may be used interchangeably. A neural network may consist of a set of interconnected computational units, which may generally be referred to as "nodes." These "nodes" may also be referred to as "neurons." A neural network is composed of at least two nodes. The nodes (or neurons) constituting neural networks may be interconnected by one or more "links."

[0077] In a neural network, two or more nodes connected via links can form a relative relationship between an input node and an output node. The concepts of input and output nodes are relative; any node in an output node relationship with respect to one node may be in an input node relationship with respect to another node, and vice versa. As previously mentioned, the input node versus output node relationship can be generated based on links. One or more output nodes may be connected to a single input node via links, and vice versa.

[0078] In a relationship between an input node and an output node connected through a single link, the value of the output node can be determined based on data input to the input node. Here, the nodes interconnecting the input node and the output node may have weights. The weights may be variable and may be varied by a user or an algorithm to enable the neural network to perform the desired function. Here, the edges or links interconnecting the input node and the output node have weights that can be variably applied by a user or an algorithm to enable the neural network to perform the desired function. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node may determine its value based on the values ​​input to the input nodes connected to the output node and the weights set on the links corresponding to each input node.

[0079] As described above, a neural network is formed in which two or more nodes are interconnected through one or more links to create input-output node relationships within the network. The characteristics of a neural network can be determined by the number of nodes and links within the network, the relationships between the nodes and links, and the weight values ​​assigned to each link. For example, if two neural networks exist with the same number of nodes and links but different weight values ​​between the links, the two neural networks can be recognized as being different from each other.

[0081] FIG. 3 is a conceptual diagram illustrating the overall configuration of a system according to one embodiment.

[0082] The embodiments of the present disclosure relate to the field of medical diagnosis and artificial intelligence technology, and to a technology for non-invasively predicting the risk of degenerative neurological diseases, including mild cognitive impairment and dementia, by analyzing retinal images of a subject acquired from various imaging devices.

[0083] Embodiments of the present disclosure can provide a method for early and non-invasively evaluating the risk of mild cognitive impairment and dementia without separate expensive equipment by utilizing a single retinal image obtained from a standard digital fundus camera or a smartphone-linked imaging device widely distributed in primary care institutions.

[0084] The embodiments of the present disclosure can provide a method that can simultaneously improve the clinical explainability and predictive accuracy of diagnostic results by extracting and combining explicit quantitative features based on anatomical structures such as density, curvature, and diameter of retinal blood vessels and high-dimensional nonlinear latent features captured by a deep learning model in parallel.

[0085] Embodiments of the present disclosure can provide a method for quantitatively calculating model-specific uncertainty statistics representing the confidence level of risk prediction values ​​produced by artificial intelligence, beyond simple class classification, by combining an ensemble learning system using multiple classification models learned in different ways with a dropout masking technique.

[0086] Embodiments of the present disclosure may provide a method for medical professionals to recognize and respond to the possibility of misdiagnosis by artificial intelligence in advance by providing warning information recommending re-imaging or additional examination to the user when the calculated uncertainty score exceeds a threshold, and by visually highlighting retinal regions that contribute to judgment or have high uncertainty.

[0087] The embodiments of the present disclosure can provide a method to help medical staff and subjects intuitively understand professional analysis results and maximize convenience of use in clinical settings by automatically generating and outputting quantified risk scores and uncertainty information as summary text in natural language form through a large language model (LLM).

[0088] Referring to FIG. 3, the system (300) may include a camera (320) for capturing a retinal image of a subject, a user terminal (310) for receiving demographic data of the subject and the retinal image, a network (330) for transmitting and receiving data with the user terminal (310), an analysis server (340) for analyzing the retinal image received through the network (330) to calculate a retinal image-based state, and an artificial intelligence learning server (360) for enhancing the analysis model of the analysis server (340). In another embodiment, the system (300) may further include a Picture Archiving and Communication System (PACS) or Electronic Medical Record (EMR) system (370) for storing and managing the retinal image and analysis results. Additionally, the system (300) may further include an edge or on-premise server (350) built on an internal network of a medical institution to perform independent inference operations for data security and low-latency processing. However, this structure is merely an example, and the system (300) may not include a camera (320) and may receive the subject's retinal image directly or through a network (330).

[0089] An electronic device (e.g., the electronic device (101) of FIG. 1) that performs a retinal image-based condition evaluation method according to the embodiments can be implemented in various forms of computing devices and network architectures depending on the usage environment or data processing load. For example, the electronic device can be implemented as an on-premise server in the form of a workstation that is directly connected to a standard fundus camera inside a hospital to perform analysis in real time, as well as as a centralized analysis server (300) that utilizes large-scale computing resources in a cloud environment to perform integrated analysis of images collected from multiple medical institutions. In addition, in environments where data security and low latency are emphasized, it can also be implemented in the form of a small edge computing device or a software medical device (SaMD) that is embedded in the device itself or performs inference directly within a smartphone for imaging. Furthermore, the aforementioned electronic device is not limited to a single piece of hardware but can be flexibly expanded through virtualization technology or a container-based distributed architecture, and can play a pivotal role in an integrated diagnostic solution that automates the entire process from the collection of medical data to the storage of diagnostic results by linking with external artificial intelligence learning servers, Picture Archiving and Communication Systems (PACS), and Electronic Medical Records (EMR) systems.

[0090] The form of the system for performing the retinal image-based condition evaluation method according to the embodiments is not limited by the details described in the foregoing detailed description and drawings. Specifically, the system may be implemented as an on-premise server in the form of a workstation that performs real-time analysis by being connected wirelessly or via wired connection to a standard fundus camera within the hospital, as well as as a remote central analysis server system that integrates and processes large-scale images collected from multiple medical institutions by utilizing cloud computing resources. Furthermore, in environments where data security and computational efficiency are emphasized, the system may be implemented as an embedded component within the retinal imaging device itself, as an edge computing device that performs inference independently within a smartphone used for imaging, or as a Software-as-a-Medical (SaMD) based on a mobile application. Moreover, the system is not limited to a single hardware configuration but can be flexibly expanded through virtualization technology or a container-based distributed processing architecture. It may also be provided as an integrated digital healthcare solution that automates the entire process from data collection to diagnostic result feedback by organically linking with external AI learning servers, Hospital Information Systems (HIS), Picture Archiving and Communication Systems (PACS), and Electronic Medical Records (EMR) systems.

[0091] According to one embodiment, the electronic device can receive an input image of the subject's retina and demographic data about the subject.

[0092] Electronic devices can be implemented with a structure in which the hardware layer and the software layer are organically combined. For example, the hardware layer may include communication modules, memory, processors, storage devices, and network interfaces, while the software layer may include data receiving modules, session management modules, file verification modules, and data structuring modules. This is merely an example, and the hardware and software layers of an electronic device can be configured in various architectures.

[0093] The input image may include source image data of the retina and demographic data of the subject. In another embodiment, the input data may include only source image data. The retinal image may be a color fundus image and may be transmitted in various formats such as JPEG, PNG, DICOM, etc. The demographic data may consist of age, gender, educational level, information on underlying diseases, family history, lifestyle information, etc., and may be provided in the form of structured JSON, XML, or database records.

[0094] The input image may not be limited to a color fundus image and may include various forms of image data including the retina or fundus region.

[0095] The electronic device can receive input images such as traditional non-mydriatic or mydriatic fundus camera images, as well as images acquired by a portable fundus imaging device, images acquired via a fundus imaging adapter attached to a smartphone, images acquired by an ultra-wide-angle retinal imaging device, or synthetic retinal images captured from multiple viewing angles. Furthermore, the input image may not be limited to a single still image and may be an image from which a specific frame is extracted from image data consisting of continuous frames.

[0096] The input image may include various image modalities, such as color images in the visible light range, near-infrared-based retinal images, fluorescein angiography (FA) images, tomographic images generated from optical coherence tomography (OCT) that include retinal surface structures, or multispectral images. In such cases, the electronic device can apply preprocessing and normalization processes appropriate for each image modality and convert them into a format suitable for a common analysis pipeline.

[0097] In addition, the input images may include not only original captured footage but also historical footage stored in hospital information systems or external storage, image data provided by cloud-based medical platforms, or retinal images collected from research databases. Even in these cases, the electronic device can perform file format conversion, resolution correction, and color normalization to meet internal analysis standards.

[0098] Therefore, the input image is not limited to specific shooting equipment or specific image formats and can encompass all forms of digital image data including retinal structures, and the electronic device can be configured to accommodate images acquired from various shooting environments and equipment.

[0099] During the processing, the communication module of the hardware layer receives input images from an external imaging device or a user terminal. The processor can reassemble the received data packets, perform file integrity verification, and determine whether there is any damage. Subsequently, the data receiving module of the software layer performs a file format check to determine whether the image format conforms to an analyzable standard and, if necessary, converts it to an internal standard format. The session management module links to the corresponding subject record by matching it with a user identifier, and the storage module can record images and demographic data in temporary or permanent storage.

[0100] The output data may be a source retinal image tensor loaded in memory for use in an internal processing pipeline, and a normalized demographic data structure. The output data becomes input data for a subsequent image preprocessing step, and the demographic information may be kept ready to be combined together in a rule-based feature vector generation step. This may correspond to an initial operation forming a common input base for integrated analysis of image-based features and non-image-based feature information.

[0101] According to one embodiment, the electronic device can preprocess the input image. For example, the electronic device can perform image standardization processing on the input image, including at least one of resolution and color space normalization, noise removal, and contrast correction.

[0102] The electronic device can identify the valid retinal area included in the input image and set a region of interest excluding the background area. The electronic device can detect the field of view using the circular boundary of the retina or the difference in brightness and contrast, and can crop or align the image based on the detected area. The electronic device can maintain a consistent analysis reference coordinate system by correcting the center coordinates of the image and can perform rotation correction or scaling if necessary. In addition, the electronic device can calculate image quality indicators by calculating the degree of focus blur, brightness distribution, ratio of overexposed areas, or the presence of reflected light, and can generate a quality flag by comparing it with a pre-set standard. The output data of the above preprocessing may be a retinal image with the region of interest set and alignment completed, along with corresponding coordinate information and quality indicators, which can be used as input data for subsequent image standardization processing.

[0103] According to one embodiment, the electronic device can produce a rule-based feature vector representing quantitative features of the subject's retinal structure for the preprocessed input image.

[0104] The electronic device can identify the retinal structure, including the retinal blood vessel structure and the optic nerve head region, from the input image on a pixel-by-pixel basis by applying a deep learning-based segmentation model.

[0105] The electronic device can use a retinal image, which has undergone preprocessing and image standardization, as input data. The electronic device can convert the input image into a tensor form to match the input format of a deep learning-based segmentation model and can perform at least one of channel alignment, size normalization, or pixel value scaling. The electronic device can input the input image into a deep learning-based segmentation model to obtain a vascular segmentation result that distinguishes vascular and non-vascular regions on a pixel-by-pixel basis. For example, the electronic device can output a vascular probability map indicating the probability that each pixel is a blood vessel, and can generate a blood vessel mask by applying a threshold value to the probability map.

[0106] Additionally, the electronic device may input the input image into the same or a separate deep learning-based segmentation model to obtain an optic disc segmentation result that distinguishes between the optic disc region and the non-optic disc region at the pixel level. The electronic device may output at least one of a disc mask representing the outer boundary of the optic disc and a cup mask representing a depression within the optic disc, and the disc mask and the cup mask may subsequently be used as input data for calculating the cup-to-disc ratio.

[0107] The electronic device can post-process the blood vessel segmentation results and the optic nerve head segmentation results. During the post-processing process, the electronic device can perform at least one of removing noise pixels, correcting connectivity through morphological operations, restoring discontinuities of thin blood vessels, filling holes within regions, or boundary smoothing. Additionally, the electronic device can generate a skeleton from a blood vessel mask, and the skeleton information can be used as input data for calculating quantitative features such as blood vessel skeleton length, blood vessel diameter, blood vessel curvature, and blood vessel branching angle.

[0108] The output data of the pixel-unit identification described above may include at least one of a vascular mask or vascular probability map representing a vascular structure, a disk mask representing the optic disc region, a cup mask representing the optic disc depression, and corresponding ROI coordinate information. The output data may subsequently be provided as input data for calculating vascular density, fractal dimension, AVR, optic disc area, optic disc depression area, optic disc rim area, etc., during a rule-based feature calculation process.

[0109] The electronic device can calculate a plurality of quantitative vascular features including at least one of a blood vessel diameter, blood vessel curvature, fractal dimension of a blood vessel network, arteriovenous diameter ratio (AVR), and blood vessel density based on the retinal structure, and a plurality of quantitative optic disc features including at least one of a cup-to-disc ratio, optic disc area, optic disc depression area, optic disc rim area, or texture feature.

[0110] Based on the retinal structure, the electronic device may calculate a plurality of quantitative vascular features including at least one of a blood vessel centerline length, blood vessel diameter, blood vessel curvature, blood vessel branching angle, fractal dimension of the blood vessel network, arteriovenous diameter ratio (AVR), and blood vessel density according to a predefined mathematical, statistical, or geometric calculation formula, and may calculate a plurality of quantitative optic disc features including at least one of an optic disc cup-to-disc ratio, optic disc area, optic disc cup-to-disc area, optic disc rim area, circularity index, color distribution statistics, or grayscale co-occurrence matrix-based texture statistics.

[0111] The calculated vascular and optic disc features can be configured as scalar values ​​or vectors, respectively, and can subsequently be used as input data to construct rule-based feature vectors. This quantification process, based on mathematical, statistical, and geometric calculations, can serve as a foundation for explicitly quantifying the anatomical structure of the retina to ensure interpretability.

[0112] The electronic device can generate the rule-based feature vector by combining the plurality of quantitative vascular features, the plurality of quantitative optic disc features, and the demographic data.

[0113] The input data may consist of a set of quantitative vascular features and a set of optic disc features derived during the retinal structure analysis step, as well as the subject's demographic data. Quantitative vascular features may include, for example, vascular centerline length, mean diameter, curvature, branching angle, fractal dimension, arteriovenous-to-vein diameter ratio (AVR), vascular density, etc., and each feature may be expressed as a scalar value or in the form of multiple statistics. Optic disc features may include the optic disc cup-to-disc ratio, disc area, cup area, rim area, roundness index, color mean and variance, and texture indices based on a grayscale co-occurrence matrix. The demographic data may consist of variables that may correlate with disease risk, such as age and gender.

[0114] For example, the input data may be 1) a set of multiple quantitative vascular features consisting of blood vessel centerline length, blood vessel diameter, blood vessel curvature, blood vessel branching angle, fractal dimension, arterial-vein diameter ratio (AVR), blood vessel density, etc., 2) a set of multiple quantitative optic disc features consisting of optic disc cup-to-disc ratio, optic disc area, optic disc cup-to-disc area, optic disc rim area, circularity index, color statistics, texture statistics, etc., and 3) demographic data consisting of age, gender, etc. However, this data format is merely an example and various forms of input data may be applied.

[0115] The electronic device can normalize each of the above features so that they can be combined in the same computational space. For example, mean and standard deviation-based standardization or min-max normalization can be applied to compensate for scale differences between features having different units. Categorical data can be converted into binary vector form, and continuous demographic data can be included as numerical dimensions as is.

[0116] The electronic device can construct a single one-dimensional vector by concatenating normalized vascular feature vectors, optic disc feature vectors, and demographic feature vectors according to a predefined order. In this case, each feature can be placed at a fixed index position, and the dimension of the vector can be determined by the total number of features included. If necessary, specific features can be multiplied by weights to reflect their importance, but rule-based feature vectors can also be generated using a simple concatenation method.

[0117] The rule-based feature vector described above may be a quantitative representation that explicitly reflects the anatomical structure of the retina and the demographic characteristics of the subject, and each dimension may maintain clinical significance in an interpretable manner. The vector may subsequently be combined with a deep learning-based latent feature vector to form an integrated feature vector, and may be used as input to provide foundational data for simultaneously utilizing interpretable information and non-linear pattern information. The rule-based feature vector may be a numerical representation that explicitly reflects the anatomical structure of the retina, and each dimension may have clinically interpretable significance. The vector may subsequently be combined with a deep learning-based latent feature vector to form an integrated feature vector, and may be used as input data. That is, the output data of this step is a quantitative representation that integrates structural features with ensured interpretability and demographic information, and this can be directly provided as input to a subsequent classification model.

[0118] According to one embodiment, the electronic device can obtain a latent feature vector of the subject's retina from the input image through a deep learning-based feature extraction model.

[0119] The input data may be a retinal image that has undergone image standardization processing. The electronic device may convert the input image into a tensor form to match the input format of a deep learning-based feature extraction model and may perform preprocessing including pixel value normalization, channel alignment, or batch dimension expansion. The deep learning-based feature extraction model may include a convolutional neural network (CNN), a Vision Transformer, or a combination thereof, and may be pre-trained or learned via supervised learning to automatically learn high-dimensional patterns from the retinal image.

[0120] The electronic device can pass the input image through the forward propagation path of the model to extract the output of an intermediate layer (feature map) or a final embedding layer. For example, the electronic device can use the result of global average pooling prior to the last fully connected layer as a feature vector, which can be represented as a fixed-dimensional latent feature vector. The latent feature vector may include non-linear characteristics that are difficult for humans to explicitly define, such as the arrangement of blood vessels in the retina, color distribution, texture patterns, and microstructural deformation.

[0121] The electronic device can normalize the aforementioned latent feature vector so that it can be utilized in subsequent steps, for example, by applying L2 normalization to maintain a constant vector size. Additionally, computational efficiency can be improved by applying dimensionality reduction techniques as needed.

[0122] The output data of the above process may be a fixed-dimensional latent feature vector, which has a generation principle different from that of a rule-based feature vector obtained by directly calculating the anatomical structure of the retina. The latent feature vector can subsequently be combined with the rule-based feature vector to form an integrated feature vector, and can complementarily provide non-linear pattern information.

[0123] According to one embodiment, an electronic device can obtain an integrated feature vector by combining the rule-based feature vector and the latent feature vector.

[0124] The electronic device can combine a rule-based feature vector calculated by mathematical, statistical, or geometric calculations from the retinal structure and a latent feature vector obtained by a deep learning-based feature extraction network from the input image to obtain an integrated feature vector containing feature information having different generation principles.

[0125] The electronic device may obtain the integrated feature vector by combining the rule-based feature vector and the latent feature vector using a vector concatenation method, or by combining them after mapping them to the same feature space through weighted combination or dimensionality reduction transformation. These methods are merely examples, and various methods may be applied to the combination.

[0126] The electronic device can obtain an integrated feature vector that simultaneously includes interpretable information and non-linear pattern information by combining the rule-based feature vector containing explicit quantitative information about the anatomical structure of the retina and the latent feature vector containing a high-dimensional pattern representation of the input image.

[0127] According to one embodiment, the electronic device can input the integrated feature vector into a plurality of classification models learned in different ways and apply dropout masking to obtain a final class probability vector and model-specific uncertainty statistics for the subject's cardiometabolic and neurological disease status.

[0128] The above plurality of classification models may include a neural network-based classification model including a dropout layer, or a classification model configured to implement dropout masking by probabilistically masking input feature dimensions.

[0129] The electronic device can calculate a probability distribution for each model by performing multiple inferences on the same input with some of the feature dimensions probabilistically disabled through dropout masking for each classification model.

[0130] For each classification model that takes the integrated feature vector as input, the electronic device applies dropout masking by randomly selecting and deactivating some of the constituent dimensions of the integrated feature vector or some of the internal operation nodes of the classification model according to a pre-set masking probability, and by performing multiple forward propagation operations on the same integrated feature vector based on multiple masking patterns in which the dropout masking is applied differently from each other, the device can calculate a model-specific probability distribution including multiple class probability vectors for each classification model.

[0131] Electronic devices may include multiple classification models having different learning principles and decision structures, using an integrated feature vector as input. For example, a Support Vector Machine (SVM) can distinguish classes based on a maximum margin hyperplane and form non-linear boundaries by applying a kernel function. A Random Forest can train multiple decision trees on a random feature subset and bootstrap samples, and perform predictions through majority or average probability. Gradient Boosting-based models (e.g., Gradient Boosting, XGBoost, LightGBM) can produce a final prediction through a weighted sum of weak learners sequentially trained to compensate for the error of the previous tree. Additionally, neural network-based classifiers, such as Multi-Layer Perceptrons (MLPs), can perform non-linear transformations through hidden layers and estimate class probabilities in a high-dimensional representation space. In this way, models based on different structures and assumptions can be configured in parallel, which can have the effect of mitigating bias regarding specific data distributions or noise conditions and improving prediction stability.

[0132] Alternatively, the plurality of classification models may be multiple models that share the same / similar model structure but are trained independently with different training data splits (cross-validation folds) and / or different initializations. In addition, for non-neural network-based classification models (SVM, Random Forest, Gradient Boosting, etc.), dropout masking may be implemented as input feature masking applied to the constituent dimensions of the integrated feature vector.

[0133] Monte Carlo Dropout is a method that approximates a probabilistic prediction distribution by activating dropout not only during the training phase but also during the inference phase. While general dropout probabilistically disables hidden layer nodes or parts of input features to prevent overfitting during training, Monte Carlo Dropout maintains the same masking during inference and can perform multiple forward propagations on the same input. In this case, different masking patterns can be applied in each iteration, and as a result, a sample set of class probability vectors can be generated. The device can use the mean of these samples as the prediction probability and calculate variance or entropy as an uncertainty indicator. This technique can have the effect of approximating the posterior distribution of a Bayesian neural network and has the advantage of quantifying the reliability of the model by providing predictions based on probability distributions rather than a single decision value.

[0134] For multiple classification models to which dropout is applied, the electronic device can activate a dropout mask and probabilistically sample said dropout mask. Here, the dropout mask may be implemented as a binary vector or binary tensor specifying “maintain (1)” or “block (0)” at the level of a hidden unit or neuron of the classification model, and nodes of the classification model to which dropout is not applied may be configured to always remain active by treating them as an identity mask where the mask is all 1. For example, in the t-th forward propagation of the k-th classification model, the sublayer to which dropout is applied Masks for It can be sampled by the Bernoulli distribution.

[0135]

[0136] Here Is It is the dropout probability of the i-th sublayer, and the larger the value, the more units are blocked, inducing the model to have various valid parameter states. The function of is to generate “multiple different predictions within a single model” by probabilistically changing the forward propagation path even when the same input is given, and its effect is to reveal epistemological uncertainty (uncertainty in areas where the model has not sufficiently generalized from the training data) as the spread of the probability distribution. In addition, since the probabilistically sampled mask can change with every inference iteration, it can provide an ensemble effect similar to different valid models being applied sequentially as t increases.

[0137] For example, an electronic device can calculate a probability distribution for each model by performing multiple inferences on the same input with some of the feature dimensions probabilistically disabled through dropout masking using the following mathematical formula for each classification model.

[0138] [Mathematical Formula]

[0139]

[0140]

[0141] Here can be an integrated feature vector, can be an index of a classification model, and There can be several classification models. Is It can be a dropout mask vector generated in the nth iteration of inference, where each element is a pre-set masking probability It can be sampled as 0 or 1 depending on. can mean element-wise product, and may mean an input with some dimensions disabled. Is It can mean the forward propagation operation of the i-th classification model, and can be a logit vector, and can be a probability vector by class. Is It can be the average probability vector of the iterative inference results for the i-th classification model, and can be the variance-based uncertainty statistic (variance vector by class) of the iterative inference result. The electronic device Uncertainty scores for each model can be calculated based on this, for example, the sum or mean of class variances can be defined as model uncertainty, or The entropy of can also be calculated and used as uncertainty.

[0142] The electronic device is the average probability vector of each classification model. Combine the final class probability vector It can calculate [the result] and perform weighted combination by assigning higher weights to models with lower uncertainty. This method is merely an example, and the weights applied to the results output by the classification model can be determined in various ways. In addition, the electronic device can quantitatively provide the reliability of the prediction results by including not only the final class probability vector but also model-specific uncertainty statistics or combined final uncertainty statistics in the output data.

[0143] The electronic device can calculate or retrieve model weights corresponding to each classification model based on a validation performance indicator that includes at least one of classification accuracy, loss value, or AUC for the validation data of each classification model. Here, the validation performance indicator is a numerical value representing the generalization performance of each classification model and is converted into a weight that reflects the relative confidence between models. For example, performance score Using (model performance score), normalize weights as follows (Model weights) can be calculated.

[0144]

[0145] Here is the number of classification models, and The function of this step is to adjust the joint weights so that the average probability vector of a high-performing classification model contributes more significantly to the final prediction. The effect of this step is to increase robustness against changes in data distribution and prediction stability by limiting the influence of models that happen to perform poorly in a specific fold or specific distribution.

[0146] The electronic device can calculate the average probability vector for each model and the uncertainty statistics for each model based on the probability distribution for each classification model.

[0147] The electronic device aggregates a plurality of class probability vectors calculated for each of the above classification models to generate an average probability vector for each model by calculating an average value for each class, and can calculate at least one of the variance, standard deviation, entropy, or probability interval of the plurality of class probability vectors to calculate uncertainty statistics for each model representing the prediction uncertainty of the above classification models.

[0148] In the above operation, the input data is a specific classification model Performed with different dropout masking applied to A set of class probability vectors obtained from iterative inference It could be. Each is the number of classes It may be a vector containing probability values ​​corresponding to, and each element may be a predicted probability for a specific disease type (e.g., normal, mild cognitive impairment, moderate cognitive impairment, etc.).

[0149] The electronic device aggregates the above probability vectors by class to obtain the average probability vector per model It can create. This applies to each class About It can be calculated as follows. The above average probability vector may correspond to the expected value of the dropout-based iterative inference result and may provide a more stable prediction probability compared to a single inference result.

[0150] Electronic devices can calculate variance or standard deviation to quantify prediction uncertainty. Class The variance for is It can be defined as such, and the standard deviation can be calculated as its square root. Since the above variance or standard deviation represents the variability of probability values ​​between iterative inferences, a larger value may indicate higher uncertainty regarding the prediction of the corresponding class.

[0151] Also, electronic devices are average probability vectors entropy The above entropy can be calculated. The entropy represents the degree of dispersion of the probability distribution and can have a lower value as the probability is concentrated in a specific class, and a higher value as it is evenly distributed across multiple classes. This can be used as an indicator reflecting the model's prediction confidence.

[0152] Electronic devices can also calculate probability intervals. For example, they can calculate the upper and lower percentile ranges (e.g., 5%–95%) for each class, or the form of adding or subtracting multiples of the standard deviation from the mean value. A confidence interval can be defined. Such probability intervals can intuitively express the range of variation in prediction results and can be used as a criterion for determining warnings or re-examination recommendations in subsequent steps.

[0153] The average probability vector and uncertainty statistics for each model calculated in this way are, respectively, classification models Representative prediction results and reliability information can be provided simultaneously. This information can be used as a basis for weight calculation in the subsequent weighted combination step between multiple models, and can be utilized as input data for calculating the final class probability vector and final uncertainty statistics.

[0154] As such, a structure that applies dropout techniques to different classification models and then ensembles them can have the effect of simultaneously securing model diversity and stochastic uncertainty estimation.

[0155] First, different classification models may differ in their learning principles, assumptions, and methods of forming decision boundaries. For example, Support Vector Machines can form margin-based decision boundaries, Random Forests can perform non-linear partitioning through multiple tree structures, and Gradient Boosting series can refine predictions by sequentially correcting residuals. Neural network-based classifiers can learn complex patterns through high-dimensional non-linear mapping. When models with such different inductive biases are configured in parallel, the likelihood of overfitting or biases regarding specific data patterns being confined to individual models increases, and the error correlation between models may decrease.

[0156] If dropout techniques are applied to each model, stochastic model transformations can occur even within a single model. That is, whenever iterative inference is performed on the same integrated feature vector, some input dimensions or hidden nodes may be randomly deactivated, thereby generating different prediction result samples. This can have the effect of using a set of multiple similar but distinct submodels rather than a single fixed model. Consequently, each classification model can yield uncertainty statistics based on variance or entropy, along with an average probability vector.

[0157] Subsequently, the electronic device can combine the average probability vectors of each classification model in an ensemble manner. Simple average, weighted average, or stacking techniques can be applied, and in particular, uncertainty statistics derived from dropout-based iterative inference can be reflected in the weight calculation. For example, relatively high weights can be assigned to models with low prediction variance, and low weights to models with high variance. This method can have the effect of automatically reducing the influence of models with high uncertainty.

[0158] This structure can have the following combined effects. First, the simultaneous existence of structural diversity among models and dropout-based stochastic diversity can lower the correlation of prediction errors and reduce total variance. Second, it can mitigate the problem of overconfidence in a single model and improve the calibration characteristics of prediction probabilities. Third, since prediction reliability can be quantified using variance or entropy calculated from iterative inference, it is possible to rationally decide whether to recommend re-imaging or additional examinations along with retinal image-based condition assessment.

[0159] Consequently, a structure that applies dropout techniques to different classification models and then ensembles them can provide an integrated effect that not only improves prediction accuracy compared to a single model but also simultaneously enhances prediction stability, generalization performance, and the ability to quantify uncertainty.

[0160] The electronic device can calculate the final class probability vector based on the average probability vector for each model.

[0161] The electronic device can calculate the final class probability vector by applying weights that are pre-set or determined during the learning process to the average probability vectors of each classification model to perform weighted summation, or by performing an operation that combines the average probability vectors of each classification model in a stacking manner.

[0162] Input data is a classification model Average probability vector calculated for each It can be. Each average probability vector can represent the expected probability distribution calculated by the corresponding model through dropout-based iterative inference, and the number of classes It can have a corresponding dimension. The electronic device integrates these average probability vectors to form a single final class probability vector It can produce.

[0163] In the case of the weighted summation method, the electronic device has weights for each classification model The above weights can be set, and these weights may be set empirically in advance or determined during the training process based on performance metrics (e.g., accuracy, AUC, etc.) for validation data. Additionally, the above weights may be set to be inversely proportional to the uncertainty statistics of each model. For example, higher weights may be assigned to models with lower variance or entropy. In this case, the final class probability vector can be calculated as follows.

[0164]

[0165] Calculated as above It can include weighted average probabilities for each class, which can mitigate bias in individual models and improve stability.

[0166] In the case of the stacking method, the electronic device is each average probability vector You can utilize a high-level meta-classifier that uses as an input feature. That is, The final class probability can be calculated by constructing it into a single expansion vector and inputting it into a meta-model. The meta-model may include logistic regression, a linear classifier, or a neural network structure, and can be trained to prevent overfitting through cross-validation. The stacking method has the advantage of being able to learn mutually complementary patterns between models compared to a simple weighted average.

[0167] As described above, the electronic device can integrate the average probability vectors of multiple models by selecting at least one combination method among weighted summation or stacking, and the final class probability vector calculated as a result can represent the final predicted probability distribution for the subject's cardiometabolic and neurological disease status. The final class probability vector can subsequently be utilized as input data for the stages of calculating risk by disease type, quantifying uncertainty, and generating warning information.

[0168] According to one embodiment, the electronic device can output output data including risk levels by disease type based on the final class probability vector.

[0169] In the above operation, the input data may be a final class probability vector calculated by combining multiple classification models, and each probability value may correspond to a specific disease type or cognitive impairment stage. The electronic device may interpret the probability value corresponding to each class as a risk level directly, or convert it into a score between 0 and 100 to aid user understanding. Additionally, the electronic device may assign grade information, such as low risk, medium risk, and high risk, according to predefined standard intervals. The output data may be configured in the form of structured data including probabilities by disease type, scored risk levels, grade information, and judgment results, and may be displayed on a user terminal or a report screen for medical staff.

[0170] The electronic device can calculate an uncertainty score by calculating at least one of entropy, standard deviation, or variance based on the final class probability vector and the model-specific uncertainty statistics, and output data including the uncertainty score together with the disease-specific risk.

[0171] In the above operation, the electronic device can quantify the reliability of the prediction using the average probability and variability calculated from the iterative inference results of each classification model. For example, a low uncertainty score can be assigned if class probabilities are concentrated in a specific disease type and the range of variability is small, while a high uncertainty score can be assigned if probabilities are distributed across multiple disease types or if there is a large deviation between iterative inferences. The uncertainty score can be expressed as a single numerical value or as an individual indicator for each disease type, and can be included in the report along with the risk level for each disease type to simultaneously provide the confidence level of the prediction result.

[0172] For example, the electronic device may generate output data including information that guides at least one of re-imaging, re-examination, additional examination, request for specialist reading, or withholding interpretation of results when model-specific uncertainty statistics or uncertainty information based thereon is greater than or equal to a threshold value.

[0173] According to the embodiments, the electronic device can non-invasively and at low cost screen for the risk of mild cognitive impairment and dementia early by utilizing a single retinal image obtained with a standard fundus camera or a smartphone-linked imaging device distributed to primary care institutions or public health centers without expensive optical coherence tomography (OCT) equipment.

[0174] According to the embodiments, by extracting and combining explicit quantitative indicators based on anatomical structures such as the density, curvature, and fractal dimension of retinal blood vessels and high-dimensional nonlinear latent features captured by deep learning in parallel, it is possible to secure higher diagnostic accuracy than simple image analysis while providing reliable clinical judgment grounds to medical professionals.

[0175] According to the embodiments, by quantitatively calculating not only the model's predicted values ​​but also the uncertainty statistics based thereon through an ensemble learning system and a dropout masking technique, the stability and reliability of the diagnosis can be dramatically improved, such as by recommending re-photography or additional detailed examinations for ambiguous cases that are difficult for artificial intelligence to judge.

[0176] According to the embodiments, by visually highlighting retinal regions that have a decisive influence on risk determination or show high uncertainty through Grad-CAM or uncertainty-based activation maps, medical professionals can intuitively understand the analysis process of artificial intelligence and review it by comparing it with the actual presence or absence of retinal lesions.

[0177] According to the embodiments, by automatically generating and providing summary text in natural language form through a large language model (LLM) for complex numerical data and analysis results, diagnostic content can be easily explained even to subjects lacking specialized knowledge, and the efficiency of writing result reports in the medical field can be significantly improved.

[0178] Terms are defined below.

[0179] The subject refers to the patient being diagnosed. The input image is an image of the subject's eyeball and may include a retinal image. For example, the input image may include a fundus image acquired by a fundus camera or a smartphone-linked imaging device, but this is merely an example and may include images of various formats. The input image is not limited to two-dimensional color fundus images acquired through a standard digital fundus camera or a smartphone-linked fundus imaging device. As long as the input image allows for the identification of structural features of the subject's eyeball or retina, it may be widely applied without restrictions on the shooting method or image data format.

[0180] For example, input images may include images acquired through various ophthalmic imaging devices such as optical coherence tomography (OCT), optical coherence tomography angiography (OCTA), scanning laser ophthalmoscopy (SLO), ultra-widefield fundus imaging, fluorescein fundus angiography (FAG), or indocyanine green angiography (ICGA). Additionally, red-free fundus images using special lighting filters, multispectral images, or general ocular images captured by combining a separate optical adapter with a smartphone's standard camera module may also be used as input images. Furthermore, the format of the input images can encompass everything from general general-purpose image formats to medical imaging standard formats such as DICOM, black-and-white or color image formats, and 2D cross-sectional or projected image formats extracted from 3D volume data. That is, the system and method according to one embodiment are implemented to receive and process any image acquired by any imaging technique or format as an input image for preprocessing and feature extraction, provided that the data is capable of extracting biomarkers for analyzing the subject's cardiac metabolic and neurological disease state, such as retinal vascular structures or optic nerve head regions.

[0181] Demographic data refers to data containing basic personal information, such as the subject's age and gender. Preprocessing is the process of correcting quality deviations in fundus images and transforming them to be suitable for analysis. Resolution normalization is a process that aligns images acquired from various equipment models to a specific standard resolution. Color space normalization is a process that standardizes image colors to meet the input requirements of deep learning models. Noise removal is a process that improves quality by removing unnecessary noise from the image. Contrast correction is a quality enhancement process that adjusts image brightness and contrast to ensure structural features are clearly visible. Image standardization is a series of processes that consistently corrects deviations across various fundus imaging equipment, including resolution and color space normalization, noise removal, and contrast correction. Retinal structure refers to major anatomical structures, including retinal vascular structures and the optic disc region. Quantitative features are indicators extracted by quantifying retinal blood vessels and optic disc structures using engineering and mathematical methods. Rule-based feature vectors are vectors generated by connecting explicit numerical values, such as quantitative vascular features and optic disc features, in a defined order.

[0182] A deep learning-based segmentation model is a neural network model that precisely separates retinal structures at the pixel level. The retinal vascular structure is the arrangement and morphological network of microvessels within the retina, including arteries and veins. The optic disc region is an anatomical area that includes the optic disc and the optic disc cupping. Vascular diameter is the average thickness of a vascular segment measured based on the midline. Vascular curvature is an indicator that calculates the degree of bending of the vascular midline using methods such as the ratio of the actual curve length to the straight-line distance. The vascular network is the branching and connection structure of all blood vessels distributed in the retina. Fractal dimension is a numerical value that quantifies the geometric complexity of the vascular network branching structure through algorithms such as box counting. The arterial-vein diameter ratio, or AVR, is the ratio between the average equivalent diameters of retinal arteries and veins. Vascular density is the ratio of the number of pixels segmented by vessels to the total number of pixels within a specific region of interest. The cup-to-disc ratio is the ratio of the area or dimension of the optic disc cupping to the area or dimension of the optic disc. The optic disc area is the total area of ​​the optic disc region calculated through elliptical fitting, etc. The optic disc depression area is the area of ​​the depressed region within the optic disc. The optic disc rim area is the area of ​​the rim region obtained by subtracting the optic disc depression area from the optic disc area. Texture features are an indicator calculated based on pixel patterns, such as those of the optic disc region, using a grayscale co-occurrence matrix, etc.

[0183] A formula refers to a mathematical, statistical, or geometric algorithm or formula applied to derive quantitative features from retinal structures. Vascular centerline length is the actual pixel length of the skeletal line extracted from the segmented vascular mask. Vascular branching angle refers to the angle at the intersection where blood vessels branch out. The circularity index is an indicator representing how close the shape of areas such as the optic disc rim or avascular regions is to a perfect circle. Color distribution statistics are statistical figures calculated based on color histograms of areas such as the optic disc region. Grayscale co-occurrence matrix-based texture statistics are indicators that quantify the spatial distribution characteristics of pixels, such as contrast, uniformity, and entropy. A deep learning-based feature extraction model is an artificial intelligence model that extracts atypical patterns from the entire input image as high-dimensional vectors.

[0184] Latent feature vectors are high-dimensional feature representation embeddings automatically extracted from images through the training of a deep learning model. Acquisition is the process of deriving feature vectors or data through computation or model inference. An integrated feature vector is a single vector formed by combining an interpretable rule-based feature vector with a latent feature vector extracted by deep learning. A deep learning-based feature extraction network is a neural network structure used to generate latent feature vectors. Feature information refers to a set of input variables derived from retinal images and patient data for risk prediction. Vector concatenation is a combination method that joins different feature vectors in a fixed order to form a single, higher-dimensional vector. Weighted concatenation is a method used when combining different feature vectors or probability values ​​by assigning different weights based on the importance or performance indicators of each element and summing them. Dimensionality reduction transformation is a process that reduces the vector size while preserving key information before inputting the integrated feature vector into a classifier.

[0185] A homogeneous feature space is a mathematical reference dimension fitted through transformations to combine vectors with different generative principles. A mapping refers to a mathematical function or operation that transforms data or vectors into a specific feature space. Anatomical structures are visually identifiable biological forms of the retina, such as blood vessels and optic discs. Explicit quantitative information is feature information calculated by algorithms into numerical values ​​that humans can clearly interpret, such as diameter and density. High-dimensional pattern representations are complex and abstract image feature embeddings internally extracted by deep learning models. Interpretable information is an indicator whose meaning and derivation basis medical professionals can intuitively understand in light of clinical knowledge. Non-linear pattern information refers to minute and complex pixel-level changes within the retina that are difficult to explain by simple mathematical relationships.

[0186] A classification model is a machine learning algorithm that takes an integrated feature vector as input and predicts class probabilities. Dropout masking is a technique that ensures diversity in inference by randomly deactivating the model's internal computation nodes or feature dimensions according to set probabilities. Cardiometabolic and neurological disease states represent information indicating the applicability and severity of cardiovascular, metabolic, and brain / cognitive-related diseases. For example, cardiometabolic and neurological disease states may include conditions related to cognitive impairment, dementia, mild cognitive impairment, cerebrovascular disease, cardiovascular disease, metabolic disease, or systemic vascular disease. The final class probability vector is the set of final predicted probabilities for each cognitive impairment class derived by combining the probabilities output from multiple classification models. Model-specific uncertainty statistics are numerical values ​​of prediction uncertainty, such as the variance, standard deviation, and entropy of the probability distribution obtained through Monte Carlo dropout inference. Feature dimensions refer to the number or axes of individual feature elements that constitute the integrated feature vector. Model-specific probability distribution is the scattering of multiple class prediction probability values ​​generated by iterative inference by a single model with dropout applied. Constituent dimensions are the detailed element items that make up the feature vector. Internal computation nodes are computational units, such as neurons, that constitute the interior of a deep learning or machine learning model. The masking probability is the probability value of a random selection to disable a specific node or dimension when dropout is applied.

[0187] A masking pattern is a specific combination of states of nodes that are activated or deactivated through dropout. The forward propagation operation is the computational process in which input data passes through a model to derive an output prediction value. The average value by class is the arithmetic mean of the probability values ​​for the same disease class among the various inference results derived through dropout. The average probability vector by model is the result vector obtained by taking the average of multiple probability vectors iteratively calculated by a single model. A probability interval is a range of values ​​where a predicted probability is estimated to exist, using percentiles or the mean and standard deviation. Prediction uncertainty is a measure indicating how uncertain an AI model's judgment is, and it is quantified using variance or entropy. Weighted summation is a combination method that adds up the probability values ​​multiplied by weights assigned based on the cross-validation performance of each model. The stacking method is a technique that derives a final conclusion by using the output probabilities of multiple base classifiers as input to a meta-model. The risk level by disease is a numerical score representing the likelihood that a subject falls into the categories of normal, mild cognitive impairment, or dementia.

[0188] The output data is the final analysis result provided, including final probabilities, risk levels, visualization maps, and text-based diagnostic descriptions. The uncertainty score is a numerical indicator designed to warn of potential misdiagnosis by calculating the spread of the predicted probability distribution. The threshold is a specific numerical value that serves as a judgment criterion for classifying risk or triggering uncertainty warnings. Percentile-based intervals are probability ranges set based on specific percentiles when the predicted distribution is arranged in order of size. The probability distribution by disease type represents the distribution of probability values ​​obtained through iterative inference by the model for a specific disease. The distribution graph is a chart that visually represents the variance and uncertainty of the probability. The probability interval graph displays the average predicted probability along with the calculated uncertainty probability intervals. The uncertainty-based activation map is a map that overlays colors to highlight retinal regions where the model perceived significant uncertainty during judgment. The large language model is an artificial intelligence model that learns from vast amounts of text data to generate summary text based on analysis results. Natural language form refers to a format with sentence structures that humans can read and understand in everyday life, rather than mechanical numerical values. The text output data is explanatory text written in sentence form, synthesizing the calculated risk scores, quantitative indicators, and other factors.

[0190] FIG. 4 is a flowchart illustrating the operation of a retinal image-based condition evaluation method according to one embodiment.

[0191] According to one embodiment, in operation (S401), the electronic device can receive an input image of the subject's retina and demographic data about the subject.

[0192] According to one embodiment, in operation (S402), the electronic device can preprocess the input image with respect to the input image.

[0193] According to one embodiment, in operation (S403), the electronic device can produce a rule-based feature vector representing quantitative features of the subject's retinal structure for the preprocessed input image.

[0194] According to one embodiment, in operation (S404), the electronic device can obtain a latent feature vector of the subject's retina from the input image through a deep learning-based feature extraction model.

[0195] According to one embodiment, in operation (S405), the electronic device can obtain an integrated feature vector by combining the rule-based feature vector and the latent feature vector.

[0196] According to one embodiment, in operation (S406), the electronic device can input the integrated feature vector into a plurality of classification models learned in different ways and apply dropout masking to obtain a final class probability vector and model-specific uncertainty statistics for the subject's cardiometabolic and neurological disease state.

[0197] The electronic device can calculate a probability distribution for each model by performing multiple inferences on the same input with some of the feature dimensions probabilistically disabled through dropout masking for each classification model.

[0198] For each classification model that takes the integrated feature vector as input, the electronic device applies dropout masking by randomly selecting and deactivating some of the constituent dimensions of the integrated feature vector or some of the internal operation nodes of the classification model according to a pre-set masking probability, and by performing multiple forward propagation operations on the same integrated feature vector based on multiple masking patterns in which the dropout masking is applied differently from each other, the device can calculate a model-specific probability distribution including multiple class probability vectors for each classification model.

[0199] According to one embodiment, the electronic device may calculate a branching decision value by inputting the inter-model prediction conflict index and the model internal variability index into a non-linear function, using the inter-model prediction conflict index calculated by utilizing the JS divergence between average probability distributions between models based on the model-specific probability distribution, and the model internal variability index generated by calculating the entropy value of the probability distribution for each iteration inference for each classification model and calculating the variance of the entropy values. The electronic device may perform a first route if the branching decision value satisfies a first criterion, and perform a second route if the branching decision value does not satisfy the first criterion.

[0200] For example, an electronic device can calculate a branching decision value using the following mathematical formula.

[0201] [Mathematical Formula]

[0202]

[0203] In the above mathematical formula may be a set of class probability vectors obtained from dropout iterative inference of the k-th classification model, and can be an indicator representing the degree of prediction conflict between models, and It can be an indicator that aggregates the variability of probability distributions by model. can be a non-linear activation function, and may be a coefficient that is set in advance or determined during the verification process, and can be a branch threshold. represents an Indicator Function. This may be a function that outputs 1 if the condition inside the parentheses is True and 0 if it is False. The electronic device [is] the result of the branch determination above In this case, Route A can be executed, and In this case, Route B can be executed.

[0204] According to one embodiment, the electronic device sets any two of the plurality of classification models as a single model pair and can calculate the difference in probability distribution between the average probability vectors for each of the two models corresponding to the model pair using an information-theoretic distance measure. The electronic device can calculate the difference in probability distribution for each of all model pairs that can be formed from the plurality of classification models. The electronic device can calculate an Inter-Model Disagreement Index, which indicates the degree of prediction conflict between models, by averaging or weighting the difference in probability distribution calculated for all model pairs.

[0205] The prediction conflict metric between models can be a value that quantifies the difference in distribution between the average probability vectors produced by K classification models for the same input as an information-theoretical distance, or it can be specified by using the average JS divergence between model pairs. The electronic device is each classification model For [the device], average probability vectors per model can be generated by aggregating the results of dropout iterative inference, and the level of agreement between models can be quantified by comparing the said average probability vectors per model. The electronic device [manages] two models and The JS divergence between two probability distributions can be calculated using the average probability vector, and the Inter-Model Disagreement Index can be generated by averaging the JS divergence for all pairs of models. The above index can reflect how different the probability distribution of a specific model is from the probability distributions of other models and how informationally inconsistent it is; the larger the value, the greater the prediction conflict between models.

[0206] For each classification model, the electronic device can calculate an entropy value for each of the multiple class probability vectors included in the model-specific probability distribution. For each classification model, the electronic device can calculate the variance of the multiple entropy values ​​to produce a model-specific variability value representing the internal variability of the corresponding classification model. The electronic device can generate an Intra-Model Variability Index by averaging or weighting the model-specific variability values ​​calculated for each of the multiple classification models, or by selecting a maximum value.

[0207] An indicator aggregating the variability of probability distributions by model may be a value that quantifies how much the uncertainty (entropy) of probability distributions generated from multiple inferences with different dropout masking applied within each classification model fluctuates with each iteration, and can be specified by using the variance of entropy per inference iteration. The electronic device is a specific classification model For each class probability vector obtained from multiple inferences, an entropy value can be calculated, and the variance of the entropy values ​​can be calculated and defined as the internal variability of the model. If the variance value is large, it may mean that the confidence level of the prediction probability distribution varies significantly depending on the dropout masking pattern even for the same input, which can be interpreted as a situation where the model cannot make a stable judgment regarding the input.

[0208] According to one embodiment, the electronic device may include an operation of deactivating some of the K classification models in Root B. At the point when Root B is selected by the branching decision, the electronic device may immediately determine the models to be deactivated using already calculated model-specific probability distributions or model-specific uncertainty-related values, and may perform deactivation without additional branching decisions or condition comparison operations.

[0209] According to one embodiment, the electronic device may perform the following operations at root B. For each of the K classification models, the electronic device may include the model as a deactivation target if the model-specific probability distribution of the model shows a form in which the variation between iterative inferences is large or the probability does not stably converge to a specific class. The electronic device may exclude the model included as a deactivation target from the ensemble calculation, and this exclusion may be performed by masking the probability vector of the model so that it is not input into the combination operation.

[0210] The electronic device may construct a reduced set of models consisting only of the remaining models that are not deactivated, and may generate an average probability vector per model only for said reduced set of models, or may calculate a final class probability vector by combining the average probability vectors of the reduced set of models. The electronic device may include the number of deactivated models or the deactivation ratio in the output data to provide state information that indirectly indicates that a conflict between models has occurred in the corresponding input.

[0211] Through this, the electronic device can reduce averaging errors that may occur when all models are weighted equally in situations where consensus between models is low, and can have the effect of reducing the volatility of the final probability vector by immediately excluding the influence of unstable models.

[0212] The electronic device is the branching result above In this case, Root A can be performed. The electronic device can calculate the average probability vector per model and the uncertainty statistics per model based on the probability distribution per model of each classification model.

[0213] The electronic device aggregates a plurality of class probability vectors calculated for each of the above classification models to generate an average probability vector for each model by calculating an average value for each class, and can calculate at least one of the variance, standard deviation, entropy, or probability interval of the plurality of class probability vectors to calculate uncertainty statistics for each model representing the prediction uncertainty of the above classification models.

[0214] The electronic device can calculate the final class probability vector based on the average probability vector for each model.

[0215] The electronic device can calculate the final class probability vector by applying weights that are pre-set or determined during the learning process to the average probability vectors of each classification model to perform weighted summation, or by performing an operation that combines the average probability vectors of each classification model in a stacking manner.

[0216] According to one embodiment, in operation (S407), the electronic device may output output data including risk levels by disease type based on the final class probability vector.

[0218] FIG. 5 is a diagram illustrating an example of preprocessing an image of a retina to calculate a rule-based vector by an electronic device according to one embodiment.

[0219] Referring to FIG. 5, a whole retinal image may be illustrated, and the whole retinal image may be illustrated along with the result of being divided into a plurality of first sections during the preprocessing process. The electronic device may define a plurality of first sections according to a pre-set spatial division criterion for the whole retinal image, and may extract a partial image corresponding to each first section and output it as a first divided image. The first sections may be set as at least one of a square grid shape, a rectangular grid shape, or a radial section shape based on a center.

[0220] FIG. 5 may further include an image illustrating an enlarged view of a specific section among the first sections. The electronic device may generate an enlarged image based on an original coordinate area corresponding to the specific section, and the enlarged image may be generated in a visually enlarged form by maintaining the original resolution or through interpolation. The enlargement process may be performed as part of a preprocessing step for precisely analyzing the detailed structure of the region of interest.

[0221] Additionally, FIG. 5 may show the result of subdividing the enlarged specific section into a plurality of second sections. The electronic device may define a plurality of second sections by applying a division unit different from the first section division or a smaller grid size to the enlarged specific section. Accordingly, the specific section may have a multi-stage section structure, and a partial image corresponding to each second section may be output as an additional divided image.

[0222] As described above, the hierarchical partitioning structure leading from the entire retinal image -> multiple first compartments -> magnified image of a specific compartment -> multiple second compartments may be a preprocessing procedure that enables stepwise analysis of the global and local structures of the retina. By assigning a unique identifier to each of the compartments and storing mapping information with the original retinal image coordinate system, the electronic device can interpret quantitative features corresponding to each local region in conjunction with the original location during the subsequent rule-based feature calculation step.

[0223] The above segmentation process can be performed optionally and may be configured to perform only the first segmentation or to perform the second segmentation only for specific segments, depending on image quality, analysis purpose, or system settings. Accordingly, multi-stage segmentation as illustrated in FIG. 5 can be performed as part of preprocessing and can perform the function of providing a basis for analysis at the local region level before calculating rule-based feature vectors.

[0224] After receiving a retinal image with completed image standardization processing as input data, the electronic device can divide the retinal image into multiple sections according to a certain spatial standard. The division may include at least one of a method of dividing the entire image into an equal grid shape, a method of dividing the image into radial or concentric circle shapes based on the center of the retina, or a method of dividing the area after establishing a relative coordinate system based on the positions of the optic nerve head and the macula (fovea).

[0225] The electronic device can detect a center point or an anatomical reference point of the retinal image and define multiple regions by setting a certain radius or angular range centered on the detected reference point. Additionally, the electronic device can generate multiple grid regions in the shape of squares or rectangles by considering the resolution and aspect ratio of the retinal image, and can generate segmented images by extracting partial images corresponding to each grid region.

[0226] The above segmentation process can be performed as part of a preprocessing step, and the segmented image can be utilized as an intermediate output prior to the subsequent rule-based feature vector generation. For example, instead of generating features directly from the entire retinal image, the electronic device may derive quantitative features by calculating vascular density, color statistics, brightness distribution, or texture characteristics individually for each segment and then aggregating them.

[0227] In addition, the above segmentation process may be optional, and its execution may be determined based on system settings, input image quality, or the purpose of analysis. For example, segmentation may be performed when the input image resolution is sufficiently high and local lesion analysis is required, while the segmentation process may be omitted when only global structure-based analysis is required.

[0228] The electronic device may store the plurality of segmented images in memory or assign unique identification information to each segmented image and transmit it to a subsequent processing module. In this case, the segmented images may be stored along with the original coordinate system information, allowing local features calculated in a subsequent step to be mapped back to their positions on the original retinal image. The process of segmenting the retinal image into multiple sections may be performed as part of the preprocessing, and it may perform the function of establishing a basis for analysis at the local region level prior to calculating rule-based feature vectors.

[0230] FIG. 6 is a diagram illustrating the structure of a software layer of an electronic device that performs a retinal image-based condition evaluation method according to one embodiment.

[0231] FIG. 6 is a diagram illustrating the software layer structure of an electronic device performing a retinal image-based condition evaluation method according to one embodiment. The software layer illustrated in FIG. 6 may be implemented in a form in which functional modules for performing a retinal image-based condition evaluation method are hierarchically configured, and may be configured to perform a series of processing steps in software, ranging from receiving input data, image preprocessing, feature extraction, artificial intelligence-based inference, and result generation. The electronic device may be a server device, an edge computing device, a medical analysis device, or a general computing device on which medical software is executed, and the functional blocks may be implemented in the form of program modules executed by a processor and memory.

[0232] Referring to FIG. 6, the software layer of the electronic device may include a data receiving unit (610), a preprocessing unit (620), a feature extraction unit (630), an inference unit (640), and an output data generation unit (650). Each of the above functional blocks may be implemented as an independent software module and may be configured in a hierarchical structure that internally includes detailed functional modules. This structure may be intended to improve the scalability and maintainability of the system by modularizing image processing, feature analysis, and artificial intelligence inference functions.

[0233] First, the data receiving unit (610) can perform the function of receiving input images and demographic data of the subject transmitted from a user terminal. The input image may be a retinal image acquired by a fundus camera or other ocular imaging device, and may be transmitted in various image file formats such as JPEG, PNG, or DICOM. The data receiving unit (610) can verify the integrity of the received file and configure a processing context at the subject level based on a session identifier, and may store or cache the input data in memory or a storage device so that it can be referenced in a subsequent analysis step. In addition, the data receiving unit (610) can generate basic attribute information required in a subsequent image processing step by checking metadata such as the resolution, color channels, and imaging device information of the image file.

[0234] The preprocessing unit (620) can perform a preprocessing process to normalize image quality and identify major retinal structures for the input image. The preprocessing unit (620) may include an image standardization processing unit (621) and a retinal structure identification unit (622). The image standardization processing unit (621) can reduce image quality deviations caused by differences in the shooting device or shooting environment by performing at least one of resolution normalization, color space normalization, noise removal, and contrast correction on the input image. For example, if the brightness or color distribution of images captured by different fundus cameras differs, normalization can be performed to minimize deviations in image quality during the subsequent analysis stage.

[0235] The retinal structure identification unit (622) can perform the function of identifying major structures of the retina from a preprocessed image. The retinal structure identification can be performed using geometric boundary detection, morphological operations, connectivity component analysis, or a learning-based segmentation algorithm, through which major anatomical structures including retinal vascular structures and optic disc regions can be identified. The identification results can be generated in the form of a vascular mask, an optic disc region mask, centerline information, or region of interest information, and such structural information can be utilized in a subsequent rule-based feature calculation step.

[0236] The feature extraction unit (630) is a functional block that generates a feature vector to be used for analysis based on the preprocessing results and the input image. The feature extraction unit (630) may include a rule-based feature calculation unit (631), a latent feature extraction unit (632), a feature combination unit (633), and optionally a feature selection unit (634). The rule-based feature calculation unit (631) can calculate structural and morphological quantitative features of the retina, such as vascular density, vascular diameter, vascular curvature, fractal dimension, arteriovenous ratio, and optic disc-related indicators, based on the retinal structure identification results. These features can be calculated according to a predefined mathematical or statistical formula and can be combined with demographic data, such as the subject's age and gender, to form a rule-based feature vector.

[0237] The latent feature extraction unit (632) can generate a high-dimensional latent feature vector from an input image using a deep learning-based feature extraction network. The feature extraction network can be implemented as a convolutional neural network or other image recognition model and can automatically extract complex patterns or non-linear image features that are difficult for humans to define.

[0238] The feature combining unit (633) can combine a rule-based feature vector and a latent feature vector to generate an integrated feature vector. The combining process can be performed using at least one of a vector combining method, a weighted combining method, or a method of combining after mapping to a common feature space. Additionally, optionally, the feature selection unit (634) can perform feature selection, dimension reduction, or normalization processing to adjust the dimension of the integrated feature vector.

[0239] The inference unit (640) is a functional block that performs risk assessment using an artificial intelligence-based classification model with an integrated feature vector as input. The inference unit (640) can perform ensemble-based inference using multiple classification models having different learning principles. Each classification model can receive an integrated feature vector as input and calculate class probabilities for cardiometabolic and neurological disease states, and a probability distribution for each model can be generated by performing repeated inference on the same input multiple times with dropout masking applied. The electronic device can calculate an average probability vector for each model based on the probability distribution and calculate a statistical value representing prediction uncertainty based on the variability of the probability distribution. In addition, the results of each model can be combined using a weighted average or stacking method to generate a final class probability vector, thereby allowing for the quantitative calculation of the subject's retinal image-based condition.

[0240] The output data generation unit (650) is a functional block that generates final output data to be provided to the user based on the inference results. The output data generation unit (650) may include a visualization generation unit (651) and a text summary generation unit (652). The visualization generation unit (651) can generate an activation map based on the final class probability vector or uncertainty statistics and superimpose it with the original retinal image to visually represent the analysis results. The text summary generation unit (652) can generate a summary of the analysis results in natural language form based on the final class probability vector, uncertainty statistics, and risk levels by disease type, and this process can be performed using a large language model. The generated summary text may include information such as risk descriptions, prediction reliability, and recommendations for additional testing.

[0241] As such, the software layer structure illustrated in FIG. 6 can be configured to perform the entire process from receiving input image data to preprocessing, feature extraction, AI-based risk inference, and result generation in a modular manner, thereby enabling the implementation of an integrated analysis system capable of evaluating retinal image-based conditions in a quantitative and explainable manner.

[0243] FIG. 7 is a diagram illustrating the structure of an inference unit according to one embodiment.

[0244] FIG. 7 is a diagram illustrating the structure of an inference unit in a retinal image-based condition evaluation method system according to one embodiment. The inference unit illustrated in FIG. 7 may be configured to receive an integrated feature vector as input, perform ensemble-based inference using a plurality of classification models, calculate average probability and uncertainty statistics from the prediction probability distribution of each model, and then combine them to generate a final class probability vector and risk level by disease type. This structure may aim to improve prediction stability and reliability compared to a single-model-based prediction method, and to provide both explainability and uncertainty information required in a medical diagnostic environment.

[0245] Referring to FIG. 7, the inference unit may include an ensemble inference unit (710), a probability combination unit (720), an uncertainty quantification unit (730), and a result generation unit (740). The components may be sequentially connected to form a series of inference pipelines that generate final output data from an integrated feature vector.

[0246] First, the ensemble inference unit (710) can receive an integrated feature vector or a dimensionally adjusted integrated feature vector generated by the feature extraction unit as input and perform a prediction operation using multiple classification models. The ensemble inference unit (710) may include a first classification model (711), a second classification model (712), and a k-th classification model (713), and the classification models may be models trained based on different learning principles or different model structures. For example, some models may be deep learning-based classification models, and other models may be statistical or tree-based classification models.

[0247] Each classification model can generate class probability vectors for cardiometabolic and neurological disease states based on the input integrated feature vector. In this case, each classification model can be configured to perform iterative inference multiple times on the same input with dropout masking applied. Dropout masking can be applied by probabilistically deactivating some computation nodes or feature dimensions within the model, and these masking patterns can be applied differently for each iterative inference process. As a result, each classification model generates multiple class probability vectors for the same input, and the set of these probability vectors can form a model-specific probability distribution.

[0248] The output of the ensemble inference unit (710) is a probability distribution per model, which can be expressed in the form of a set of multiple class probability vector samples generated by each classification model. For example, the first classification model (711) can generate multiple probability vectors during the iterative inference process, and the second classification model (712) and the kth classification model (713) can also generate multiple probability vectors in the same way. This set of probability vectors can then be used as basic data for calculating average probability and uncertainty statistics in the probability combination unit (720).

[0249] The probability combination unit (720) can calculate the average probability vector of each classification model and the uncertainty statistics of each model based on the probability distribution of each model transmitted from the ensemble inference unit (710). Specifically, the average probability vector of each model can be calculated by calculating the average value of each class for a plurality of probability vectors generated in each classification model. In addition, statistical values ​​representing the prediction uncertainty of each model can be calculated by analyzing the variability of the probability vectors. These statistical values ​​can be calculated in the form of the variance, standard deviation, entropy, or probability interval of the probability distribution.

[0250] The probability combining unit (720) can generate a final class probability vector by combining the average probability vectors for each model calculated as above. The combining method may be a weighted combining method that calculates the average by applying weights to the prediction results of each classification model, or a stacking-based combining method that uses the average probability vectors of each model as input. This combining process can enable the generation of a more stable final prediction result by combining the prediction results of different models.

[0251] The uncertainty quantification unit (730) can perform the function of calculating the final uncertainty statistics based on the model-specific uncertainty statistics calculated by the probability combination unit (720). Specifically, the final uncertainty statistics can be generated by aggregating the uncertainty statistics calculated from multiple classification models. The aggregation process can be performed using the average, maximum value, or weighted average of the model-specific uncertainty statistics.

[0252] Additionally, the uncertainty quantification unit (730) can calculate a probability interval for each class. The probability interval can be calculated using a percentile-based interval of the probability distribution or the relationship between the mean value and the standard deviation, thereby expressing the confidence range of the probability for each class. This probability interval information can be used by medical staff to determine the reliability of the prediction results.

[0253] Additionally, the uncertainty quantification unit (730) may optionally determine the reliability of the prediction result based on an uncertainty threshold. For example, if the final uncertainty statistics exceed a preset threshold, it may determine that the reliability of the prediction result is low and generate a flag recommending re-imaging or additional examination. This function can be utilized as a safeguard to reduce the risk of incorrect prediction results in a medical environment and to induce additional verification procedures.

[0254] The result generation unit (740) can perform the function of generating a final analysis result to be provided to the user based on the final class probability vector and the uncertainty quantification result. Specifically, the result generation unit (740) can calculate the risk level for each disease or disease type by converting the class probability for each disease or disease type into a risk score form. These risk scores can be normalized or scaled so that they can be expressed in a form that is easy to understand in a medical analysis system or user interface.

[0255] Additionally, the result generation unit (740) can generate structured output data including a final class probability vector, final uncertainty statistics, probability interval information, and risk scores by disease type. The structured output data may optionally include a recommendation flag based on the uncertainty threshold judgment result, and may include messages such as a recommendation for re-imaging or a recommendation for additional examination. This output data may be provided in the form of data transmitted to a medical analysis report, a user interface screen, or an external medical information system.

[0256] As such, the inference unit structure illustrated in FIG. 7 can be configured to perform ensemble-based inference using multiple classification models, probability distribution analysis by model, uncertainty quantification, and result generation processes in a stepwise manner, and thereby can be implemented to provide retinal image-based condition evaluation results in a more stable and reliable manner.

[0257] The inference unit (640) may be configured to receive an integrated feature vector as input during the learning phase to generate multiple classification models that classify the state of cardiovascular and neurological diseases, and to prepare models and parameters so that dropout masking-based iterative inference and ensemble combination are possible during the inference phase. The learning process may be performed not by learning a single model, but by securing multiple classification models having different learning principles and generalization characteristics, and by including a dropout structure so that each model can reflect the variability of the prediction probability, and then determining a method for combining the prediction results of multiple models.

[0258] First, the training data may consist of an input image received through the data receiving unit (610), demographic data, and correct labels. The input image may be organized into a preprocessed image form through the preprocessing unit (620) by undergoing the processing of the image standardization processing unit (621) and the retinal structure identification unit (622). Through the feature extraction unit (630), a rule-based feature vector and a latent feature vector may be generated by the rule-based feature calculation unit (631) and the latent feature extraction unit (632), respectively. The two vectors may be combined by the feature combining unit (633) to generate an integrated feature vector, and optionally, a dimensionally adjusted integrated feature vector may be generated by the feature selection unit (634). The correspondence between the integrated feature vector generated in this way and the correct labels may be composed of training data and used in the learning process of the inference unit (640).

[0259] The above correct answer label may be assigned as one of the predefined cognitive state classifications, such as normal, mild cognitive impairment, or dementia, and the inference unit (640) may be configured to learn the correspondence relationship between the integrated feature vector and the label. Additionally, the training data may be divided by subject so that data of the same subject is not simultaneously included in the training and verification processes, and a data division strategy may be applied so that bias due to imaging equipment or imaging conditions is not concentrated in a specific data division.

[0260] The inference unit (640) may apply different learning settings for each model to learn multiple classification models having different learning principles. For example, some classification models may be learned by using the entire dimension of the integrated feature vector as input, and some classification models may be learned by using the integrated feature vector whose dimension has been adjusted by the feature selection unit (634) as input. In addition, the normalization method of input features, the learning rate, the model structure, the method of reflecting feature importance, or the overfitting prevention strategy may be set differently for each classification model. Multiple classification models learned with such different settings can form different prediction patterns even for the same input data, and this diversity can serve as a basis for improving prediction stability when combined in an ensemble.

[0261] Additionally, the inference unit (640) can be configured to include a dropout masking structure during the training process of each classification model. Specifically, a method of probabilistically deactivating some computation nodes or parts of the input feature dimensions within the model can be applied during the training process, thereby preventing excessive reliance on specific features or specific nodes and improving the generalization performance of the model. This dropout structure can be maintained not only during the training phase but also during the inference phase, and the model structure can be designed to perform iterative inference with different dropout patterns applied to the same input. Furthermore, the dropout application location or deactivation probability can be set differently for each classification model, thereby ensuring additional structural diversity between models.

[0262] The inference unit (640) can evaluate the performance of each learned classification model on the validation data. In this process, not only simple classification accuracy but also class-specific sensitivity, specificity, stability of prediction probability, and variability of the prediction distribution can be evaluated. For example, if a specific model shows excessive variability in prediction probability only in a specific class or unstable prediction only under specific image quality conditions, the model may be configured to adjust its training settings or perform retraining. Additionally, the inference unit (640) can calculate the reliability of each model by comparing the prediction stability and performance of each classification model, and this information can be utilized in the subsequent ensemble combination step.

[0263] When the method of combining multiple classification models is a weighted combination method, the inference unit (640) can determine the weight of each classification model based on the performance evaluation results obtained during the verification process. For example, a relatively high weight may be assigned to a classification model with excellent prediction performance and high stability of the prediction distribution, and a relatively low weight may be assigned to a model with high prediction variability. These weights may be stored as fixed values ​​by system settings, and may also be managed in the form of multiple profiles so that different sets of weights are applied according to specific environmental conditions, such as the type of shooting equipment or image quality conditions.

[0264] In addition, if the classification model combination method is a stacking method, the inference unit (640) may configure separate training data for training the meta-combination model. At this time, meta-training data may be configured using the prediction probability generated by each classification model as an input feature, and the meta-combination model may be trained to predict the final class based on the input. In the training process of the meta-combination model, prediction results generated through cross-validation may be used to prevent overfitting, and through this, the meta-combination model may be configured to learn the strengths and weaknesses of each classification model to perform a more stable final prediction.

[0265] The inference unit (640) can store the parameters, dropout settings, input feature dimension information, and model combination policy of each classification model together after training is completed. The information stored at this time may include model version information containing compatibility information with the processing settings of the feature extraction unit (630), thereby allowing the preprocessing of input data and the feature configuration method during the inference stage to be managed to match the settings during training. Additionally, a version management structure may be applied to enable comparative evaluation with a previous version model or rollback when updating the model.

[0266] Additionally, the inference unit (640) can set operational parameters to be used for uncertainty evaluation based on iterative inference during the learning phase. For example, the number of iterative inferences, the method for calculating uncertainty statistics, the uncertainty score threshold, and a policy recommending re-shooting or additional inspection when the threshold is exceeded can be set based on verification data. These policies may be set as a single value, or they may be stored in the form of multiple policies so that different thresholds are applied depending on conditions such as image quality indicators or types of shooting equipment.

[0267] In this way, the learning process of the inference unit (640) can be performed by learning multiple classification models based on integrated feature vectors generated by the data receiving unit (610), the preprocessing unit (620), and the feature extraction unit (630), configuring the model structure to enable uncertainty evaluation based on iterative inference using a dropout structure, and determining the model combination method and operation policy together. Through this, the electronic device can provide not only a single prediction result but also information on the reliability of the prediction, and can be configured to evaluate the retinal image-based state in a more stable and explainable manner.

[0270] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0271] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0272] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software devices to perform the operation of the embodiment, and vice versa.

[0273] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0274] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below. Explanation of the symbols

[0276] 300 : System 310 : User terminal 320 : Camera 330 : Network 340: Analysis Server 350: Edge or on-premises server 360: AI Learning Server 370 : Medical imaging storage and transmission system or electronic medical record system

Claims

Claim 1 An operation of receiving an input image of the front of a subject's retina and demographic data regarding the subject by means of an electronic device; an operation of preprocessing the input image by means of the electronic device; an operation of calculating a rule-based feature vector representing quantitative features of the subject's retinal structure related to the subject's cardiometabolic and neurological diseases for the preprocessed input image by means of the electronic device; an operation of obtaining a latent feature vector related to the subject's retinal cardiometabolic and neurological diseases from the input image through a deep learning-based feature extraction model by means of the electronic device; an operation of obtaining an integrated feature vector by combining the rule-based feature vector and the latent feature vector by means of the electronic device; and an operation of obtaining a final class probability vector and model-specific uncertainty statistics regarding the subject's cardiometabolic and neurological disease status by means of the electronic device, by inputting the integrated feature vector into a plurality of classification models trained in different ways and applying dropout masking. The operation of outputting output data including evaluation results for cardiometabolic and neurological disease states based on the final class probability vector by the electronic device, and the operation of obtaining the final class probability vector and model-specific uncertainty statistics comprises: an operation of calculating a model-specific probability distribution for each classification model by applying a sampled dropout mask vector according to a preset masking probability, by performing multiple inferences on the same input while probabilistically deactivating a portion of the feature dimension through dropout masking for each classification model by the electronic device; and an operation of calculating a model-specific average probability vector and model-specific uncertainty statistics based on the model-specific probability distribution of each classification model by the electronic device.A retinal image-based condition evaluation method comprising the operation of applying weights calculated based on model-specific uncertainty statistics to the model-specific average probability vector by means of the electronic device to calculate the final class probability vector, wherein the weights calculated based on model-specific uncertainty statistics are set lower as the variance included in the model-specific uncertainty statistics is higher, and are set higher as the variance included in the model-specific uncertainty statistics is lower. Claim 2 delete Claim 3 A retinal image-based state evaluation method according to claim 1, wherein the operation of calculating the probability distribution for each model comprises, by means of the electronic device, applying dropout masking to each classification model that takes the integrated feature vector as input, by randomly selecting and deactivating some of the constituent dimensions of the integrated feature vector or some of the internal operation nodes of the classification model according to a preset masking probability, and performing multiple forward propagation operations on the same integrated feature vector based on multiple masking patterns in which the dropout masking is applied differently from each other, thereby calculating the probability distribution for each model including multiple class probability vectors for each classification model. Claim 4 A retinal image-based state evaluation method according to claim 1, wherein the operation of calculating the average probability vector and the model-specific uncertainty statistics comprises, by means of the electronic device, aggregating a plurality of class probability vectors calculated for each classification model and calculating a class-specific average value to generate the average probability vector for the model, and calculating at least one of the variance, standard deviation, entropy, or probability interval of the plurality of class probability vectors to calculate the model-specific uncertainty statistics representing the prediction uncertainty of the classification model. Claim 5 A retinal image-based state evaluation method according to claim 1, wherein the operation of calculating the final class probability vector comprises, by means of the electronic device, performing weighted summation by applying weights that are pre-set or determined during the learning process to the average probability vector of each classification model, or performing an operation of combining the average probability vectors of each classification model in a stacking manner.

Images