Ai-based blinding disease detection system and method

Abstract

The present invention relates to an AI-based blinding disease detection system and method, the system comprising: one or more cameras for photographing the eyes of a patient; a device management module for collecting retinal images and monitoring the state of each camera; a database for storing the retinal images; an AI management module for evaluating the risk of blinding diseases by analyzing the retinal images on the basis of an AI algorithm; a result provision module for providing, to a customer terminal, the result of evaluating the risk of blinding diseases; and a third-party connecting module linked to a hospital information system (IHS) or a cloud-based external system so as to connect to an external tool for additional analysis or report generation.

Description

AI-based blindness detection system and method

[0001] The present invention relates to an AI-based blindness disease detection system and method, and more specifically, to an AI-based blindness disease detection system and method that analyzes fundus images using an AI-based trained model to extract pathological characteristics of various blindness diseases and detects signs of various blindness diseases based thereon.

[0002] Generally, the three major causes of blindness are broadly divided into diabetic retinopathy (DR), glaucoma (GLC), and age-related macular degeneration (AMD).

[0003] Diabetic retinopathy (DR) is caused by damage to the blood vessels of the retina, and its main characteristics include retinal hemorrhage, microaneurysms, and exudates. Glaucoma (GLC) is caused by damage to the optic nerve, and its main characteristic is visual field defects associated with elevated intraocular pressure. Age-related macular degeneration (AMD) is caused by the degeneration of the macula, and central visual field loss appears as a major symptom.

[0004] Diabetic retinopathy is a vascular complication of the retina that damages the retina and leads to severe vision loss if not treated immediately.

[0005] The global growth rate of the diabetes population is estimated to increase from 2.8% to 4.4% from 2000 to 2030, corresponding to approximately 171 million to 195 million patients suffering from diabetes. Among diabetes patients, there is a risk of 2% blindness and about 10% vision loss over the next 15 years.

[0006] In addition, glaucoma (GLC) occurs due to the influence of intraocular pressure difference, which damages the optic nerve head and causes vision loss.

[0007] Age-related macular degeneration (AMD) also causes vision loss and blindness in older adults aged 60 and over, and has been shown to have a significant impact on older adults aged 60 and over.

[0008] Many of these patients may experience difficulties in their lives due to their eyesight, one of the five basic human senses.

[0009] Meanwhile, there are various techniques used by experts or doctors to diagnose the aforementioned eye diseases, and these techniques determine the eye diseases by utilizing fundus images captured through cross-sectional imaging such as Optical Coherence Tomography (OCT) and fundus imaging.

[0010] In addition, to enable rapid and accurate diagnosis of eye diseases, diagnostic devices for eye diseases based on image classification applying various artificial intelligence models in the respective diagnostic areas of DR, GLC, and AMD are being researched and developed.

[0011] Typically, to diagnose ophthalmic diseases, fundus images are obtained by performing optical coherence tomography (OCT) and fundus imaging.

[0012] In this context, fundus images are used to diagnose and screen for many ophthalmic diseases, such as diabetic retinopathy (DR), glaucoma (GLC), and age-related macular degeneration (AMD). Therefore, the quality of fundus images can affect a clinician's ability to perform accurate examinations and diagnoses.

[0013] Although it is possible to diagnose multiple ophthalmic diseases from fundus images as described above, conventional ophthalmic disease diagnostic devices applying artificial intelligence were only able to detect a single specialized eye disease from fundus images. In particular, conventional eye disease diagnostic devices using artificial intelligence had the problem of detecting only one eye disease from fundus images and being unable to detect two or more eye diseases.

[0014] The present invention aims to solve the aforementioned problems by providing an AI-based blindness disease detection system and method that analyzes fundus images using an AI-based trained model to extract pathological characteristics of various blindness diseases, and based on this, enables the detection of signs of various blindness diseases.

[0015] An AI-based blindness detection system (100) according to one embodiment of the present invention may include one or more cameras (110) for photographing a patient's eye, a device management module (120) for collecting retinal images captured through each of the one or more cameras (110) and monitoring the status of each camera (110), a database (130) for storing the retinal images, an AI management module (140) for analyzing the retinal images stored in the database (130) based on an AI algorithm to evaluate the risk of blindness, a result providing module (150) for providing the results of the evaluation of the risk of blindness to a customer terminal, and a third-party connecting module (160) that is linked with a hospital information system (IHS) or a cloud-based external system and connected to an external tool for additional analysis or report generation.

[0016] According to one embodiment of the present invention, the camera (110) may include a UI controller module (111) for managing a user interface, a camera controller module (112) for capturing a retinal image during the shooting process through the camera (110), a personal service provision module (113) for creating an account for each patient, providing personalized services based on feedback requested for each patient account, and managing health data for each patient account, an access controller module (114) for managing access rights for each patient account, a rational judgment module (115) for analyzing the captured retinal image, a database manager module (116) for storing user information, device owner information, retinal image, and device information, and a remote communicator (117) for communicating with an external service module or an external remote server.

[0017] According to one embodiment of the present invention, the UI controller module (111) may include a user identification module (111-1) that retrieves basic information about a user and then verifies the user's identity, an authority extraction module (111-2) that verifies the role and device control authority previously assigned to the user whose identity has been verified, a use case selection module (111-3) that determines the scope of work that the user whose identity has been verified can perform, a UI configuration module (111-4) that generates a UI to be provided to the user based on the information provided through the authority extraction module (111-2) and the use case selection module (111-3), and a UI front-end module (111-5) that provides the generated UI to a device so that it is output.

[0018] According to one embodiment of the present invention, the user identification module (111-1) retrieves basic information about the user from the user information database, the authority extraction module (111-2) checks the role and device control authority assigned to the user from the RBAC information database and the device control information database, and the use case selection module (111-3) can determine the scope of work by obtaining the user's authority and role information from the use case database.

[0019] According to one embodiment of the present invention, the rational judgment module (115) may include a local judgment module (115-1) that analyzes a retinal image captured in a local environment and provides an analysis result, and a remote judgment module (115-2) that runs in a remote server or cloud environment and analyzes the retinal image at a faster speed than the local judgment module (115-1) and provides an analysis result.

[0020] According to one embodiment of the present invention, the database (130) may include a retinal dataset storage unit (131) for storing a retinal dataset, a data fetcher (132) for retrieving a specific retinal dataset from the retinal dataset storage unit (131), a device synchronization module (133) linked with the data fetcher (132) for synchronizing user information, device control information, and the retinal dataset, a serial number management module (134) for managing and tracking a unique serial number for each device, and a firmware version management module (135) for managing a firmware version for each device.

[0021] According to one embodiment of the present invention, the AI ​​management module (140) may include a raw database (141) for storing raw data required for learning and inference, a preprocessing module (142) for preprocessing the raw data into a format for model training by refining and normalizing the raw data and extracting features, a trainer module (143) for training an AI model using the preprocessed data, a model storage module (144) for storing the trained AI model, a verification module (145) for evaluating and verifying the performance of the AI ​​model by loading specific raw data from the raw database (141) and inputting it into the trained AI model, an inference unit configuration module (146) for configuring one or more inference units based on the AI ​​model after loading the AI ​​model from the model storage module (144), an ensemble judgment module (147) for aggregating the prediction results of each of the one or more inference units, and an inference pipeline management module (148) for deriving a final inference result by inputting input data into the ensemble judgment module (147) and providing the derived final inference result to a device.

[0022] According to one embodiment of the present invention, the AI ​​management module (140) performs contrast-limited adaptive histogram equalization (CLAHE) on the input retinal image before inputting the retinal image into the CNN model, performs downsampling so that the size of the retinal image is reduced to fit the CNN model, and can augment the dataset of the retinal image using a random rotation technique, a scaling technique, or a flipping technique.

[0023] The AI ​​management module (140) according to one embodiment of the present invention uses any one of InceptionResNetV2, DenseNet201 and ResNet50, which are pre-trained CNN architecture models, and can train the CNN architecture models through K-fold cross-validation, augmented data training, and model ensemble processes.

[0024] According to one embodiment of the present invention, the AI ​​management module (140) can evaluate the performance of the AI ​​model using one or more of the area under the ROC curve (AUC) indicator, the sensitivity indicator, and the specificity indicator.

[0025] According to one embodiment of the present invention, the AI ​​management module (140) inputs a fundus image adjusted to a size of 299*299*3 as input data to an Inception-v3 model, extracts features of the fundus image using the Inception-v3 model and generates a feature map, reduces the dimensionality by averaging pooling the feature map using an Average Pooling 2D Layer technique, flattens the fundus image after averaging pooling and converts it into a 1D vector form, performs learning for final classification based on the features of the fundus image in a Fully Connected Layer, and generates an output that classifies quality using a Softmax activation function.

[0026] According to one embodiment of the present invention, the AI ​​management module (140) performs image preprocessing by resizing an input retinal image to 632 pixels, automatically detecting a region of interest (ROI), and applying an elliptical mask, classifies the transposed retinal image into a training set and a validation set using a 10-fold cross-validation split method as training data, mixes the training data and the validation data together to prevent bias based on the order of the data, and can perform a validation test after storing the AI ​​model that has completed training in a knowledge base according to a pre-learned classification rule.

[0027] According to one embodiment of the present invention, the third-party connecting module (160) may include a third-party access controller module (161) that performs data communication with an external system, a PHR and HL7 FHIR parser module (162) that parses PHR and HL7 FHIR data received from the external system and converts it into a usable format, a PHR and HL7 FHIR request handler module (163) that searches for and provides the requested data when the external system requests specific data, a consistency check module (164) that checks the consistency of the data received from the external system and determines whether there is a conflict with existing data, a database controller module (165) that stores the PHR and HL7 FHIR data, and a PHR and HL7 FHIR database (166) that manages data stored according to personal health records and HL7 FHIR standards.

[0028] According to one embodiment of the present invention, the AI ​​management module (140) can detect details related to retinal diseases, including diabetic retinopathy (DR), glaucoma (GLC), and age-related macular degeneration (AMD), by extracting features from the retinal image using a convolutional neural network (CNN) model.

[0029]

[0030] An AI-based blindness detection method according to another embodiment of the present invention may include the steps of: photographing a patient's eye through one or more cameras; collecting retinal images captured through each of the one or more cameras and monitoring the status of each camera through a device management module; storing the retinal images in a database; analyzing the retinal images stored in the database based on an AI algorithm through an AI management module to evaluate the risk of blindness; providing the results of the evaluation of the risk of blindness to a customer terminal through a result providing module; and linking with a hospital information system (IHS) or a cloud-based external system through a third-party connecting module to generate additional analysis or reports through connection with external tools.

[0031] According to the present invention, by analyzing fundus images using an AI-based trained model, pathological characteristics of various blinding diseases are extracted, and based thereon, signs of various blinding diseases can be detected.

[0032] FIG. 1 is a schematic diagram showing the configuration of an AI-based blindness disease detection system (100) according to one embodiment of the present invention.

[0033] Figure 2 is a drawing showing the configuration of the camera (110) illustrated in Figure 1 in more detail.

[0034] FIG. 3 is a diagram showing the configuration of the UI controller module (111) illustrated in FIG. 2 in more detail.

[0035] Figure 4 is a diagram showing the configuration of the database (130) in more detail.

[0036] FIG. 5 is a diagram showing the configuration of the AI ​​management module (140) in more detail.

[0037] FIG. 6 is a diagram conceptually illustrating an image preprocessing method through an AI management module (140).

[0038] FIG. 7 is a diagram conceptually illustrating a method for training verification and test data splitting through an AI management module (140).

[0039] FIG. 8 is a diagram conceptually illustrating the process of quality evaluation and classification of fundus images through an AI management module (140).

[0040] FIG. 9 is a diagram conceptually illustrating the process of classifying fundus images using the Inception-v3 model of the AI ​​management module (140).

[0041] FIG. 10 is a conceptual diagram illustrating the disease classification process based on various pathological characteristics through an AI management module (140).

[0042] FIG. 11 is a diagram conceptually illustrating the training and testing process of an AI model through an AI management module (140).

[0043] FIG. 12 is a drawing showing the configuration of a third-party connecting module (160) in more detail.

[0044] FIG. 13 is a flowchart illustrating an AI-based blindness disease detection method according to one embodiment of the present invention in a series of sequences.

[0045] Hereinafter, specific details for implementing the present invention will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present invention may be unnecessarily obscured.

[0046] In the attached drawings, identical or corresponding components are given the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0047] The advantages and features of the invented embodiments and the methods for achieving them will become clear by referring to the embodiments described below together with the accompanying drawings. However, the present invention is not limited to the embodiments described below but can be implemented in various different forms, and these embodiments are provided merely to make the present invention complete and to fully inform a person skilled in the art of the scope of the invention.

[0048] The terms used in this specification will be briefly explained, and the invented embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in the present invention; however, these terms may vary depending on the intent of those skilled in the relevant field, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on the meanings they possess and the content of the invention as a whole.

[0049] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0050]

[0051] This invention was developed as a result of a national research and development project, and specific details are as follows.

[0052] [Project ID] TASK00000000834

[0053] [Assignment No.] 2-2-1

[0054] [Ministry Name] Ministry of Trade, Industry and Energy

[0055] [Name of Project Management (Specialized) Agency] Korea Institute for Industrial Technology Promotion

[0056] [Research Project Name] Support for the Globalization of Ophthalmic Optical Medical Devices

[0057] [Research Project Title] Deep Learning-Based Fundus Image Classification Device and Method for Diagnosis of Ophthalmic Diseases

[0058] [Contribution Rate] 1 / 1

[0059] [Name of Project Performing Organization] Chosun University Industry-Academic Cooperation Foundation

[0060] [Research Period] March 18, 2024 – October 31, 2024

[0061]

[0062] FIG. 1 is a schematic diagram showing the configuration of an AI-based blindness disease detection system (100) according to one embodiment of the present invention.

[0063] Referring to FIG. 1, an AI-based blindness detection system (100) according to one embodiment of the present invention may largely include one or more cameras (110), a device management module (120), a database (130), an AI management module (140), a result providing module (150), and a third-party connecting module (160).

[0064] First, one or more cameras (110) are cameras owned by a medical institution and can capture retinal images by photographing the patient's eye. The retinal images captured through each camera (110) can be transmitted in real time to a device management module (120). This is described in more detail as follows.

[0065] FIG. 2 is a drawing showing the configuration of the camera (110) shown in FIG. 1 in more detail, and FIG. 3 is a drawing showing the configuration of the UI controller module (111) shown in FIG. 2 in more detail.

[0066] Referring to FIGS. 2 and FIGS. 3, a camera (110) according to one embodiment of the present invention may include a UI controller module (111) for managing a user interface, a camera controller module (112) for capturing a retinal image during the shooting process through the camera (110), a personal service provision module (113) for creating an account for each patient and providing personalized services based on feedback requested for each patient account, and managing health data for each patient account, an access controller module (114) for managing access rights for each patient account, a rational judgment module (115) for analyzing the captured retinal image, a database manager module (116) for storing user information, device owner information, retinal image, and device information, and a remote communicator (117) for communicating with an external service module or an external remote server.

[0067] Here, the UI controller module (111) is a component that manages the user interface and helps the user interact with the camera (110), thereby allowing the user or operator to adjust the camera (110) functions or check the results.

[0068] Additionally, in one embodiment, the UI controller module (111) may include a user identification module (111-1) that retrieves basic information about a user and then verifies the user's identity, an authority extraction module (111-2) that verifies the role and device control authority assigned to the user whose identity has been verified, a use case selection module (111-3) that determines the scope of work that the user whose identity has been verified can perform, a UI configuration module (111-4) that generates a UI to be provided to the user based on the information provided through the authority extraction module (111-2) and the use case selection module (111-3), and a UI front-end module (111-5) that provides the generated UI to the device so that it is output.

[0069] Here, in one embodiment, the user identification module (111-1) retrieves basic information about the user from the user information database, the authority extraction module (111-2) checks the role and device control authority assigned to the user from the RBAC information database and the device control information database, and the use case selection module (111-3) can determine the scope of work by obtaining the user's authority and role information from the use case database.

[0070] Additionally, the camera controller module (112) can capture a retinal image from the camera (110) and process the captured retinal image to prepare it for use in the next step of analysis.

[0071] In addition, the personal service provision module (113) can provide personalized services tailored to the needs of individual users, manage the user's health data, and provide customized feedback or recommendations based on this.

[0072] Additionally, the access controller module (114) manages access rights between the user and the device to maintain the security of the system and can ensure that only authorized users can access sensitive data.

[0073] Additionally, in one embodiment, the rational judgment module (115) may include a local judgment module (115-1) that analyzes a retinal image captured in a local environment and provides an analysis result, and a remote judgment module (115-2) that runs in a remote server or cloud environment and analyzes a retinal image at a faster speed than the local judgment module (115-1) and provides an analysis result.

[0074] At this time, the remote judgment module (115-2) is an AI model running in a remote server or cloud environment to perform more complex analysis than the local judgment module (115-1), and has the ability to process large-scale data or complex algorithms, and can be used in situations that cannot be resolved by the local judgment module (115-1).

[0075] Additionally, the database manager module (116) is a database system that manages various data, and may include user information, device owner information, retina images, device info, etc.

[0076]

[0077] The device management module (120) manages retinal images collected from all cameras (110) and allows them to be stored in a database (130), monitors the status of each camera (110), and may separately display equipment errors if necessary.

[0078] The database (130) can store retinal images and can serve as a central storage facility for all data related to the collected images. This database (130) can serve as a central hub for all information necessary for AI analysis and other service provision. A more detailed look at this is as follows.

[0079] Figure 4 is a diagram showing the configuration of the database (130) in more detail.

[0080] Referring to FIG. 4, the database (130) may include a retinal dataset storage unit (131) that stores a retinal dataset, a data fetcher (132) that retrieves a specific retinal dataset from the retinal dataset storage unit (131), a device synchronization module (133) that synchronizes user information, device control information, and retinal datasets in conjunction with the data fetcher (132), a serial number management module (134) that manages and tracks a unique serial number for each device, and a firmware version management module (135) that manages a firmware version for each device.

[0081] The data fetcher (132) is a component that retrieves necessary data from the retinal dataset, and the retrieved data can then be transferred to the device synchronization module (133) for analysis and processing.

[0082] The device synchronization module (133) performs synchronization with User Info, Device Control Info, and Retina Dataset, and can maintain data consistency with other modules and reflect the latest information.

[0083] The serial number management module (134) manages and tracks the unique serial number of the device and synchronizes the identification information of each device with a database, thereby enabling effective device management and maintenance.

[0084] The firmware version management module (135) manages the firmware version of the device and performs updates when necessary, thereby ensuring that the AI-based blindness detection system (100) is always kept up to date and enabling the addition of new features or bug fixes.

[0085]

[0086] The AI ​​management module (140) can evaluate the risk of blindness by analyzing retinal images stored in the database (130) described earlier based on an AI algorithm. This is described in more detail as follows.

[0087] FIG. 5 is a diagram showing the configuration of the AI ​​management module (140) in more detail, FIG. 6 is a diagram conceptually explaining the image preprocessing method through the AI ​​management module (140), FIG. 7 is a diagram conceptually explaining the training verification and test data splitting method through the AI ​​management module (140), FIG. 8 is a diagram conceptually explaining the quality evaluation and classification process of fundus images through the AI ​​management module (140), FIG. 9 is a diagram conceptually explaining the fundus image classification process using the Inception-v3 model of the AI ​​management module (140), FIG. 10 is a diagram conceptually showing the disease classification process based on various pathological features through the AI ​​management module (140), and FIG. 11 is a diagram conceptually showing the training and testing process of the AI ​​model through the AI ​​management module (140).

[0088] Referring to FIGS. 5 to 11, the AI ​​management module (140) may include a raw database (141) that stores raw data required for learning and inference, a preprocessing module (142) that refines and normalizes the raw data, extracts features, and preprocesses them into a format for model training, a trainer module (143) that trains an AI model using the preprocessed data, a model storage module (144) that stores the trained AI model, a verification module (145) that loads specific raw data from the raw database (141) and inputs it into the trained AI model to evaluate and verify the performance of the AI ​​model, an inference unit configuration module (146) that loads an AI model from the model storage module (144) and configures one or more inference units based on the AI ​​model, an ensemble judgment module (147) that aggregates the prediction results of each of the one or more inference units, and an inference pipeline management module (148) that inputs input data into the ensemble judgment module (147) to derive a final inference result and provides the derived final inference result to a device.

[0089] The raw database (141) is an initial data storage that stores raw data required for learning and inference, and can provide data for pre-processing and validation.

[0090] The preprocessing module (142) converts raw data into a format suitable for model training, and in this process, tasks such as data cleaning, normalization, and feature extraction may be performed. In addition, the preprocessed data can be transmitted to the trainer module (143).

[0091] The trainer module (143) is a component that trains an AI model using pre-processed data, and the trained AI model can be stored in the model storage module (144).

[0092] The model storage module (144) is a storage for storing trained AI models, and the AI ​​models stored in the model storage module (144) can be utilized in the inferencing unit composer module (146).

[0093] The verification module (145) can perform the role of evaluating and verifying the performance of a pre-trained AI model after loading data from a raw database. In this process, the verification module (145) can evaluate the accuracy and reliability of the AI ​​model and determine the model to be used for final inference.

[0094] The inference unit configuration module (146) can configure an inference unit based on a learned AI model stored in the model storage module (144). At this time, the inference unit can be made into a unit capable of performing inference operations in real time on various input data.

[0095] The ensemble judgment module (147) is a component in which multiple inference units cooperate to derive a final conclusion, and the ensemble judgment module (147) can improve accuracy by synthesizing the prediction results of multiple models to provide a more accurate prediction.

[0096] The inference pipeline management module (148) can manage the entire inference process, receiving input data and outputting the final result through the inference unit and the ensemble judgment module (147). At this time, the inference pipeline management module (148) can process the input data and deliver the inference result to the end user.

[0097] Additionally, in one embodiment, the AI ​​management module (140) performs contrast-limited adaptive histogram equalization (CLAHE) on the input retinal image before inputting the retinal image into the CNN model, performs downsampling so that the size of the retinal image is reduced to fit the CNN model, and can augment the dataset of the retinal image using a random rotation technique, a scaling technique, or a flipping technique.

[0098] Here, the CLAHE (contrast-limited adaptive histogram equalization) method enhances image contrast, enabling the CNN to detect fine features in retinal images more effectively.

[0099] In addition, the downsampling method performs downsampling to reduce the size of the image to fit the CNN model, and reduces the image size while preserving key features.

[0100] In addition, ensemble techniques combine multiple models to further improve performance and utilize the strengths of each model to perform more accurate predictions.

[0101] In addition, in one embodiment, the AI ​​management module (140) uses any one of InceptionResNetV2, DenseNet201 and ResNet50 pre-trained CNN architecture models and can train the CNN architecture models through K-fold cross-validation, augmented data training and model ensemble processes.

[0102] Additionally, in one embodiment, the AI ​​management module (140) may evaluate the performance of the AI ​​model using one or more of the area under the ROC curve (AUC) metric, the sensitivity metric, and the specificity metric.

[0103] In this context, AUC (Area Under the ROC Curve) is an important metric in classification tasks, indicating how well the model distinguishes between different classes.

[0104] Furthermore, sensitivity refers to the ability to correctly identify patients with a specific disease, and specificity refers to the ability to correctly identify people without the disease.

[0105] Looking at FIG. 6, an AI management module (140) according to one embodiment of the present invention can improve contrast and enhance the performance of the model by preprocessing a retinal image using the CLAHE (contrast-limited adaptive histogram equalization) method.

[0106] Additionally, the AI ​​management module (140) can remove backgrounds that are not important information, such as eyelashes and eyelids, from the ultra-wide angle retinal image (UWF). To do this, the AI ​​management module (140) performs a rectangular crop of the image and then removes the background by applying a circular mask using OpenCV Bitwise AND.

[0107] Additionally, the AI ​​management module (140) may utilize augmentation techniques including horizontal and vertical flipping, rotation, and brightness adjustment to increase the dataset. In this case, each image is made into four augmented images, and augmentation may be applied only to the training data and validation data.

[0108] Additionally, the AI ​​management module (140) can perform data splitting, wherein the data is first split into a training set (80%) and a test set (20%), and the training data can be further divided into a validation set through 5-fold cross-validation. The AI ​​management module (140) can reliably evaluate the performance of the model through cross-validation, and five split sets are generated through 5-fold cross-validation and can be used for training and validation.

[0109] Looking at Figure 7, during the training validation and test data splitting process, the entire data can be divided into training-validation data (80%) and test data (20%).

[0110] In this case, training and validation data can be used to train the model and evaluate its performance, while test data, which is data the model has never seen during the training and validation process, can be used to evaluate the final model performance.

[0111] In addition, the training-validation data (80%) can be further divided into training data and validation data.

[0112] In this case, training data can be used to train the model, and validation data can be used to evaluate performance and optimize the model after it has been trained.

[0113] Additionally, the AI ​​management module (140) can perform 5-fold cross-validation. Here, 5-fold cross-validation divides the entire training-validation data into 5 folds. This method prevents the model from overfitting when the dataset is small and can help improve the model's generalization performance.

[0114] Therefore, as shown in Figure 7, for 5-fold cross-validation, the data is divided into 5 folds, and each fold can be used as verification data once.

[0115] For example, in Split 1, Fold 1 is used as validation data and Folds 2 through 5 are used as training data, and in Split 2, Fold 2 is used as validation data and the remaining folds are used as training data, and this process is repeated so that each fold is used as validation data once, and a total of 5 model training and evaluations can be performed.

[0116] Looking at FIG. 8, FIG. 8 is a summary diagram of the entire process of ensuring data quality through image quality evaluation and classifying diseases using a CNN-based classification model, and the AI ​​management module (140) according to the present invention can evaluate the quality of fundus images. More specifically, the AI ​​management module (140) can evaluate the quality of fundus images, at which time images of good quality proceed to the classification process, and images of poor quality can be filtered out.

[0117] In addition, high-quality images are processed and then divided into training data using 5-fold cross-validation. At this time, data diversity is ensured by applying CLAHE (Adaptive Histogram Equalization) and data augmentation techniques to the processed images. Afterwards, the AI ​​management module (140) can classify them into DR (Diabetic Retinopathy), GLC (Glaucoma), AMD (Age-Related Macular Degeneration), and NR (Normal) through the trained CNN classifier.

[0118] Next, the AI ​​management module (140) can pass the processed images through a classifier during the test process to classify them into DR, GLC, AMD, and NR.

[0119] Additionally, referring to FIG. 9, an AI management module (140) according to one embodiment of the present invention inputs a fundus image adjusted to a size of 299*299*3 as input data to an Inception-v3 model, extracts features of the fundus image using the Inception-v3 model and generates a feature map, reduces the dimensionality by averaging the feature map using an Average Pooling (2D Layer) technique, flattens the fundus image after averaging pooling and converts it into a 1D vector form, performs learning for final classification based on the features of the fundus image in a Fully Connected Layer, and generates an output that classifies quality using a Softmax activation function.

[0120] In this process, scaled-up fundus images are input into the Inception-v3 model. Inception-v3 is a pre-trained CNN model that extracts key features from the fundus images and reduces dimensionality by mean pooling the feature maps extracted by the CNN model. This reduces model complexity and optimizes performance.

[0121] Additionally, the AI ​​management module (140) flattens the image data after average pooling and converts it into a one-dimensional vector form. The converted data can be input into a fully connected layer.

[0122] Additionally, the AI ​​management module (140) can perform training for final classification based on the features of the image in a fully connected layer, and can generate an output classified into good quality and bad quality using a Softmax activation function.

[0123]

[0124] Looking at Fig. 10, the process of classifying specific diseases based on various pathological features extracted from fundus images is as follows.

[0125] The fundus image is an image showing the internal structure of the eye, including the retina, and the AI ​​management module (140) can analyze the fundus image to detect signs of various eye diseases.

[0126] Additionally, the AI ​​management module (140) can extract pathological characteristics from the fundus image. Major characteristics include exudate (a substance caused by the accumulation of fluid in the retina), hemorrhage (a phenomenon in which bleeding occurs in blood vessels within the retina), neovascularization (the abnormal formation of new blood vessels), macular edema (swelling occurring in the macula), and optic disc damage (a phenomenon in which damage occurs around the optic nerve).

[0127] Additionally, the AI ​​management module (140) can classify specific diseases based on extracted pathological characteristics. Major diseases include diabetic retinopathy associated with exudate, hemorrhage, neovascularization, and macular edema, glaucoma associated with damage to the optic disc, and age-related macular degeneration (age-related macular degeneration).

[0128] In addition, the AI ​​management module (140) can automatically detect and classify the characteristics of the disease through a classifier that has learned the characteristics associated with each disease.

[0129] For example, an exudate classifier determines whether exudate is present, a hemorrhage classifier detects the presence of hemorrhage, and a renal angiogenesis classifier, macular edema classifier, and optic disc classifier can each learn and analyze the corresponding pathological characteristics.

[0130] Referring to FIG. 11, an AI management module (140) according to one embodiment of the present invention can perform image preprocessing by resizing an input retinal image to 632 pixels, automatically detecting a region of interest (ROI), and applying an elliptical mask.

[0131] Additionally, the AI ​​management module (140) classifies the transposed retinal images into a training set and a validation set using a 10-fold cross-validation split method as training data, and mixes the training data and the validation data together to prevent bias based on the order of the data.

[0132] Additionally, the AI ​​management module (140) can store the trained AI model in a knowledge base according to pre-learned classification rules and then conduct a verification test.

[0133] At this stage, the trained model stores the learned classification rules in a knowledge base. Rules capable of classifying diseases can be learned at this stage, and classification criteria can be defined to be used in the subsequent testing phase to classify fundus images.

[0134] Once training is complete, the AI ​​management module (140) moves on to the testing phase, and new test images are preprocessed in the same way as the training data. This means the process of preparing the trained model to predict new data.

[0135] Therefore, users can check the classified results through the user interface (UI) and determine which class the test image belongs to among NR (normal), GLC (glaucoma), AMD (age-related macular degeneration), and DR (diabetic retinopathy).

[0136]

[0137] The third-party connecting module (160) can be linked with the hospital information system (IHS) or a cloud-based external system and connected to external tools for additional analysis or report generation. This is described in more detail as follows.

[0138] FIG. 12 is a drawing showing the configuration of a third-party connecting module (160) in more detail.

[0139] Referring to FIG. 12, a third-party connecting module (160) according to one embodiment may include a third-party access controller module (161) that performs data communication with an external system, a PHR and HL7 FHIR parser module (162) that parses PHR and HL7 FHIR data received from an external system and converts it into a usable format, a PHR and HL7 FHIR request handler module (163) that searches for and provides the requested data when a specific data request is made by an external system, a consistency check module (164) that checks the consistency of data received from an external system and determines whether there is a conflict with existing data, a database controller module (165) that stores PHR and HL7 FHIR data, and a PHR and HL7 FHIR database (166) that manages data stored according to personal health records and HL7 FHIR standards.

[0140] Here, the third-party access controller module (161) is a component that manages data communication with an external system, processes data requests coming from the outside, and can safely transmit only the necessary data into the system.

[0141] Additionally, the PHR and HL7 FHIR parser module (162) parses (modifies) PHR and HL7 FHIR data received from an external system and converts it into a format that can be used within the system. In this process, the structure of the data is understood, and only the necessary parts can be extracted and processed.

[0142] Additionally, the PHR and HL7 FHIR request handler module (163) is a component that processes data requests from an external system, checks whether the requested data exists within the system, and, if necessary, retrieves and returns the data through the DB controller.

[0143] Additionally, the consistency check module (164) is a component that checks the consistency of data entering the system, and verifies whether the parsed data conflicts with existing data and whether consistency between data is maintained, thereby storing only the data with guaranteed consistency in the database.

[0144] Additionally, the database controller module (165) can control the database that stores and manages PHR and HL7 FHIR data. The database controller module (165) performs operations such as storing, updating, and deleting data, and can maintain the integrity of the database.

[0145] Additionally, the PHR and HL7 FHIR database (166) is a database that manages personal health records and data stored according to the HL7 FHIR standard, and may contain all health-related data that can be used within the system or retrieved by external requests.

[0146]

[0147] Next, we will examine in order how to detect a blinding disease using the AI-based blinding disease detection system (100) that was previously examined.

[0148] FIG. 13 is a flowchart illustrating an AI-based blindness disease detection method according to one embodiment of the present invention in a series of sequences.

[0149] Referring to Fig. 13, first, the patient's eye is photographed through one or more cameras (S1301), retinal images captured through each of the one or more cameras are collected, and the status of each camera is monitored (S1302).

[0150] Next, the collected retinal images are stored in a database (S1303), and the risk of blindness is assessed by analyzing the retinal images stored in the database based on an AI algorithm (S1304).

[0151] Next, the results of the assessment of the risk of blindness are provided to the customer terminal (S1305), and additionally, the system is linked with a hospital information system (IHS) or a cloud-based external system to generate additional analysis or reports through connection with external tools (S1306).

[0152]

[0153] Although the present invention has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the invention as understood by a person skilled in the art to which the invention pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification.

[0154] Explanation of the symbols

[0155] 100: AI-based blindness detection system

[0156] 110: Camera

[0157] 111: UI Controller Module

[0158] 111-1: User Identification Module

[0159] 111-2: Permission Extraction Module

[0160] 111-3: Use Case Selection Module

[0161] 111-4: UI Configuration Module

[0162] 111-5: UI Frontend Module

[0163] 112: Camera controller module

[0164] 113: Personal Service Provision Module

[0165] 114: Access controller module

[0166] 115: Rational Judgment Module

[0167] 115-1: Local Judgment Module

[0168] 115-2: Remote Judgment Module

[0169] 116: Database Manager Module

[0170] 117: Remote communicator

[0171] 120: Device Management Module

[0172] 130: Database

[0173] 131: Retina dataset storage

[0174] 132: Data Fetcher

[0175] 133: Device Synchronization Module

[0176] 134: Serial Number Management Module

[0177] 135: Firmware Version Management Module

[0178] 140: AI Management Module

[0179] 141: Raw database

[0180] 142: Preprocessing Module

[0181] 143: Trainer Module

[0182] 144: Model saving module

[0183] 145: Verification Module

[0184] 146: Inference Unit Configuration Module

[0185] 147: Ensemble Judgment Module

[0186] 148: Inference Pipeline Management Module

[0187] 150: Result providing module

[0188] 160: Third-party connecting module

[0189] 161: Third-party access controller module

[0190] 162: PHR and HL7 FHIR Parser Module

[0191] 163: PHR and HL7 FHIR Request Handler Module

[0192] 164: Consistency Check Module

[0193] 165: Database Controller Module

[0194] 166: PHR and HL7 FHIR databases

Claims

1. One or more cameras (110) for photographing the patient's eyes; A device management module (120) that collects retinal images captured through each of the one or more cameras (110) and monitors the status of each camera (110); A database (130) for storing the above retinal image; An AI management module (140) that evaluates the risk of blindness by analyzing the retinal images stored in the database (130) based on an AI algorithm; A result providing module (150) that provides the result of evaluating the risk of the above-mentioned blindness disease to a customer terminal; and A third-party connecting module (160) that is linked to a hospital information system (IHS) or a cloud-based external system and is connected to an external tool for additional analysis or report generation, comprising AI-based blindness detection system.

2. In Paragraph 1, The above camera (110) is, UI controller module (111) for managing the user interface; A camera controller module (112) that captures a retinal image during the shooting process through the above camera (110); A personal service provision module (113) that creates an account for each patient, provides personalized services based on feedback requested for each patient account, and manages health data for each patient account; An access controller module (114) that manages access rights for each patient account; A rational judgment module (115) that analyzes the captured retinal image; A database manager module (116) that stores the above user information, device owner information, retinal image and device information; and A remote communicator (117) that communicates with an external service module or an external remote server; comprising AI-based blindness detection system.

3. In Paragraph 2, The above UI controller module (111) is, A user identification module (111-1) that retrieves basic information about the user and verifies the user's identity; For a user whose user identity has been verified, an authority extraction module (111-2) that verifies the role and device control authority previously assigned to the user; A use case selection module (111-3) that determines the range of tasks that a user whose user identity has been verified can perform; A UI configuration module (111-4) that generates a UI to be provided to a user based on information provided through the above-mentioned authorization extraction module (111-2) and the above-mentioned use case selection module (111-3); and A UI frontend module (111-5) that provides the generated UI to a device for output, comprising AI-based blindness detection system.

4. In Paragraph 3, The above user identification module (111-1) retrieves basic information about the user from the user information database, and The above-mentioned authority extraction module (111-2) checks the roles and device control authority assigned to the user from the RBAC information database and the device control information database, and The above use case selection module (111-3) determines the scope of work by obtaining user authority and role information from the use case database. AI-based blindness detection system.

5. In Paragraph 4, The above rational judgment module (115) is, A local judgment module (115-1) that analyzes a retinal image captured in a local environment and provides an analysis result; and A remote judgment module (115-2) that runs in a remote server or cloud environment and analyzes the retinal image at a faster speed than the local judgment module (115-1) to provide an analysis result; comprising AI-based blindness detection system.

6. In Paragraph 5, The above database (130) is, A retinal dataset storage unit (131) that stores a retinal dataset; A data fetcher (132) that retrieves a specific retinal dataset from the above retinal dataset storage unit (131); A device synchronization module (133) that is linked with the data fetcher (132) and synchronizes user information, device control information, and the retinal dataset; A serial number management module (134) for managing and tracking unique serial numbers for each device; and A firmware version management module (135) for managing firmware versions for each device; including AI-based blindness detection system.

7. In Paragraph 6, The above AI management module (140) is, Raw database (141) for storing raw data required for learning and inference; A preprocessing module (142) that refines the above raw data, normalizes it, extracts features, and preprocesses it into a format for model training; A trainer module (143) that trains an AI model using pre-processed data; A model storage module (144) for storing the learned AI model; A verification module (145) that retrieves specific raw data from the raw database (141), inputs it into the learned AI model, and evaluates and verifies the performance of the AI ​​model; After loading the AI ​​model from the above model storage module (144), an inference unit configuration module (146) that configures one or more inference units based on the AI ​​model; An ensemble judgment module (147) that aggregates the prediction results of each of the above one or more inference units; and Including an inference pipeline management module (148) that inputs input data into the ensemble judgment module (147) to derive a final inference result, and then provides the derived final inference result to a device. AI-based blindness detection system.

8. In Paragraph 7, The above AI management module (140) is, Before inputting the retinal image into the above CNN model, contrast-limited adaptive histogram equalization (CLAHE) is performed on the input retinal image, downsampling is performed so that the size of the retinal image is reduced to fit the above CNN model, and the dataset of the retinal image is augmented using a random rotation technique, a scaling technique, or a flipping technique. AI-based blindness detection system.

9. In Paragraph 8, The above AI management module (140) is, Using any one of InceptionResNetV2, DenseNet201, and ResNet50 pre-trained CNN architecture models, and training said CNN architecture models through K-fold cross-validation, augmented data training, and model ensemble processes, AI-based blindness detection system.

10. In Paragraph 9, The above AI management module (140) is, Evaluating the performance of the above AI model using one or more of the area under the ROC curve (AUC) metric, the sensitivity metric, and the specificity metric, AI-based blindness detection system.

11. In Paragraph 10, The above AI management module (140) is, A fundus image adjusted to a size of 299*299*3 is input as input data to an Inception-v3 model, features of the fundus image are extracted using the Inception-v3 model to generate a feature map, the dimensionality is reduced by averaging the feature map using an Average Pooling (2D Layer) technique, the fundus image is flattened after averaging to convert it into a 1D vector form, training for final classification is performed in a Fully Connected Layer based on the features of the fundus image, and an output classifying quality is generated using a Softmax activation function. AI-based blindness detection system.

12. In Paragraph 11, The above AI management module (140) is, The method involves resizing the input retinal image to 632 pixels, performing image preprocessing by automatically detecting the Region of Interest (ROI) and applying an elliptical mask, classifying the transposed retinal image into a training set and a validation set using a 10-fold cross-validation split as training data, mixing the training and validation data to prevent bias based on data order, storing the trained AI model in a knowledge base according to pre-learned classification rules, and then conducting a validation test. AI-based blindness detection system.

13. In Paragraph 12, The above third-party connecting module (160) is, A third-party access controller module (161) that performs data communication with an external system; A PHR and HL7 FHIR parser module (162) that parses PHR and HL7 FHIR data received from the above external system and converts it into a usable format; A PHR and HL7 FHIR request handler module (163) that searches for and provides the requested data when a specific data request is made by the external system above; A consistency check module (164) that checks the consistency of data received from the above external system and determines whether there is a conflict with existing data; A database controller module (165) that stores the above PHR and HL7 FHIR data; and PHR and HL7 FHIR databases (166) for managing personal health records and data stored in accordance with HL7 FHIR standards; AI-based blindness detection system.

14. In Paragraph 13, The above AI management module (140) is, Using a Convolutional Neural Network (CNN) model to extract features from the retinal images to detect details related to retinal diseases, including diabetic retinopathy (DR), glaucoma (GLC), and age-related macular degeneration (AMD), AI-based blindness detection system.

15. A step of photographing the patient's eyes through one or more cameras; A step of collecting retinal images captured by each of the one or more cameras through a device management module and monitoring the status of each camera; Step of storing the above retinal image in a database; A step of evaluating the risk of blindness by analyzing the retinal images stored in the database based on an AI algorithm through an AI management module; A step of providing the results of the assessment of the risk of the blindness disease to a customer terminal through a result provision module; and A step comprising: connecting with a hospital information system (IHS) or a cloud-based external system through a third-party connecting module to generate additional analysis or reports through connection with external tools; AI-based blindness detection method.

Images