Method, device, and computer program for classifying keratitis activity using artificial intelligence

Abstract

The present invention relates to a method for classifying keratitis activity using artificial intelligence, comprising: an image receiving step of receiving an anterior segment image of a subject; a preprocessing step of obtaining a pupil region image from the received anterior segment image; and a training step of training an artificial intelligence model using the pupil region image labeled with the activity level of keratitis.

Description

Method, device and computer program for classifying keratitis activity using artificial intelligence

[0001] This invention was made with the support of the Ministry of Science and ICT under the project identification number 1711180602 and project number 2021R1C1C1007795. The research management specialized institution of the said project is the National Research Foundation of Korea, the research project name is "Individual Basic Research (Ministry of Science and ICT)", the research project name is "Machine learning-based cataract clinical diagnosis technology for national eye examination and development of treatment algorithm according to severity", the main institution is the Samsung Life Public Welfare Foundation, and the research period is from March 1, 2023 to February 29, 2024.

[0002] Keratitis is a condition in which the cornea becomes inflamed due to various causes. Depending on the pathogen and the patient's condition, keratitis can lead to perforation, decreased vision, and even vision loss. Therefore, appropriate treatment, timing, and duration are crucial. Determining the activity level of keratitis is crucial for determining the appropriate treatment, timing, and duration.

[0003] In existing clinical settings, the standard for assessing the activity of keratitis is the results of a culture test of pathogens collected from the lesion. However, this method has the drawback of requiring considerable time for culture, making immediate assessment and response difficult.

[0004] In this regard, Korean Patent No. 10-2329313 discloses a corneal lesion analysis system and method using an anterior segment image and a computer-readable recording medium. The prior art document is characterized by including an image acquisition unit that acquires an anterior segment image from the eye of a subject, and a feature extraction unit that extracts feature information regarding the location and cause of the corneal lesion from the anterior segment image by applying a convolution layer with the anterior segment image through machine learning based on a database storing clinical information analyzing the location and cause of the corneal lesion of the subjects previously acquired.

[0005] However, the above prior literature has limitations in that it takes a long time to perform calculations because it inputs the anterior segment image directly into artificial intelligence for learning and feature extraction without a preprocessing step, and it only extracts feature information about the location and cause of corneal lesions and does not extract the activity of keratitis.

[0006] Therefore, there is a need for a method, device, and computer program for classifying keratitis activity using artificial intelligence that can solve this problem.

[0007] The present invention aims to provide a method, device and computer program for classifying corneal activity using artificial intelligence, which trains artificial intelligence using an image of the pupil region extracted from an anterior segment image.

[0008] In addition, the present invention aims to classify the activity of keratitis by inputting an image of the patient's pupil area into a learned artificial intelligence model.

[0009] In order to achieve the above object, the present invention is characterized by a method for classifying the activity of keratitis using artificial intelligence, comprising: an image receiving step of receiving an anterior segment image of a subject; a preprocessing step of obtaining an image of a pupil region from the received anterior segment image; and a learning step of training an artificial intelligence model with the image of the pupil region labeled with the activity of keratitis and whether or not it is present.

[0010] Preferably, the preprocessing step may include a position information extraction step for extracting pupil position and pupil radius information from the anterior segment image; and a region extraction step for extracting a pupil region from the anterior segment image using the extracted pupil position and radius information.

[0011] Preferably, the location information extraction step is a method in which the anterior segment image is converted to grayscale, the histogram distribution is evenly converted through normalization, and the value difference between pixels is smoothly converted by applying blur.

[0012] Preferably, the preprocessing step can group a plurality of pixels constituting the anterior segment image into a predetermined number of units, obtain an average pixel value for each group, and then calculate a difference value between the average pixel values ​​and those of adjacent groups.

[0013] Preferably, the preprocessing step may binarize the anterior segment image using a threshold value selected through the Otsu method based on a histogram of differences in the calculated pixel average values.

[0014] Preferably, the preprocessing step may apply a contour to the binarized anterior segment image to emphasize the boundary, and use the Hough Circle Transform (CHT) to obtain pupil position and pupil radius information.

[0015] Preferably, the above region extraction step can extract a circular region that is wider by a predetermined range than the extracted pupil radius based on the extracted pupil position.

[0016] Preferably, the region extraction step can obtain a first region by inputting the range of the extracted circle into the prompt mode of SAM (Segment Anything Model), and can obtain a plurality of second regions by inputting the range of the extracted circle into the auto mode of SAM (Segment Anything Model).

[0017] Preferably, the region extraction step can compare the first region and the second region in an IoU (Intersection over Union) manner, and extract the second region if it is greater than a specific value, and extract the first region if it is less than a specific value.

[0018] Preferably, the method may further include an activity classification step of inputting an image of the subject's pupil area into a learned artificial intelligence model and outputting the activity level and whether or not keratitis is cured.

[0019] In addition, the present invention is a device for classifying keratitis activity using artificial intelligence, comprising: a processor including one or more cores; and a memory; wherein the processor receives an anterior segment image of a subject, acquires an image of a pupil region from the received anterior segment image, trains an artificial intelligence model with the image of the pupil region labeled with the activity of keratitis and whether or not it has been cured, and inputs the image of the pupil region of the subject into the trained artificial intelligence model to output the activity of keratitis and whether or not it has been cured.

[0020] In addition, the present invention is a computer program including commands stored in a computer-readable storage medium that cause a computer to perform the following operations, wherein the operations include: an image receiving operation for receiving an anterior segment image of a subject; a preprocessing operation for obtaining a pupil region image from the received anterior segment image; and a learning operation for training an artificial intelligence model with the pupil region image labeled with the activity level of keratitis; and another feature of the present invention is that the preprocessing operation includes a position information extraction operation for extracting pupil position and pupil radius information from the anterior segment image; and a region extraction operation for extracting a pupil region from the anterior segment image using the extracted pupil position and radius information.

[0021] The present invention has the advantage of being able to resolve the inconvenience of existing manual work and improve the learning efficiency of artificial intelligence models by utilizing pupil region images extracted from anterior segment images for learning.

[0022] In addition, the present invention has the advantage of being able to provide information that can be referenced by doctors by classifying the activity of keratitis using only camera photographs commonly taken by patients with keratitis.

[0023] In addition, the present invention has the advantage of enabling appropriate responses in a timely manner through activity classification using artificial intelligence instead of culture test results that take a relatively long time.

[0024] The present invention has the advantage of being able to resolve the inconvenience of existing manual work and improve the learning efficiency of artificial intelligence models by utilizing pupil region images extracted from anterior segment images for learning.

[0025] In addition, the present invention has the advantage of being able to provide information that can be referenced by doctors by classifying the activity of keratitis using only camera photographs commonly taken by patients with keratitis.

[0026] In addition, the present invention has the advantage of enabling appropriate responses in a timely manner through activity classification using artificial intelligence instead of culture test results that take a relatively long time.

[0027] A method for classifying corneal activity using artificial intelligence, comprising: an image receiving step for receiving an anterior segment image of a subject; a preprocessing step for obtaining a pupil region image from the received anterior segment image; and a learning step for training an artificial intelligence model using the pupil region image labeled with the activity and presence or absence of keratitis.

[0028] Hereinafter, the present invention will be described in detail with reference to the contents described in the attached drawings. However, the present invention is not limited or restricted by the exemplary embodiments. The same reference numerals in each drawing indicate components that perform substantially the same functions.

[0029] The purpose and effects of the present invention can be naturally understood or made clearer by the following description, and the purpose and effects of the present invention are not limited solely by the following description. Furthermore, in describing the present invention, if a detailed description of known technologies related to the present invention is deemed to unnecessarily obscure the gist of the present invention, such detailed description will be omitted.

[0030] The terminology used herein is merely used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the description of the invention, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0031] While terms like "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0032] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0033] When interpreting components, even if there is no explicit description, it is interpreted as including the margin of error. When describing temporal relationships, for example, when temporal continuity is described with phrases such as "after," "following," "next to," or "before," this also includes cases where the relationship is not continuous, unless "immediately" or "directly" is used.

[0034] Hereinafter, the technical configuration of the present invention will be described in detail with reference to the attached drawings.

[0035] Figure 1 illustrates a flowchart of a method for classifying corneal activity using artificial intelligence according to an embodiment of the present invention. Referring to Figure 1, the method for classifying corneal activity using artificial intelligence may include an image receiving step (S100), a preprocessing step (S300), a learning step (500), and an activity classification step (S700).

[0036] Anterior segment images are photographs taken of the eye with a camera. In addition to the pupil, which is the area affected by keratitis, they also include areas such as the eyelids and eyebrows, which are unnecessary information for model learning. Typically, removing these areas requires manual processing of the entire data set or the development of a segmentation model using mask data created from a subset of the data. However, a method for classifying keratitis activity using AI can automatically extract the pupil region for use in AI learning without this process.

[0037] This AI-based method for classifying keratitis activity provides a novel algorithm by appropriately combining previously proposed fragmented functions (Otsu method, Hough circle) and large-scale AI (Segment Anything Medel; SAM). This AI-based method for classifying keratitis activity can identify and extract the location and area of ​​the pupil, even when the border between the iris and conjunctiva is unclear due to disease progression or imaging techniques. This AI-based method for classifying keratitis activity can classify keratitis activity and whether it has been resolved by pathogen.

[0038] The method for classifying keratitis activity using artificial intelligence utilizes pupil images extracted from anterior segment images for learning, thereby eliminating the inconvenience of existing manual work and improving the learning efficiency of artificial intelligence models. Furthermore, the method for classifying keratitis activity using artificial intelligence has the advantage of providing doctors with useful information by classifying keratitis activity using only camera-based examination photos commonly performed on keratitis patients. Furthermore, the present invention enables prompt and appropriate response through activity classification using artificial intelligence, instead of relying on culture test results, which take a relatively long time.

[0039] Fig. 2 is a summary diagram of a method for classifying corneal activity using artificial intelligence according to an embodiment of the present invention. Referring to Fig. 2, the method for classifying corneal activity using artificial intelligence can obtain an image of the pupil region through a preprocessing step (S300) of the anterior segment image of the subject received in the image receiving step (S100), and input the obtained image of the pupil region into an artificial intelligence model.

[0040] The image receiving step (S100) can receive an image of the anterior segment of the subject. The anterior segment image can be acquired using a slit lamp microscope in a clinic, hospital, or other location, or an imaging device installed in a user device. The user device can include any device that includes a photographing module and can take pictures, such as a smartphone, tablet, or digital camera.

[0041] The image receiving step (S100) can transmit the received anterior segment image to be preprocessed in the preprocessing step (S300), and can also be input into an artificial intelligence model for learning in the learning step (S500).

[0042] The preprocessing step (S300) can acquire an image of the pupil region from the received anterior segment image. The preprocessing step (S300) can perform preprocessing to input the anterior segment image received in the image receiving step (S100) into an artificial intelligence model. The preprocessing step (S300) can provide the preprocessed image as training data for the artificial intelligence model. In addition, the preprocessing step (S300) can provide the preprocessed image to the trained artificial intelligence model to classify the keratitis activity level.

[0043] An anterior segment image is a photograph taken of the eye area, and includes the pupil, which is the area where keratitis occurs, as well as the eyelids and eyebrows. The preprocessing step (S300) according to an embodiment of the present invention can acquire only the image of the pupil, which is the area where keratitis occurs, from the anterior segment image. In this way, when an anterior segment image containing unnecessary information is used for learning or classification of an artificial intelligence model, the model may make predictions using information at a location other than the affected area (cornea). By removing the image around the pupil, which is unnecessary information, through the preprocessing step (S300) according to an embodiment of the present invention, the artificial intelligence model can accurately see and learn the affected area (cornea), thereby improving the reliability and validity of the artificial intelligence model.

[0044] The preprocessing step (S300) may include a location information extraction step (S310) and an area extraction step (S330).

[0045] The position information extraction step (S310) can extract pupil location and pupil radius information from the anterior segment image. The pupil location and pupil radius information extracted in the position information extraction step (S310) can serve as a reference for the location and range of the region extracted in the region extraction step (S330). In addition, the pupil location and pupil radius information extracted in the position information extraction step (S310) can be input into the SAM (Segment Anything Model) along with the anterior segment image to extract the pupil region.

[0046] FIG. 3 is a diagram illustrating a location information extraction step (S310) according to an embodiment of the present invention. Referring to FIG. 3, each function of the location information extraction step (S310) will be described.

[0047] The location information extraction step (S310) can convert the anterior segment image to grayscale, normalize it to evenly transform the histogram distribution, and apply blur to smoothly transform the value difference between pixels. The leftmost photo in Fig. 3 is the original anterior segment image, the second photo from the left is the photo converted to grayscale and normalized, and the third photo from the left is the photo with blur applied. The reason the location information extraction step (S310) performs such a function is to exclude unnecessary boundary parts caused by shadows, skin texture, blood vessels, fine hairs, etc. before extracting information on the location and radius of the pupil.

[0048] The position information extraction step (S310) can group multiple pixels constituting the anterior segment image into a predetermined number of units, obtain the pixel average value for each group, and then calculate the difference value between the pixel average value and the adjacent group. The position information extraction step (S310) can apply grayscale conversion, normalization, blurring, etc. to the anterior segment image prior to performing the present embodiment.

[0049] The position information extraction step (S310) can binarize the anterior segment image using a threshold value selected through the Otsu method based on a histogram of the differences in the calculated pixel average values. The position information extraction step (S310) has a function of calculating the pixel average value for each group after grouping pixels and then calculating the difference value of the pixel average value with that of the adjacent group, and can obtain a clear boundary line from the anterior segment image through binarization. The third photo from the left in Fig. 3 is a photo to which binarization has been applied.

[0050] Here, the Otsu method refers to an algorithm that randomly sets a boundary value to divide pixels into two groups, repeatedly calculates the intensity distributions of the two groups, and then selects the boundary value that most evenly distributes the intensity between the two groups. In other words, the Otsu method can mean finding the optimal threshold value that minimizes the difference in the proportion of pixels classified into binary categories based on a specific threshold value.

[0051] The location information extraction step (S310) applies a contour to the binarized anterior segment image to emphasize the boundary line, and extracts pupil location and pupil radius information using Hough Circle Transform (CHT). Here, the contour refers to boundary information of an area with the same color or the same pixel value. In addition, Hough Circle Transform refers to a method of detecting a circle using the extracted edge information. The second photo from the left in Fig. 3 is a photo to which the contour has been applied, and the rightmost photo is a photo to which the Hough Circle Transform has been applied.

[0052] Fig. 4 is a diagram illustrating an area extraction step (S330) according to an embodiment of the present invention. Referring to Fig. 4, each function of the area extraction step (S330) will be described.

[0053] The region extraction step (S330) can extract the pupil region from the anterior segment image using the extracted pupil location and radius information.

[0054] The area extraction step (S330) can extract a circular area that is wider by a predetermined range than the extracted pupil radius based on the extracted pupil position.

[0055] The region extraction step (S330) can obtain a first region by inputting the extracted circle range into the prompt method of the SAM (Segment Anything Model), and can obtain multiple second regions by inputting the extracted circle range into the auto method of the SAM (Segment Anything Model). At this time, when obtaining multiple second pupil regions using the auto method of the SAM (Segment Anything Model), all possible regions can be obtained, and among these, only regions larger than a specific width can be obtained. The first region obtained by the prompt method can be a pupil region, and the second region obtained by the auto method of the SAM can be multiple pupil region candidates.

[0056] The region extraction step (S330) compares the first region and the second region using the Intersection over Union (IoU) method, and extracts the second region if the value is greater than a certain value, and extracts the first region if the value is less than a certain value. Preferably, the specific value may be 0.7, but is not limited thereto.

[0057] The learning step (500) can train an artificial intelligence model using images of the pupil region labeled with the activity and presence of keratitis. In this case, the images of the pupil region may refer to a dataset constructed in the region extraction step (S330).

[0058] The artificial intelligence model used in the embodiment of the present invention may be a deep neural network, and preferably a convolutional neural network (CNN).

[0059] A convolutional neural network (CNN) is a type of deep learning model inspired by the structure of the visual cortex of animals, designed to process data with grid patterns, such as images. A convolutional neural network typically includes convolutional layers, pooling layers, and fully connected layers. Convolutional and pooling layers can exist repeatedly within a neural network, and input data can be transformed into output through these layers. Convolutional layers utilize kernels (or masks) to extract features. The element-wise product between each element of the kernel and the input value is computed at each location and summed to obtain the output, which is called a feature map. This process can be repeated, applying multiple kernels to form any number of feature maps. In a convolutional neural network, convolutional and pooling layers perform feature extraction, while fully connected layers map the extracted features to the final output, such as a classification operation.

[0060] Convolutional neural networks, such as neural networks, can be trained to minimize output errors. Separate from the forward propagation process, which extracts values ​​from the input layer to the output layer, backpropagation occurs within the neural network, calculating the error between the input training data and the corresponding neural network output values ​​and updating the weights of the nodes in each layer to reduce this error. The training process in convolutional neural networks can be summarized as finding the kernel that extracts the output values ​​with the lowest error based on the given training data. The kernel is the only parameter that is automatically learned during the training of the convolutional layer. On the other hand, in convolutional neural networks, kernel size, number of kernels, padding, etc. are hyperparameters that must be set before the training process begins. Therefore, convolutional neural network models can be distinguished based on the kernel size, number of kernels, and number of convolutional and pooling layers.

[0061] A neural network can be trained by at least one of supervised learning, which uses training data with labeled answers for each training data, unsupervised learning, semi-supervised learning, or reinforcement learning, in which the training data are not labeled with answers. In this case, an error can be calculated by comparing the output from the neural network with the label or training data, and the calculated error is backpropagated in the backward direction (i.e., from the output layer to the input layer) in the neural network, and the connection weights of each node in each layer of the neural network can be updated according to the backpropagation. The amount of change in the connection weights of each node that are updated can be determined according to a learning rate.

[0062] Overfitting occurs when neural networks learn excessively from training data, resulting in increased errors even as the number of training data increases. Overfitting can increase errors in machine learning algorithms, and various optimization methods can be used to prevent it. Methods to prevent overfitting include increasing the training data, regularization, dropout (inactivating some network nodes during the learning process), and the use of batch normalization layers.

[0063] The activity classification step (S700) can input an image of the subject's pupil area into a learned artificial intelligence model and output the activity level and status of keratitis.

[0064] FIG. 5 illustrates a configuration diagram of a device (100) for classifying keratitis activity using artificial intelligence according to an embodiment of the present invention. Referring to FIG. 5, the configuration of the device (100) for classifying keratitis activity using artificial intelligence illustrated is merely a simplified example. In one embodiment of the present invention, the device (100) for classifying keratitis activity using artificial intelligence may include other configurations for implementing the computing environment of the device (100), and only some of the disclosed configurations may constitute the device (100).

[0065] A device (100) for classifying corneal activity using artificial intelligence may include a processor (110) including one or more cores, a memory (120), and a network (130).

[0066] The processor (110) may be configured with one or more cores, and may include a processor for data analysis and deep learning, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device. The processor (110) may read a computer program stored in the memory (120) and perform data processing for machine learning according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor (110) may perform operations for learning a neural network. The processor (110) may perform calculations for learning a neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating errors, and updating weights of a neural network using backpropagation. At least one of the CPU, GPGPU, and TPU of the processor (110) may process learning of a network function. For example, a CPU and a GPGPU can jointly process network function learning and data classification using network functions. Furthermore, in one embodiment of the present disclosure, processors of multiple computing devices can be jointly used to process network function learning and data classification using network functions. Furthermore, a computer program executed on a computing device according to one embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.

[0067] The processor (110) can receive an image of the anterior segment of the subject. The processor (110) can perform the image receiving step (S100) described above.

[0068] The processor (110) can obtain an image of the pupil region from the received anterior segment image. The processor (110) can perform the preprocessing step (S300) described above.

[0069] The processor (110) can train an artificial intelligence model using the image of the pupil region labeled with the activity and presence of keratitis. The processor (110) can perform the aforementioned learning step (S300).

[0070] The processor (110) can input an image of the subject's pupil area into a learned artificial intelligence model and output the activity level and presence of keratitis. The processor (110) can perform the activity classification step (S700) described above.

[0071] The memory (120) can store any form of information generated or determined by the processor (110) and any form of information received by the network (130).

[0072] The memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The computing device (100) may also operate in relation to web storage that performs the storage function of the memory (120) on the internet. The description of the above-described memory is merely an example, and the present disclosure is not limited thereto.

[0073] The network (130) may use any known wired or wireless communication system. The network (130) may receive endoscope images and the like from related devices or systems.

[0074] The network (130) can transmit and receive information, user interfaces, etc. processed by the processor (110) through communication with other terminals. For example, the network (130) can provide a user interface generated by the processor (100) to a client (e.g., a user terminal). In addition, the network (130) can receive external input from a user authorized as a client and transmit it to the processor (110). At this time, the processor (110) can process operations such as outputting, modifying, changing, and adding information provided through the user interface based on the external input of the user received from the network (130).

[0075] Meanwhile, a device (100) for classifying keratitis activity using artificial intelligence according to one embodiment of the present disclosure may include a server as a computing system that transmits and receives information through communication with a client. In this case, the client may be any type of terminal capable of accessing the server.

[0076] In an additional embodiment, a device (100) for classifying corneal activity using artificial intelligence may include any type of terminal that receives data resources generated from any server and performs additional information processing.

[0077] Another embodiment of the present invention, a computer program for classifying corneal activity using artificial intelligence, may perform image reception operations, preprocessing operations, learning operations, and activity classification operations. The computer program for classifying corneal activity using artificial intelligence may be stored in a computer-readable storage medium and include commands that cause the computer to perform the following operations.

[0078] The image receiving operation can receive an image of the anterior segment of the subject. The image receiving operation refers to the operation performed in the image receiving step (S100) described above.

[0079] The preprocessing operation can acquire an image of the pupil region from the received anterior segment image. The preprocessing operation may include a location information extraction operation and a region extraction operation. The preprocessing operation refers to the operation performed in the aforementioned preprocessing step (S300).

[0080] The location information extraction operation can extract pupil location and pupil radius information from the anterior segment image. The location information extraction operation refers to the operation performed in the location information extraction step (S310) described above.

[0081] The region extraction operation can extract the pupil region from the anterior segment image using the extracted pupil location and radius information. The region extraction operation refers to the operation performed in the region extraction step (S330) described above.

[0082] The learning action can be used to train an artificial intelligence model using the above-mentioned pupil area images labeled with the activity and presence of keratitis. The learning action refers to the action performed in the aforementioned learning step (S500).

[0083] The activity classification operation inputs an image of the subject's pupil area into a learned artificial intelligence model to output the activity level and status of keratitis. The activity classification operation refers to the operation performed in the aforementioned activity classification step (S700).

[0084] Figure 6 illustrates a schematic diagram of a computing environment according to an embodiment of the present invention.

[0085] Although the present disclosure has been described above as being generally implemented by a computing device, those skilled in the art will appreciate that the present disclosure may be implemented in combination with computer-executable instructions and / or other program modules that may be executed on one or more computers and / or as a combination of hardware and software.

[0086] Generally, program modules include routines, programs, components, data structures, and the like that perform particular tasks or implement particular abstract data types. Furthermore, those skilled in the art will appreciate that the methods of the present disclosure can be implemented with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which may be operatively connected to one or more associated devices.

[0087] The described embodiments of the present disclosure can also be practiced in distributed computing environments, where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0088] Computers typically include a variety of computer-readable media. Computer-readable media can be any media that can be accessed by a computer, and includes both volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media. By way of example, and not limitation, computer-readable media can include computer-readable storage media and computer-readable transmission media. Computer-readable storage media includes both volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be accessed by a computer and used to store the desired information.

[0089] Computer-readable transmission media typically includes any information delivery media that embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism. The term modulated data signal means a signal that has one or more of its characteristics set or changed so as to encode information in the signal. By way of example, and not limitation, computer-readable transmission media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, or other wireless media. Combinations of any of the above are also intended to be included within the scope of computer-readable transmission media.

[0090] An exemplary environment for implementing various aspects of the present disclosure is illustrated, including a computer (1000), which includes a processing unit (1020), a system memory (1030), and a system bus (1010). The system bus (1010) connects system components, including but not limited to the system memory (1030), to the processing unit (1020). The processing unit (1020) may be any of a variety of commercially available processors. Dual processors and other multiprocessor architectures may also be utilized as the processing unit (1020).

[0091] The system bus (1010) may be any of several types of bus structures that may be additionally interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. The system memory (1030) includes read-only memory (ROM) (1034) and random access memory (RAM) (1032). A basic input / output system (BIOS) is stored in non-volatile memory (1034), such as ROM, EPROM, or EEPROM, and includes basic routines that help transfer information between components within the computer (1000), such as during start-up. The RAM (1032) may also include high-speed RAM, such as static RAM, for caching data.

[0092] The computer (1000) also includes an internal hard disk drive (HDD) (1050) (e.g., EIDE, SATA) - which may also be configured for external use within a suitable chassis (not shown), a magnetic floppy disk drive (FDD) (1060) (e.g., for reading from or writing to removable diskettes), and an optical disk drive (1070) (e.g., for reading from or writing to CD-ROM disks or other high-capacity optical media such as DVDs). The hard disk drive (1050), the magnetic disk drive (1060), and the optical disk drive (1070) may be connected to the system bus (1010) by a hard disk drive interface, a magnetic disk drive interface, and an optical drive interface, respectively. Interfaces for implementing external drives include at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

[0093] These drives and their associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, and the like. In the case of the computer (1000), the drives and media correspond to storing any data in a suitable digital format. While the description of computer-readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will appreciate that other types of computer-readable media, such as zip drives, magnetic cassettes, flash memory cards, cartridges, and the like, may also be used in the exemplary operating environment, and that any such media may contain computer-executable instructions for performing the methods of the present disclosure.

[0094] A number of program modules, including an operating system (1092), one or more application programs (1094), other program modules (1096), and a database (1098), may be stored in the drive and RAM (1032). All or portions of the operating system, applications, modules, and / or data may also be cached in RAM (1032). It will be appreciated that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

[0095] A user may enter commands and information into the computer (1000) via one or more wired / wireless input devices (1042), such as a keyboard and a pointing device such as a mouse. Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, and the like. These and other input devices are often connected to the processing unit (1020) via an input / output interface (1040) that is connected to the system bus (1010), but may be connected by other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, and the like.

[0096] A monitor or other type of display device is also connected to the system bus (1010) via an interface such as a video adapter. In addition to the monitor, the computer typically includes other peripheral output devices (not shown) such as speakers, a printer, and so on.

[0097] The computer (1000) may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) (1082), via wired and / or wireless communications. The remote computer(s) (1082) may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other conventional network node, and may generally include many or all of the components described for the computer (1000). The logical connections include wired / wireless connections to a local area network (LAN) and / or a larger network, such as a wide area network (WAN). Such LAN and WAN networking environments are common in offices and companies and facilitate enterprise-wide computer networks, such as intranets, all of which may be connected to a worldwide computer network, such as the Internet.

[0098] When used in a LAN networking environment, the computer (1000) is connected to a local network (not shown) via a wired and / or wireless communication network interface or adapter (not shown). The adapter (not shown) may facilitate wired or wireless communication to the LAN (not shown), which may also include a wireless access point installed therein for communicating with the wireless adapter (not shown). When used in a WAN networking environment, the computer (1000) may include a modem (not shown), be connected to a communication computing device on the WAN (not shown), or have other means for establishing communications over the WAN (not shown), such as via the Internet. The modem (not shown), which may be internal or external and wired or wireless, is connected to the system bus (1010) via a serial port interface (not shown). In a networked environment, program modules described for the computer (1000), or portions thereof, may be stored in a remote memory / storage device (not shown). It will be appreciated that the network connections shown are exemplary and that other means of establishing a communications link between computers may be used.

[0099] The computer (1000) operates to communicate with any wireless device or object that is arranged and operates via wireless communication, such as a printer, a scanner, a desktop and / or portable computer, a portable data assistant (PDA), a communication satellite, any equipment or location associated with a radio-detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure, as in a conventional network, or simply an ad hoc communication between at least two devices.

[0100] Wi-Fi (Wireless Fidelity) enables connections to the Internet and other devices without wires. Wi-Fi is a wireless technology that allows devices, such as computers, to send and receive data anywhere within the coverage area of ​​a base station, both indoors and outdoors, similar to cell phones. Wi-Fi networks use wireless technologies called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in the unlicensed 2.4 and 5 GHz radio bands, at data rates of, for example, 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual-band).

[0101] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0102] Those skilled in the art will appreciate that the various illustrative logical blocks, modules, processors, means, circuits, and model steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, various forms of programs or design code (referred to herein, for convenience, as software), or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0103] The various embodiments presented herein can be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term article of manufacture includes a computer program, carrier, or media accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Furthermore, various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0104] It should be understood that the specific order or hierarchy of steps in the presented processes is merely an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but are not intended to be limited to the specific order or hierarchy presented.

[0105] The description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments disclosed herein, but is to be construed in the broadest scope consistent with the principles and novel features disclosed herein.

[0106] The embodiments of the present invention described above are not implemented solely through devices and methods. They may also be implemented through programs that implement functions corresponding to the configurations of the embodiments of the present invention, or through recording media containing such programs. Such recording media may be executed not only on servers but also on user terminals.

[0107] Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims also fall within the scope of the present invention.

[0108] While the present invention has been described in detail through representative examples above, those skilled in the art will understand that various modifications to the above-described embodiments are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims described below but also by all changes or modifications derived from the claims and equivalent concepts.

[0109] The present invention aims to provide a method, device and computer program for classifying corneal activity using artificial intelligence, which trains artificial intelligence using an image of the pupil region extracted from an anterior segment image.

[0110] In addition, the present invention aims to classify the activity of keratitis by inputting an image of the patient's pupil area into a learned artificial intelligence model.

Claims

1. A method for classifying keratitis activity using artificial intelligence, An image receiving step for receiving an image of the anterior segment of the subject; A preprocessing step for obtaining an image of the pupil region from the received anterior segment image; and A learning step for training an artificial intelligence model with the above pupil area images labeled with the activity and presence of keratitis; A method comprising:

2. In paragraph 1, The above preprocessing step is, A position information extraction step for extracting pupil position and pupil radius information from the above anterior segment image; and A method comprising: a region extraction step for extracting a pupil region from the anterior segment image using extracted pupil position and radius information; 3. In paragraph 2, The above location information extraction step is, A method of converting the above anterior segment image to grayscale, converting the histogram distribution to be even through normalization, and applying a blur to smoothly convert the value difference between pixels.

4. In paragraph 2, The above location information extraction step is, A method of grouping a plurality of pixels constituting the above anterior segment image into a predetermined number of units, calculating an average pixel value for each group, and then calculating the difference value between the average pixel values and those of adjacent groups.

5. In paragraph 4, The above location information extraction step is, A method for binarizing the anterior segment image using a threshold value selected through the Otsu method based on a histogram of differences from the produced pixel average values.

6. In paragraph 5, The above location information extraction step is, A method of applying a contour to a binarized anterior segment image to emphasize a boundary line and extracting pupil location and pupil radius information using a Hough Circle Transform (CHT).

7. In paragraph 2, The above area extraction step is, A method of extracting a circular area that is wider by a predetermined range than the radius of the extracted pupil based on the extracted pupil location.

8. In paragraph 7, The above area extraction step is, The first region is obtained by entering the range of the extracted circle into the prompt mode of SAM (Segment Anything Model), A method of obtaining multiple second regions by inputting the range of an extracted circle into the auto mode of SAM (Segment Anything Model).

9. In paragraph 8, The above area extraction step is, A method for comparing the first region and the second region using an IoU (Intersection over Union) method, extracting the second region if it is greater than a specific value, and extracting the first region if it is less than a specific value.

10. In paragraph 1, A method further comprising an activity classification step of inputting an image of the subject's pupil area into a learned artificial intelligence model and outputting the activity level and whether or not keratitis is cured.

11. A device for classifying keratitis activity using artificial intelligence. a processor comprising one or more cores; and memory; Including, The above processor, Receive an image of the subject's anterior segment, Obtain an image of the pupil region from the received anterior segment image, An artificial intelligence model is trained with the above pupil area images labeled with the activity and presence of keratitis, A device that inputs an image of a subject's pupil area into a learned artificial intelligence model and outputs the activity level and status of keratitis.

12. A computer program including commands stored in a computer-readable storage medium that cause a computer to perform the following operations, wherein the operations are: Image receiving operation for receiving an image of the anterior segment of the subject; Preprocessing operation for obtaining an image of the pupil region from the received anterior segment image; and A learning operation for training an artificial intelligence model with the image of the pupil region labeled with the activity and presence of keratitis; The above preprocessing operation is, A computer program stored in a computer-readable storage medium, comprising: a position information extraction operation for extracting pupil position and pupil radius information from the anterior segment image; and a region extraction operation for extracting a pupil region from the anterior segment image using the extracted pupil position and radius information.

Images