Labeling system and method based on identification and prediction of multiple ophthalmic diseases

Abstract

The present invention relates to a labeling system and method based on identification and prediction of multiple ophthalmic diseases, wherein according to one embodiment of the present invention, the labeling system based on identification and prediction of multiple ophthalmic diseases may be configured to comprise: a collection unit for collecting a plurality of fundus images; a memory in which a multi-label classification program for classifying labels of a plurality of ophthalmic diseases is stored; and a processor for executing the program to classify labels representing respective ophthalmic diseases from the plurality of fundus images. The processor includes: a preprocessing unit for extracting optic disc and macula regions from each of the plurality of fundus images to generate and store a data set; a multi-label prediction unit for learning on the basis of the data set and calculating a probability that each fundus image belongs to a label representing each specific disease classification by using the learned result; and a multi-label classification unit for classifying labels for respective fundus images by using the calculated probabilities of each fundus image to derive an ophthalmic disease identification result.

Description

Multiple ophthalmic disease identification and prediction-based labeling system and method

[0001] The present invention relates to a system and method for multiple ophthalmic disease identification and prediction-based labeling, and more particularly, to a system and method for multiple ophthalmic disease identification and prediction-based labeling that calculates the probability of belonging to each label for each region of a fundus image through a plurality of classification models and integrates the results to derive ophthalmic disease results.

[0002] Early detection of ocular diseases can prevent vision loss. Ultra-wide-field fundus imaging (UFI) is an advanced imaging technique that has revolutionized the diagnosis and treatment of ophthalmic diseases. UFI provides an ultra-wide-angle view of the retina, up to 300 degrees, allowing the identification of peripheral diseases that are difficult to detect with conventional fundus imaging.

[0003] Much existing research has focused on techniques for identifying individual diseases, such as diabetic retinopathy, glaucoma, and retinal detachment. Some studies have attempted to classify multiple diseases, but their data is limited to images containing only a single disease, making them impractical for real-world applications.

[0004] The purpose of the present invention is to solve the above problems, and to construct a data set by extracting the optic nerve head and macular region from a plurality of fundus images, and to calculate the probability of belonging to each label representing each specific disease classification of the entire fundus image, the optic nerve head region image, and the macular region image through a plurality of classification models, and to derive and provide a result of an ophthalmic disease by integrating the calculation results of each classification model.

[0005] According to the present invention, it is intended to improve diagnostic accuracy, reduce the cost and time required for managing retinal diseases, and provide effective treatment through early diagnosis of the disease.

[0006] According to an embodiment of the present invention for solving the above-described problem, a multi-ocular disease identification and prediction-based labeling system may be configured to include: a collection unit for collecting a plurality of fundus images; a memory for storing a multi-label classification program for classifying labels of a plurality of ocular diseases; and a processor for executing the program to classify labels representing each ocular disease from the plurality of fundus images; wherein the processor comprises: a preprocessing unit for extracting an optic nerve head and a macular region from each of the plurality of fundus images to generate and store a data set; a multi-label prediction unit for learning based on the data set and calculating a probability that each fundus image belongs to each label representing each specific disease classification using the learned result; and a multi-label classification unit for classifying labels for each fundus image using the calculated probability of each fundus image to derive a result of the ocular disease.

[0007] According to another embodiment of the present invention, the fundus image may be composed of an Ultra-wide-field Fundus Image (UFI).

[0008] According to another embodiment of the present invention, the preprocessing unit includes an object detector that learns a method of detecting the optic nerve head and the macular region from each of the plurality of fundus images, and the object detector can extract a result indicating the optic nerve head region and the macular region detected from each of the plurality of fundus images.

[0009] According to another embodiment of the present invention, the data set may include a full area image of each of the fundus images, an optic disc area image obtained by extracting the optic disc area from each of the fundus images, and a macular area image obtained by extracting the macular area from each of the fundus images.

[0010] According to another embodiment of the present invention, the multi-label prediction unit learns a method of predicting a probability of being classified into at least one corresponding label among a plurality of CNNs (Convolutional Neural Networks) based on the data set, and can predict the probability of being classified into each label using the learned plurality of CNNs.

[0011] According to another embodiment of the present invention, the multi-label prediction unit may predict a probability of being classified as at least one first ophthalmic disease among the ophthalmic diseases by utilizing the full region image from a first CNN among the plurality of CNNs, predict a probability of being classified as at least one second ophthalmic disease among the ophthalmic diseases by utilizing the optic nerve head region image from a second CNN among the plurality of CNNs, and predict a probability of being classified as at least one third ophthalmic disease among the ophthalmic diseases by utilizing the macular region image from a third CNN among the plurality of CNNs.

[0012] According to another embodiment of the present invention, the multi-label classification unit can predict the probability of being classified as each of the first ophthalmic disease, each of the second ophthalmic disease, and each of the third ophthalmic disease by combining all of the probabilities of being classified as each of the third ophthalmic disease from the multi-label prediction unit.

[0013] According to another embodiment of the present invention, the multi-label classification unit can predict the probability of being classified as at least one of the various ocular diseases by multiplying the probability of being classified as each of the various ocular diseases by a corresponding weight among a plurality of probability weights and adding all the resulting values.

[0014] According to another embodiment of the present invention, the multi-label classification unit learns a first probability weight corresponding to a probability of being classified as each of the first ophthalmic diseases, learns a second probability weight corresponding to a probability of being classified as each of the second ophthalmic diseases, and learns a third probability weight corresponding to a probability of being classified as each of the third ophthalmic diseases, and the sum of the first probability weight, the second probability weight, and the third probability weight may be configured not to exceed a threshold value.

[0015] A method for identifying and predicting multiple ophthalmic diseases according to an embodiment of the present invention comprises: a method for classifying labels representing each ophthalmic disease from a plurality of fundus images by executing a multi-label classification program in which a processor classifies labels of a plurality of ophthalmic diseases stored in a memory, the method comprising: a step of collecting a plurality of fundus images; a preprocessing step of extracting an optic disc and a macular region from each of the plurality of fundus images to generate and store a data set; a multi-label prediction step of learning based on the data set and calculating a probability that each fundus image belongs to each label representing each specific disease classification using the learned result; and a multi-label classification step of classifying labels for each fundus image using the calculated probability of each fundus image to derive a result of the ophthalmic disease.

[0016] According to another embodiment of the present invention, the preprocessing step may extract a result indicating the optic disc area and the macular area detected from each of the plurality of fundus images by using an object detector that learns a method of detecting the optic disc and the macular area from each of the plurality of fundus images.

[0017] According to another embodiment of the present invention, the multi-label prediction step trains a method of predicting a probability of classification into at least one corresponding label among a plurality of CNNs (Convolutional Neural Networks) based on the data set, and predicts the probability of classification into each label using the trained plurality of CNNs.

[0018] According to another embodiment of the present invention, the label prediction step may be configured to include: a step of predicting a probability of being classified as at least one first ophthalmic disease among the ophthalmic diseases by utilizing the full region image from a first CNN among the plurality of CNNs; a step of predicting a probability of being classified as at least one second ophthalmic disease among the ophthalmic diseases by utilizing the optic nerve head region image from a second CNN among the plurality of CNNs; and a step of predicting a probability of being classified as at least one third ophthalmic disease among the ophthalmic diseases by utilizing the macular region image from a third CNN among the plurality of CNNs.

[0019] According to another embodiment of the present invention, the multi-label classification step can predict the probability of being classified as at least one ophthalmic disease among the various ophthalmic diseases by combining the probability of being classified as each of the first ophthalmic disease, the probability of being classified as each of the second ophthalmic disease, and the probability of being classified as each of the third ophthalmic disease.

[0020] According to another embodiment of the present invention, the multi-label classification step multiplies the probability of being classified as each of the various ocular diseases by a corresponding weight among a plurality of probability weights, and adds all the resulting values ​​to predict the probability of being classified as at least one ocular disease among the various ocular diseases.

[0021] According to another embodiment of the present invention, the multi-label classification step includes: a step of learning a first probability weight corresponding to a probability of being classified as each of the first ophthalmic diseases; a step of learning a second probability weight corresponding to a probability of being classified as each of the second ophthalmic diseases; and a step of learning a third probability weight corresponding to a probability of being classified as each of the third ophthalmic diseases; and the sum of the first probability weight, the second probability weight, and the third probability weight may be configured not to exceed a threshold value.

[0022] According to the present invention, an accurate early diagnosis can be provided when diagnosing an ophthalmic disease, and more specifically, a data set is constructed by extracting the optic disc and macular region from a plurality of fundus images, and a plurality of classification models are used to calculate the probability of belonging to each label representing each specific disease classification of the entire fundus image, the optic disc region image, and the macular region image, and an ophthalmic disease result can be derived by integrating the calculation results of each classification model.

[0023] According to the present invention, it is possible to improve diagnostic accuracy, reduce the cost and time required for managing retinal diseases, and enable effective treatment through early diagnosis of diseases.

[0024] FIG. 1 is a block diagram schematically illustrating the configuration of a multiple ophthalmic disease identification and prediction-based labeling system according to one embodiment of the present invention.

[0025] Figure 2 is an example diagram showing the distribution of lesions according to multiple ophthalmic diseases.

[0026] Figure 3 is a block diagram schematically showing the detailed configuration of the processor illustrated in Figure 1.

[0027] FIG. 4 is a flowchart of a multiple ophthalmic disease identification and prediction-based labeling method according to one embodiment of the present invention.

[0028] FIG. 5 is a diagram illustrating a multiple ophthalmic disease identification and prediction-based labeling method according to one embodiment of the present invention.

[0029] FIG. 7 is an example diagram of a classification result produced by a multiple ophthalmic disease identification and prediction-based labeling server according to one embodiment.

[0030] FIGS. 8 to 11 are diagrams comparing the results of a multiple ophthalmic disease identification and prediction-based labeling method according to one embodiment of the present invention with the results of a prior art.

[0031]

[0032] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention.

[0033] However, when describing embodiments, if a detailed description of a related known function or configuration is judged to unnecessarily obscure the gist of the present invention, a detailed description thereof will be omitted. Furthermore, the sizes of each component in the drawings may be exaggerated for illustrative purposes and do not necessarily represent the sizes actually applied.

[0034] Additionally, throughout the specification, when a component is referred to as being "connected" or "connected" to another component, it should be understood that the component may be directly connected or connected to the other component, but may also be connected or connected via another component in between, unless otherwise specifically stated. Additionally, throughout the specification, when it is said that a part "includes" a component, this does not exclude other components, but rather includes other components, unless otherwise specifically stated.

[0035] FIG. 1 is a block diagram schematically illustrating the configuration of a multiple ophthalmic disease identification and prediction-based labeling system according to one embodiment of the present invention.

[0036] Referring to FIG. 1, a multiple ophthalmic disease identification and prediction-based labeling system (1) may include a multiple ophthalmic disease identification and prediction-based labeling server (10) and a user terminal (20). The medical image classification server (10) may include a collection unit (100), a memory (200), and a processor (300).

[0037] The collection unit (100) can collect multiple fundus images (hereinafter, "multiple fundus images") taken of the fundus of multiple subjects. The collection unit (100) can receive multiple fundus images from the user terminal (20) by communicating with the user terminal (20) via a wired or wireless network.

[0038] Each of the multiple fundus images may be an Ultra-wide-field Fundus Image (UFI). Each of the multiple fundus images may be an image of the fundus of a patient diagnosed with multiple ophthalmic diseases. An ophthalmologist may label each of the multiple fundus images with at least one of the multiple ophthalmic diseases.

[0039] The multiple ophthalmic diseases may include diabetic retinopathy (DR), epiretinal membrane (ERM), glaucoma suspect (GS), macular degeneration (MD), retinal break (RB), retinal vein occlusion (RVO), etc. Hereinafter, the multiple ophthalmic diseases are described as the six ophthalmic diseases described above, but this is for convenience of explanation and the invention is not limited thereto.

[0040] The memory (200) may include volatile memory and / or non-volatile memory. The memory (200) may store, for example, commands or data related to the collection unit (100) and the processor (300), one or more programs and / or software, an operating system, etc., to implement and / or provide operations, functions, etc. provided by the multiple ophthalmic disease identification and prediction-based labeling system (1).

[0041] The program stored in the memory (200) may include a multi-label classification program (hereinafter, "multi-label classification program") that classifies labels of multiple ophthalmic diseases. The multi-label classification program may provide a service of classifying a fundus image into labels representing each of multiple ophthalmic diseases.

[0042] Figure 2 is an example diagram showing the distribution of lesions according to multiple ophthalmic diseases.

[0043] Hereinafter, lesions corresponding to each of multiple ophthalmic diseases will be described with reference to FIG. 2.

[0044] Diabetic retinopathy (DR) can be a diabetic complication that affects the blood vessels of the retina. DR can present with lesions such as microaneurysms, hemorrhages, hard exudates, cotton wool spots, neovascularization, and vitreous hemorrhages. Referring to Figure 2, lesions associated with DR typically appear in the periphery of the retina.

[0045] Retinal detachment (RB) can be a type of retinal disorder in which the retina detaches or separates from the underlying tissue. RB is characterized by a retinal tear or hole. Referring to Figure 2, lesions corresponding to RB can occur anywhere in the UFI.

[0046] Retinal vein occlusion (RVO) can be a blockage of a retinal vein, causing blood and fluid to accumulate in the retina. RVO can manifest as retinal hemorrhages, cotton wool spots, macular edema, and neovascularization. Referring to Figure 2, lesions associated with RVO typically appear in the central portion of the UFI and the peripheral retina of the central portion.

[0047] ERM can be a condition in which a thin layer of tissue grows on the surface of the retina, causing visual distortion. ERM can manifest as lesions such as a wrinkled or folded retina, cystic spaces, and macular distortion. Referring to Figure 2, lesions corresponding to ERM typically appear in the macular region of the UFI.

[0048] AMD is a disease in which the macula, responsible for central vision, deteriorates over time. AMD can be characterized by lesions such as drusen, pigmentary changes, geographic atrophy, and neovascularization. Referring to Figure 2, lesions associated with AMD typically appear in the macula of the UFI.

[0049] GSM can indicate conditions that may lead to glaucoma. Glaucoma is a group of ophthalmic diseases that damage the optic nerve and can lead to blindness. Some UFIs indicating GS may exhibit lesions such as optic disc changes and retinal nerve fiber layer defects. Referring to Figure 2, lesions corresponding to GS can be found around the optic disc.

[0050] Therefore, in one embodiment, in the multi-label classification program, the processor (300) can determine a region (hereinafter, "discrimination region") to be a discrimination target of a label corresponding to each ophthalmic disease based on the spatial distribution of the plurality of ophthalmic diseases. For example, among the plurality of ophthalmic diseases, the labels corresponding to each of DR, RB, and RVO may have the entire region of the UFI as a discrimination region, the labels corresponding to each of AMD and ERM may have mainly the macular region among the UFI as a discrimination region, and the label corresponding to GS may have mainly the area near the optic nerve head among the UFI as a discrimination region. Hereinafter, the discrimination region may represent a region of interest (RoI).

[0051] In this way, in the present invention, labels representing each ophthalmic disease can be classified using images of the entire area of ​​each fundus image, the optic nerve head area extracted from each fundus image, and the macular area extracted from each fundus image.

[0052] That is, the processor (300) can classify multiple labels representing each ophthalmic disease from multiple fundus images by executing a multi-label classification program. The processor (300) can generate result data representing the probability of each fundus image being classified into each of the multiple ophthalmic diseases as a result of executing the multi-label classification program and transmit the result data to the user terminal (20). The processor (300) can control the operation of each of the collection unit (100) and the memory (200). The processor (300) may be a computing device. The processor (300) may include at least one of a processing unit (Processor) such as an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a microcontroller, and a microprocessor.

[0053] Figure 3 is a block diagram schematically showing the detailed configuration of the processor illustrated in Figure 1.

[0054] Referring to FIG. 3, the processor (300) may be configured to include a preprocessing unit (310), a multi-label prediction unit (320), and a multi-label classification unit (330).

[0055] The preprocessing unit (310) extracts the optic nerve head and macular region from each of multiple fundus images to create and store a data set.

[0056] At this time, the data set may be configured to include a full area image of each of the above fundus images, an optic disc area image obtained by extracting the optic disc area from each of the above fundus images, and a macular area image obtained by extracting the macular area from each of the above fundus images.

[0057] More specifically, the preprocessing unit (310) may be configured to include an object detector (311).

[0058] The object detector (311) can learn a method of detecting the optic nerve head and the macular region from each of the multiple fundus images, and thereby extract a result indicating the optic nerve head region and macular region detected from each of the multiple fundus images.

[0059] Additionally, the multi-label prediction unit (320) learns based on the above data set and uses the learned result to calculate the probability that each fundus image belongs to each label representing each specific disease classification.

[0060] More specifically, the multi-label prediction unit (320) learns a method of predicting the probability of being classified into at least one corresponding label among the plurality of labels for a corresponding CNN among the plurality of CNNs based on the data set, and can predict the probability of being classified into each label using the plurality of learned CNNs.

[0061] At this time, the multi-label prediction unit (320) can predict the probability of being classified as at least one first ophthalmic disease among the ophthalmic diseases by utilizing the entire region image from the first CNN among the plurality of CNNs, predict the probability of being classified as at least one second ophthalmic disease among the ophthalmic diseases by utilizing the optic nerve head region image from the second CNN among the plurality of CNNs, and predict the probability of being classified as at least one third ophthalmic disease among the ophthalmic diseases by utilizing the macular region image from the third CNN among the plurality of CNNs.

[0062] Additionally, the multi-label classification unit (330) classifies labels for each fundus image using the calculated probabilities of each fundus image to derive the results of ophthalmic diseases.

[0063] More specifically, the multi-label classification unit (330) can predict the probability of being classified as at least one ophthalmic disease among the various ophthalmic diseases by combining all of the probabilities of being classified as each of the first ophthalmic disease, each of the probabilities of being classified as each of the second ophthalmic disease, and each of the probabilities of being classified as each of the third ophthalmic disease from the multi-label prediction unit (320), and further, the multi-label classification unit (330) can predict the probability of being classified as at least one ophthalmic disease among the various ophthalmic diseases by multiplying the probability of being classified as each of the various ophthalmic diseases by a corresponding weight among a plurality of probability weights and adding all of the resulting values.

[0064] To this end, the multi-label classification unit (330) can learn a first probability weight corresponding to the probability of being classified as each of the first ophthalmic diseases, learn a second probability weight corresponding to the probability of being classified as each of the second ophthalmic diseases, and learn a third probability weight corresponding to the probability of being classified as each of the third ophthalmic diseases, and at this time, the sum of the first probability weight, the second probability weight, and the third probability weight is configured not to exceed a threshold value.

[0065] FIG. 4 is a flowchart of a multiple ophthalmic disease identification and prediction-based labeling method according to one embodiment of the present invention, and FIG. 5 is a diagram for explaining a multiple ophthalmic disease identification and prediction-based labeling method according to one embodiment of the present invention.

[0066] Additionally, FIG. 6 is a drawing illustrating a fundus image showing the optic nerve head and macular area according to one embodiment of the present invention.

[0067] Hereinafter, a method for identifying multiple ophthalmic diseases and labeling based on prediction according to one embodiment will be described with reference to FIGS. 4 to 6.

[0068] According to a method for identifying and predicting multiple ophthalmic diseases and labeling them based on a prediction, a processor first collects multiple fundus images (S110). At this time, the fundus images are ultra-wide-field fundus images (UFIs).

[0069] Thereafter, the processor extracts the optic nerve head and macular region from each of the multiple fundus images to create and store a data set (S120).

[0070] More specifically, the processor can extract results indicating the optic disc region and macular region detected from each of the plurality of fundus images by using an object detector that learns a method of detecting the optic disc and macular region from each of the plurality of fundus images.

[0071] At this time, the data set may be configured to include a full area image of each of the above fundus images, an optic disc area image obtained by extracting the optic disc area from each of the above fundus images, and a macular area image obtained by extracting the macular area from each of the above fundus images.

[0072] When cropping to detect the optic disc and macular region, either manual or automatic cropping can be used. The macular region must include the macula, and the optic disc region must include the optic disc.

[0073] Referring to Figure 3, the macula is indicated by a dark area near the center of the UFI, and the optic disc is indicated by a bright yellow area.

[0074] In this case of automatic cropping, the macula and optic disc are located by an object detector such as Faster RCNN or YOLO, and then the macula and optic disc can be cropped based on the locations of the macula and optic disc.

[0075] Thereafter, the processor learns based on the above data set, and uses the learned result to calculate the probability that each fundus image belongs to each label representing each specific disease classification (S130).

[0076] At this time, the processor learns a method of predicting the probability of being classified into at least one corresponding label among the plurality of labels for a corresponding CNN among the plurality of CNNs (Convolutional neural networks) based on the data set, and can predict the probability of being classified into each label using the plurality of learned CNNs.

[0077] More specifically, the processor may be configured to predict a probability of being classified as at least one first ophthalmic disease among the ophthalmic diseases by utilizing the full region image from a first CNN among the plurality of CNNs, to predict a probability of being classified as at least one second ophthalmic disease among the ophthalmic diseases by utilizing the optic nerve head region image from a second CNN among the plurality of CNNs, and to predict a probability of being classified as at least one third ophthalmic disease among the ophthalmic diseases by utilizing the macular region image from a third CNN among the plurality of CNNs.

[0078] Afterwards, the processor classifies labels for each fundus image using the calculated probabilities of each fundus image to derive the results of an ophthalmic disease (S140).

[0079] At this time, the processor can predict the probability of being classified as at least one ophthalmic disease among the various ophthalmic diseases by combining the probability of being classified as each of the first ophthalmic disease, the probability of being classified as each of the second ophthalmic disease, and the probability of being classified as each of the third ophthalmic disease, and more specifically, the processor can predict the probability of being classified as at least one ophthalmic disease among the various ophthalmic diseases by multiplying the probability of being classified as each of the various ophthalmic diseases by a corresponding weight among a plurality of probability weights and adding all the resulting values.

[0080] To this end, the processor may learn a first probability weight corresponding to the probability of being classified as each of the first ophthalmic diseases, learn a second probability weight corresponding to the probability of being classified as each of the second ophthalmic diseases, and learn a third probability weight corresponding to the probability of being classified as each of the third ophthalmic diseases, wherein the sum of the first probability weight, the second probability weight, and the third probability weight is configured not to exceed a threshold value, and the derivation of the result of the ophthalmic disease through such label classification may be expressed as in the following mathematical expression 1.

[0081]

[0082] [Mathematical Formula 1]

[0083] Pout_i = w 1_i ×p 1_i +w 2_i ×p 2_i +w 3_i ×p 3_i

[0084] At this time, p 1_i , p 2_i , p 3_i represents the probability of three classification models for ophthalmic disease i, and w 1_i , w 2_i , w 3_iare the probability weights for the first, second, and third ophthalmic diseases, respectively, w 1_i + w 2_i +w 3_i= Satisfies 1.

[0085] FIG. 7 is an example diagram of a classification result produced by a multiple ophthalmic disease identification and prediction-based labeling server according to one embodiment.

[0086] Referring to FIG. 7, for each of a plurality of ophthalmic diseases for one fundus image, the probability output by a plurality of CNNs (e.g., a plurality of CNNs (CNN #1(f), CNN #2(g), CNN #3(h)) of FIG. 4) and the result of comparison with a predetermined threshold value (e.g., Th1, Th2-1, Th2-2, Th3-1, Th3-2, Th3-3 of FIG. 4) can indicate whether each of the plurality of ophthalmic diseases is present (Positive, red in FIG. 7) or not (negative, green in FIG. 7).

[0087] The fundus image shown in the example of Fig. 7 was output with the probability of being classified as AMD, DR, ERM, GS, RB, and RVO, respectively, as 7.38%, 72.41%, 8.35%, 72.34%, 1.45%, and 3.19%, respectively. In addition, the fundus image shown in the example of Fig. 7 was classified as having DR and GS, and not having AMD, ERM, RB, or RVO among multiple ophthalmic diseases.

[0088] FIGS. 8 to 11 are diagrams comparing the results of a multiple ophthalmic disease identification and prediction-based labeling method according to one embodiment of the present invention with the results of a prior art.

[0089] At this time, the AUC, accuracy, sensitivity, and specificity indices of the proposed invention and the prior art were compared and evaluated for each class. In addition, EfficientNetB3 was used as a classification model, and the parameters used for fusion were w1_i = w 2_i = w 3_i = 1 / 3.

[0090] In Figures 8 to 11, AUC, accuracy, sensitivity, and specificity for each disease are compared, and it can be seen that the present invention (Proposed) enables more accurate diagnosis of ophthalmic diseases compared to the prior art.

[0091] The detailed description of the present invention, as described above, has described specific embodiments. However, various modifications are possible without departing from the scope of the present invention. The technical spirit of the present invention should not be limited to the aforementioned embodiments, but should be defined not only by the claims but also by equivalents thereof.

Claims

1. A collection unit that collects multiple fundus images; A memory storing a multi-label classification program that classifies labels of multiple ophthalmic diseases; and A processor that executes the above program to classify labels representing each ophthalmic disease from the plurality of fundus images; The above processor, A preprocessing unit that extracts the optic disc and macular region from each of the above multiple fundus images to create and store a data set; A multi-label prediction unit that learns based on the above data set and uses the learned results to calculate the probability that each fundus image belongs to each label representing each specific disease classification; and A multi-label classifier that classifies labels for each fundus image using the calculated probability of each fundus image to derive the results of an ophthalmic disease; A multi-ocular disease identification and prediction-based labeling system.

2. In paragraph 1, The above fundus image is, UFI (Ultra-wide-field Fundus Image), A multi-ocular disease identification and prediction-based labeling system.

3. In paragraph 1, The above preprocessing unit, An object detector that learns a method of detecting the optic nerve head and the macular region from each of the plurality of fundus images; Including, The above object detector, Extracting results indicating the optic nerve head area and macular area detected from each of the multiple fundus images, A multi-ocular disease identification and prediction-based labeling system.

4. In paragraph 1, The above data set is, Each of the above fundus images includes a full area image, an optic disc area image extracted from the optic disc area from each of the above fundus images, and a macular area image extracted from the macular area from each of the above fundus images. A multi-ocular disease identification and prediction-based labeling system.

5. In paragraph 1, The above multi-label prediction unit, A method of predicting the probability of being classified into at least one corresponding label among the plurality of labels for a corresponding CNN among a plurality of CNNs (Convolutional neural networks) based on the above data set is learned, and the probability of being classified into each label is predicted using the learned plurality of CNNs. A multi-ocular disease identification and prediction-based labeling system.

6. In paragraph 5, The above multi-label prediction unit, Predicting the probability of being classified as at least one first ophthalmic disease among the ophthalmic diseases by utilizing the entire region image from the first CNN among the plurality of CNNs, Predicting the probability of being classified as at least one second ophthalmic disease among the ophthalmic diseases by utilizing the optic nerve head region image from the second CNN among the plurality of CNNs, Predicting the probability of being classified as at least one third ophthalmic disease among the ophthalmic diseases by utilizing the macular region image from the third CNN among the plurality of CNNs. A multi-ocular disease identification and prediction-based labeling system.

7. In paragraph 1, The above multi-label classification unit is, Predicting the probability of being classified as at least one of the various ocular diseases by combining the probability of being classified as each of the first ocular diseases, the probability of being classified as each of the second ocular diseases, and the probability of being classified as each of the third ocular diseases from the multi-label prediction unit. A multi-ocular disease identification and prediction-based labeling system.

8. In paragraph 7, The above multi-label classification unit is, Multiplying the probability of being classified as each of the various ocular diseases by the corresponding weight among multiple probability weights and adding all the resulting values, predicting the probability of being classified as at least one of the various ocular diseases. A multi-ocular disease identification and prediction-based labeling system.

9. In paragraph 8, The above multi-label classification unit is, Learn a first probability weight corresponding to the probability of being classified into each of the above first ophthalmic diseases, Learn a second probability weight corresponding to the probability of being classified as each of the above second ophthalmic diseases, Learn the third probability weight corresponding to the probability of being classified into each of the above three ophthalmic diseases, The sum of the first probability weight, the second probability weight, and the third probability weight does not exceed a threshold value. A multi-ocular disease identification and prediction-based labeling system.

10. A method for classifying labels representing each ophthalmic disease from a plurality of fundus images by executing a multi-label classification program that classifies labels of a plurality of ophthalmic diseases stored in a memory by a processor, A step of collecting multiple fundus images; A preprocessing step of extracting the optic disc and macular region from each of the above multiple fundus images to create and store a data set; A multi-label prediction step for learning based on the above data set and using the learned result to calculate the probability that each fundus image belongs to each label representing each specific disease classification; and A multi-label classification step for classifying labels for each fundus image using the calculated probability of each fundus image to derive the results of an ophthalmic disease; A multi-ocular disease identification and prediction-based labeling method.

11. In paragraph 10, The above fundus image is, UFI (Ultra-wide-field Fundus Image), A multi-ocular disease identification and prediction-based labeling method.

12. In paragraph 10, The above preprocessing step is, Using an object detector that learns how to detect the optic disc and the macular region from each of the multiple fundus images, Extracting results indicating the optic nerve head area and macular area detected from each of the multiple fundus images, A multi-ocular disease identification and prediction-based labeling method.

13. In paragraph 10, The above data set is, Each of the above fundus images includes a full area image, an optic disc area image extracted from the optic disc area from each of the above fundus images, and a macular area image extracted from the macular area from each of the above fundus images. A multi-ocular disease identification and prediction-based labeling method.

14. In paragraph 10, The above multi-label prediction step is, A method of predicting the probability of being classified into at least one corresponding label among the plurality of labels for a corresponding CNN among a plurality of CNNs (Convolutional neural networks) based on the above data set is learned, and the probability of being classified into each label is predicted using the learned plurality of CNNs. A multi-ocular disease identification and prediction-based labeling method.

15. In paragraph 14, The above label prediction step is, A step of predicting the probability of being classified as at least one first ophthalmic disease among the ophthalmic diseases by utilizing the entire region image from the first CNN among the plurality of CNNs; A step of predicting the probability of being classified as at least one second ophthalmic disease among the ophthalmic diseases by utilizing the optic nerve head region image from the second CNN among the plurality of CNNs; and A step of predicting the probability of being classified as at least one third ophthalmic disease among the ophthalmic diseases by utilizing the macular region image from the third CNN among the plurality of CNNs; A multi-ocular disease identification and prediction-based labeling method.

16. In paragraph 10, The above multi-label classification step is, Predicting the probability of being classified as at least one of the various ocular diseases by combining the probability of being classified as each of the first ocular diseases, the probability of being classified as each of the second ocular diseases, and the probability of being classified as each of the third ocular diseases. A multi-ocular disease identification and prediction-based labeling method.

17. In paragraph 16, The above multi-label classification step is, Multiplying the probability of being classified as each of the various ocular diseases by the corresponding weight among multiple probability weights and adding all the resulting values, predicting the probability of being classified as at least one of the various ocular diseases. A multi-ocular disease identification and prediction-based labeling method.

18. In paragraph 17, The above multi-label classification step is, A step of learning a first probability weight corresponding to the probability of being classified into each of the first ophthalmic diseases; A step of learning a second probability weight corresponding to the probability of being classified as each of the second ophthalmic diseases; and A step of learning a third probability weight corresponding to the probability of being classified as each of the third ophthalmic diseases; The sum of the first probability weight, the second probability weight, and the third probability weight does not exceed a threshold value. A multi-ocular disease identification and prediction-based labeling method.

Images