Ophthalmic information processing device, ophthalmic device, ophthalmic information processing method, and program

Abstract

To provide a new technique for applying a proper treatment at an early stage to a disease of a subject eye.SOLUTION: An ophthalmologic information processing device comprises: a feature amount extractor; and a classifier. The feature amount extractor extracts a feature amount of each of two or more mutually-different images of eyes of a subject. The classifier outputs two or more pieces of confidence degree information for estimating the classification result of the disease state of the disease of the eyes of the subject on the basis of the feature amount of each of the two or more images extracted by the feature amount extractor and one or more pieces of background data expressing the subject or the state of the eyes of the subject.SELECTED DRAWING: Figure 1A

Description

[Technical field]

[0001] The present invention relates to an ophthalmologic information processing device, an ophthalmologic apparatus, an ophthalmologic information processing method, and a program. [Background technology]

[0002] In recent years, the rapid progress of machine learning techniques, such as deep learning, has led to the practical application of artificial intelligence technology in various fields. In particular, in the medical field, deep learning has improved the accuracy of detection of diseased sites or tissue features in diagnostic images, making it possible to perform accurate and high-precision medical diagnoses quickly.

[0003] For example, Patent Document 1 discloses a method of performing machine learning using feature points in a plurality of past time-series fundus images as training data, and diagnosing glaucoma from a plurality of newly acquired time-series fundus images using the obtained trained model.For example, Patent Document 2 discloses a method of classifying the pathology of glaucoma according to the shape of the optic disc using a neural network. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 211688 [Patent Document 2] JP 2019-5319 A Summary of the Invention [Problem to be solved by the invention]

[0005] For example, ocular fundus diseases such as glaucoma, age-related macular degeneration, and diabetic retinopathy are diseases whose symptoms will progress unless appropriate treatment is administered, even if they are discovered early by screening, etc. In this case, by properly understanding the pathology of the disease, appropriate treatment for the disease can be administered.

[0006] In this manner, it is required to detect fundus diseases with high accuracy and at an early stage, and to specify the pathology of the detected fundus disease with high accuracy. This applies not only to fundus diseases but also to diseases of the subject eye in general.

[0007] The present invention has been made in view of the above circumstances, and one of its objects is to provide a new technique for providing appropriate treatment for a disease of an examinee's eye at an early stage. [Means for solving the problem]

[0008] One aspect of the embodiment includes a feature extractor that extracts features of two or more different images of a subject's eye; a feature selector that selects one or more background data with the highest correct answer rate, the one or more background data being obtained by sequentially repeating a process of selecting one or more background data by a statistical test for two or more background data representing each of a plurality of subjects with an eye disease or the eye condition of the subject, and a process of calculating a correct answer rate by cross-validation; The feature amounts of the two or more images extracted by the feature extractor; The feature selector The above 1 Based on the above background data, disease and a classifier that outputs two or more pieces of certainty information for estimating a classification result of a pathology of a patient.

[0009] Another aspect of the embodiment is an ophthalmologic apparatus including an optical system that acquires the two or more images, and the above-described ophthalmologic information-processing device.

[0010] Another aspect of the embodiment is a feature extraction step of extracting feature amounts of each of two or more different images of a subject's eye; a feature selection step of selecting one or more background data items having the highest correct answer rate obtained by sequentially repeating a process of selecting one or more background data items by a statistical test for two or more background data items that represent each of a plurality of subjects with an eye disease or the eye condition of the subject, and a process of calculating a correct answer rate by cross-validation; the feature amounts of the two or more images extracted in the feature amount extraction step; Selected in the feature selection step The above 1 Based on the above background data, disease and a classification step of outputting two or more pieces of confidence information for estimating a classification result of the pathology of the patient.

[0011] Another aspect of the embodiment is a program for causing a computer to execute each step of the above-described ophthalmologic information processing method. Effect of the Invention

[0012] According to some embodiments of the present invention, a new technique for providing appropriate treatment for a disease of an examinee's eye at an early stage can be provided. [Brief description of the drawings]

[0013] [Figure 1A] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. [Figure 1B] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. [Diagram 2] 1 is a schematic diagram illustrating an example of a configuration of an optical system of an ophthalmic apparatus according to an embodiment. [Diagram 3] 1 is a schematic diagram illustrating an example of a configuration of an optical system of an ophthalmic apparatus according to an embodiment. [Figure 4] 2 is a schematic diagram illustrating an example of a configuration of a processing system of an ophthalmologic apparatus according to an embodiment. FIG. [Diagram 5] 2 is a schematic diagram illustrating an example of a configuration of a processing system of an ophthalmologic apparatus according to an embodiment. FIG. [Figure 6] 2 is a schematic diagram illustrating an example of a configuration of a processing system of an ophthalmologic apparatus according to an embodiment. FIG. [Figure 7A] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 7B] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 7C] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 7D] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 7E] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 7F] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 7G] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 7H] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 8]2 is a schematic diagram illustrating an example of a configuration of a processing system of an ophthalmologic apparatus according to an embodiment. FIG. [Figure 9] 2 is a schematic diagram illustrating an example of a configuration of a processing system of an ophthalmologic apparatus according to an embodiment. FIG. [Figure 10] 5 is a schematic diagram illustrating an example of an operation flow of the ophthalmologic apparatus according to the embodiment. FIG. [Figure 11] 5 is a schematic diagram illustrating an example of an operation flow of the ophthalmologic apparatus according to the embodiment. FIG. [Figure 12] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 13] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 14] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 15] 5A and 5B are schematic diagrams for explaining the operation of the ophthalmologic apparatus according to the embodiment. [Figure 16] FIG. 11 is a schematic diagram illustrating an example of a configuration of an ophthalmologic system according to a modified example of the embodiment. [Figure 17] FIG. 11 is a schematic diagram illustrating an example of a configuration of an ophthalmologic system according to a modified example of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Examples of an ophthalmological information processing apparatus, an ophthalmological apparatus, an ophthalmological information processing method, and a program according to some embodiments of the present invention will be described in detail with reference to the drawings. Note that matters described in documents cited in this specification and any publicly known techniques can be incorporated in the embodiments.

[0015] The ophthalmological information processing device according to the embodiment extracts feature amounts from two or more different images of a subject's eye (test eye) and outputs two or more pieces of confidence information for estimating a classification result of a pathological condition of a disease of the test eye based on the two or more extracted feature amounts and one or more pieces of background data. For example, the ophthalmological information processing device extracts feature amounts for each image using a trained model obtained by performing supervised machine learning for each image. For example, the ophthalmological information processing device outputs two or more pieces of confidence information from the two or more extracted feature amounts for each image and one or more pieces of background data using a classification model obtained by performing supervised machine learning. In some embodiments, the ophthalmological information processing device outputs a classification result of a pathological condition of a disease of the test eye as estimated information based on the above two or more pieces of confidence information.

[0016] Here, the two or more images are images of a predetermined portion (e.g., fundus) of the subject's eye captured in different directions. For example, the two or more images include at least one of one or more OCT images of a desired cross-sectional direction generated from three-dimensional OCT data captured by using Optical Coherence Tomography (OCT) for the subject's eye, and a front image (e.g., fundus image) captured by a capturing optical system. The OCT image may be a tomographic image or a front image.

[0017] Furthermore, the feature may be a feature in an intermediate layer of the trained model, or confidence information for estimating the classification result of the pathological condition of a disease of the test eye in the output layer of the trained model.

[0018] The background data is data representing the condition of the subject or the eye to be examined. Examples of the data representing the condition of the subject include age, sex, nationality, place of birth, data representing the association of the disease with genetic factors, and data representing the association of the disease with environmental factors. Examples of the data representing the condition of the eye to be examined include data representing the shape of the eye to be examined, and data representing the visual function of the eye to be examined. The data representing the condition of the eye to be examined is obtained by objectively or subjectively examining the eye to be examined.

[0019] Furthermore, the certainty information is information that indicates the certainty of the estimated pathological condition of the disease accompanied by the subject's eye (for example, the probability that the condition is estimated to be present) for each of a plurality of pathological conditions of a predetermined disease.

[0020] Hereinafter, a case where the ophthalmological information processing device, the ophthalmological device, the ophthalmological information processing method, and the program according to the embodiment are applied to fundus diseases will be described. However, the application field of the ophthalmological information processing device, the ophthalmological device, the ophthalmological information processing method, and the program according to the embodiment is not limited to fundus diseases, and can be applied to diseases of the subject eye in general.

[0021] For example, an ophthalmologic information processing device outputs two or more pieces of confidence information for estimating a classification result of a pathological condition of a fundus disease from multiple images of the fundus of a subject's eye and one or more pieces of background data of the subject or the subject's eye.

[0022] Ocular fundus diseases include glaucoma, age-related macular degeneration, and diabetic retinopathy.

[0023] When the fundus disease is glaucoma, for example, confidence information (classification information) for estimating the classification result based on the shape of the optic disc is output in order to classify the pathological condition according to the classification by Nicolela et al. (Nicolella classification) as disclosed in Patent Document 2.

[0024] When the fundus disease is age-related macular degeneration, for example, confidence information is output for estimating a classification result based on the morphology of the retinal pigment epithelium layer (e.g., the morphology of choroidal neovascularization) to classify the disease as atrophic or exudative.

[0025] If the fundus disease is diabetic retinopathy, for example, confidence information is output to estimate the classification result based on the morphology and distribution of microaneurysms, hemorrhages, white spots, neovascularization, and hemorrhagic areas in order to classify the disease as simple diabetic retinopathy, preproliferative diabetic retinopathy, or proliferative diabetic retinopathy.

[0026] In this way, by providing doctors and other medical professionals with confidence information regarding the pathological classification of fundus diseases, it becomes possible to determine an appropriate treatment for the fundus disease at an early stage, thereby increasing the possibility of curing the fundus disease.

[0027] For example, in a case where the ocular fundus disease is determined to be glaucoma, an appropriate treatment method can be selected from among drug therapy, laser treatment, surgery, etc. according to the pathology of the disease.

[0028] For example, in cases where the fundus disease is determined to be age-related macular degeneration, an appropriate treatment for exudative age-related macular degeneration can be selected from among drug dosage, photodynamic therapy, laser coagulation, surgery, etc., depending on the pathology of the disease.

[0029] For example, in a case where the ocular fundus disease is determined to be diabetic retinopathy, an appropriate treatment method can be selected from among retinal photocoagulation, vitreous surgery, etc. according to the pathology of the disease.

[0030] In the following embodiment, a case will be described in which the fundus disease is glaucoma, and the ophthalmologic information processing device outputs two or more pieces of confidence information for estimating a classification result of a pathology according to the Nicollella classification from two or more images of the subject's eye and one or more background data of the subject or the subject's eye. Here, the two or more images of the subject's eye include one or more OCT images generated from three-dimensional OCT data of the subject's eye, and a color fundus image. The one or more background data include the subject's age and one or more measurement values ​​obtained by subjectively or objectively measuring the subject's eye.

[0031] The ophthalmic information processing device can perform a predetermined analysis process and a predetermined display process on three-dimensional OCT data of the subject's eye acquired by the ophthalmic device using Optical Coherence Tomography (OCT). The ophthalmic device according to some embodiments has not only the function of an OCT device for performing OCT, but also at least one function of a fundus camera, a scanning laser ophthalmoscope, a slit lamp microscope, and a surgical microscope. Furthermore, the ophthalmic device according to some embodiments has a function of measuring optical characteristics of the subject's eye. Examples of ophthalmic devices having a function of measuring optical characteristics of the subject's eye include a refractometer, a keratometer, a tonometer, a wavefront analyzer, a specular microscope, and a perimeter. The ophthalmic device according to some embodiments has a function of a laser treatment device used for laser treatment.

[0032] In some embodiments, the ophthalmological apparatus includes an ophthalmological information processing device (ophthalmological image processing device, ophthalmological analysis device) according to an embodiment. The ophthalmological information processing method according to an embodiment is executed by the ophthalmological information processing device. The program according to an embodiment causes a processor (computer) to execute each step of the ophthalmological information processing method. The recording medium according to an embodiment is a computer-readable non-transitory recording medium (storage medium) on which the program according to an embodiment is recorded.

[0033] In this specification, the term "processor" refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA)), etc. The processor realizes the functions according to the embodiment by, for example, reading and executing a program stored in a memory circuit or a storage device.

[0034] FIG. 1A shows a block diagram of an example of the configuration of an ophthalmologic information processing apparatus according to an embodiment.

[0035] The ophthalmologic information processing device 1000 of the embodiment outputs two or more pieces of confidence information OUT-1 to OUT-m (m is an integer greater than or equal to 2) from two or more images IMG-1 to IMG-n (n is an integer greater than or equal to 2) of the test eye and background data BG of the subject or the test eye.

[0036] The images IMG-1 to IMG-n include one or more OCT images generated from three-dimensional OCT data obtained by performing OCT on the subject's eye, and a color fundus image acquired using an imaging optical system. The one or more OCT images include, for example, at least one of a horizontal (lateral, x-direction, first direction) B-scan image passing through a representative position of the optic disc, a vertical (vertical, y-direction, second direction intersecting with the first direction) B-scan image passing through a representative position of the optic disc, a vertical B-scan image passing through a representative position of the fovea, a projection image, an en-face image, and a B-scan image passing through the middle between the representative position of the optic disc and the representative position of the fovea. Examples of the representative position of the optic disc include the center of the optic disc, the center of gravity, etc. Examples of the representative position of the fovea include the center of the fovea, the center of gravity, etc.

[0037] The background data BG includes at least one of the age of the subject, the sex of the subject, the axial length of the subject's eye, the MD (Mean Defect) value of the subject's eye, the spherical equivalent power of the subject's eye, the nerve head tissue blood flow value (MT (Mean of Tissue area) value) of the subject's eye, the minimum intraocular pressure value in a predetermined period of the past, the maximum intraocular pressure value in a predetermined period of the past, and the average intraocular pressure value in a predetermined period of the past. The axial length is acquired using a known optical coherence tomography or a known axial length measuring device. The MD value is a value indicating the degree of visual field defect and is acquired using a perimeter. The spherical equivalent power is acquired using a known ocular refractive power measuring device. The MT value is acquired using a known laser speckle flowgraphy (LSFG). The minimum intraocular pressure value in the predetermined period of the past may be, for example, the minimum intraocular pressure value in the past year. The maximum intraocular pressure value in the predetermined period of the past may be, for example, the maximum intraocular pressure value in the past year. The average intraocular pressure value for a predetermined period in the past may be, for example, the average intraocular pressure value for the past year.

[0038] In some embodiments, the ophthalmological information-processing device 1000 performs feature selection processing based on the distribution of each of a plurality of background data including the above-mentioned background data BG, and outputs the above-mentioned two or more pieces of confidence information OUT-1 to OUT-m using the background data selected by the feature selection processing.

[0039] The ophthalmologic information processing device 1000 includes a feature extractor 1100 and a classifier 1200. The feature extractor 1100 includes extractors 1110-1 to 1110-n corresponding to the images IMG-1 to IMG-n, respectively, and extracts feature amounts of the images IMG-1 to IMG-n. The feature amounts extracted by the feature extractor 1100 may be confidence information for estimating the classification results according to the Nicollella classification of the images IMG-1 to IMG-n. The classifier 1200 outputs confidence information OUT-1 to OUT-m based on the feature amounts of the images IMG-1 to IMG-n extracted by the feature extractor 1100 and the background data BG.

[0040] Fig. 1B is a block diagram showing a specific configuration example of the ophthalmologic information-processing device 1000 shown in Fig. 1A. In Fig. 1B, the same components as those in Fig. 1A are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0041] The ophthalmologic information-processing device 1000 outputs confidence level information OUT-1 to OUT-4 for estimating the classification result of the pathology of glaucoma according to the Nicollella classification from the images IMG-1 to IMG-7 of the subject eye and the background data BG.

[0042] Image IMG-1 is a horizontal B-scan image passing through the center of the optic disc. Image IMG-2 is a vertical B-scan image passing through the center of the optic disc. Image IMG-3 is a vertical B-scan image passing through the center of the fovea. Image IMG-4 is a projection image. Image IMG-5 is an en-face image. Image IMG-6 is a B-scan image passing midway between the center of the optic disc and the center of the fovea. Image IMG-7 is a color fundus image. Images IMG-1 to IMG-6 are acquired from three-dimensional OCT data of the test eye. Image IMG-7 is acquired using a fundus camera.

[0043] In some embodiments, as a pre-processing before input to the ophthalmologic information-processing device 1000, a scaling process or a cropping process is performed so that the image sizes of the images IMG-1 to IMG-7 are the same. In some embodiments, as the pre-processing, at least one of a scaling process, a cropping process, and a shifting process is performed on the B-scan images of the images IMG-1 to IMG-7 so that reference positions in the images are the same. In some embodiments, as the pre-processing, at least one of a scaling process, a cropping process, and a shifting process is performed on the B-scan images of the images IMG-1 to IMG-7 so that at least one of the sizes and positions of the commonly depicted characteristic parts are the same. In some embodiments, as the pre-processing, at least one of a scaling process, a cropping process, and a shifting process is performed on the en-face images, the projection images, and the color fundus images of the images IMG-1 to IMG-7 so that reference positions in the images are the same. In some embodiments, as this pre-processing, at least one of a scaling process, a cropping process, and a shifting process is performed on the en-face images, projection images, and color fundus images among images IMG-1 to IMG-7 so that at least one of the sizes and positions of commonly depicted features is consistent.

[0044] The background data BG includes, for example, the age of the subject, the axial length of the subject's eye, the MD value of the subject's eye, the spherical equivalent power of the subject's eye, the MT value of the subject's eye, and the minimum intraocular pressure value during a predetermined period in the past.

[0045] The certainty information OUT-1 to OUT-4 includes information indicating the certainty of each of the four types of glaucoma pathology according to the Nicollella classification.

[0046] The Nicolela classification defines four types of optic disc glaucoma (focal ischemic, myopic, age-related sclerotic, and global enlargement) according to the shape of the optic disc. Focal ischemic (FI) is a type in which a notch is present in part of the rim and a localized defect in the optic nerve fiber layer is seen. Focal ischemic type is more common in women and is often accompanied by migraines and seizures. Myopic (MY) is a type in which the optic disc is inclined and crescent-shaped peripapillary chorioretinal atrophy (PPA) with a depression on the temporal side is seen. Myopic type is more common in young people and is often accompanied by myopia. Senile sclerotic (SS) is a type in which the optic disc is circular, the depression is shallow, and a halo is seen around the optic disc. Age-related sclerotic type is more common in elderly people and is often accompanied by cardiovascular disorders. Generalized enlargement (GE) is a type that presents a large, deep circular depression. Generalized enlargement is common in subjects with high intraocular pressure.

[0047] The certainty information OUT-1 is information that indicates the certainty that the glaucoma developed in the test eye is focal ischemic (FI) type. The certainty information OUT-2 is information that indicates the certainty that the glaucoma developed in the test eye is myopic (MY) type. The certainty information OUT-3 is information that indicates the certainty that the glaucoma developed in the test eye is age-related sclerosis (SS) type. The certainty information OUT-4 is information that indicates the certainty that the glaucoma developed in the test eye is global expanding (GE) type.

[0048] In this case, the feature extractor 1100 includes extractors 1110-1 to 1110-7 corresponding to the images IMG-1 to IMG-7, respectively, and extracts the features of the images IMG-1 to IMG-7. The classifier 1200 outputs confidence information OUT-1 to OUT-4 based on the features of the images IMG-1 to IMG-7 extracted by the feature extractor 1100 and the background data BG.

[0049] The extractor 1110-1 extracts feature amounts of the image IMG-1 of the subject's eye using an individual estimation model (trained model) obtained by performing supervised machine learning using a training image IMG-1 as training data and a classification result of the pathology of glaucoma determined by a doctor or the like according to the Nicolella classification as teacher data. The extractor 1110-2 extracts feature amounts of the image IMG-2 of the subject's eye using an individual estimation model obtained by performing supervised machine learning using a training image IMG-2 as training data and a classification result of the pathology of glaucoma determined by a doctor or the like according to the Nicolella classification as teacher data. The extractor 1110-3 extracts feature amounts of the image IMG-3 of the subject's eye using an individual estimation model obtained by performing supervised machine learning using a training image IMG-3 as training data and a classification result of the pathology of glaucoma determined by a doctor or the like according to the Nicolella classification as teacher data. The extractor 1110-4 extracts feature quantities of the image IMG-4 of the subject's eye using an individual estimation model obtained by performing supervised machine learning using the training image IMG-4 as training data and the classification result of the pathology of glaucoma determined by a doctor or the like according to the Nicolella classification as training data. The extractor 1110-5 extracts feature quantities of the image IMG-5 of the subject's eye using an individual estimation model obtained by performing supervised machine learning using the training image IMG-5 as training data and the classification result of the pathology of glaucoma determined by a doctor or the like according to the Nicolella classification as training data. The extractor 1110-6 extracts feature quantities of the image IMG-6 of the subject's eye using an individual estimation model obtained by performing supervised machine learning using the training image IMG-6 as training data and the classification result of the pathology of glaucoma determined by a doctor or the like according to the Nicolella classification as training data. The extractor 1110-7 uses the training image IMG-7 as training data and the classification results of the pathological condition of glaucoma determined by a doctor or the like according to the Nicolella classification as training data to obtain an individual estimation model, and extracts features of the image IMG-7 of the test eye using the individual estimation model obtained by performing supervised machine learning.

[0050] The classifier 1200 uses the feature values ​​of each of the training images IMG-1 to IMG-7 and the background data BG as training data, and outputs confidence level information OUT-1 to OUT-4 using a classification model (trained model) obtained by performing supervised machine learning using the classification results of the glaucoma pathology determined by a doctor or the like according to the Nicolella classification as training data.

[0051] Hereinafter, a specific description will be given of a case where an ophthalmic apparatus having an optical coherence tomography function and a fundus camera function has the functions of the ophthalmic information processing apparatus 1000 according to the embodiment. However, it is also possible to incorporate the functions of the ophthalmic information processing apparatus 1000 according to the embodiment into a single optical coherence tomography.

[0052] In the following, an ophthalmic device capable of performing OCT measurement of the fundus of the subject eye will be described as an example, but the ophthalmic device according to the embodiment may be capable of performing OCT measurement of the anterior segment of the subject eye. In some embodiments, the range of OCT measurement and the measurement site are changed by moving a lens that changes the focal position of the measurement light. In some embodiments, by adding one or more attachments (objective lens, front lens, etc.), OCT measurement of the fundus, OCT measurement of the anterior segment, and OCT measurement of the entire eye including the fundus and the anterior segment are possible. In some embodiments, in an ophthalmic device for fundus measurement, a front lens is placed between the objective lens and the subject eye, and measurement light that has been made into a parallel beam is made incident on the subject eye to perform OCT measurement of the anterior segment.

[0053] <Configuration> [Optical system] As shown in FIG. 2, the ophthalmic apparatus 1 includes a fundus camera unit 2, an OCT unit 100, and an arithmetic and control unit 200. The fundus camera unit 2 is provided with an optical system and a mechanism for acquiring a front image of the subject's eye E. The OCT unit 100 is provided with a part of the optical system and a mechanism for performing OCT. The other part of the optical system and the mechanism for performing OCT is provided in the fundus camera unit 2. The arithmetic and control unit 200 includes one or more processors for performing various calculations and controls. In addition to these, any elements or units such as a member for supporting the subject's face (chin rest, forehead rest, etc.) and a lens unit for switching the target site of OCT (for example, an attachment for anterior segment OCT) may be provided in the ophthalmic apparatus 1. In some embodiments, the lens unit is configured to be manually inserted and removed between the subject's eye E and an objective lens 22 described later. In some embodiments, the lens unit is configured to be automatically inserted and removed between the subject's eye E and an objective lens 22 described later under the control of a control unit 210 described later.

[0054] [Fundus camera unit] The fundus camera unit 2 is provided with an optical system for photographing the fundus Ef of the subject's eye E. The acquired image of the fundus Ef (called a fundus image, fundus photograph, etc.) is a front image such as an observation image or a capture image. The observation image is obtained by capturing a moving image using near-infrared light. The capture image is a still image using flash light. Furthermore, the fundus camera unit 2 can photograph the anterior segment Ea of the subject's eye E to obtain a front image (anterior segment image).

[0055] The fundus camera unit 2 includes an illumination optical system 10 and an imaging optical system 30. The illumination optical system 10 irradiates illumination light onto the subject's eye E. The imaging optical system 30 detects return light of the illumination light from the subject's eye E. The measurement light from the OCT unit 100 is guided to the subject's eye E through an optical path in the fundus camera unit 2, and the return light is guided to the OCT unit 100 through the same optical path.

[0056] Light (observation illumination light) output from an observation light source 11 of an illumination optical system 10 is reflected by a reflecting mirror 12 having a curved reflecting surface, passes through a condenser lens 13, and passes through a visible light cut filter 14 to become near-infrared light. The observation illumination light is then focused once near a photographing light source 15, reflected by a mirror 16, and passes through relay lenses 17 and 18, an aperture 19, and a relay lens 20. The observation illumination light is then reflected by the peripheral portion (area surrounding the hole) of a holed mirror 21, passes through a dichroic mirror 46, and is refracted by an objective lens 22 to illuminate the subject's eye E (fundus Ef or anterior eye Ea). Return light of the observation illumination light from the subject's eye E is refracted by the objective lens 22, passes through a dichroic mirror 46, passes through a hole formed in the central area of ​​the holed mirror 21, and passes through a dichroic mirror 55. The return light transmitted through the dichroic mirror 55 passes through the photographing focusing lens 31 and is reflected by the mirror 32. The return light further passes through the half mirror 33A and is reflected by the dichroic mirror 33, and is imaged on the light receiving surface of the image sensor 35 by the condenser lens 34. The image sensor 35 detects the return light at a predetermined frame rate. The focus of the photographing optical system 30 is adjusted to match the fundus Ef or the anterior segment Ea.

[0057] Light (imaging illumination light) output from the imaging light source 15 is irradiated onto the fundus Ef through the same path as the observation illumination light. Return light of the imaging illumination light from the subject's eye E is guided to the dichroic mirror 33 through the same path as the return light of the observation illumination light, passes through the dichroic mirror 33, is reflected by a mirror 36, and is imaged on the light receiving surface of an image sensor 38 by a condenser lens 37. For example, a color fundus image IMG-7, which will be described later, is acquired by the image sensor 38.

[0058] The LCD (Liquid Crystal Display) 39 displays a fixation target and a visual target for visual acuity measurement. A part of the light beam output from the LCD 39 is reflected by the half mirror 33A, reflected by the mirror 32, passes through the photographing focusing lens 31 and the dichroic mirror 55, and passes through the hole of the aperture mirror 21. The light beam that passes through the hole of the aperture mirror 21 passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef.

[0059] The fixation position of the subject's eye E can be changed by changing the display position of the fixation target on the screen of the LCD 39. Examples of the fixation position include a fixation position for acquiring an image centered on the macula, a fixation position for acquiring an image centered on the optic disc, a fixation position for acquiring an image centered on the fundus center between the macula and the optic disc, and a fixation position for acquiring an image of a site (periphery of the fundus) far away from the macula. The ophthalmic device 1 according to some embodiments includes a GUI (Graphical User Interface) or the like for designating at least one of such fixation positions. The ophthalmic device 1 according to some embodiments includes a GUI or the like for manually moving the fixation position (display position of the fixation target).

[0060] The configuration for presenting a movable fixation target to the subject's eye E is not limited to a display device such as an LCD. For example, a movable fixation target can be generated by selectively turning on a plurality of light sources in a light source array (such as a light emitting diode (LED) array). Also, a movable fixation target can be generated by one or more movable light sources.

[0061] The ophthalmologic apparatus 1 may also be provided with one or more external fixation light sources. One of the one or more external fixation light sources can project fixation light onto the fellow eye of the subject's eye E. The projection position of the fixation light onto the fellow eye can be changed. By changing the projection position of the fixation light onto the fellow eye, the fixation position of the subject's eye E can be changed. The fixation position by the external fixation light source may be the same as the fixation position of the subject's eye E using the LCD 39. For example, a movable fixation target can be generated by selectively turning on a plurality of external fixation light sources. Also, a movable fixation target can be generated by one or more movable external fixation light sources.

[0062] The alignment optical system 50 generates an alignment index used for aligning the optical system with respect to the subject's eye E. Alignment light output from an LED 51 passes through apertures 52 and 53 and a relay lens 54, is reflected by a dichroic mirror 55, and passes through a hole in the aperture mirror 21. The light that passes through the hole in the aperture mirror 21 passes through a dichroic mirror 46 and is projected onto the subject's eye E by the objective lens 22. The corneal reflected light of the alignment light is guided to the image sensor 35 via the same path as the return light of the observation illumination light. Manual alignment or auto alignment can be performed based on the received light image (alignment index image).

[0063] The focusing optical system 60 generates a split index used for focus adjustment of the subject's eye E. The focusing optical system 60 is moved along the optical path (illumination optical path) of the illumination optical system 10 in conjunction with the movement of the photographing focusing lens 31 along the optical path (photographing optical path) of the photographing optical system 30. The reflecting rod 67 can be inserted into and removed from the illumination optical path. When performing focus adjustment, the reflecting surface of the reflecting rod 67 is tilted and disposed in the illumination optical path. The focusing light output from the LED 61 passes through the relay lens 62, is split into two light beams by the split index plate 63, passes through the two-hole diaphragm 64, is reflected by the mirror 65, and is once imaged and reflected on the reflecting surface of the reflecting rod 67 by the condenser lens 66. Furthermore, the focusing light passes through the relay lens 20, is reflected by the aperture mirror 21, passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef. The fundus reflection light of the focusing light is guided to the image sensor 35 through the same path as the cornea reflection light of the alignment light. Manual focusing or autofocusing can be performed based on the received light image (split target image).

[0064] The dichroic mirror 46 combines the imaging optical path and the OCT optical path (optical path of the interference optical system). The dichroic mirror 46 reflects light in the wavelength band used for OCT and transmits light for fundus imaging. The OCT optical path (optical path of the measurement light) is provided with a collimator lens unit 40, an optical path length changer 41, an optical scanner 42, an OCT focusing lens 43, a mirror 44, and a relay lens 45 in this order from the OCT unit 100 side to the dichroic mirror 46 side.

[0065] The optical path length changing unit 41 is movable in the direction of the arrow shown in Fig. 1 and changes the length of the optical path for OCT. This change in the optical path length is used for correcting the optical path length according to the axial length, adjusting the interference state, etc. The optical path length changing unit 41 includes a corner cube and a mechanism for moving it.

[0066] The optical scanner 42 is disposed at a position optically conjugate with the pupil of the subject's eye E. The optical scanner 42 deflects the measurement light LS passing through the OCT optical path. The optical scanner 42 can deflect the measurement light LS one-dimensionally or two-dimensionally.

[0067] In the case of one-dimensional deflection, the optical scanner 42 includes a galvanometer scanner that deflects the measurement light LS in a predetermined deflection direction within a predetermined deflection angle range. In the case of two-dimensional deflection, the optical scanner 42 includes a first galvanometer scanner and a second galvanometer scanner. The first galvanometer scanner deflects the measurement light LS so as to scan the imaging site (fundus Ef or anterior segment Ea) in a horizontal direction perpendicular to the optical axis of the interference optical system (OCT optical system). The second galvanometer scanner deflects the measurement light LS deflected by the first galvanometer scanner so as to scan the imaging site in a vertical direction perpendicular to the optical axis of the interference optical system. Examples of the scanning mode of the measurement light LS by the optical scanner 42 include horizontal scan, vertical scan, cross scan, radial scan, circular scan, concentric circle scan, and spiral scan.

[0068] The OCT focusing lens 43 is moved along the optical path of the measurement light LS to adjust the focus of the OCT optical system. The OCT focusing lens 43 is movable within a movement range including a first lens position for locating the focal position of the measurement light LS on or near the fundus Ef of the subject's eye E, and a second lens position for making the measurement light LS irradiated to the subject's eye E into a parallel beam. The movement of the photographing focusing lens 31, the movement of the focus optical system 60, and the movement of the OCT focusing lens 43 can be controlled in a coordinated manner.

[0069] [OCT unit] An example of the configuration of the OCT unit 100 is shown in Fig. 3. The OCT unit 100 is provided with an optical system for acquiring an OCT image of the subject's eye E. This optical system is an interference optical system that splits light from a wavelength sweep type (wavelength scanning type) light source into measurement light and reference light, generates interference light by causing return light of the measurement light from the subject's eye E to interfere with the reference light that has passed through a reference light path, and detects this interference light. The detection result (detection signal) of the interference light by the interference optical system is an interference signal indicating the spectrum of the interference light, and is sent to the arithmetic and control unit 200.

[0070] The light source unit 101 includes a wavelength-swept (wavelength-scanning) light source capable of sweeping (scanning) the wavelength of emitted light, similar to a general swept-source type ophthalmic device. The wavelength-swept light source includes a laser light source including a resonator. The light source unit 101 changes the output wavelength over time in the near-infrared wavelength band that is not visible to the human eye.

[0071] Light L0 output from the light source unit 101 is guided by an optical fiber 102 to a polarization controller 103, where the polarization state of the light L0 is adjusted. The polarization controller 103 adjusts the polarization state of the light L0 guided through the optical fiber 102, for example, by applying stress from the outside to the looped optical fiber 102.

[0072] The light L0, whose polarization state has been adjusted by the polarization controller 103, is guided by an optical fiber 104 to a fiber coupler 105, where it is split into a measurement light LS and a reference light LR.

[0073] The reference light LR is guided by an optical fiber 110 to a collimator 111, where it is converted into a parallel light beam, and is guided to an optical path length changer 114 via an optical path length correction member 112 and a dispersion compensation member 113. The optical path length correction member 112 acts to match the optical path length of the reference light LR with the optical path length of the measurement light LS. The dispersion compensation member 113 acts to match the dispersion characteristics between the reference light LR and the measurement light LS. The optical path length changer 114 includes, for example, a corner cube and a moving mechanism for moving the corner cube, and is capable of moving the corner cube by the moving mechanism in the incident direction of the reference light LR. This changes the optical path length of the reference light LR.

[0074] The reference light LR that has passed through the optical path length changing unit 114 passes through the dispersion compensation member 113 and the optical path length correction member 112, is converted from a parallel beam to a focused beam by the collimator 116, and enters the optical fiber 117. The reference light LR that has entered the optical fiber 117 is guided to a polarization controller 118 where its polarization state is adjusted, is guided by the optical fiber 119 to an attenuator 120 where the amount of light is adjusted, and is guided by the optical fiber 121 to a fiber coupler 122.

[0075] On the other hand, the measurement light LS generated by the fiber coupler 105 is guided by the optical fiber 127 and converted into a parallel beam by the collimator lens unit 40, and passes through the optical path length changing unit 41, the optical scanner 42, the OCT focusing lens 43, the mirror 44, and the relay lens 45. The measurement light LS passing through the relay lens 45 is reflected by the dichroic mirror 46, refracted by the objective lens 22, and enters the subject's eye E. The measurement light LS is scattered and reflected at various depth positions in the subject's eye E. The return light of the measurement light LS from the subject's eye E travels in the opposite direction along the same path as the outward path and is guided to the fiber coupler 105, and reaches the fiber coupler 122 via the optical fiber 128.

[0076] The fiber coupler 122 generates interference light by combining (causing interference between) the measurement light LS incident via the optical fiber 128 and the reference light LR incident via the optical fiber 121. The fiber coupler 122 splits the interference light at a predetermined splitting ratio (for example, 1:1) to generate a pair of interference lights LC. The pair of interference lights LC are guided to a detector 125 via optical fibers 123 and 124, respectively.

[0077] The detector 125 is, for example, a balanced photodiode having a pair of photodetectors that detect a pair of interference lights LC, respectively, and outputting the difference between the detection results by these. The detector 125 sends the detection result (interference signal) to a DAQ (Data Acquisition System) 130. A clock KC is supplied to the DAQ 130 from the light source unit 101. The clock KC is generated in synchronization with the output timing of each wavelength swept (scanned) in a predetermined wavelength range by the wavelength swept light source in the light source unit 101. For example, the light source unit 101 optically delays one of two branched lights obtained by branching the light L0 of each output wavelength, and then generates the clock KC based on the result of detecting the combined light. The DAQ 130 samples the detection result of the detector 125 based on the clock KC. The DAQ 130 sends the sampled detection result of the detector 125 to the arithmetic and control unit 200. The arithmetic and control unit 200 forms a reflection intensity profile for each A-line by performing a Fourier transform or the like on the spectral distribution based on the detection result obtained by the detector 125, for example, for each series of wavelength scans (for each A-line). Furthermore, the arithmetic and control unit 200 forms image data by imaging the reflection intensity profile of each A-line.

[0078] 2 and 3, both an optical path length changer 41 for changing the length of the optical path (measurement optical path, measurement arm) of the measurement light LS and an optical path length changer 114 for changing the length of the optical path (reference optical path, reference arm) of the reference light LR are provided. However, only one of the optical path length changers 41 and 114 may be provided. In addition, it is also possible to change the difference between the reference optical path length and the measurement optical path length using optical members other than these.

[0079] [Calculation and control unit] The arithmetic and control unit 200 analyzes the detection signal input from the DAQ 130 to form an OCT image of the subject's eye E. The arithmetic and control process for this purpose is similar to that of a conventional swept-source type OCT device.

[0080] In addition, the arithmetic and control unit 200 controls each part of the fundus camera unit 2, the display device 3, and the OCT unit 100.

[0081] To control the fundus camera unit 2, the arithmetic and control unit 200 controls the operation of the observation light source 11, the imaging light source 15, and the LEDs 51 and 61, controls the operation of the LCD 39, controls the movement of the imaging focusing lens 31, controls the movement of the OCT focusing lens 43, controls the movement of the reflecting rod 67, controls the movement of the focus optical system 60, controls the movement of the optical path length changing unit 41, and controls the operation of the optical scanner 42.

[0082] As the control of the display device 3, the arithmetic and control unit 200 causes the display device 3 to display an OCT image or a fundus image of the subject's eye E.

[0083] To control the OCT unit 100, the arithmetic and control unit 200 controls the operation of the light source unit 101, the movement of the optical path length changing unit 114, the operation of the attenuator 120, the operation of the polarization controllers 103 and 118, the operation of the detector 125, and the operation of the DAQ 130.

[0084] The arithmetic control unit 200 includes a processor, a RAM, a ROM, a hard disk drive, a communication interface, and the like, similar to a conventional computer. A computer program for controlling the ophthalmic apparatus 1 is stored in a storage device such as a hard disk drive. The arithmetic control unit 200 may include various circuit boards, for example, a circuit board for forming an OCT image. The arithmetic control unit 200 may also include operation devices (input devices) such as a keyboard and a mouse, and a display device such as an LCD.

[0085] The fundus camera unit 2, the display device 3, the OCT unit 100, and the arithmetic and control unit 200 may be configured integrally (i.e., within a single housing) or may be configured separately in two or more housings.

[0086] [Processing system] Fig. 4 shows an example of the configuration of a processing system of the ophthalmic apparatus 1. In Fig. 4, some of the components included in the ophthalmic apparatus 1 are omitted.

[0087] The arithmetic and control unit 200 includes a control unit 210 , and controls each part of the fundus camera unit 2 , the display device 3 , and the OCT unit 100 .

[0088] (Control unit) The control unit 210 executes various types of control. The control unit 210 includes a main control unit 211 and a storage unit 212.

[0089] (Main control unit) The main control unit 211 includes a processor and controls each unit of the ophthalmologic apparatus 1. For example, the main control unit 211 controls the focusing drive units 31A and 43A, the image sensors 35 and 38, the LCD 39, the optical path length change unit 41, the optical scanner 42, and the movement of the optical system (movement mechanism 150) of the fundus camera unit 2. Furthermore, the main control unit 211 controls the light source unit 101, the optical path length change unit 114, the attenuator 120, the polarization controllers 103 and 118, the detector 125, the DAQ 130, and the like of the OCT unit 100.

[0090] For example, the main control unit 211 displays the fixation target at a position on the screen of the LCD 39 that corresponds to a manually or automatically set fixation position. The main control unit 211 can also change (continuously or stepwise) the display position of the fixation target displayed on the LCD 39. This makes it possible to move the fixation target (i.e., to change the fixation position). The display position and movement manner of the fixation target are set manually or automatically. Manual setting is performed using, for example, a GUI. Automatic setting is performed by, for example, the data processing unit 300.

[0091] The focusing drive unit 31A moves the photographing focusing lens 31 in the optical axis direction of the photographing optical system 30, and moves the focus optical system 60 in the optical axis direction of the illumination optical system 10. This changes the focus position of the photographing optical system 30. In some embodiments, the focusing drive unit 31A has a mechanism for moving the photographing focusing lens 31 and a mechanism for moving the focus optical system 60 separately. The focusing drive unit 31A is controlled when performing focus adjustment, etc.

[0092] The focusing driver 43A moves the OCT focusing lens 43 in the optical axis direction of the measurement light path. This changes the focusing position of the measurement light LS. For example, by moving the OCT focusing lens 43 to a first lens position, the focusing position of the measurement light LS can be located at the fundus Ef or in its vicinity. For example, by moving the OCT focusing lens 43 to a second lens position, the focusing position of the measurement light LS can be located at a far point position to make the measurement light LS a parallel beam. The focusing position of the measurement light LS corresponds to the depth position (z position) of the beam waist of the measurement light LS.

[0093] The moving mechanism 150, for example, moves at least the fundus camera unit 2 (optical system) three-dimensionally. In a typical example, the moving mechanism 150 includes at least a mechanism for moving the fundus camera unit 2 in the x direction (left-right direction), a mechanism for moving in the y direction (up-down direction), and a mechanism for moving in the z direction (depth direction, front-back direction). The mechanism for moving in the x direction includes, for example, an x ​​stage that can move in the x direction, and an x ​​moving mechanism that moves the x stage. The mechanism for moving in the y direction includes, for example, a y stage that can move in the y direction, and a y moving mechanism that moves the y stage. The mechanism for moving in the z direction includes, for example, a z stage that can move in the z direction, and a z moving mechanism that moves the z stage. Each moving mechanism includes an actuator such as a pulse motor, and operates under the control of the main controller 211.

[0094] The control of the moving mechanism 150 is used for alignment and tracking. Tracking refers to moving the device optical system in accordance with the eye movement of the subject's eye E. When tracking is performed, alignment and focus adjustment are performed in advance. Tracking is a function of maintaining a suitable positional relationship where alignment and focus are achieved by making the position of the device optical system follow the eye movement. In some embodiments, the moving mechanism 150 is configured to be controlled in order to change the optical path length of the reference light (hence, the optical path length difference between the optical path of the measurement light and the optical path of the reference light).

[0095] In the case of manual alignment, the user operates a user interface 240 described below to cancel the displacement of the eye E relative to the optical system, thereby moving the optical system and the eye E relatively. For example, the main controller 211 controls the moving mechanism 150 by outputting a control signal corresponding to the operation content on the user interface 240 to the moving mechanism 150, thereby moving the optical system and the eye E relatively.

[0096] In the case of auto-alignment, the main controller 211 controls the moving mechanism 150 so that the displacement of the subject's eye E relative to the optical system is cancelled, thereby moving the optical system and the subject's eye E relatively. In some embodiments, the main controller 211 controls the moving mechanism 150 by outputting a control signal to the moving mechanism 150 so that the optical axis of the optical system approximately coincides with the axis of the subject's eye E and the distance of the optical system to the subject's eye E is a predetermined working distance, thereby moving the optical system and the subject's eye E relatively. Here, the working distance is a preset value also called the working distance of the objective lens 22, and corresponds to the distance between the subject's eye E and the optical system during measurement (photography) using the optical system.

[0097] The main controller 211 controls the fundus camera unit 2 and the like to control fundus photography and anterior eye photography. The main controller 211 also controls the fundus camera unit 2 and the OCT unit 100 and the like to control OCT measurement. The main controller 211 can execute multiple preliminary operations before performing OCT measurement. The preliminary operations include alignment, coarse focus adjustment, polarization adjustment, fine focus adjustment, and the like. The multiple preliminary operations are executed in a predetermined order. In some embodiments, the multiple preliminary operations are executed in the above order.

[0098] The types and order of the preliminary operations are not limited to the above, and are arbitrary. For example, a preliminary operation (small pupil determination) for determining whether the subject's eye E is a small pupil eye can be added to the preliminary operations. The small pupil determination is performed, for example, between the coarse focus adjustment and the optical path length difference adjustment. In some embodiments, the small pupil determination includes the following series of processes: a process of acquiring a front image (anterior eye image) of the subject's eye E; a process of identifying an image area corresponding to the pupil; a process of obtaining a size (diameter, circumference, etc.) of the identified pupil area; a process of determining whether the eye has a small pupil based on the obtained size (threshold processing); and a process of controlling the aperture 19 when the eye is determined to have a small pupil. In some embodiments, the process further includes a process of approximating the pupil area to a circle or an ellipse in order to obtain the pupil size.

[0099] The coarse focus adjustment is a focus adjustment using a split index. Note that the coarse focus adjustment can also be performed by determining the position of the focusing lens 31 based on information relating the eye refractive power and the position of the focusing lens 31 obtained in advance and the measured value of the refractive power of the subject's eye E.

[0100] The focus fine adjustment is performed based on the interference sensitivity of the OCT measurement. For example, by monitoring the interference intensity (interference sensitivity) of the interference signal acquired by the OCT measurement of the subject's eye E, the position of the OCT focusing lens 43 where the interference intensity is maximized is obtained, and the OCT focusing lens 43 is moved to that position, thereby performing the focus fine adjustment.

[0101] In the optical path length difference adjustment, a predetermined position in the subject's eye E is controlled to become a reference position of the measurement range in the depth direction. This control is performed on at least one of the optical path length changing units 41, 114. This adjusts the optical path length difference between the measurement optical path and the reference optical path. By setting the reference position by adjusting the optical path length difference, it becomes possible to perform OCT measurement with high accuracy for a desired measurement range in the depth direction simply by changing the wavelength sweep speed.

[0102] In the polarization adjustment, the polarization state of the reference light LR is adjusted to optimize the interference efficiency between the measurement light LS and the reference light LR.

[0103] (Storage part) The storage unit 212 stores various types of data. Examples of data stored in the storage unit 212 include image data of an OCT image, image data of a fundus image, image data of an anterior segment image, and information about the subject's eye. The information about the subject's eye includes information about the subject, such as a patient ID and name, and information about the subject's eye, such as identification information for the left eye / right eye.

[0104] Moreover, the storage unit 212 stores various programs and data for operating the ophthalmologic apparatus 1.

[0105] The control unit 210 is capable of controlling the image forming unit 230 and the data processing unit 300 .

[0106] (Image forming section) The image forming unit 230 forms an OCT image (image data) of the subject's eye E based on sampling data obtained by sampling the detection signal from the detector 125 with the DAQ 130. The OCT image formed by the image forming unit 230 includes an A-scan image. The image forming unit 230 may form a B-scan image (tomographic image) and a C-scan image. This processing includes processes such as noise removal (noise reduction), filtering, dispersion compensation, and FFT (Fast Fourier Transform) as in conventional swept-source type OCT. In the case of other types of OCT devices, the image forming unit 230 executes known processing according to the type.

[0107] Image forming section 230 is configured to include, for example, the circuit board described above. Note that in this specification, "image data" and an "image" based on the image data may be regarded as being the same thing.

[0108] (Data Processing Unit) The data processing unit 300 processes data acquired by photographing the subject's eye E and performing OCT measurement. For example, the data processing unit 300 performs various image processing and analysis processing on the image formed by the image forming unit 230. Specifically, the data processing unit 300 executes various correction processing such as brightness correction of the image. In addition, the data processing unit 300 can perform various image processing and analysis processing on the images (fundus images, anterior eye images, etc.) obtained by the fundus camera unit 2.

[0109] The data processing unit 300 executes known image processing such as an interpolation process for interpolating pixels between tomographic images to form image data of a three-dimensional image of the fundus oculi Ef or the anterior segment Ea. The image data of the three-dimensional image means image data in which the positions of pixels are defined by a three-dimensional coordinate system. The image data of the three-dimensional image includes image data consisting of three-dimensionally arranged voxels. This image data is called volume data or voxel data. When an image based on the volume data is to be displayed, the data processing unit 300 performs a rendering process (such as volume rendering or MIP (Maximum Intensity Projection)) on the volume data to form image data of a pseudo three-dimensional image as viewed from a specific line of sight. The pseudo three-dimensional image is displayed on a display device such as the display unit 240A.

[0110] It is also possible to form stack data of multiple tomographic images as image data of a three-dimensional image. Stack data is image data obtained by arranging multiple tomographic images obtained along multiple scan lines three-dimensionally based on the positional relationship of the scan lines. In other words, stack data is image data obtained by expressing multiple tomographic images that were originally defined by individual two-dimensional coordinate systems using one three-dimensional coordinate system (i.e., embedding them in one three-dimensional space).

[0111] The data processing unit 300 can perform various renderings on the acquired three-dimensional data set (volume data, stack data, etc.) to form a B-mode image (longitudinal section image, axial section image) in an arbitrary cross section, a C-mode image (transverse section image, horizontal section image) in an arbitrary cross section, a projection image, a shadowgram, etc. An image of an arbitrary cross section such as a B-mode image or a C-mode image is formed by selecting pixels (voxels) on a specified cross section from the three-dimensional data set. A projection image is formed by projecting the three-dimensional data set in a predetermined direction (z direction, depth direction, axial direction). A shadowgram is formed by projecting a part of the three-dimensional data set (for example, partial data corresponding to a specific layer) in a predetermined direction. An image with a viewpoint on the front side of the subject's eye, such as a C-mode image, a projection image, or a shadowgram, is called an en-face image.

[0112] The data processing unit 300 can construct a B-mode image or a front image (vessel-enhanced image, angiogram) in which retinal blood vessels and choroidal blood vessels are emphasized based on data (e.g., B-scan image data) collected in a time series by OCT. For example, time series OCT data can be collected by repeatedly scanning approximately the same part of the subject's eye E.

[0113] In some embodiments, the data processor 300 compares time-series B-scan images obtained by B-scanning approximately the same region, and constructs an enhanced image in which the changed portion is emphasized by converting pixel values ​​of the changed portion of signal intensity into pixel values ​​corresponding to the changed portion. Furthermore, the data processor 300 extracts information of a predetermined thickness of a desired region from the constructed multiple enhanced images, and constructs the information as an en-face image to form an OCTA image.

[0114] Images generated by the data processing unit 300 (for example, a three-dimensional image, a B-mode image, a C-mode image, a projection image, a shadowgram, and an OCTA image) are also included in the OCT image.

[0115] Furthermore, the data processing unit 300 analyzes the detection result of the interference light obtained by the OCT measurement to determine the focus state of the measurement light LS in the focus fine adjustment control. For example, the main control unit 211 performs repetitive OCT measurement while controlling the focus driving unit 43A according to a predetermined algorithm. The data processing unit 300 calculates a predetermined evaluation value related to the image quality of the OCT image by analyzing the detection result of the interference light LC repeatedly obtained by the OCT measurement. The data processing unit 300 determines whether the calculated evaluation value is equal to or less than a threshold value. In some embodiments, the focus fine adjustment is continued until the calculated evaluation value is equal to or less than a threshold value. That is, when the evaluation value is equal to or less than the threshold value, it is determined that the focus state of the measurement light LS is appropriate, and the focus fine adjustment is continued until it is determined that the focus state of the measurement light LS is appropriate.

[0116] In some embodiments, the main controller 211 performs the above-described repetitive OCT measurements to acquire interference signals, while monitoring the intensity (interference intensity, interference sensitivity) of the interference signals acquired sequentially. Furthermore, while performing this monitoring process, the OCT focusing lens 43 is moved to search for the position of the OCT focusing lens 43 where the interference intensity is maximized. By performing such fine focus adjustment, the OCT focusing lens 43 can be guided to a position where the interference intensity is optimized.

[0117] Moreover, the data processing unit 300 analyzes the detection result of the interference light obtained by the OCT measurement to determine the polarization state of at least one of the measurement light LS and the reference light LR. For example, the main control unit 211 performs repetitive OCT measurement while controlling at least one of the polarization controllers 103 and 118 according to a predetermined algorithm. In some embodiments, the main control unit 211 controls the attenuator 120 to change the attenuation of the reference light LR. The data processing unit 300 calculates a predetermined evaluation value regarding the image quality of the OCT image by analyzing the detection result of the interference light LC repeatedly acquired by the OCT measurement. The data processing unit 300 determines whether the calculated evaluation value is equal to or less than a threshold value. This threshold value is set in advance. The polarization adjustment is continued until the calculated evaluation value is equal to or less than the threshold value. That is, when the evaluation value is equal to or less than the threshold value, it is determined that the polarization state of the measurement light LS is appropriate, and the polarization adjustment is continued until it is determined that the polarization state of the measurement light LS is appropriate.

[0118] In some embodiments, the main controller 211 can also monitor the interference strength during polarization adjustment.

[0119] 5 to 9H are explanatory diagrams of the data processing unit 300 according to the embodiment. FIG. 5 shows a block diagram of a configuration example of the data processing unit 300 according to the embodiment. FIG. 6 shows a block diagram of a configuration example of the feature selection unit 362 in FIG. 5. FIGS. 7A to 7H are explanatory diagrams of the operation of the statistical test unit 362A in FIG. 6. FIG. 8 shows a schematic diagram of the relationship between the feature extractor 340, the classifier 350, and the learning unit 360 in FIG. 5. FIG. 9 is a schematic diagram showing a configuration example of the first extractor 341 in FIG. 8.

[0120] The data processing unit 300 includes a B-scan image forming unit 310, a projection image forming unit 320, an en-face image forming unit 330, a feature extractor 340, a classifier 350, a learning unit 360, and an analysis unit 370.

[0121] The B-scan image forming unit 310 passes through a desired site and forms a B-scan image in a desired cross-sectional direction by arranging the A-scan images formed by the image forming unit 230 in a predetermined direction. Alternatively, the B-scan image forming unit 310 passes through a desired site and forms a B-scan image in a desired cross-sectional direction by selecting pixels on a desired cross section from the image data (three-dimensional OCT data) of the three-dimensional image obtained as described above. The B-scan image forming unit 310 passes through a site specified by the analysis unit 370 described below, for example, and forms a B-scan image in a desired cross-sectional direction. The above-mentioned B-scan image may be formed by the image forming unit 230.

[0122] By performing the above processing, the B-scan image forming unit 310 is able to form, from the three-dimensional image data of the test eye E, a horizontal B-scan image IMG-1 passing through the center of the optic disc, a vertical B-scan image IMG-2 passing through the center of the optic disc, a vertical B-scan image IMG-3 passing through the center of the fovea, and a vertical B-scan image IMG-6 passing midway between the center of the optic disc and the center of the fovea.

[0123] The projection image forming section 320 forms a projection image of the fundus Ef of the subject's eye E. For example, the projection image forming section 320 forms a projection image IMG-4 of the fundus Ef by projecting three-dimensional image data of the fundus Ef in the z direction.

[0124] The en-face image forming section 330 forms an en-face image IMG-5 of the fundus Ef of the subject's eye E. For example, the en-face image forming section 330 analyzes three-dimensional image data of the fundus Ef to identify the top layer of the retina (e.g., the internal limiting membrane), and forms the image by integrating pixels for a predetermined depth (e.g., 50 micrometers) in the depth direction (z direction) from the identified top layer. That is, the en-face image forming section 330 forms an integrated image projecting a layer region from the layer region corresponding to the internal limiting membrane to a deep layer region as the en-face image IMG-5.

[0125] The feature extractor 340 extracts feature amounts from each of the B-scan images IMG-1 to IMG-3, IMG-6 formed by the B-scan image forming unit 310, the projection image IMG-4 formed by the projection image forming unit 320, the en-face image IMG-5 formed by the en-face image forming unit 330, and the color fundus image IMG-7 acquired using the photographing optical system 30. The feature extractor 340 according to the embodiment extracts, as a feature amount, certainty information representing the certainty of each of the four types of glaucoma pathology according to the Nicollella classification for each of the images IMG-1 to IMG-7, similar to the certainty information OUT-1 to OUT-4 output by the classifier 350 described later.

[0126] The feature extractor 340 uses an individual estimation model obtained by machine learning executed in advance corresponding to each of the images IMG-1 to IMG-7 to extract feature amounts of each of the images IMG-1 to IMG-7 of the subject's eye E. The feature extractor 340 includes a plurality of extractors (first extractor 341 to seventh extractor 347, see FIG. 8) corresponding to the images IMG-1 to IMG-7.

[0127] The individual estimation model (trained model) according to the embodiment is used in a computer (processor) equipped with a CPU and a memory. The function of the feature extractor 340 is realized, for example, by a convolutional neural network (CNN). That is, the CPU operates in accordance with an instruction from the individual estimation model stored in the memory to perform calculations based on trained weighting coefficients and response functions in the CNN for pixel values ​​of an image input to the convolution layer, and to output confidence information (broadly defined feature) from the output layer or feature from the intermediate layer. The feature extractor 340 having such a configuration can extract local correlation patterns while gradually lowering the resolution of the input image, and output feature based on the extracted correlation pattern.

[0128] The feature extractor 340 realizes the functions of the feature extractor 1100 shown in FIGS. 1A and 1B.

[0129] The classifier 350 outputs the above-mentioned certainty information OUT-1 to OUT-4 based on the feature amounts of the images IMG-1 to IMG-7 extracted by the feature extractor 340 and the background data BG. The classifier 350 uses a classification model obtained by machine learning executed in advance to output the certainty information OUT-1 to OUT-4 corresponding to the feature amounts of the images IMG-1 to IMG-7 extracted by the feature extractor 340 and the background data BG.

[0130] The classification model according to the embodiment is used in a computer (processor) including a CPU and a memory. The function of the classifier 350 is realized by, for example, a support vector machine (SVM).

[0131] Classifier 350 implements the functionality of classifier 1200 shown in FIGS. 1A and 1B.

[0132] The learning unit 360 executes machine learning to generate an individual estimation model (trained model) in the feature extractor 340 and a classification model (trained model) in the classifier 350. The learning unit 360 includes a first learning unit 361, a feature selection unit 362, and a second learning unit 363.

[0133] The first learning unit 361 performs supervised machine learning for each type of image IMG-1 to IMG-7, using training images as training data and the classification results of the pathological condition of glaucoma according to the Nicolella classification by a doctor or the like as teacher data, to generate an individual estimation model corresponding to each image.

[0134] The feature selection unit 362 selects background data that contributes to improving the classification accuracy based on the distribution of each of the plurality of background data. Specifically, the feature selection unit 362 selects background data BG that contributes to improving the classification accuracy of the pathology by performing a statistical test on two or more background data BG0 of each of the plurality of subjects with glaucoma (disease).

[0135] As shown in FIG. 6, the feature selection unit 362 includes a statistical testing unit 362A, a background data selection unit 362B, and a cross-validation unit 362C.

[0136] The statistical test unit 362A performs a statistical test on each of the two or more background data BG0. The background data BG0 includes, for example, the age of the subject, the sex of the subject, the axial length of the subject's eye, the MD value of the subject's eye, the spherical equivalent power of the subject's eye, the MT value of the subject's eye, the minimum intraocular pressure value in the past year, the maximum intraocular pressure value in the past year, and the average intraocular pressure value in the past year. Specifically, the statistical test unit 362A obtains the distribution of each of the four types of glaucoma pathology according to the Nicollera classification for each background data, and identifies background data BG1 that is determined to have a high possibility of improving the classification accuracy of the type. For example, the statistical test unit 362A performs a one-way analysis of variance (one-way ANOVA) analysis process for each background data BG0 to analyze the difference in average values ​​between the background data, and identifies background data BG1 whose p-value is less than a predetermined threshold value (for example, 0.01).

[0137] Fig. 7A shows an example of the distribution of axial lengths of a number of test eyes with glaucoma in four types of glaucoma pathology according to the Nicollella classification (FI, GE, MY, and SS types). In Fig. 7A, box plots showing the first quartile, second quartile, and third quartile are shown for each type.

[0138] Focusing on the second quartile, there is a difference in the average value of the distribution of axial length in each type. In the case of Fig. 7A, the statistic is 27.95 and the p-value is less than 0.001. The statistical test unit 362A identifies the axial length with a p-value less than 0.01 as background data BG1.

[0139] Figure 7B shows an example of the distribution of MD values ​​of multiple test eyes with glaucoma in four types of glaucoma pathology according to the Nicollella classification. In Figure 7B, box plots showing the first, second, and third quartiles are shown for each type, as in Figure 7A.

[0140] 7B, the statistic is 16.23, and the p-value is less than 0.001. The statistical test unit 362A specifies the MD value of the subject's eye having the p-value less than 0.01 as background data BG1.

[0141] 7C shows an example of the distribution of the spherical equivalent power of a plurality of test eyes with glaucoma in four types of glaucoma pathology according to the Nicollella classification. In the case of FIG. 7C, the statistic is 19.90, and the p-value is less than 0.001. The statistical test unit 362A specifies the spherical equivalent power of the test eye with a p-value less than 0.01 as background data BG1.

[0142] FIG. 7D shows an example of the distribution of ages of multiple subjects with glaucoma in four types of glaucoma pathology according to the Nicollella classification. In the case of FIG. 7D, the statistic is 24.36 and the p-value is less than 0.001. The statistical test unit 362A identifies the ages of subjects with p-values ​​less than 0.01 as background data BG1.

[0143] FIG. 7E shows an example of the distribution of MT values ​​of a plurality of test eyes with glaucoma in four types of glaucoma pathology according to the Nicollella classification. In the case of FIG. 7E, the statistic is 34.56, and the p-value is less than 0.001. The statistical test unit 362A identifies the MT value of the test eye with a p-value less than 0.01 as background data BG1.

[0144] 7F shows an example of the distribution of maximum intraocular pressure values ​​over the past year of multiple test eyes with glaucoma in four types of glaucoma pathology according to the Nicollella classification. In the case of FIG. 7F, the statistic is 1.35 and the p-value is 0.259. The statistical test unit 362A excludes the maximum intraocular pressure values ​​over the past year of test eyes whose p-value is not less than 0.01 from the background data BG1.

[0145] 7G shows an example of the distribution of average intraocular pressure values ​​over the past year of multiple test eyes with glaucoma in four types of glaucoma pathology according to the Nicollella classification. In the case of FIG. 7G, the statistic is 1.44 and the p-value is 0.23081. The statistical test unit 362A excludes the average intraocular pressure values ​​over the past year of test eyes with p-values ​​other than less than 0.01 from the background data BG1.

[0146] FIG. 7H shows an example of the distribution of minimum intraocular pressure values ​​over the past year for multiple test eyes with glaucoma in four types of glaucoma pathology according to the Nicollella classification. In the case of FIG. HG, the statistic is 5.89 and the p-value is 0.000596. The statistical test unit 362A identifies the minimum intraocular pressure value over the past year for test eyes with a p-value of less than 0.01 as background data BG1.

[0147] As described above, the statistical testing unit 362A specifies background data BG1 by excluding the maximum intraocular pressure value of the subject's eye over the past year and the average intraocular pressure value of the subject's eye over the past year from the background data BG0.

[0148] The background data selection unit 362B selects background data BG from the combination of two or more background data BG1 identified by the statistical testing unit 362A. Specifically, the background data 32B selects one of one or more combinations of background data from the two or more background data BG1 identified by the statistical testing unit 362A, and acquires the accuracy rate obtained by the cross-validation executed by the cross-validation unit 362C using the selected combination of one or more background data. The background data selection unit 362B sequentially repeats the selection of one or more background data and the acquisition of the accuracy rate from the cross-validation unit 362C, and selects the combination of one or more background data with the highest accuracy rate as the background data BG, thereby performing feature selection.

[0149] The cross-validation unit 362C performs cross-validation using one or more combinations of background data selected by the background data selection unit 362B and the feature amounts FA of each of the images IMG-1 to IMG-7 extracted by the feature extractor 340, and calculates a correct answer rate. For example, k-fold cross validation is performed as the cross validation (e.g., k=5). In this case, the cross-validation unit 362C outputs the average value of the k correct answer rates obtained by dividing into k as the correct answer rate of the cross-validation result using one or more combinations of background data selected by the background data selection unit 362B.

[0150] In some embodiments, the feature selection unit 362 performs the above feature selection using a linear model (e.g., SVM) as a classifier in which the feature FA of each of the images IMG-1 to IMG-7 extracted by the feature extractor 340 is penalized by the L1 norm.

[0151] For example, if the features extracted by the feature extractor 340 are the confidence information OUT-1 to OUT-4 of the images IMG-1 to IMG-7, respectively, and the background data BG1 identified by the statistical testing unit 362A is the above-mentioned six types of background data, the feature selection unit 362 performs feature selection processing on combinations of 34 (=7×4+6)-dimensional data.

[0152] The second learning unit 363 uses the features of the training images IMG-1 to IMG-7 and the background data selected by the feature selection unit 362 as training data, and generates a classification model by performing supervised machine learning using the classification results of the pathological condition of glaucoma according to the Nicolella classification by a doctor or the like as teaching data.

[0153] The analysis unit 370 performs a predetermined analysis process on the image data of the image formed by the image forming unit 230 or the data processing unit 300. The analysis process includes a process for identifying a predetermined site, a process for identifying a predetermined layer region, a process for generating a tomographic image in a desired cross-sectional direction at a desired cross-sectional position, and the like.

[0154] Examples of the predetermined location include the center of the optic disc, the fovea, and a location midway between the optic disc and the fovea.

[0155] For example, the technology disclosed in Japanese Patent Application Laid-Open No. 2008-206684 can be applied to the process of identifying the center (or center of gravity) of the optic disc. Typical identification of the optic disc center includes a process of identifying an optic disc image (e.g., an image region corresponding to the edge of the optic disc) from three-dimensional OCT data or a fundus image, a process of obtaining a circle (or an ellipse) that approximates the edge of the optic disc image, and a process of obtaining the center of the obtained approximate circle (or approximate ellipse).

[0156] The process of identifying the center (or center of gravity) of the fovea can be performed in the same manner as the process of identifying the center of the optic disc.

[0157] The analysis unit 370 can also specify a blood vessel, a lesion, etc. as the predetermined site. Examples of the lesion include a detachment, edema, bleeding, a tumor, and drusen.

[0158] Examples of specified layer regions include the inner limiting membrane, nerve fiber layer, ganglion cell layer, inner plexiform layer, inner nuclear layer, outer plexiform layer, outer nuclear layer, external limiting membrane, photoreceptor layer, retinal pigment epithelium (RPE), Bruch's membrane, choroid, sclera, and vitreous body.

[0159] The process of identifying a specific layer of tissue is a known segmentation process. The analysis unit 370 identifies a plurality of layer regions in the A-scan direction by analyzing the three-dimensional OCT data. The segmentation process is an image process for identifying a specific tissue or tissue boundary. For example, the analysis unit 370 obtains the gradient of pixel values ​​(brightness values) in each A-scan image included in the OCT data, and identifies the position with a large gradient as the tissue boundary. For example, the analysis unit 370 identifies a layer of tissue of a predetermined number of pixels on the sclera side of the identified RPE as Bruch's membrane. The A-scan image is one-dimensional image data extending in the depth direction of the fundus. The depth direction of the fundus is defined as, for example, the z direction, the incident direction of the OCT measurement light, the axial direction, the optical axis direction of the interference optical system, or the like.

[0160] The process of generating a tomographic image in a desired cross-sectional direction at a desired cross-sectional position is realized, for example, by specifying a cross-sectional position in the 3D OCT data based on a predetermined portion or layer region specified by the above-mentioned specification process, and selecting or interpolating pixels in the desired cross-sectional direction at the specified cross-sectional position. Note that, in the process of generating this tomographic image, a tomographic image in a desired cross-sectional direction at a cross-sectional position specified by a user using the operation unit 240B may be generated.

[0161] The analysis unit 370 can perform predetermined analysis processing on the three-dimensional OCT data, the OCT image, or the fundus image. The predetermined analysis processing includes, in addition to identifying a predetermined portion (tissue, lesion) in the subject's eye E, calculation of the distance (interlayer distance), area, angle, ratio, and density between the specified portions, calculation using a specified formula, identifying the shape of the predetermined portion, calculation of the statistical values ​​thereof, calculation of the distribution of the measured values ​​and statistical values, and image processing based on the results of these analysis processing.

[0162] The data processing unit 300 that functions as described above includes, for example, the above-mentioned processor, RAM, ROM, a hard disk drive, a circuit board, etc. A computer program that causes the processor to execute the above-mentioned functions is stored in advance in a storage device such as a hard disk drive.

[0163] Fig. 8 is a schematic diagram for explaining the operation of the data processing unit 300 in Fig. 5. Fig. 8 shows a schematic relationship between the feature extractor 340, the classifier 350, and the learning unit 360. Fig. 8 shows the feature extractor 340, the classifier 350, the first learning unit 361, and the second learning unit 363. In Fig. 8, the same parts as in Fig. 5 are given the same reference numerals, and the description will be omitted as appropriate.

[0164] The first learning unit 361 generates an individual estimation model for estimating whether the subject's eye has one of the four types of glaucoma pathology for each of the images IMG-1 to IMG-6 generated from the 3D OCT data and the color fundus image IMG-7. The first learning unit 361 generates the individual estimation model by executing supervised machine learning for each of the multiple images. In some embodiments, the first learning unit 361 generates the multiple individual estimation models by transfer learning.

[0165] For example, for a horizontal B-scan image IMG-1 passing through the center (or its vicinity) of the optic disc, the first learning unit 361 performs supervised machine learning using B-scan images IMG-1 of multiple test eyes other than the test eye from which features are to be extracted as training data TR1 and labels representing classification results of the pathology of glaucoma according to the Nicollella classification by a doctor or the like for each B-scan image as teacher data SD1 to generate an individual estimation model for the B-scan image IMG-1. The function of the individual estimation model for the B-scan image IMG-1 is realized by the first extractor 341 shown in FIG. 8.

[0166] For example, for a vertical B-scan image IMG-2 passing through the center (or near) of the optic disc, the first learning unit 361 performs supervised machine learning using B-scan images IMG-2 of multiple test eyes other than the test eye from which features are to be extracted as training data TR1 and labels representing classification results of the pathology of glaucoma according to the Nicollella classification by a doctor or the like for each B-scan image as teacher data SD1 to generate an individual estimation model for the B-scan image IMG-2. The function of the individual estimation model for the B-scan image IMG-2 is realized by the second extractor 342 shown in FIG. 8.

[0167] For example, for a vertical B-scan image IMG-3 passing through the fovea (or its vicinity), the first learning unit 361 performs supervised machine learning using B-scan images IMG-3 of multiple test eyes other than the test eye from which features are to be extracted as training data TR1 and labels representing classification results of the pathology of glaucoma according to the Nicollella classification by a doctor or the like for each B-scan image as teacher data SD1 to generate an individual estimation model for the B-scan image IMG-3. The function of the individual estimation model for the B-scan image IMG-3 is realized by the third extractor 343 shown in FIG. 8.

[0168] For example, for the projection image IMG-4, the first learning unit 361 generates an individual estimation model for the projection image IMG-4 by performing supervised machine learning using the projection images IMG-4 of a plurality of test eyes other than the test eye from which the feature is to be extracted as training data TR1 and the labels representing the classification results of the pathology of glaucoma according to the Nicollella classification by a doctor or the like for each projection image as teacher data SD1. The function of the individual estimation model for the projection image IMG-4 is realized by the fourth extractor 344 shown in FIG. 8.

[0169] For example, for the en-face image IMG-5, the first learning unit 361 generates an individual estimation model for the en-face image IMG-5 by performing supervised machine learning using the en-face images IMG-5 of multiple test eyes other than the test eye from which the feature is to be extracted as training data TR1 and the labels representing the classification results of the pathology of glaucoma according to the Nicollella classification by a doctor or the like for each en-face image as teacher data SD1. The function of the individual estimation model for the en-face image IMG-5 is realized by the fifth extractor 345 shown in FIG. 8.

[0170] For example, for a vertical B-scan image IMG-6 passing midway between the center of the optic disc and the center of the fovea, the first learning unit 361 performs supervised machine learning using the B-scan images IMG-6 of multiple test eyes other than the test eye from which the feature values ​​are to be extracted as training data TR1 and labels representing the classification results of the pathology of glaucoma according to the Nicollella classification by a doctor or the like for each B-scan image as teacher data SD1 to generate an individual estimation model for the B-scan image IMG-6. The function of the individual estimation model for the B-scan image IMG-6 is realized by the sixth extractor 346 shown in FIG. 8.

[0171] For example, for a color fundus image IMG-7, the first learning unit 361 performs supervised machine learning using fundus images IMG-7 of a plurality of test eyes other than the test eye from which features are extracted as training data TR1, and labels representing classification results of the pathology of glaucoma according to the Nicollella classification by a doctor or the like for each B-scan image as teacher data SD1 to generate an individual estimation model for the fundus image IMG-7. The function of the individual estimation model for the fundus image IMG-7 is realized by a seventh extractor 347 shown in FIG. 8.

[0172] Each of the above multiple individual estimation models may have a similar configuration.

[0173] 9 shows a block diagram of a configuration example of the first extractor 341 according to the embodiment. Each of the second extractor 342 to the seventh extractor 347 may have the same configuration as that in FIG.

[0174] The function of the first extractor 341 is realized by, for example, a convolutional neural network (CNN). That is, the first extractor 341 operates in accordance with an instruction from an individual estimation model stored in a computer memory to perform calculations based on learned weighting coefficients and response functions in the convolutional neural network for pixel values ​​of the image IMG-1 input to the convolutional layer 3411 of the feature extractor 341A, which is an input layer, and to output a classification result from the classifier 341B, which is an output layer. The first extractor 341 having such a configuration can extract a local correlation pattern while gradually lowering the resolution of the image, and output a classification result based on the extracted correlation pattern.

[0175] The first extractor 341 includes a feature extractor 341A and a classifier 341B. The feature extractor 341A extracts features of the judgment image by repeating extraction of features and downsampling (filtering) for each predetermined image region of the input image IMG-1. The classifier 341B outputs certainty information indicating the certainty of each of the four types of glaucoma pathology according to the Nicollella classification based on the features extracted by the feature extractor 341A.

[0176] The feature extractor 341A includes a plurality of units, each of which includes a convolution layer and a pooling layer, connected in multiple stages. In each unit, the input of the pooling layer is connected to the output of the convolution layer. The pixel value of the corresponding pixel in the image IMG-1 is input to the input of the first convolution layer. The input of the subsequent convolution layer is connected to the output of the previous pooling layer.

[0177] 9, feature extractor 341A includes two units connected in two stages. That is, in feature extractor 341A, a unit including a convolution layer 3411 and a pooling layer 3412 is connected to a unit including a convolution layer 3413 and a pooling layer 3414 in the subsequent stage. The output of the pooling layer 3412 is connected to the input of the convolution layer 3413.

[0178] The classifier 341B includes fully connected layers 3415 and 3416, and the output of the fully connected layer 3415 is connected to the input of the fully connected layer 3416.

[0179] In the feature extractor 341A and the classifier 341B, learned weighting coefficients are assigned between the neurons in the two connected layers. Each neuron performs a calculation using a response function on the calculation result that takes into account the weighting coefficients from one or more input neurons, and outputs the obtained calculation result to the neuron in the next stage.

[0180] The weighting coefficients are updated by performing known machine learning using two or more previously acquired images IMG-1 of the subject's eye (horizontal B-scan images passing through the center (or near) the optic disc) as training data and labels attached to each image by a doctor or the like as teacher data. The existing weighting coefficients are updated by machine learning using two or more previously acquired images as training data. In some embodiments, the weighting coefficients are updated by transfer learning.

[0181] The first extractor 341 may have a known layer structure such as VGG16, VGG19, InceptionV3, ResNet18, ResNet50, Xception, or DenseNet201. The classifier 341B may have a known configuration such as Random Forest or Support Vector Machine (SVM).

[0182] In Figure 8, the second learning unit 363 generates a classification model by performing supervised machine learning using the feature values ​​TR2 of the training images IMG-1 to IMG-7 and the background data BG selected by the feature selection unit 362 as training data, and the classification result of the pathological condition of glaucoma according to the Nicolella classification by a doctor or the like as teacher data SD2.

[0183] As described above, the feature extractor 340 extracts features for each of the seven types of images IMG-1 to IMG-7 of the subject's eye E using the first extractor 341 to the seventh extractor 347 that realize the function of the individual estimation model generated for each image by the first learning unit 361. The feature may be confidence level information for each of the four types into which the pathology of glaucoma is classified according to the Nicollera classification for each image. The classifier 350 uses the classification model generated by the second learning unit 363 to output confidence level information (information representing the probability that each type is estimated) OUT1 to OUT4 for each of the four types into which the pathology of glaucoma is classified according to the Nicollera classification, from the feature levels extracted by the first extractor 341 to the seventh extractor 347 and one or more background data selected by the feature selection unit 362. The confidence level information OUT-1 is confidence level information representing the confidence level that the glaucoma developed in the subject's eye is focal ischemic type (FI). The certainty information OUT-2 is certainty information that indicates the certainty that the glaucoma developed in the subject's eye is myopic (MY) type. The certainty information OUT-3 is certainty information that indicates the certainty that the glaucoma developed in the subject's eye is age-related sclerosis (SS) type. The certainty information OUT-4 is certainty information that indicates the certainty that the glaucoma developed in the subject's eye is global expanding (GE) type.

[0184] In some embodiments, the classifier 350 (or the data processing unit 300) outputs estimation information for estimating the type of glaucoma pathology that the subject's eye develops based on the certainty information OUT-1 to OUT-4. For example, the classifier 350 outputs the type with the highest certainty among the certainty information OUT-1 to OUT-4 as the estimation result of the type of glaucoma pathology that the subject's eye develops.

[0185] (User Interface) As shown in FIG. 4, the user interface 240 includes a display unit 240A and an operation unit 240B. The display unit 240A includes the display device 3 and a display device. The operation unit 240B includes the operation device of the arithmetic control unit 200 described above. The operation unit 240B may include various buttons and keys provided on the housing of the ophthalmic apparatus 1 or on the outside. For example, when the fundus camera unit 2 has a housing similar to that of a conventional fundus camera, the operation unit 240B may include a joystick, an operation panel, etc. provided on this housing. In addition, the display unit 240A may include various display devices such as a touch panel provided on the housing of the fundus camera unit 2.

[0186] It is not necessary for the display unit 240A and the operation unit 240B to be configured as separate devices. For example, it is possible to use a device in which the display function and the operation function are integrated, such as a touch panel. In that case, the operation unit 240B is configured to include this touch panel and a computer program. The operation content on the operation unit 240B is input to the control unit 210 as an electric signal. Also, operations and information input may be performed using a graphical user interface (GUI) displayed on the display unit 240A and the operation unit 240B.

[0187] The data processing unit 300 is an example of an "ophthalmological information processing device" according to the embodiment. The first extractor 341 to the seventh extractor 347 are examples of "two or more extractors" according to the embodiment. The feature selection unit 362 is an example of a "feature selector" according to the embodiment. The second learning unit 363 is an example of a "learning unit" according to the embodiment. The images IMG-1 to IMG-6 are an example of an "OCT image" according to the embodiment. The color fundus image IMG-7 is an example of a "color image of the subject's eye" according to the embodiment. The projection image IMG-4 and the en-face image IMG-5 are an example of a "front image" according to the embodiment. The B-scan images IMG-1 to IMG-3 and IMG-6 are an example of a "tomographic image" according to the embodiment. The optical system of the fundus camera unit 2 and the OCT unit 100 is an example of an "optical system" according to the embodiment.

[0188] [Operation] The operation of the ophthalmologic apparatus 1 according to the embodiment will be described.

[0189] 10 and 11 show flow diagrams of an example of the operation of the ophthalmologic apparatus 1 according to the embodiment. FIG. 10 shows a flow diagram of an example of a generation process of an individual estimation model of the feature extractor 340 and a classification model of the classifier 350. FIG. 11 shows a flow diagram of an example of a classification process of a pathological condition of glaucoma using an individual estimation model and a classification model generated by the generation process shown in FIG. 10. The storage unit 212 stores a computer program for realizing the processes shown in FIG. 10 and FIG. 11. The control unit 210 (main control unit 211) can execute the processes shown in FIG. 10 and FIG. 11 by operating according to the computer program.

[0190] The process shown in Fig. 10 is executed prior to the execution of the process shown in Fig. 11. In Fig. 10, it is assumed that a plurality of images IMG-1 to IMG7 of the subject's eyes for training purposes have been acquired in advance in the ophthalmologic apparatus 1.

[0191] (S1: Statistical testing process) First, the main control unit 211 controls the feature selection unit 362 to execute a statistical test process on two or more pieces of background data.

[0192] 7A to 7H for one of the two or more pieces of predetermined background data BG0, and calculates a p-value. For example, the two or more pieces of background data BG0 include eight-dimensional data including the subject's age, the subject's sex, the axial length of the subject's eye, the MD value of the subject's eye, the spherical equivalent power of the subject's eye, the MT value of the subject's eye, the minimum intraocular pressure value in the past year, the maximum intraocular pressure value in the past year, and the average intraocular pressure value in the past year.

[0193] (S2: Is the statistical test process complete?) Next, the main control unit 211 determines whether or not the statistical testing process in step S1 is completed. For example, the main control unit 211 determines whether or not step S1 has been executed for all of the two or more pieces of background data BG0, thereby determining whether or not the statistical testing process in step S1 is completed.

[0194] When it is determined that the statistical testing process in step S1 has ended (step S2: Y), the operation of the ophthalmic apparatus 1 proceeds to step S3. When it is determined that the statistical testing process in step S1 has not ended (step S2: N), the operation of the ophthalmic apparatus 1 proceeds to step S1.

[0195] (S3: Feature selection) In step S2, when it is determined that the statistical testing process in step S1 has ended (step S2: Y), the main control unit 211 causes the statistical testing unit 362A to select background data that is determined to be effective for classifying the pathology of glaucoma from among two or more background data obtained by repeatedly executing the statistical testing process in step S1.

[0196] For example, the statistical testing unit 362A identifies background data having a p-value of less than 0.01 as background data BG1 from among two or more background data BG0. For example, the background data BG1 includes six-dimensional data including the subject's age, the axial length of the subject's eye, the MD value of the subject's eye, the spherical equivalent power of the subject's eye, the MT value of the subject's eye, and the minimum intraocular pressure value for the past year. Note that the type of data identified as the background data BG1 may differ depending on the distribution of the background data used in the statistical testing process.

[0197] (S3: Generate a classification model) Next, the main control unit 211 controls the background data selection unit 362B to select one of the combinations of s (s is an integer equal to or greater than 1)-dimensional data from the six-dimensional background data BG1 identified in step S2.

[0198] For example, when selecting one-dimensional data, the background data selection unit 362B selects one of six types of data from the six-dimensional background data BG1. For example, when selecting two-dimensional data, the background data selection unit 362B selects one of 15 (=6×5 / 2) types of data from the six-dimensional background data BG1. For example, when selecting three-dimensional data, the background data selection unit 362B selects one of 20 (=6×5×4 / 6) types of data from the six-dimensional background data BG1. The same applies to the cases of four-, five-, or six-dimensional data.

[0199] Thereafter, the main control unit 211 controls the second learning unit 363 to generate a classification model by performing machine learning using the features of the training images IMG-1 to IMG-7 extracted by the feature extractor 340 and the s-dimensional background data selected by the background data selection unit 362B.

[0200] (S5: Cross-validation) Next, the main control unit 211 controls the cross-validation unit 362C to perform cross-validation using the feature amounts FA of each of the training images IMG-1 to IMG-7 extracted by the feature extractor 340 and the s-dimensional background data selected by the background data selection unit 362B.

[0201] The cross-validation unit 362C executes, for example, 5-fold cross-validation and calculates the average value of the 5 correct answer rates.

[0202] (S6: Cross-validation complete?) Next, the main control unit 211 determines whether or not the cross-validation in step S5 is completed. For example, the main control unit 211 determines whether or not the cross-validation in step S5 is completed by determining whether or not step S5 has been executed for all combinations of s dimensions from among two or more background data BG1 (for example, six-dimensional background data).

[0203] When it is determined that the cross-validation in step S5 is completed (step S6: Y), the operation of the ophthalmic apparatus 1 proceeds to step S7. When it is determined that the cross-validation in step S5 is not completed (step S6: N), the operation of the ophthalmic apparatus 1 proceeds to step S4.

[0204] (S7: Select background data) In step S6, when it is determined that the cross-validation in step S5 is completed (step S6: Y), the main control unit 211 controls the background data selection unit 362B to select the combination of background data with the highest accuracy rate among all combinations of the six-dimensional background data as the background data BG. Note that the type of data identified as the background data BG may differ depending on the distribution of the training image and background data.

[0205] (S8: Generate classification model) The main control unit 211 again controls the second learning unit 363 to generate a classification model by performing machine learning using the features of the training images IMG-1 to IMG-7 extracted by the feature extractor 340 and the background data BS selected in step S7.

[0206] This is the end of the operation of the ophthalmologic apparatus 1 (END).

[0207] When the classification model is generated as shown in FIG. 10, the ophthalmologic apparatus 1 outputs confidence information OUT-1 to OUT-4 for estimating the classification result of the pathology of glaucoma for the image of the subject eye E to be classified into the pathology of glaucoma, according to the flow shown in FIG. 11.

[0208] In FIG. 11, it is assumed that an individual estimation model is generated in advance in the ophthalmologic apparatus 1.

[0209] (S11: Creating an image from OCT data) First, the main controller 211 controls the OCT unit 100 and the like to perform OCT on the subject's eye E and acquire three-dimensional OCT data of the subject's eye E. Next, the main controller 211 controls the image forming part 230 and the data processing part 300 to form OCT images of images IMG-1 to IMG-6 from the three-dimensional OCT data.

[0210] (S12: Acquire fundus image) Next, the main controller 211 controls the fundus camera unit 2 and the like to photograph the fundus Ef of the subject's eye E and acquire a color fundus image IMG-7 of the fundus Ef.

[0211] (S13: Get background data) Next, the main controller 211 acquires background data of the subject's eye E for each of the background data selected in step S7 of Fig. 10. In some embodiments, the main controller 211 controls a communication unit (not shown) to acquire background data of the subject's eye E from electronic medical record information or an external ophthalmic device via a network. In some embodiments, the main controller 211 acquires background data input by a user using the operation unit 240B.

[0212] (S14: Classification of glaucoma pathology) Next, the main control unit 211 controls the feature extractor 340 and the classifier 350 to input the images IMG-1 to IMG-7 of the test eye E acquired in steps S11 and S12 and the background data acquired in step S13 using the individual estimation model already generated and the classification model generated as shown in FIG. 10, and output the confidence level information OUT-4 to OUT-4 for estimating the pathological condition of glaucoma.

[0213] This is the end of the operation of the ophthalmologic apparatus 1 (END).

[0214] Here, the accuracy of the classification result of the pathology of glaucoma by the ophthalmologic apparatus 1 according to the embodiment will be described.

[0215] 12 is a diagram illustrating the accuracy of the classification result of the pathology of glaucoma by the ophthalmologic apparatus 1 according to the embodiment when the types of the input image of the subject's eye are increased. In FIG. 12, a κ value (kappa coefficient, kappa statistic) is used as an index of the accuracy of the classification result.

[0216] In Fig. 12, when the ophthalmologic apparatus 1 according to the embodiment is used to classify the pathology of glaucoma using images IMG-1, IMG-2, IMG-4, and IMG-5, the κ value is "0.654", and when the images IMG1 to IMG5 are used to classify the pathology of glaucoma, the κ value is "0.661". Similarly, when the images IMG-1, IMG-2, IMG-4 to IMG-6 are used to classify the pathology of glaucoma, the κ value is "0.663", and when the images IMG-1, IMG-2, IMG-4, IMG-5, and IMG-7 are used to classify the pathology of glaucoma, the κ value is "0.728". Also, when the images IMG-1 to IMG-7 are used to classify the pathology of glaucoma, the κ value is "0.733".

[0217] That is, by increasing the types of images of the subject's eye and classifying the pathology of glaucoma using images IMG-1 to IMG-7, it is possible to improve the accuracy of the classification result of the pathology of glaucoma.

[0218] FIG. 13 shows an example of the precision rate, recall rate, and accuracy rate for each type when the pathology of glaucoma is classified according to the Nicollella classification using the images IMG-1 to IMG-7 by the ophthalmologic apparatus 1 according to the embodiment.

[0219] FIG. 13 shows that when the pathology of glaucoma is classified according to the Nicollella classification using images IMG-1 to IMG-7, the accuracy rate is "82.4%."

[0220] FIG. 14 shows an example of a classification result when background data is added to the classification of pathological conditions of glaucoma by the ophthalmologic apparatus 1 as in the above embodiment.

[0221] 14, when the pathology of glaucoma is classified without adding background data, the accuracy rate is 78.2%. In contrast, when the pathology of glaucoma is classified with adding background data as in the above embodiment, the accuracy rate is 80.2%.

[0222] That is, by classifying the pathology of glaucoma by adding background data in addition to increasing the types of images as described above, the accuracy of the classification result of the pathology of glaucoma can be improved.

[0223] FIG. 15 shows an example of a classification result when background data selected by performing feature selection as in the above embodiment is added to the classification of pathological conditions of glaucoma by the ophthalmologic apparatus 1.

[0224] As shown in Fig. 15, when the pathology of glaucoma is classified by adding background data without performing feature selection, the accuracy rate is "80.2%" (similar to Fig. 14). In contrast, when the pathology of glaucoma is classified by adding background data selected by performing feature selection as in the above embodiment, the accuracy rate is "81.4%".

[0225] That is, by classifying the pathology of glaucoma by adding background data selected by feature selection in addition to increasing the types of images as described above, the accuracy of the classification results of the pathology of glaucoma can be improved.

[0226] <Modification> In the above embodiment, the case where an ophthalmic apparatus including an OCT optical system and an imaging optical system for imaging a fundus has the function of the data processing unit 300 has been described, but the configuration according to the embodiment is not limited to this. For example, the function of the data processing unit 300 according to the embodiment may be provided outside the ophthalmic apparatus.

[0227] FIG. 16 is a block diagram of a configuration example of an ophthalmologic system according to a modified example of the embodiment.

[0228] An ophthalmological system 2000 according to this modification includes an ophthalmological apparatus 2100 and an ophthalmological information processing device 2200. The ophthalmological system 2000 realizes functions similar to those of the ophthalmological apparatus 1 according to the embodiment by the ophthalmological apparatus 2100 and the ophthalmological information processing device 2200. The ophthalmological information processing device 2200 has at least a part of the functions of the data processing unit 300 in FIG. 5. For example, the ophthalmological information processing device 2200 has functions of a feature extractor 340, a classifier 350, and a learning unit 360. The functions of the ophthalmological information processing device 2200 are realized, for example, by a computer including a processor.

[0229] For example, the ophthalmological apparatus 2100 has a configuration in which the configuration of the ophthalmological information-processing device 2200 is omitted from the configuration of the ophthalmological apparatus 1 according to the embodiment.

[0230] FIG. 17 is a block diagram of a configuration example of an ophthalmologic system according to another modified example of the embodiment.

[0231] An ophthalmological system 2000 according to this modification includes ophthalmological apparatuses 2100-1 to 2100-T (T is an integer equal to or greater than 2) and an ophthalmological information-processing device 2200, and the ophthalmological information-processing device 2200 is configured to be communicatively connected to any of the ophthalmological apparatuses 2100-1 to 2100-T via a network 2300. As in FIG. 16, the ophthalmological information-processing device 2200 has at least some of the functions of the data processing unit 300 in FIG. 5. For example, the ophthalmological information-processing device 2200 has the functions of a feature extractor 340, a classifier 350, and a learning unit 360. The functions of the ophthalmological information-processing device 2200 are implemented, for example, by a computer including a processor.

[0232] For example, the ophthalmic devices 2100-1 to 2100-T have the same functions as the ophthalmic device 2100.

[0233] <effect> An ophthalmological information processing apparatus, an ophthalmological apparatus, an ophthalmological information processing method, and a program according to an embodiment will be described.

[0234] An ophthalmological information processing device (data processing unit 300, ophthalmological information processing device 1000, 2200) according to some embodiments includes a feature extractor (1100, 340) and a classifier (1200, 350). The feature extractor extracts features of two or more different images (IMG-1 to IMG-7) of the subject's eye. The classifier outputs two or more pieces of confidence information (OUT-1 to OUT-4) for estimating a classification result of a pathological condition of a disease in the subject's eye based on the features of the two or more images extracted by the feature extractor and one or more background data (BG) representing the condition of the subject or the subject's eye.

[0235] According to this embodiment, two or more pieces of confidence information for estimating the classification result of the pathology of the eye disease of the subject are output using one or more pieces of background data, so it is possible to improve the accuracy of the pathology classification result compared to the case where confidence information is output only from the feature amount of the image, and therefore it is possible to provide an appropriate treatment for the disease of the subject's eye at an early stage.

[0236] In some embodiments, the feature extractor includes two or more extractors (first extractor 341 to seventh extractor 347) that extract features using two or more trained models obtained by performing supervised machine learning for each image. The classifier uses features (TR2) of images of the eyes of multiple subjects for training and background data (BG) of the multiple subjects or the eyes of the multiple subjects as training data, and outputs two or more pieces of confidence information using a classification model obtained by performing supervised machine learning with classification results of the images as supervised data (SD2).

[0237] According to this aspect, a classification model obtained by machine learning is used to output certainty information from the features and background data for each type of multiple images, making it possible to output highly accurate certainty information.

[0238] Some embodiments include a feature selector (feature selector 362) that selects one or more background data by performing a statistical test on two or more background data of each of a plurality of subjects with a disease.

[0239] According to this aspect, the certainty information is output using background data that is likely to contribute to outputting highly accurate certainty information, so that the accuracy of the certainty information can be further improved.

[0240] Some embodiments include a learning unit (second learning unit 363) that generates a classification model by performing supervised machine learning using features of images of the eyes of multiple subjects for training and one or more background data as training data and classification results of the images as teacher data.

[0241] According to this aspect, it becomes possible to output highly accurate certainty information from the feature amounts and background data for each type of a plurality of images.

[0242] In some embodiments, the features include confidence information for estimating a classification result of a pathology.

[0243] According to this aspect, the certainty information for each image and the one or more background data are used to output two or more pieces of certainty information for estimating the classification result of the pathology of the subject's eye disease, thereby making it possible to improve the accuracy of the pathology classification result.

[0244] In some embodiments, the two or more images include OCT images (images IMG-1 to IMG-6) obtained by performing optical coherence tomography on the subject's eye and a color image of the subject's eye (color fundus image IMG-7).

[0245] According to this embodiment, it is possible to output a highly accurate classification result of a pathological condition from an OCG image and a color image.

[0246] In some embodiments, the OCT image includes at least one of a front image of the fundus (projection image IMG-4, en-face image IMG-5) and a tomographic image of the fundus (B-scan images IMG-1 to IMG-3, IMG-6). The color image includes a color image of the fundus (fundus image IMG-7). The two or more pieces of confidence information include confidence information (OUT-1 to OUT-4) for estimating the classification result of the type of glaucoma pathology according to the Nicollella classification.

[0247] According to this embodiment, it is possible to output confidence level information that enables a highly accurate estimation of the classification result of the pathology of glaucoma according to the Nicollella classification.

[0248] In some embodiments, the OCT images include a B-scan image in a first direction (horizontal direction) passing through the center of the optic disc (IMG-1), a B-scan image in a second direction (vertical direction) intersecting the first direction passing through the center of the optic disc (IMG-2), a B-scan image in the second direction passing through the center of the fovea (IMG-3), a projection image (IMG-4), an en-face image (IMG-5), and a B-scan image passing halfway between the center of the optic disc and the center of the fovea (IMG-6).

[0249] According to this aspect, it is possible to output confidence information that reflects the morphology of the optic disc and the fovea and enables the classification result of the pathology of glaucoma according to the Nicollella classification to be estimated with high accuracy.

[0250] In some embodiments, the background data includes at least one of axial length, MD value, spherical equivalent, age, nerve head tissue blood flow value, and minimum intraocular pressure value in the past year.

[0251] According to this aspect, it becomes possible to output more accurate certainty information.

[0252] In some embodiments, the two or more pieces of confidence information include confidence information representing a confidence that the glaucoma is of focal ischemic type, confidence information representing a confidence that the glaucoma is of myopic type, confidence information representing a confidence that the glaucoma is of age-related sclerosis type, and confidence information representing a confidence that the glaucoma is of global magnification type.

[0253] According to this embodiment, it is possible to output certainty information indicating the certainty of each of the four types of glaucoma pathology.

[0254] An ophthalmic apparatus (1) according to some embodiments includes an optical system (an optical system provided in the fundus camera unit 2 and the OCT unit 100) for acquiring the above-mentioned two or more images, and any one of the above-mentioned ophthalmic information-processing devices.

[0255] According to this aspect, one or more of the background data are used to output two or more pieces of certainty information for estimating the classification result of the pathology of the subject's eye disease, making it possible to provide an ophthalmologic device that can improve the accuracy of the pathology classification result compared to when certainty information is output based only on the image features.

[0256] An ophthalmologic information processing method according to some embodiments includes a feature extraction step of extracting features of each of two or more different images of a subject's eye, and a classification step of outputting two or more pieces of confidence information for estimating a classification result of a pathological condition of a disease of the subject's eye based on the features of each of the two or more images extracted in the feature extraction step and one or more background data representing the condition of the subject or the subject's eye.

[0257] According to this embodiment, two or more pieces of confidence information for estimating the classification result of the pathology of the eye disease of the subject are output using one or more pieces of background data, so it is possible to improve the accuracy of the pathology classification result compared to the case where confidence information is output only from the feature amount of the image, and therefore it is possible to provide an appropriate treatment for the disease of the subject's eye at an early stage.

[0258] In some embodiments, the feature extraction step extracts features using two or more trained models obtained by performing supervised machine learning for each image, and the classification step outputs two or more pieces of confidence information using a classification model obtained by performing supervised machine learning using features of images of the eyes of multiple subjects for training and background data of the multiple subjects or the eyes of the multiple subjects as training data, and classification results of the images as supervised data.

[0259] According to this aspect, a classification model obtained by machine learning is used to output certainty information from the features and background data for each type of multiple images, making it possible to output highly accurate certainty information.

[0260] Some embodiments include a feature selection step of selecting one or more background data by performing a statistical test on two or more background data of each of a plurality of subjects with a disease.

[0261] According to this aspect, the certainty information is output using background data that is likely to contribute to outputting highly accurate certainty information, so that the accuracy of the certainty information can be further improved.

[0262] Some embodiments include a learning step of performing supervised machine learning using features of images of the eyes of multiple subjects for training and one or more background data as training data, and using classification results of the images as training data to generate a classification model.

[0263] According to this aspect, it becomes possible to output highly accurate certainty information from the feature amounts and background data for each type of a plurality of images.

[0264] In some embodiments, the features include confidence information for estimating a classification result of a pathology.

[0265] According to this aspect, the certainty information for each image and the one or more background data are used to output two or more pieces of certainty information for estimating the classification result of the pathology of the subject's eye disease, thereby making it possible to improve the accuracy of the pathology classification result.

[0266] In some embodiments, the two or more images include an OCT image of the fundus obtained by performing optical coherence tomography on the subject's eye and a color image of the fundus, and the two or more pieces of confidence information include confidence information for estimating a classification result of a type of glaucoma pathology according to the Nicollella classification.

[0267] According to this embodiment, it is possible to output a highly accurate classification result of a pathological condition according to the Nicollella classification from an OCG image and a color image.

[0268] In some embodiments, the OCT images include a B-scan image in a first direction passing through the center of the optic disc, a B-scan image in a second direction intersecting the first direction passing through the center of the optic disc, a B-scan image in a second direction passing through the center of the fovea, a projection image, an en-face image, and a B-scan image passing halfway between the center of the optic disc and the center of the fovea.

[0269] According to this aspect, it is possible to output confidence information that reflects the morphology of the optic disc and the fovea and enables the classification result of the pathology of glaucoma according to the Nicollella classification to be estimated with high accuracy.

[0270] In some embodiments, the background data includes at least one of axial length, MD value, spherical equivalent, age, nerve head tissue blood flow value, and minimum intraocular pressure value in the past year.

[0271] According to this aspect, it becomes possible to output more accurate certainty information.

[0272] In some embodiments, the two or more pieces of confidence information include confidence information representing a confidence that the glaucoma is of focal ischemic type, confidence information representing a confidence that the glaucoma is of myopic type, confidence information representing a confidence that the glaucoma is of age-related sclerosis type, and confidence information representing a confidence that the glaucoma is of global magnification type.

[0273] According to this embodiment, it is possible to output certainty information indicating the certainty of each of the four types of glaucoma pathology.

[0274] A program according to some embodiments causes a computer to execute each step of any of the ophthalmologic information processing methods described above.

[0275] According to this aspect, one or more of the background data are used to output two or more pieces of confidence information for estimating the classification result of the pathology of the subject's eye disease, making it possible to provide a program that can improve the accuracy of the pathology classification result compared to when confidence information is output based only on the image features.

[0276] A program for implementing the ophthalmologic information processing method according to some embodiments can be stored in any non-transitory computer-readable recording medium. The recording medium may be an electronic medium using magnetism, light, magneto-optical, semiconductor, etc. Typically, the recording medium is a magnetic tape, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, a solid-state drive, etc.

[0277] It is also possible to transmit and receive computer programs via a network such as the Internet or a LAN.

[0278] The above-described embodiments are merely examples for carrying out the present invention. Anyone who wishes to carry out the present invention may make any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention. [Explanation of symbols]

[0279] 1 Ophthalmology equipment 2 Fundus camera unit 100 OCT units 230 Image forming section 300 Data Processing Unit 310 B-scan imaging unit 320 Projection Image Formation Department 330 en-face Image Formation Department 340, 1100 Feature Extractor 350, 1200 classifier 360 Learning Department 361 First Learning Section 362 Feature Selection Unit 362A Statistical Testing Department 362C Cross-validation section 363 Second Learning Section 370 Analysis Department 1000, 2200 Ophthalmology information processing device 1110-1~1110-n, 1110-1~1110-7 Extractor BG, BG0, BG1 background data IMG-1~IMG-n, IMG-1~IMG-7 images OUT-1~OUT-m, OUT-1~OUT-4 confidence information

Claims

1. A feature extractor that extracts features of two or more different images of the subject's eye; a feature selector that selects one or more background data with the highest correct answer rate, the one or more background data being obtained by sequentially repeating a process of selecting one or more background data by a statistical test for two or more background data that represent each of a plurality of subjects with an eye disease or the eye condition of the subject, and a process of calculating a correct answer rate by cross-validation; a classifier that outputs two or more pieces of confidence information for estimating a classification result of a pathological condition of the disease based on each of the two or more images extracted by the feature extractor and the one or more pieces of background data selected by the feature selector; An ophthalmological information processing device comprising:

2. The feature extractor includes two or more extractors that extract the feature using two or more trained models obtained by performing supervised machine learning for each of the images; The classifier uses feature amounts of images of the eyes of a plurality of subjects for training and background data of the eyes of the plurality of subjects or the plurality of subjects as training data, and performs supervised machine learning using classification results of the images as supervised data to output the two or more pieces of confidence information using a classification model obtained by the above. The ophthalmological information processing apparatus according to claim 1 .

3. a learning unit that generates the classification model by performing supervised machine learning using the feature amounts of the images of the eyes of the plurality of subjects for training and the one or more background data as training data and using classification results of the images as supervised data.

3. The ophthalmologic information processing apparatus according to claim 2.

4. The feature amount includes confidence information for estimating the classification result of the pathology.

4. The ophthalmologic information processing apparatus according to claim 1, wherein the ophthalmologic information processing apparatus is a computer.

5. The two or more images include an OCT image obtained by performing optical coherence tomography on the subject's eye, and a color image of the subject's eye.

5. The ophthalmological information processing apparatus according to claim 1, wherein the ophthalmological information processing apparatus is a computer.

6. The OCT image includes at least one of a front image of a fundus and a tomographic image of the fundus, the color image includes a color image of the fundus; The two or more pieces of confidence information include confidence information for estimating a classification result of the pathological condition of glaucoma according to the Nicollera classification.

6. An ophthalmologic information processing apparatus according to claim 5.

7. The OCT image includes a B-scan image in a first direction passing through the center of the optic disc, a B-scan image in a second direction intersecting the first direction passing through the center of the optic disc, a B-scan image in the second direction passing through the center of the fovea, a projection image, an en-face image, and a B-scan image passing midway between the center of the optic disc and the center of the fovea.

7. An ophthalmologic information processing apparatus according to claim 6.

8. The background data includes at least one of an axial length, an MD value, an equivalent spherical power, an age, a nerve head tissue blood flow value, and a minimum intraocular pressure value in the past year.

8. An ophthalmologic information processing apparatus according to claim 6 or 7.

9. The two or more pieces of certainty information include certainty information indicating a certainty that glaucoma is of local ischemic type, certainty information indicating a certainty that glaucoma is of myopic type, certainty information indicating a certainty that glaucoma is of age-related sclerosis type, and certainty information indicating a certainty that glaucoma is of global expansion type. The ophthalmological information processing apparatus according to any one of claims 6 to 8.

10. an optical system for acquiring the two or more images; An ophthalmological information processing device according to any one of claims 1 to 9, 13. An ophthalmic device comprising:

11. A feature extraction step of extracting feature amounts of each of two or more different images of the subject's eye; a feature selection step of selecting one or more background data items with the highest correct answer rate obtained by sequentially repeating a process of selecting one or more background data items by a statistical test for two or more background data items that represent each of a plurality of subjects with an eye disease or the eye condition of the subject, and a process of calculating a correct answer rate by cross-validation; a classification step of outputting two or more pieces of confidence information for estimating a classification result of a pathological condition of the disease based on each of the two or more images extracted in the feature extraction step and the one or more pieces of background data selected in the feature selection step; An ophthalmological information processing method comprising:

12. The feature extraction step extracts the feature using two or more trained models obtained by performing supervised machine learning for each of the images; The classification step uses feature amounts of images of the eyes of a plurality of subjects for training and background data of the eyes of the plurality of subjects or the plurality of subjects as training data, and outputs the two or more pieces of confidence information using a classification model obtained by performing supervised machine learning using classification results of the images as training data. The ophthalmologic information processing method according to claim 11 .

13. A learning step of generating the classification model by performing supervised machine learning using the feature amounts of the images of the eyes of the plurality of subjects for training and the one or more background data as training data and using classification results of the images as supervised data. The ophthalmologic information processing method according to claim 12 .

14. The feature amount includes confidence information for estimating the classification result of the pathology. The ophthalmologic information processing method according to any one of claims 11 to 13.

15. The two or more images include an OCT image of a fundus obtained by performing optical coherence tomography on the subject's eye, and a color image of the fundus; The two or more pieces of confidence information include confidence information for estimating a classification result of the pathological condition of glaucoma according to the Nicollera classification. The ophthalmologic information processing method according to any one of claims 11 to 14.

16. The OCT image includes a B-scan image in a first direction passing through the center of the optic disc, a B-scan image in a second direction intersecting the first direction passing through the center of the optic disc, a B-scan image in the second direction passing through the center of the fovea, a projection image, an en-face image, and a B-scan image passing midway between the center of the optic disc and the center of the fovea. The ophthalmologic information processing method according to claim 15 .

17. The background data includes at least one of an axial length, an MD value, an equivalent spherical power, an age, a nerve head tissue blood flow value, and a minimum intraocular pressure value in the past year.

17. An ophthalmologic information processing method according to claim 15 or 16.

18. The two or more pieces of certainty information include certainty information indicating a certainty that glaucoma is of local ischemic type, certainty information indicating a certainty that glaucoma is of myopic type, certainty information indicating a certainty that glaucoma is of age-related sclerosis type, and certainty information indicating a certainty that glaucoma is of global expansion type. The ophthalmologic information processing method according to any one of claims 15 to 17.

19. A program for causing a computer to execute each step of the ophthalmologic information processing method according to any one of claims 11 to 18.

Images