Ophthalmologic information processing device, ophthalmologic device, operation method of ophthalmologic information processing device and program

The ophthalmological information processing apparatus uses machine learning on OCT images to accurately detect and classify fundus diseases, facilitating early and appropriate treatment by analyzing multiple cross-sectional images of the eye.

JP2024159925AActive Publication Date: 2024-11-08TOPCON CORPORATION +2
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2024147068
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-08
Estimated Expiration
2040-03-18

AI Technical Summary

Technical Problem

Existing methods struggle to accurately and promptly detect fundus diseases such as glaucoma, age-related macular degeneration, and diabetic retinopathy, and provide appropriate treatment based on understanding the pathology of these diseases.

Method used

An ophthalmological information processing apparatus that acquires multiple images of the eye in different cross-sectional directions using optical coherence tomography (OCT), employs machine learning to generate trained models for each image type, and estimates the presence and pathology of diseases using a combination of disease estimation and pathology estimation models.

Benefits of technology

Enables early and accurate detection of fundus diseases, allowing for timely and appropriate treatment strategies based on the disease's pathology, improving treatment outcomes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024159925000001_ABST
    Figure 2024159925000001_ABST
Patent Text Reader

Abstract

To provide a new technique for performing appropriate treatment for a disease of a subject eye at an early stage.SOLUTION: An ophthalmologic information processing device includes an acquisition unit, multiple estimators, and a classifier. The acquisition unit acquires multiple images of a subject eye with mutually-different cross-sectional directions. The multiple estimators use respective learned models obtained through machine learning for each type of multiple images to output a feature amount or certainty information representing certainty that the subject eye is an eye with age-related macular degeneration for each type of images. The classifier uses a classification model obtained through machine learning to output estimation information for estimating whether or not the subject eye is the eye with the age-related macular degeneration on the basis of the multiple feature amounts or the multiple pieces of certainty information output from the multiple estimators.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an ophthalmologic information-processing device, an ophthalmologic device, an operation method for an ophthalmologic information-processing device, and a program. [Background technology]

[0002] In recent years, the rapid progress of machine learning techniques, such as deep learning, has led to the practical application of artificial intelligence technology in various fields. In particular, in the medical field, deep learning has improved the accuracy of detection of diseased sites or tissue features in diagnostic images, making it possible to perform accurate and high-precision medical diagnoses quickly.

[0003] For example, Patent Document 1 discloses a method of performing machine learning using feature points in a plurality of past time-series fundus images as training data, and diagnosing glaucoma from a plurality of newly acquired time-series fundus images using the obtained trained model.For example, Patent Document 2 discloses a method of classifying the pathology of glaucoma according to the shape of the optic disc using a neural network. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 211688 [Patent Document 2] JP 2019-5319 A Summary of the Invention [Problem to be solved by the invention]

[0005] For example, ocular fundus diseases such as glaucoma, age-related macular degeneration, and diabetic retinopathy are diseases whose symptoms will progress unless appropriate treatment is administered, even if they are discovered early by screening, etc. In this case, by properly understanding the pathology of the disease, appropriate treatment for the disease can be administered.

[0006] In this manner, it is required to detect fundus diseases with high accuracy and at an early stage, and to specify the pathology of the detected fundus disease with high accuracy. This applies not only to fundus diseases but also to diseases of the subject's eye in general.

[0007] The present invention has been made in consideration of the above circumstances, and has an object to provide a new technique for providing appropriate treatment for a disease of an examinee's eye at an early stage. [Means for solving the problem]

[0008] A first aspect of some embodiments is an ophthalmological information processing device that includes an acquisition unit that acquires multiple images of a test eye with different cross-sectional directions, and a disease estimation unit that uses multiple trained models obtained by machine learning for each type of the multiple images to output estimation information for estimating whether the test eye is a diseased eye from the multiple images.

[0009] In a second aspect of some embodiments, in the first aspect, the disease estimation unit includes a plurality of estimators that use each of the plurality of trained models for each type of the plurality of images to output features or confidence information representing a confidence that the test eye is a diseased eye, and a classifier that uses a classification model obtained by machine learning to output the estimation information from the plurality of features or the plurality of confidence information output from the plurality of estimators.

[0010] In a third aspect of some embodiments, in the first or second aspect, the plurality of images include a tomographic image or a front image of a fundus of the subject's eye, and the disease is a fundus disease.

[0011] In a fourth aspect of some embodiments, in the third aspect, the plurality of images includes B-scan images in a cross-sectional direction intersecting a line connecting the optic disc and the fovea.

[0012] In a fifth aspect of some embodiments, in the third or fourth aspect, the plurality of images include a B-scan image in a first cross-sectional direction passing through the optic disc, a B-scan image in a second cross-sectional direction passing through the optic disc and intersecting the first cross-sectional direction, and a B-scan image passing through the fovea.

[0013] In a sixth aspect of some embodiments, in any of the third to fifth aspects, the front image includes at least one of an en-face image projected from a layer area corresponding to the internal limiting membrane to a deep layer area, and a projection image.

[0014] In a seventh aspect of some embodiments, in any one of the third to sixth aspects, the fundus disease includes at least one of glaucoma, age-related macular degeneration, and diabetic reticuloretinopathy.

[0015] An eighth aspect of some embodiments, in any of the first to seventh aspects, includes a pathology estimation unit that outputs classification information for estimating a classification result of the pathology of the disease from the multiple images, using a trained model for classification obtained by supervised machine learning using features obtained by the multiple trained models.

[0016] In a ninth aspect of some embodiments, in the eighth aspect, the disease includes glaucoma, and the classification information is information for estimating classification results of multiple types of optic disc shapes.

[0017] A tenth aspect of some embodiments, in the eighth or ninth aspect, includes a first learning unit that generates the trained model for classification by supervised machine learning using features obtained by the multiple trained models.

[0018] An eleventh aspect of some embodiments, in any of the first to tenth aspects, includes a second learning unit configured to generate the plurality of trained models by supervised machine learning for each type of the plurality of images.

[0019] A twelfth aspect of some embodiments, in any one of the first to eleventh aspects, further includes an image generating unit that generates at least one of the plurality of images based on three-dimensional OCT data of the subject's eye.

[0020] A thirteenth aspect of some embodiments is an ophthalmic device including an OCT unit that performs optical coherence tomography on the test eye, an image generation unit that generates at least one of the plurality of images based on the three-dimensional data acquired by the OCT unit, and an ophthalmic information processing device of any of the first to eleventh aspects.

[0021] A fourteenth aspect of some embodiments is an ophthalmic apparatus including an OCT unit that performs optical coherence tomography on the subject's eye, and the ophthalmic information processing device of the twelfth aspect.

[0022] A fifteenth aspect of some embodiments is a method for operating an ophthalmological information processing device, including an acquisition step of acquiring multiple images of a test eye having different cross-sectional directions, and a disease estimation step of outputting estimated information for estimating whether the test eye is a diseased eye from the multiple images using multiple trained models obtained by machine learning for each type of the multiple images.

[0023] In a 16th aspect of some embodiments, in the 15th aspect, the disease estimation step includes a plurality of estimation steps of outputting features or confidence information representing a confidence that the test eye is a diseased eye, using each of the plurality of trained models for each type of the plurality of images, and a classification step of outputting the estimation information from the plurality of features or the plurality of confidence information output in the plurality of estimation steps, using a classification model obtained by machine learning.

[0024] In a seventeenth aspect of some embodiments, in the fifteenth or sixteenth aspect, the plurality of images include a tomographic image or a front image of a fundus of the test eye, and the disease is a fundus disease.

[0025] In an eighteenth aspect of some embodiments, in the seventeenth aspect, the plurality of images includes B-scan images in a cross-sectional direction intersecting a line connecting the optic disc and the fovea.

[0026] In a 19th aspect of some embodiments, in the 17th or 18th aspect, the plurality of images include a B-scan image in a first cross-sectional direction passing through the optic disc, a B-scan image in a second cross-sectional direction passing through the optic disc and intersecting the first cross-sectional direction, and a B-scan image passing through the fovea.

[0027] In a twentieth aspect of some embodiments, in any of the seventeenth to nineteenth aspects, the front image includes at least one of an en-face image projected from a layer area corresponding to the internal limiting membrane to a deep layer area, and a projection image.

[0028] In a twenty-first aspect of some embodiments, in any one of the seventeenth to twentieth aspects, the fundus disease includes at least one of glaucoma, age-related macular degeneration, and diabetic reticuloretinopathy.

[0029] A 22nd aspect of some embodiments, in any of the 15th to 21st aspects, includes a pathology estimation step of outputting classification information for estimating a classification result of the pathology of the disease from the multiple images, using a trained model for classification obtained by supervised machine learning using features obtained by the multiple trained models.

[0030] In a 23rd aspect of some embodiments, in the 22nd aspect, the disease includes glaucoma, and the classification information is information for estimating classification results of multiple types of optic disc shapes.

[0031] A 24th aspect of some embodiments includes, in the 22nd or 23rd aspect, a first learning step of generating the trained model for classification by supervised machine learning using features obtained by the multiple trained models.

[0032] A twenty-fifth aspect of some embodiments, in any of the fifteenth to twenty-fourth aspects, includes a second learning step of generating the plurality of trained models by supervised machine learning for each type of the plurality of images.

[0033] A 26th aspect of some embodiments, in any of the 15th to 25th aspects, further includes an image generating step of generating at least one of the plurality of images based on three-dimensional OCT data of the subject's eye.

[0034] A twenty-seventh aspect of some embodiments is a program for causing a computer to execute each step of the method for operating an ophthalmologic information-processing device according to any one of the fifteenth to twenty-sixth aspects.

[0035] The configurations according to the above-described multiple aspects can be combined in any desired manner. Effect of the Invention

[0036] According to some embodiments of the present invention, a new technique for providing appropriate treatment for a disease of an examinee's eye at an early stage can be provided. [Brief description of the drawings]

[0037] [Figure 1] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic system according to an embodiment. [Diagram 2] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic apparatus according to an embodiment. [Diagram 3] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. [Figure 4] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. [Diagram 5] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. [Figure 6] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. [Figure 7] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. [Figure 8] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. [Figure 9] 4 is a schematic diagram illustrating an example of an operation flow of the ophthalmologic information processing apparatus according to the embodiment. FIG. [Figure 10] 4 is a schematic diagram illustrating an example of an operation flow of the ophthalmologic information processing apparatus according to the embodiment. FIG. [Figure 11] FIG. 11 is a schematic diagram illustrating an operation of an ophthalmologic information processing apparatus according to a comparative example of the embodiment. [Figure 12] FIG. 2 is a schematic diagram illustrating the operation of the ophthalmologic information processing apparatus according to the embodiment. [Figure 13] FIG. 13 is a schematic diagram illustrating an example of a configuration of an ophthalmologic apparatus according to a modified example of an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] Examples of an ophthalmological information processing apparatus, an ophthalmological apparatus, an ophthalmological information processing method, and a program according to some embodiments of the present invention will be described in detail with reference to the drawings. Note that matters described in documents cited in this specification and any publicly known techniques can be incorporated in the embodiments.

[0039] The ophthalmic information processing apparatus according to the embodiment can perform a predetermined analysis process and a predetermined display process on three-dimensional OCT data of a subject's eye acquired by an ophthalmic apparatus using Optical Coherence Tomography (OCT). The ophthalmic apparatus according to some embodiments has not only a function of an OCT apparatus for performing OCT, but also at least one function of a fundus camera, a scanning laser ophthalmoscope, a slit lamp microscope, and a surgical microscope. Furthermore, the ophthalmic apparatus according to some embodiments has a function of measuring optical characteristics of the subject's eye. Examples of ophthalmic apparatuses having a function of measuring optical characteristics of the subject's eye include a refractometer, a keratometer, a tonometer, a wavefront analyzer, a specular microscope, and a perimeter. The ophthalmic apparatus according to some embodiments has a function of a laser treatment apparatus used for laser treatment.

[0040] The ophthalmologic information processing device generates a plurality of images (tomographic images or front images) having different cross-sectional directions from the acquired 3D OCT data of the subject's eye, and outputs estimation information for estimating whether the subject's eye is a diseased eye from the generated plurality of images. Furthermore, the ophthalmologic information processing device outputs classification information for estimating a classification result of the pathological condition of the disease from the generated plurality of images (or their feature amounts).

[0041] Hereinafter, a case where the ophthalmological information processing device, the ophthalmological device, the ophthalmological information processing method, and the program according to the embodiment are applied to fundus diseases will be described. However, the application field of the ophthalmological information processing device, the ophthalmological device, the ophthalmological information processing method, and the program according to the embodiment is not limited to fundus diseases, and can be applied to diseases of the subject eye in general.

[0042] Ocular fundus diseases include glaucoma, age-related macular degeneration, and diabetic retinopathy.

[0043] When the fundus disease is glaucoma, for example, classification information for estimating a classification result based on the shape of the optic disc is output in order to classify the pathological condition according to the classification by Nicolela et al. (Nicolella classification) as disclosed in Patent Document 2.

[0044] When the fundus disease is age-related macular degeneration, for example, classification information is output for estimating a classification result based on the morphology of the retinal pigment epithelium layer (e.g., the morphology of choroidal neovascularization) to classify the disease as atrophic or exudative.

[0045] If the fundus disease is diabetic retinopathy, classification information is output to estimate the classification result based on the morphology and distribution of microaneurysms, hemorrhages, white spots, neovascularization, and hemorrhagic areas in order to classify the disease as simple diabetic retinopathy, preproliferative diabetic retinopathy, or proliferative diabetic retinopathy, for example.

[0046] In this way, by providing doctors and others with not only estimated information on fundus diseases but also classification information on pathological classification of fundus diseases, it becomes possible to determine an appropriate treatment for fundus diseases at an early stage, thereby increasing the possibility of curing fundus diseases.

[0047] For example, in a case where the ocular fundus disease is determined to be glaucoma, an appropriate treatment method can be selected from among drug therapy, laser treatment, surgery, etc. according to the pathology of the disease.

[0048] For example, in cases where the fundus disease is determined to be age-related macular degeneration, an appropriate treatment for exudative age-related macular degeneration can be selected from among drug dosage, photodynamic therapy, laser coagulation, surgery, etc., depending on the pathology of the disease.

[0049] For example, in a case where the ocular fundus disease is determined to be diabetic retinopathy, an appropriate treatment method can be selected from among retinal photocoagulation, vitreous surgery, etc. according to the pathology of the disease.

[0050] The ophthalmologic information processing device of the embodiment is capable of estimating whether or not the subject eye has a fundus disease and classifying the pathology of the fundus disease by using a trained model obtained by machine learning for each of multiple images generated from 3D OCT data.

[0051] An ophthalmological system according to an embodiment includes an ophthalmological information processing device. An ophthalmological information processing method according to an embodiment is executed by the ophthalmological information processing device. A program according to an embodiment causes a computer to execute each step of the ophthalmological information processing method.

[0052] In the following, glaucoma will be described as an example of a fundus disease, but the following embodiment can also be applied to other fundus diseases.

[0053] [Ophthalmology System] 1 shows a block diagram of a configuration example of an ophthalmological system according to an embodiment. The ophthalmological system 1 according to the embodiment includes an ophthalmological apparatus 10, an ophthalmological information processing device (ophthalmological image processing device, ophthalmological analysis device) 100, an operation device 180, and a display device 190.

[0054] The ophthalmic apparatus 10 performs OCT on the subject's eye and collects three-dimensional OCT data of the subject's eye. In this embodiment, the ophthalmic apparatus 10 collects fundus OCT data by OCT scanning the fundus of the subject's eye. The ophthalmic apparatus 10 can acquire images of the fundus from the acquired OCT data of the fundus. The images of the fundus include tomographic images and frontal images of the fundus. The tomographic images of the fundus include B-scan images, etc. The frontal images of the fundus include C-scan images, en-face images, shadowgrams, projection images, etc. The ophthalmic apparatus 10 transmits the acquired OCT data of the subject's eye or data of the acquired images to the ophthalmic information processing apparatus 100.

[0055] In some embodiments, the ophthalmological apparatus 10 and the ophthalmological information processing apparatus 100 are connected via a data communication network. The ophthalmological information processing apparatus 100 according to some embodiments receives the above-mentioned data from one of the multiple ophthalmological apparatuses 10 selectively connected via the data communication network.

[0056] The ophthalmologic information processing device 100 generates first estimation information for estimating whether or not the subject eye is a glaucomatous eye using an individual estimation model (trained model) obtained by machine learning for each of a plurality of images generated from the three-dimensional OCT data. The ophthalmologic information processing device 100 generates second estimation information for estimating whether or not the subject eye is a glaucomatous eye from a plurality of first estimation information generated for each of a plurality of images using a classification model (trained model) obtained by machine learning. Furthermore, the ophthalmologic information processing device 100 outputs classification information for estimating a pathology classification result from the above-mentioned plurality of images (or its feature amount) using a pathology estimation model (trained model) obtained by machine learning. The pathology estimation model is obtained by executing machine learning using the feature amount of the above-mentioned plurality of individual estimation models (or a plurality of individual estimation models and classification models). As a result, it is possible to acquire highly accurate classification information using a pathology estimation model obtained by machine learning using less training data compared to a case where a pathology estimation model is generated by performing machine learning from natural images or the like.

[0057] Hereinafter, a case where five types of images are generated from three-dimensional OCT data will be described. The five types of images are a horizontal B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the fovea (or its vicinity), a projection image, and an en-face image. However, the configuration according to the embodiment is not limited to the types and number of types of images generated from the three-dimensional OCT data. For example, a plurality of images including a B-scan image in a cross-sectional direction intersecting a line connecting the optic disc and the fovea may be generated from the three-dimensional OCT data, and the above-mentioned estimated information or classification information may be acquired based on the generated plurality of images.

[0058] In some embodiments, the ophthalmological information-processing device 100 constructs the above-mentioned trained model and outputs estimated information or classified information using the constructed trained model. In some embodiments, the ophthalmological information-processing device 100 outputs estimated information or classified information using an externally constructed trained model. In some embodiments, the ophthalmological information-processing device 100 constructs the above-mentioned trained model and outputs the constructed trained model to an external device.

[0059] The operation device 180 and the display device 190 provide functions for exchanging information between the ophthalmologic information-processing device 100 and its user, such as displaying information, inputting information, and inputting operation instructions, as a user interface unit. The operation device 180 includes operation devices such as a lever, a button, a key, and a pointing device. The operation device 180 according to some embodiments includes a microphone for inputting information by sound. The display device 190 includes a display device such as a flat panel display. In some embodiments, the functions of the operation device 180 and the display device 190 are realized by a device in which a device having an input function, such as a touch panel display, and a device having a display function are integrated. In some embodiments, the operation device 180 and the display device 190 include a graphical user interface (GUI) for inputting and outputting information.

[0060] [Ophthalmological equipment] FIG. 2 shows a block diagram of an example of the configuration of an ophthalmologic apparatus 10 according to the embodiment.

[0061] The ophthalmic device 10 is provided with an optical system for acquiring OCT data of the subject's eye. The ophthalmic device 10 has a function for performing swept-source OCT, but the embodiment is not limited thereto. For example, the type of OCT is not limited to swept-source OCT, and may be spectral domain OCT or the like. Swept-source OCT is a method of splitting light from a wavelength-swept (wavelength-scanning) light source into measurement light and reference light, causing return light of the measurement light passing through an object to be measured to interfere with the reference light to generate interference light, detecting the interference light with a balanced photodiode or the like, and forming an image by applying Fourier transform or the like to detection data collected in response to the sweeping of the wavelength and the scanning of the measurement light. Spectral domain OCT is a method of splitting light from a low-coherence light source into measurement light and reference light, causing return light of the measurement light passing through an object to be measured to interfere with the reference light to generate interference light, detecting the spectral distribution of the interference light with a spectroscope, and forming an image by applying Fourier transform or the like to the detected spectral distribution.

[0062] The ophthalmologic apparatus 10 includes a control unit 11, an OCT unit 12, an OCT data processing unit 13, and a communication unit 14.

[0063] The control unit 11 controls each unit of the ophthalmologic apparatus 10. In particular, the control unit 11 controls the OCT unit 12, the OCT data processing unit 13, and the communication unit 14.

[0064] The OCT unit 12 collects three-dimensional OCT data of the subject's eye by scanning the subject's eye using OCT. The OCT unit 12 includes an interference optical system 12A and a scanning optical system 12B.

[0065] The interference optical system 12A splits light from a light source (wavelength swept light source) into measurement light and reference light, generates interference light by making the return light of the measurement light passing through the test eye interfere with the reference light passing through the reference light path, and detects the generated interference light. The interference optical system 12A includes at least a fiber coupler and a photoreceiver such as a balanced photodiode. The fiber coupler splits light from a light source into measurement light and reference light, and generates interference light by making the return light of the measurement light passing through the test eye interfere with the reference light passing through the reference light path. The photoreceiver detects the interference light generated by the fiber coupler. The interference optical system 12A may include a light source.

[0066] The scanning optical system 12B is controlled by the control unit 11 and changes the incident position of the measurement light on the fundus of the test eye by deflecting the measurement light generated by the interference optical system 12A. The scanning optical system 12B includes, for example, an optical scanner arranged at a position optically approximately conjugate with the pupil of the test eye. The optical scanner includes, for example, a first galvanometer mirror that deflects the measurement light in the horizontal direction, a second galvanometer mirror that deflects the measurement light in the vertical direction, and a mechanism for independently driving these. For example, the second galvanometer mirror is configured to further deflect the measurement light deflected by the first galvanometer mirror. This allows the measurement light to be scanned in any direction on the fundus plane.

[0067] The detection result (detection signal) of the interference light by the interference optical system 12A is an interference signal indicating the spectrum of the interference light.

[0068] The OCT data processing unit 13 is controlled by the control unit 11 and forms three-dimensional data (image data) of the fundus based on the data of the subject's eye collected by the OCT unit 12.

[0069] For example, the OCT data processing unit 13 forms a reflection intensity profile for each A line, and arranges the formed multiple reflection intensity profiles in the B scan direction (a direction intersecting the A scan direction (e.g., z direction), e.g., x direction) and a direction intersecting the A scan direction and B scan direction (e.g., y direction) to form three-dimensional OCT data (scan data).

[0070] Also, for example, the OCT data processing unit 13 can image the reflection intensity profile in the A-line to form an A-scan image of the subject's eye E. The OCT data processing unit 13 can form a three-dimensional image by arranging a plurality of A-scan images formed for each A-line in the B-scan direction and in a direction intersecting the A-scan direction and the B-scan direction.

[0071] The processes executed by the OCT data processor 13 include noise removal (noise reduction), filtering, FFT (Fast Fourier Transform), etc. The image data acquired in this manner is a data set including a group of image data formed by imaging the reflection intensity profiles in multiple A-lines (paths of each measurement light in the subject's eye). In order to improve image quality, multiple data sets collected by repeating the scan with the same pattern multiple times can be superimposed (averaged).

[0072] In some embodiments, the OCT data processor 13 performs various data processing (image processing) and analysis processing on the image. For example, the OCT data processor 13 performs correction processing such as image brightness correction and dispersion correction. The OCT data processor 13 can form volume data (voxel data) of the subject's eye by performing known image processing such as an interpolation process that interpolates pixels between tomographic images. When an image based on the volume data is to be displayed, the OCT data processor 13 performs a rendering process on the volume data to form a pseudo three-dimensional image as viewed from a specific line of sight.

[0073] Each of the control unit 11 and the OCT data processing unit 13 includes a processor. The processor includes a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), or a field programmable gate array (FPGA)). The function of the OCT data processing unit 13 is realized by a data processing processor. In some embodiments, the functions of both the control unit 11 and the OCT data processing unit 13 are realized by one processor.

[0074] The processor realizes the functions according to the embodiment by, for example, reading and executing a program stored in a memory circuit or a storage device. At least a part of the memory circuit or the storage device may be included in the processor. Also, at least a part of the memory circuit or the storage device may be provided outside the processor.

[0075] The storage device etc. stores various data. The data stored in the storage device etc. includes data acquired by the OCT unit 12 (measurement data, photographing data, etc.), information on the subject and the subject's eye, etc. The storage device etc. may store various computer programs and data for operating each unit of the ophthalmic apparatus 10.

[0076] The communication unit 14 is controlled by the control unit 11 and executes a communication interface process for transmitting or receiving information to or from the ophthalmologic information-processing device 100 .

[0077] The ophthalmologic apparatus 10 according to some embodiments transmits data of the subject's eye formed by the OCT data processing unit 13 to the ophthalmologic information-processing device 100.

[0078] The ophthalmic apparatus 10 according to some embodiments is provided with a fundus camera, a scanning laser ophthalmoscope, and a slit lamp microscope for acquiring an image of the fundus of the subject's eye. In some embodiments, the fundus image acquired by the fundus camera is a fluorescein fundus angiography image or a fundus autofluorescence examination image.

[0079] [Ophthalmology information processing device] 3 to 6 show block diagrams of configuration examples of an ophthalmologic information-processing device 100 according to an embodiment. Fig. 3 shows a functional block diagram of the ophthalmologic information-processing device 100. Fig. 4 shows a functional block diagram of the analysis unit 200 in Fig. 3. Fig. 5 shows a functional block diagram of the estimation model construction unit 210 in Fig. 3. Fig. 6 shows a functional block diagram of the estimation unit 220 in Fig. 3.

[0080] The ophthalmological information-processing device 100 includes a control unit 110, a data processing unit 130, and a communication unit 140. In some embodiments, the ophthalmological information-processing device 100 includes an image forming unit having a function similar to that of the OCT data processing unit 13 of the ophthalmological device 10.

[0081] The control unit 110 controls each unit of the ophthalmologic information-processing device 100. In particular, the control unit 110 controls the data processing unit 130 and the communication unit 140. The control unit 110 includes a main control unit 111 and a storage unit 112.

[0082] The control unit 110 controls each unit of the ophthalmologic system 1 based on an operation instruction signal corresponding to the content of a user's operation on the operation device 180.

[0083] Each of the control unit 110 and the data processing unit 130 includes a processor. The functions of the data processing unit 130 are realized by the data processor. In some embodiments, the functions of the control unit 110 and the data processing unit 130 are realized by a single processor.

[0084] The storage unit 112 stores various types of data. The data stored in the storage unit 112 include data acquired by the ophthalmic apparatus 10 (measurement data, photographed data, etc.), data processing results by the data processing unit 130, information on the subject and the subject's eye, etc. The storage unit 112 may store various computer programs and data for operating each unit of the ophthalmic information-processing device 100.

[0085] The communication unit 140 is controlled by the control unit 110 and executes a communication interface process for transmitting or receiving information to or from the communication unit 14 of the ophthalmologic information-processing device 100 .

[0086] The data processing unit 130 can generate an image in a specified arbitrary cross-sectional direction by performing various renderings on the three-dimensional OCT data from the ophthalmologic apparatus 10. In this embodiment, the data processing unit 130 generates a plurality of images having different cross-sectional directions. The plurality of images include a B-scan image, a C-scan image, an en-face image, a projection image, a shadowgram, and the like. An image of an arbitrary cross-section such as a B-scan image or a C-scan image is formed by selecting pixels (pixels, voxels) on a specified cross-section from three-dimensional OCT data (image data). An en-face image is formed by flattening a part of the three-dimensional OCT data (image data). A projection image is formed by projecting the three-dimensional OCT data in a predetermined direction (z direction, depth direction, A-scan direction). A shadowgram is formed by projecting a part of the three-dimensional OCT data (for example, partial data corresponding to a specific layer) in a predetermined direction.

[0087] In some embodiments, at least one of the cross-sectional position and the cross-sectional direction is specified by the user operating the operation device 180. The user can specify at least one of the cross-sectional position and the cross-sectional direction using the operation device 180 while referring to a front image of the fundus generated from three-dimensional OCT data or a fundus image acquired using a fundus camera (not shown).

[0088] In some embodiments, the data processing unit 130 identifies at least one of the cross-sectional position and the cross-sectional direction by analyzing the OCT data. In this case, the data processing unit 130 can identify a characteristic region by a known analysis process and identify at least one of the cross-sectional position and the cross-sectional direction so as to pass through the identified characteristic region. Alternatively, the data processing unit 130 can identify a characteristic region by a known analysis process and identify at least one of the cross-sectional position and the cross-sectional direction so as to avoid the identified characteristic region.

[0089] The data processing unit 130 performs a predetermined data processing on the formed image of the subject's eye. Like the OCT data processing unit 13, the data processing unit 130 can perform various data processing (image processing) and analysis processing on the formed image. For example, the data processing unit 130 performs correction processing such as image brightness correction and dispersion correction. The data processing unit 130 can form volume data (voxel data) of the subject's eye by performing known image processing such as an interpolation process that interpolates pixels between tomographic images. When an image based on the volume data is to be displayed, the data processing unit 130 performs a rendering process on the volume data to form a pseudo three-dimensional image as viewed from a specific line of sight.

[0090] Note that the OCT data processing unit 13 in the ophthalmic apparatus 10 may generate the above-mentioned image group instead of the data processing unit 130. In this case, the ophthalmic information-processing device 100 acquires the above-mentioned image group from the ophthalmic apparatus 10.

[0091] The data processing unit 130 also generates a disease estimation model for estimating whether the subject eye is a glaucomatous eye from a plurality of images generated from the 3D OCT data. Furthermore, the data processing unit 130 generates a pathology estimation model for estimating a pathology classification result from the plurality of images or the feature amounts of each of the plurality of images.

[0092] The data processing unit 130 uses the above-mentioned disease estimation model and pathology estimation model to output estimation information for estimating whether the test eye is a glaucoma eye from multiple images of the test eye, and classification information for estimating the classification result of the glaucoma pathology.

[0093] Such a data processing unit 130 includes an analysis unit 200 , an estimation model construction unit 210 , and an estimation unit 220 .

[0094] The analysis unit 200 performs a predetermined analysis process on the image data of the fundus (or the image data of the fundus acquired by the ophthalmologic device 10). The analysis process includes a process of generating a tomographic image in a desired cross-sectional direction at a desired cross-sectional position. The analysis process according to some embodiments includes a process of identifying a predetermined site such as the optic disc, the fovea, etc.

[0095] 4, the analysis unit 200 includes an image generation unit 201. The image generation unit 201 generates a plurality of images having different cross-sectional directions from the three-dimensional OCT data acquired by the ophthalmologic apparatus 10. The plurality of images include a B-scan image, a C-scan image, an en-face image, a projection image, a shadowgram, and the like.

[0096] The image generating unit 201 can generate at least one image in which a cross-sectional position and a cross-sectional direction are specified by a user from the three-dimensional OCT data. The image generating unit 201 can also generate at least one image in which a cross-sectional position and a cross-sectional direction are specified so as to pass through a specific site identified in the analysis process in the analysis unit 200. The image generating unit 201 can also generate at least one image in which a cross-sectional position and a cross-sectional direction are specified so as to avoid a specific site identified in the analysis process in the analysis unit 200.

[0097] In this embodiment, the image generating unit 201 generates five types of images with different cross-sectional directions from three-dimensional OCT data of the subject's eye. The five types of images include a horizontal B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the fovea (or its vicinity), a projection image, and an en-face image. The projection image is an integrated image of all layers in the depth direction. The en-face image is an integrated image of a predetermined depth (for example, 50 micrometers) in the depth direction from the top layer of the retina (for example, the internal limiting membrane). That is, the en-face image is, for example, an image obtained by projecting a layer region from a layer region corresponding to the internal limiting membrane to a deep layer region.

[0098] As described above, the estimation model construction unit 210 constructs a disease estimation model (the above estimation model and classification model) for estimating whether the subject eye is a glaucomatous eye or not, and a pathology estimation model for estimating the classification result of the pathology.

[0099] 5, the estimation model construction unit 210 includes a disease estimation learning unit 211 and a pathological condition estimation learning unit 212. The estimation unit 220 includes a disease estimation unit 221 and a pathological condition estimation unit 222, as shown in FIG.

[0100] Fig. 7 is a schematic diagram for explaining the operation of the ophthalmological information processing device 100 according to the embodiment. Fig. 7 shows a schematic relationship between the estimation model construction unit 210 and the estimation unit 220. Fig. 7 shows a relationship between the disease estimation unit 221 and the pathological condition estimation unit 222 and the disease estimation learning unit 211 and the pathological condition estimation learning unit 212. In Fig. 7, the same parts as those in Fig. 5 or Fig. 6 are denoted by the same reference numerals, and the description will be omitted as appropriate.

[0101] The disease estimation learning unit 211 generates a plurality of individual estimation models for estimating whether or not the subject eye is a glaucomatous eye for each of a plurality of images generated from the 3D OCT data. The disease estimation learning unit 211 generates the plurality of individual estimation models by performing supervised machine learning for each of the plurality of images. In some embodiments, the disease estimation learning unit 211 generates the plurality of individual estimation models by transfer learning.

[0102] When five types of images are generated from the three-dimensional OCT data as described above, the disease estimation learning section 211 generates an individual estimation model for each type of image.

[0103] For example, for a horizontal B-scan image IMG1 passing through the center (or its vicinity) of the optic disc, the disease estimation learning unit 211 performs supervised machine learning using B-scan images IMG1 of multiple test eyes other than the test eye that is the estimation target as training data TR1 and labels representing the judgment results of a doctor or the like as to whether or not each B-scan image has glaucoma as teacher data SD1 to generate an individual estimation model for the B-scan image IMG1. The function of the individual estimation model for the B-scan image IMG1 is realized by a first estimator 301 shown in FIG. 7.

[0104] For example, for a vertical B-scan image IMG2 passing through the center (or its vicinity) of the optic disc, the disease estimation learning unit 211 performs supervised machine learning using B-scan images IMG2 of multiple test eyes other than the test eye that is the estimation target as training data TR1 and labels representing the judgment results of whether or not each B-scan image has glaucoma by a doctor or the like as teacher data SD1 to generate an individual estimation model for the B-scan image IMG2. The function of the individual estimation model for the B-scan image IMG2 is realized by the second estimator 302 shown in FIG. 7.

[0105] For example, for a B-scan image IMG3 in the vertical direction passing through the fovea (or its vicinity), the disease estimation learning unit 211 performs supervised machine learning using B-scan images IMG3 of multiple test eyes other than the test eye that is the estimation target as training data TR1 and labels representing the judgment results of whether or not each B-scan image has glaucoma by a doctor or the like as teacher data SD1 to generate an individual estimation model for the B-scan image IMG3. The function of the individual estimation model for the B-scan image IMG3 is realized by a third estimator 303 shown in FIG. 7.

[0106] For example, for the projection image IMG4, the disease estimation learning unit 211 generates an individual estimation model for the projection image IMG4 by performing supervised machine learning using the projection images IMG4 of a plurality of test eyes other than the test eye to be estimated as training data TR1 and the labels representing the judgment results of whether or not each projection image has glaucoma by a doctor or the like as teacher data SD1. The function of the individual estimation model for the projection image IMG4 is realized by the fourth estimator 304 shown in FIG. 7.

[0107] For example, for the en-face image IMG5, the disease estimation learning unit 211 generates an individual estimation model for the en-face image IMG5 by performing supervised machine learning using the en-face images IMG5 of multiple test eyes other than the test eye to be estimated as training data TR1 and labels representing the judgment results of whether or not each en-face image has glaucoma by a doctor or the like as teacher data SD1. The function of the individual estimation model for the en-face image IMG5 is realized by the fifth estimator 305 shown in FIG. 7.

[0108] In addition, the disease estimation learning unit 211 generates a classification model for estimating whether or not the subject eye is a glaucomatous eye from a plurality of pieces of estimation information output from the generated plurality of individual estimation models. The disease estimation learning unit 211 generates the classification model by performing unsupervised machine learning or supervised machine learning.

[0109] For example, the disease estimation learning unit 211 generates a classification model by performing unsupervised machine learning or supervised machine learning using a group of images IMG1 to IMG5 of a plurality of test eyes other than the test eye to be estimated as training data TR1. When performing supervised machine learning, a label indicating whether the test eye for training is a glaucomatous eye or not is used as the training data SR1. The function of this classification model is realized by a classifier 306 shown in FIG. 7.

[0110] A disease estimation model is configured by a plurality of individual estimation models and a classification model for images IMG1 to IMG5. That is, the estimation unit 220 outputs estimation information for estimating whether or not the subject eye is a glaucomatous eye from a plurality of images of the subject eye (a plurality of images generated from OCT data of the subject eye) using the disease estimation model generated by the disease estimation learning unit 211. In some embodiments, the estimation information includes information indicating whether or not the subject eye is a glaucomatous eye. In some embodiments, the estimation information includes information (certainty information) indicating a certainty that the subject eye is a glaucomatous eye (e.g., a probability that it is estimated to have glaucoma).

[0111] Each of the above multiple individual estimation models may have a similar configuration.

[0112] 8 shows a block diagram of a configuration example of the first estimator 301 according to the embodiment. Each of the second estimator 302 to the fifth estimator 305 may have the same configuration as that in FIG.

[0113] The function of the first estimator 301 is realized by, for example, a convolutional neural network (CNN). That is, the computer operates in accordance with an instruction from an individual estimation model (trained model) stored in a memory to perform calculations based on trained weighting coefficients and response functions in the convolutional neural network for pixel values ​​of an image IMG1 input to a convolutional layer 311 of a feature extractor 310, which is an input layer, and to output a determination result from a classifier 320, which is an output layer, in accordance with an instruction from an individual estimation model (trained model) stored in a memory, in accordance with an instruction from an individual estimation model (trained model) stored in a memory, in accordance with an instruction from a convolutional neural network. The first estimator 301 having such a configuration can extract a local correlation pattern while gradually lowering the resolution of an image, and output a determination result based on the extracted correlation pattern.

[0114] The first estimator 301 includes a feature extractor 310 and a classifier 320. The feature extractor 310 extracts features of the determination image by repeating extraction of features and downsampling (filtering) for each predetermined image region of the input image IMG1. The classifier 320 outputs estimation information (e.g., confidence level) for estimating whether the subject eye is a glaucomatous eye or not, based on the features extracted by the feature extractor 310.

[0115] The feature extractor 310 includes a plurality of units, each of which includes a convolution layer and a pooling layer, connected in multiple stages. In each unit, the output of the convolution layer is connected to the input of the pooling layer. The pixel value of the corresponding pixel in the image IMG1 is input to the input of the first convolution layer. The input of the subsequent convolution layer is connected to the output of the previous pooling layer.

[0116] 8, feature extractor 310 includes two units connected in two stages. That is, in feature extractor 310, a unit including convolution layer 311 and pooling layer 312 is connected to a unit including convolution layer 313 and pooling layer 314 in the subsequent stage. The output of pooling layer 312 is connected to the input of convolution layer 313.

[0117] The classifier 320 includes fully connected layers 321 and 322, and the output of the fully connected layer 321 is connected to the input of the fully connected layer 322.

[0118] In the feature extractor 310 and the classifier 320, learned weighting coefficients are assigned between the neurons in the two connected layers. Each neuron performs a calculation using a response function on the calculation result that takes into account the weighting coefficients from one or more input neurons, and outputs the obtained calculation result to the neuron in the next stage.

[0119] The weighting coefficients are updated by performing known machine learning using two or more previously acquired images IMG1 of the subject's eye (horizontal B-scan images passing through the center (or near) the optic disc) as training data and labels attached to each image by a doctor or the like as teacher data. The existing weighting coefficients are updated by machine learning using two or more previously acquired images as training data. In some embodiments, the weighting coefficients are updated by transfer learning.

[0120] The first estimator 301 may have a known layer structure such as VGG16, VGG19, InceptionV3, ResNet18, ResNet50, etc. The classifier 320 may have a known configuration such as Random Forest or a Support Vector Machine (SVM).

[0121] The classifier 306 in the disease estimation unit 221 is generated by performing known machine learning using, for example, two or more images IMG1 to IMG5 of the subject's eye acquired in the past as training data and labels attached to each image by a doctor or the like as teacher data. The classifier 306 outputs information OUT indicating whether the subject's eye is a glaucomatous eye or not, or certainty information OUT indicating the certainty that the subject's eye is a glaucomatous eye, based on a plurality of pieces of estimated information from the first estimator 301 to the fifth estimator 305. In some embodiments, the classifier 306 outputs information OUT indicating whether the subject's eye is a glaucomatous eye or not, or certainty information OUT indicating the certainty that the subject's eye is a glaucomatous eye, based on a plurality of feature amounts extracted from each of the predetermined layers of the first estimator 301 to the fifth estimator 305. The classifier 306 may have a known configuration such as a Random Forest or a Support Vector Machine, similar to the classifier 320 shown in FIG. 8.

[0122] The pathology estimation learning unit 212 generates a pathology estimation model for estimating a classification result of glaucoma from a plurality of images generated by the image generating unit 201. The pathology estimation learning unit 212 generates the pathology estimation model by performing supervised machine learning. In some embodiments, when the subject eye is determined to be a glaucomatous eye from the estimation information output from the classification model, the pathology estimation learning unit 212 generates classification information for estimating a classification result of the pathology of glaucoma.

[0123] For example, five types of images IMG1 to IMG5 generated by the image generation unit 201 are used as training data TR2, and labels representing the judgment results of a doctor or other person as to whether or not the patient has glaucoma are used as teacher data SD2 to perform supervised machine learning to generate a pathology classification estimation model 401.

[0124] At this time, the disease estimation learning unit 212 generates a pathology classification estimation model 401 by machine learning using feature amounts (weighting coefficients, etc.) of the individual estimation models for images IMG1 to IMG5 obtained by the disease estimation learning unit 211. In some embodiments, the pathology estimation learning unit 212 generates a pathology classification estimation model 401 by machine learning using feature amounts (weighting coefficients, etc.) of the individual estimation models and classification models for images IMG1 to IMG5.

[0125] The pathology estimation unit 222 uses the pathology estimation learning model generated by the pathology estimation learning unit 212 to output classification information OUTa to OUTd for classifying the pathology of glaucoma from five types of images IMG1 to IMG5 of the subject's eye. In some embodiments, the classification information includes information representing any one of a plurality of types of pathology of glaucoma into which the pathology is classified. In some embodiments, the classification information includes information representing the certainty of each of a plurality of types of pathology of glaucoma (e.g., the probability of being estimated as each type). For example, the classification information OUTa is certainty information representing the certainty that the glaucoma developed in the subject's eye is focal ischemic type (FI). The classification information OUTb is certainty information representing the certainty that the glaucoma developed in the subject's eye is myopic type (MY). The classification information OUTc is certainty information representing the certainty that the glaucoma developed in the subject's eye is age-related sclerosis type (SS). The classification information OUTd is certainty information that indicates the certainty that the glaucoma developed in the subject's eye is generalized expanding (GE) type.

[0126] In this embodiment, the pathological condition inferring unit 222 outputs classification information according to the Nicollella classification.

[0127] The Nicolela classification defines four types of optic disc glaucoma (focal ischemic, myopic, age-related sclerotic, and global enlargement) according to the shape of the optic disc. Focal ischemic (FI) is a type in which a notch is present in part of the rim and a localized defect in the optic nerve fiber layer is seen. Focal ischemic type is more common in women and is often accompanied by migraines and seizures. Myopic (MY) is a type in which the optic disc is inclined and crescent-shaped peripapillary chorioretinal atrophy (PPA) with a depression on the temporal side is seen. Myopic type is more common in young people and is often accompanied by myopia. Senile sclerotic (SS) is a type in which the optic disc is circular, the depression is shallow, and a halo is seen around the optic disc. Age-related sclerotic type is more common in elderly people and is often accompanied by cardiovascular disorders. Generalized enlargement (GE) is a type that presents a large, deep circular depression. Generalized enlargement is common in subjects with high intraocular pressure.

[0128] When the subject's eye is a glaucomatous eye, the pathology estimation unit 222 can output classification information indicating a type classified as the pathology of glaucoma that has developed in the subject's eye among the multiple types of the Nicollera classification. The pathology estimation unit 222 can also output classification information indicating the degree of certainty of each of the multiple types of the Nicollera classification.

[0129] The ophthalmic device 10 or the communication unit 140 that performs a receiving process of OCT data from the ophthalmic device 10 is an example of an "acquisition unit" according to the embodiment. As the five types of images, a horizontal B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the fovea (or its vicinity), a projection image, and an en-face image are examples of "multiple images" according to the embodiment. An individual estimation model for the B-scan image IMG1, an individual estimation model for the B-scan image IMG2, an individual estimation model for the B-scan image IMG3, an individual estimation model for the projection image IMG4, and an individual estimation model for the en-face image IMG5 are examples of "multiple trained models" according to the embodiment. The B-scan image is an example of a "tomographic image" according to the embodiment. The projection image or the en-face image is an example of a "front image" according to the embodiment. The disease estimation learning unit 211 is an example of a "second learning unit" according to the embodiment. The pathology estimation learning unit 212 is an example of a "first learning unit" according to an embodiment. The horizontal direction is an example of a "first cross-sectional direction" according to an embodiment. The vertical direction is an example of a "second cross-sectional direction" according to an embodiment.

[0130] [Example] An operation example of the ophthalmological information-processing apparatus 100 according to some embodiments will be described.

[0131] 9 and 10 show flow diagrams of an example of the operation of the ophthalmological information processing device 100 according to the embodiment. FIG. 9 shows a flow diagram of an example of a generation process of a trained model for performing disease estimation and pathological state estimation. FIG. 10 shows a flow diagram of an example of a disease estimation process and pathological state estimation process using a trained model generated by the generation process shown in FIG. 9. The storage unit 112 stores a computer program for realizing the processes shown in FIG. 9 and 10. The control unit 110 (main control unit 111) can execute the processes shown in FIG. 9 and 10 by operating according to this computer program.

[0132] In FIG. 9, it is assumed that OCT has been performed in advance on the funduses of a plurality of examinee's eyes in the ophthalmologic apparatus 10, and a plurality of pieces of three-dimensional OCT data have been acquired.

[0133] (S1: Creating a training data set from 3D OCT data) First, the main controller 111 controls the communication unit 140 to acquire a plurality of pieces of three-dimensional OCT data for a plurality of examinee's eyes acquired by the ophthalmologic apparatus 10.

[0134] Next, the main controller 111 controls the image generator 201 to generate the above-mentioned five types of images as a training data group from each of the multiple 3D OCT data. That is, the image generator 201 generates a horizontal B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the fovea (or its vicinity), a projection image, and an en-face image from the 3D OCT data. A doctor or the like interprets the generated images and attaches a label indicating whether the subject eye is a glaucomatous eye or not. The main controller 111 associates the interpreted images with the labels and stores them in the storage unit 112.

[0135] (S2: Generate a disease prediction model) Next, the main controller 111 controls the disease estimation learning unit 211 to generate an individual estimation model (disease estimation model) for one of the above five types of images. The disease estimation learning unit 211 generates the individual estimation model by performing supervised machine learning as described above.

[0136] (S3:Next?) Next, the main control unit 111 determines whether or not to perform the generation process of an individual estimation model for the next image. For example, the main control unit 111 determines whether or not to perform the generation process of an individual estimation model for the next image by counting the number of predetermined image types or the number of image types generated in step S1.

[0137] When it is determined that the generation process of an individual estimation model is to be performed for the next image (step S3: Y), the operation of the ophthalmologic information-processing device 100 proceeds to step S2. When it is not determined that the generation process of an individual estimation model is to be performed for the next image (step S3: N), the operation of the ophthalmologic information-processing device 100 proceeds to step S4.

[0138] (S4: Generate a pathology estimation model) In step S3, when it is not determined that the generation process of the individual estimation model is to be performed for the next image (step S3: N), the main controller 111 controls the pathology estimation learning unit 212 to generate a pathology classification estimation model 401 for estimating the classification result of the pathology of glaucoma based on the above five types of images. The pathology estimation learning unit 212 generates the pathology classification estimation model 401 as described above.

[0139] This is the end of the operation of the ophthalmologic information-processing device 100 (END).

[0140] As shown in Figure 10, the ophthalmological information processing device 100 uses a trained model generated according to the flow shown in Figure 9 to generate estimation information for estimating whether the test eye is a glaucomatous eye or not, and classification information for estimating the classification result of the glaucoma pathology.

[0141] In FIG. 10, it is assumed that OCT has been performed in advance on the fundus of the subject's eye, which is the estimation target, in the ophthalmologic apparatus 10, and three-dimensional OCT data has been acquired.

[0142] (S11: Creating images from 3D OCT data) First, the main controller 111 controls the communication unit 140 to acquire three-dimensional OCT data of the subject's eye, which is the estimation target, acquired by the ophthalmologic apparatus 10.

[0143] Next, the main controller 111 controls the image generator 201 to generate the above five types of images as an image group from the three-dimensional OCT data. That is, the image generator 201 generates, from the three-dimensional OCT data, a horizontal B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the fovea (or its vicinity), a projection image, and an en-face image.

[0144] (S12: Disease prediction, pathological condition prediction) Next, the main control unit 111 controls the disease estimation unit 221 to output estimation information for estimating whether or not the subject eye is a glaucomatous eye, using a disease estimation model (individual estimation model for each image) generated by repeating steps S2 to S3 of Figure 9 for the five types of image groups generated in step S11.

[0145] In addition, the main control unit 111 controls the pathology estimation unit 222 to output classification information for estimating the classification result of the pathology of glaucoma for the five types of image groups generated in step S11, using the pathology estimation model generated in step S4 of Figure 9.

[0146] In some embodiments, the main controller 111 operates the disease inferring section 221 and the pathological condition inferring section 222 in parallel to output the inferred information and the classified information simultaneously.

[0147] This is the end of the operation of the ophthalmologic information-processing device 100 (END).

[0148] Here, in order to evaluate the accuracy of glaucoma diagnosis by the ophthalmologic information-processing device 100 according to the embodiment, a comparison with a comparative example of the embodiment will be made. The ophthalmologic information-processing device according to the comparative example of the embodiment includes a 5-input convolutional neural network that uses the same 5 types of images as the embodiment as input data, and is configured to determine whether or not the subject eye has glaucoma based on the 5 types of images of the fundus of the subject eye by performing known supervised machine learning.

[0149] FIG. 11 shows an example of the area under the receiver operating characteristic curve (AUROC) obtained by an ophthalmologic information processing device according to a comparative example of the embodiment.

[0150] In Figure 11, the results of evaluation using 143 cases of OCT data from normal eyes and 672 cases of OCT data from glaucoma eyes with 5-fold cross-validation are shown as AUROC. In Figure 11, the AUC (Area Under Curve) is 0.950 ± 0.023.

[0151] FIG. 12 shows an example of the AUROC obtained by the ophthalmological information-processing device 100 according to the embodiment.

[0152] In Fig. 12, the results of evaluation using 5-fold cross-validation using the same OCT data as in the comparative example (143 normal eyes, 672 glaucoma eyes) are shown as AUROC. In Fig. 12, the AUC is 0.983 ± 0.009.

[0153] As described above, the configuration according to the embodiment makes it possible to improve the accuracy of determination compared to a case where a convolutional neural network is simply used to determine whether or not a patient has glaucoma.

[0154] <Modification> The configuration according to the embodiment is not limited to the above configuration.

[0155] An ophthalmological apparatus according to some embodiments has at least one of the functions of an ophthalmological information-processing device 100, the functions of an operation device 180, and the functions of a display device 190, in addition to the functions of the ophthalmological apparatus 10.

[0156] Hereinafter, ophthalmic apparatuses according to some modified embodiments will be described, focusing on the differences from the ophthalmic apparatus according to the above embodiment.

[0157] A block diagram of a configuration example of an ophthalmologic apparatus 10a according to a modified example of the embodiment is shown in Fig. 13. In Fig. 13, the same parts as those in Fig. 2 are given the same reference numerals, and the description thereof will be omitted as appropriate.

[0158] The configuration of the ophthalmic apparatus 10a according to this modification is different from the configuration of the ophthalmic apparatus 10 according to the above embodiment in that the ophthalmic apparatus 10a has the functions of an ophthalmic information processing device 100, an operation device 180, and a display device 190. The ophthalmic apparatus 10a includes a control unit 11a, an OCT unit 12, an OCT data processing unit 13, an ophthalmic information processing unit 15a, an operation unit 16a, and a display unit 17a.

[0159] The control unit 11a controls each unit of the ophthalmologic apparatus 10a, in particular, the control unit 11a controls the OCT unit 12, the OCT data processing unit 13, the ophthalmologic information processing unit 15a, the operation unit 16a, and the display unit 17a.

[0160] The ophthalmological information processing unit 15a has a configuration similar to that of the ophthalmological information processing device 100, and has the same functions as the ophthalmological information processing device 100. The operation unit 16a has a configuration similar to that of the operation device 180, and has the same functions as the operation device 180. The display unit 17a has a configuration similar to that of the display device 190, and has the same functions as the display device 190.

[0161] According to this modification, it is possible to obtain, with a compact configuration, estimation information for estimating whether or not the subject eye is a glaucoma eye (diseased eye) and classification information for estimating the classification result of the pathology of glaucoma with high accuracy by machine learning using a smaller amount of training data. As a result, it is possible to provide a new technology for providing appropriate treatment for the disease of the subject eye at an early stage.

[0162] <Effects> Hereinafter, an ophthalmological information processing apparatus, an ophthalmological apparatus, an ophthalmological information processing method, and a program according to some embodiments will be described.

[0163] An ophthalmological information processing device (ophthalmological information processing device 100, ophthalmological information processing unit 15a) according to some embodiments includes an acquisition unit (ophthalmological device 10, or a communication unit 140 that performs reception processing of OCT data from the ophthalmological device 10) and a disease estimation unit (221). The acquisition unit acquires multiple images of the subject eye having different cross-sectional directions. The disease estimation unit uses multiple trained models obtained by machine learning for each type of the multiple images to output estimation information for estimating whether the subject eye is a diseased eye from the multiple images.

[0164] According to this configuration, since estimation information for estimating whether the subject's eye is diseased or not is obtained for each of a plurality of images in different cross-sectional directions using a trained model obtained by machine learning, diseases can be detected early with high accuracy, and appropriate treatment for the disease of the subject's eye can be provided early.

[0165] In some embodiments, the disease estimation unit includes a plurality of estimators (first estimator 301 to fifth estimator 305) that use a plurality of trained models for each of a plurality of image types to output features or confidence information representing the confidence that the test eye is a diseased eye, and a classifier (306) that uses a classification model obtained by machine learning to output estimation information from the plurality of features or the plurality of confidence information output from the plurality of estimators.

[0166] According to this configuration, a trained model obtained by machine learning is used to output feature values ​​or confidence information for each type of multiple images, and a classification model obtained by machine learning is used to output estimated information from the feature values ​​or confidence information, so that diseases can be detected early with high accuracy, and appropriate treatment for the disease of the subject's eye can be performed early.

[0167] In some embodiments, the plurality of images includes a tomographic or en face image of a fundus of the test eye, and the disease is an ocular fundus disease.

[0168] According to this configuration, fundus diseases can be detected with high accuracy and at an early stage, and appropriate treatment for the fundus disease of the subject's eye can be provided at an early stage.

[0169] In some embodiments, the plurality of images includes B-scan images in a cross-sectional orientation across a line connecting the optic disc and the fovea.

[0170] According to this configuration, estimated information is obtained using multiple images including a B-scan image in a cross-sectional direction intersecting with a line connecting the optic disc and the fovea, so that diseases affecting the morphology of the cross section intersecting with the line connecting the optic disc and the fovea can be detected early and with high accuracy.

[0171] In some embodiments, the multiple images include a B-scan image in a first cross-sectional direction (horizontal direction) passing through the optic disc, a B-scan image in a second cross-sectional direction (vertical direction) passing through the optic disc and intersecting the first cross-sectional direction, and a B-scan image passing through the fovea.

[0172] With this configuration, diseases that affect the cross-sectional morphology of the optic disc or fovea can be detected early with high accuracy.

[0173] In some embodiments, the front image includes at least one of an en-face image projecting a layer region corresponding to the internal limiting membrane to a layer region of a deeper layer, and a projection image.

[0174] According to this configuration, diseases that affect the morphology of the front surface of the examinee's eye can be detected with high accuracy and at an early stage.

[0175] In some embodiments, the ocular fundus disease comprises at least one of glaucoma, age-related macular degeneration, and diabetic reticuloretinopathy.

[0176] According to this configuration, glaucoma and other diseases can be detected with high accuracy and at an early stage, and appropriate treatment for the disease in the examined eye can be provided at an early stage.

[0177] Some embodiments include a pathology estimation unit (222) that uses a trained model for classification (pathology classification estimation model 401) obtained by supervised machine learning using features obtained from multiple trained models to output classification information for estimating a classification result of the pathology of a disease from multiple images.

[0178] According to this configuration, it is possible to estimate the classification result of the pathology of the disease with high accuracy. As a result, it is possible to provide appropriate treatment for the disease of the subject eye at an early stage. In addition, it is possible to obtain highly accurate classification information by machine learning using less training data compared to the case where a trained model obtained by performing machine learning from natural images, etc. is used.

[0179] In some embodiments, the disease includes glaucoma and the classification information is information for estimating a classification result of a plurality of types of optic disc shapes.

[0180] According to this configuration, it is possible to estimate the classification result of the pathology of glaucoma with high accuracy, and it is therefore possible to provide an appropriate treatment for glaucoma in the examinee's eye at an early stage.

[0181] Some embodiments include a first learning unit (pathology estimation learning unit 212) that generates a trained model for classification by supervised machine learning using features obtained from multiple trained models.

[0182] With this configuration, it is possible to provide an ophthalmologic information processing device that can generate a trained classification model for estimating with high accuracy the classification result of the pathological condition of a disease developed in the test eye, through machine learning using less training data compared to using a trained model obtained by performing machine learning on natural images, etc.

[0183] Some embodiments include a second learning unit (disease estimation learning unit 211) that generates multiple trained models by supervised machine learning for each of multiple image types.

[0184] With this configuration, it is possible to provide an ophthalmologic information processing device capable of generating a trained model for detecting diseases in the test eye with high accuracy and at an early stage.

[0185] Some embodiments include an image generating unit (201) that generates at least one of a plurality of images based on three-dimensional OCT data of the subject's eye.

[0186] According to this configuration, it is possible to provide an ophthalmologic information processing device that can obtain three-dimensional OCT data of the subject's eye, thereby enabling early detection of diseases in the subject's eye with high accuracy.

[0187] An ophthalmic apparatus (10, 10a) according to some embodiments includes an OCT unit (12) that performs optical coherence tomography on a test eye, an image generating unit (201) that generates at least one of a plurality of images based on three-dimensional data acquired by the OCT unit, and an ophthalmic information processing device (100, ophthalmic information processing unit 15a) described in any one of the above.

[0188] According to this configuration, it is possible to provide an ophthalmologic apparatus that performs optical coherence tomography on a subject's eye and detects diseases of the subject's eye at an early stage with high accuracy from the obtained OCT data.

[0189] An ophthalmic apparatus (10, 10a) according to some embodiments includes an OCT unit (12) that performs optical coherence tomography on a subject's eye, and the above-described ophthalmic information processing device (100, ophthalmic information processing unit 15a).

[0190] According to this configuration, it is possible to provide an ophthalmologic apparatus that performs optical coherence tomography on a subject's eye and detects diseases of the subject's eye at an early stage with high accuracy from the obtained OCT data.

[0191] An ophthalmologic information processing method according to some embodiments includes an acquisition step of acquiring multiple images of a test eye having different cross-sectional directions, and a disease estimation step of outputting estimation information for estimating whether or not the test eye is a diseased eye from the multiple images using multiple trained models obtained by machine learning for each type of the multiple images.

[0192] According to this method, since estimation information for estimating whether the subject's eye is diseased or not is obtained for each of a plurality of images in different cross-sectional directions using a trained model obtained by machine learning, diseases can be detected early with high accuracy, and appropriate treatment for the disease of the subject's eye can be provided early.

[0193] In some embodiments, the method includes multiple estimation steps of outputting features or confidence information representing confidence that the test eye is a diseased eye using multiple trained models for each of multiple image types, and a classification step of outputting estimation information from the multiple features or the multiple confidence information output in the multiple estimation steps using a classification model obtained by machine learning.

[0194] According to this method, a trained model obtained by machine learning is used to output feature values ​​or confidence information for each type of multiple images, and a classification model obtained by machine learning is used to output estimated information from the feature values ​​or confidence information, so that diseases can be detected early with high accuracy, and appropriate treatment for the disease of the subject's eye can be performed early.

[0195] In some embodiments, the plurality of images includes a tomographic or en face image of a fundus of the test eye, and the disease is an ocular fundus disease.

[0196] According to such a method, fundus diseases can be detected with high accuracy and at an early stage, and appropriate treatment for the fundus disease of the examined eye can be provided at an early stage.

[0197] In some embodiments, the plurality of images includes B-scan images in a cross-sectional orientation across a line connecting the optic disc and the fovea.

[0198] According to this method, estimated information is obtained using multiple images including a B-scan image in a cross-sectional direction intersecting the line connecting the optic disc and the fovea, so that diseases affecting the morphology of the cross section intersecting the line connecting the optic disc and the fovea can be detected early and with high accuracy.

[0199] In some embodiments, the multiple images include a B-scan image in a first cross-sectional direction passing through the optic disc, a B-scan image in a second cross-sectional direction passing through the optic disc and intersecting the first cross-sectional direction, and a B-scan image passing through the fovea.

[0200] According to such a method, diseases that affect the cross-sectional morphology of the optic disc or fovea can be detected with high accuracy and at an early stage.

[0201] In some embodiments, the front image includes at least one of an en-face image projecting a layer region corresponding to the internal limiting membrane to a layer region of a deeper layer, and a projection image.

[0202] According to this method, diseases that affect the morphology of the front surface of the examinee's eye can be detected with high accuracy and at an early stage.

[0203] In some embodiments, the ocular fundus disease comprises at least one of glaucoma, age-related macular degeneration, and diabetic reticuloretinopathy.

[0204] According to such a method, glaucoma and other diseases can be detected with high accuracy and at an early stage, which allows appropriate treatment for the disease in the examined eye to be performed at an early stage.

[0205] Some embodiments include a pathology estimation step of outputting classification information for estimating a classification result of the pathology of a disease from multiple images, using a trained classification model (pathology classification estimation model 401) obtained by supervised machine learning using features obtained from multiple trained models.

[0206] According to this method, it is possible to estimate the classification result of the pathology of a disease with high accuracy. As a result, it is possible to provide appropriate treatment for the disease of the subject's eye at an early stage. In addition, it is possible to obtain highly accurate classification information by machine learning using less training data than when using a trained model obtained by performing machine learning from natural images, etc.

[0207] In some embodiments, the disease includes glaucoma and the classification information is information for estimating a classification result of a plurality of types of optic disc shapes.

[0208] According to this method, it is possible to estimate the classification result of the pathology of glaucoma with high accuracy, and it is therefore possible to provide an appropriate treatment for glaucoma in an early stage in the examined eye.

[0209] Some embodiments include a first learning step of generating a trained model for classification using supervised machine learning using features obtained from multiple trained models.

[0210] According to this method, it is possible to provide an ophthalmologic information processing method that can generate a trained classification model for estimating with high accuracy the classification result of the pathology of a disease developed in the test eye, through machine learning using less training data, compared to using a trained model obtained by performing machine learning on natural images, etc.

[0211] Some embodiments include a second learning step of generating a plurality of trained models using supervised machine learning for each of a plurality of image types.

[0212] According to this method, it is possible to provide an ophthalmologic information processing method capable of generating a trained model for detecting diseases in the test eye with high accuracy and at an early stage.

[0213] Some embodiments include an image generating step of generating at least one of a plurality of images based on three-dimensional OCT data of the test eye.

[0214] According to this method, it is possible to provide an ophthalmologic information processing method that can detect diseases in the subject's eye with high accuracy and at an early stage by acquiring three-dimensional OCT data of the subject's eye.

[0215] A program according to some embodiments causes a computer to execute each step of any of the ophthalmologic information processing methods described above.

[0216] According to such a program, for each of a plurality of images in different cross-sectional directions, a trained model obtained by machine learning is used to obtain inferred information for inferring whether or not the subject's eye is diseased, so that diseases can be detected early with high accuracy, and appropriate treatment for the disease of the subject's eye can be provided early.

[0217] A program for implementing the ophthalmologic information processing method according to some embodiments can be stored in any non-transitory computer-readable recording medium. The recording medium may be an electronic medium using magnetism, light, magneto-optical, semiconductor, etc. Typically, the recording medium is a magnetic tape, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, a solid-state drive, etc.

[0218] It is also possible to transmit and receive computer programs via a network such as the Internet or a LAN.

[0219] The above-described embodiments are merely examples for carrying out the present invention. Anyone who wishes to carry out the present invention may make any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention. [Explanation of symbols]

[0220] 1 Ophthalmology System 10, 10a ophthalmological equipment 11, 11a, 110 Control unit 12 OCT section 12A Interferometric Optical System 12B Scanning Optical System 14, 140 Communications Department 15a Ophthalmology Information Processing Department 16a Control section 17a Display section 100 Ophthalmology information processing device 111 Main control unit 112 Storage section 130 Data Processing Unit 180 Operating device 190 Display device 200 Analysis Department 201 Image Generation Unit 210 Estimation Model Construction Department 211 Disease Prediction Learning Department 212 Pathological condition estimation learning unit 220 Estimation section 221 Disease Prediction Department 222 Pathology Estimation Department

Claims

1. an acquisition unit that acquires a plurality of images of a subject's eye having cross-sectional directions different from each other; a plurality of estimators that output, for each type of image, a feature amount or certainty information representing a certainty that the test eye is an eye suffering from age-related macular degeneration, using each of a plurality of trained models obtained by machine learning for each type of image; a classifier that uses a classification model obtained by machine learning to output inference information for inferring whether the subject's eye is an eye suffering from age-related macular degeneration from the plurality of feature amounts or the plurality of pieces of confidence information output from the plurality of estimators; and An ophthalmological information processing device comprising:

2. and a pathology estimation unit that uses a trained model for classification obtained by supervised machine learning using feature amounts obtained from the plurality of trained models to output classification information for estimating a classification result based on the morphology of the retinal pigment epithelium layer from the plurality of images. The ophthalmological information processing apparatus according to claim 1 .

3. an acquisition unit that acquires a plurality of images of a subject's eye having cross-sectional directions different from each other; a plurality of estimators that output, for each type of image, a feature amount or certainty information representing a certainty that the test eye is an eye with diabetic retinopathy, using each of a plurality of trained models obtained by machine learning for each type of image; a classifier that uses a classification model obtained by machine learning to output inference information for inferring whether the subject's eye is an eye with diabetic retinopathy from the plurality of feature amounts or the plurality of pieces of confidence information output from the plurality of estimators; and An ophthalmological information processing device comprising:

4. and a pathology estimation unit that uses a trained model for classification obtained by supervised machine learning using feature amounts obtained from the plurality of trained models to output classification information for estimating a classification result based on at least one of the form or distribution of microaneurysm, hemorrhage, vitiligo, neovascularization, and hemorrhage region from the plurality of images.

4. An ophthalmologic information processing apparatus according to claim 3.

5. A first learning unit that generates the trained model for classification by supervised machine learning using features obtained from the plurality of trained models.

5. An ophthalmologic information processing apparatus according to claim 2 or 4.

6. The plurality of images include a tomographic image or a front image of a fundus of the subject's eye, 6. The ophthalmological information processing apparatus according to claim 1,

7. a second learning unit that generates the plurality of trained models by supervised machine learning for each of the plurality of types of images.

7. The ophthalmologic information processing apparatus according to claim 1, wherein the ophthalmologic information processing apparatus is a computer.

8. and an image generating unit that generates at least one of the plurality of images based on three-dimensional OCT data of the subject's eye. The ophthalmological information processing apparatus according to any one of claims 1 to 7.

9. an OCT unit that performs optical coherence tomography on the subject's eye; an image generating unit that generates at least one of the plurality of images based on the three-dimensional data acquired by the OCT unit; An ophthalmological information processing device according to any one of claims 1 to 8, 13. An ophthalmic device comprising:

10. an OCT unit that performs optical coherence tomography on the subject's eye; The ophthalmological information processing apparatus according to claim 8 ; 13. An ophthalmic device comprising:

11. An acquisition step in which a processor acquires a plurality of images of the subject's eye having cross-sectional directions different from each other; a plurality of estimation steps in which the processor outputs, for each type of image, a feature amount or certainty information representing a certainty that the test eye is an eye with age-related macular degeneration, using each of a plurality of trained models obtained by machine learning for each type of image; a classification step in which the processor uses a classification model obtained by machine learning to output inference information for inferring whether or not the subject eye is an eye suffering from age-related macular degeneration from the plurality of feature amounts or the plurality of pieces of confidence information output in the plurality of estimation steps; A method for operating an ophthalmic information processing device, comprising:

12. and a pathology estimation step of outputting classification information for estimating a classification result based on a morphology of the retinal pigment epithelium layer from the plurality of images, using a trained model for classification obtained by supervised machine learning using feature amounts obtained by the plurality of trained models. A method for operating an ophthalmic information device according to claim 11.

13. An acquisition step in which a processor acquires a plurality of images of the subject's eye having cross-sectional directions different from each other; A plurality of estimation steps in which the processor outputs, for each type of image, a feature amount or certainty information representing a certainty that the test eye is an eye with diabetic retinopathy, using each of a plurality of trained models obtained by machine learning for each type of image; a classification step in which the processor uses a classification model obtained by machine learning to output inference information for inferring whether or not the subject eye is an eye with diabetic retinopathy from the plurality of feature amounts or the plurality of pieces of confidence information output in the plurality of estimation steps; and A method for operating an ophthalmic information processing device, comprising:

14. The method includes a pathology estimation step of outputting classification information for estimating a classification result based on at least one of a shape or distribution of microaneurysm, bleeding, vitiligo, neovascularization, and bleeding region from the plurality of images, using a trained model for classification obtained by supervised machine learning using feature amounts obtained by the plurality of trained models, by the processor.

14. A method for operating an ophthalmologic information processing apparatus according to claim 13.

15. The processor includes a first learning step of generating the trained model for classification by supervised machine learning using features obtained by the plurality of trained models. A method for operating an ophthalmologic information processing apparatus according to claim 12 or 14.

16. A second learning step in which the processor generates the plurality of trained models by supervised machine learning for each of the plurality of image types. A method for operating an ophthalmologic information processing apparatus according to claim 15.

17. The plurality of images include a tomographic image or a front image of a fundus of the subject's eye, A method for operating an ophthalmologic information processing apparatus according to any one of claims 11 to 16.

18. The processor includes an image generating step of generating at least one of the plurality of images based on three-dimensional OCT data of the subject's eye. A method for operating an ophthalmologic information processing apparatus according to any one of claims 11 to 17.

19. A program for causing a computer to execute each step of the method for operating an ophthalmologic information processing apparatus according to any one of claims 11 to 18.

Citation Information

Patent Citations

  • Image processing device, image processing system, and image processing program

    JP2018121885A

  • Multivariate and multi-resolution retinal image anomaly detection system

    JP2020032190A

  • Electronic parts heat resistance performance test system with loader

    KR102114678B1

  • Diagnosis assistance system and control method thereof

    US20190221313A1

  • Ophthalmic image processing device and ophthalmic image processing program

    WO2019207800A1