Ophthalmic information processing device, ophthalmic device, operation method of ophthalmic information processing device, and program

Abstract

To provide a new technique for accurately detecting an optic disk hemorrhagic area with high reproductivity.SOLUTION: An ophthalmologic information processing device comprises: a determiner and a detector. The determiner uses an optic disk hemorrhagic determination model which is obtained by machine learning with first teacher data being front images of a plurality of eyegrounds each attached with a label indicating presence / absence of optic disk hemorrhagic, to determine presence / absence of optic disk hemorrhagic with respect to a front image of the eyeground of an eye to be examined. The detector uses an optic disk hemorrhagic area detection model which is obtained by machine learning with second teacher data being a plurality of pairs of image groups, each pair of which is formed of the front image of the eyeground and an optic disk hemorrhagic area image indicating an optic disk hemorrhagic area drawn in the front image, to detect an optic disk hemorrhagic area drawn in the front image determined to include optic disk hemorrhagic by the determiner.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an ophthalmic information processing device, an ophthalmic device, and an ophthalmic information processing Operation of the device The present invention relates to a method and a program. [Background technology]

[0002] Glaucoma is one of the leading causes of blindness. The optic nerve damage and visual field disorder caused by glaucoma are basically progressive and irreversible, and progress gradually without the patient's awareness. Therefore, early detection and early treatment of glaucoma to prevent or suppress the progression of the disorder are very important.

[0003] One of the risk factors for the progression of glaucoma is disk hemorrhage. For example, doctors determine the presence or absence of disk hemorrhage by observing fundus images taken with a fundus camera. Such doctors are required to use their sufficient experience and knowledge to determine the presence or absence of disk hemorrhage with high accuracy.

[0004] For example, Patent Document 1 discloses a method for extracting fundus blood vessels, hemorrhagic areas, and optic disc regions by utilizing differences in pixel values ​​in a color fundus image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2008-73280 A Summary of the Invention [Problem to be solved by the invention]

[0006] The overall color of the fundus image and the pixel values ​​of the parts and blood vessels vary depending on the eye to be examined. Therefore, even if only the difference in pixel values ​​is used as in the conventional method, it is considered that it is often difficult to extract the bleeding part with high accuracy. Furthermore, it is considered that it becomes even more difficult to specify the position of the bleeding part with high accuracy, such as whether the bleeding part is located within the optic disc region. If the bleeding part cannot be detected with high accuracy, the doctor cannot judge the presence or absence of the optic disc bleeding with high accuracy and high reproducibility.

[0007] The present invention has been made in view of the above circumstances, and one of its objectives is to provide a new technique for detecting a nipple bleeding region with high reproducibility and high accuracy. [Means for solving the problem]

[0008] A first aspect of the embodiment is an ophthalmologic information processing device that includes a determiner that determines the presence or absence of nipple hemorrhage in a frontal image of the fundus of a test eye using a nipple hemorrhage determination model obtained by machine learning with first training data being multiple frontal images of the fundus labeled with a label indicating the presence or absence of nipple hemorrhage, and a detector that detects a nipple hemorrhage area depicted in the front image that has been determined to have nipple hemorrhage by the determiner using a nipple hemorrhage area detection model obtained by machine learning with second training data being a group of multiple pairs of images, each pair being a frontal image of the fundus and a nipple hemorrhage area image representing the nipple hemorrhage area depicted in the frontal image.

[0009] In a second aspect of the embodiment, in the first aspect, the second teacher data includes a front image included in the first teacher data.

[0010] A third aspect of the embodiment is the first or second aspect, further including a first learning unit that generates the nipple bleeding determination model by performing supervised machine learning using the first training data.

[0011] A fourth aspect of the embodiment is any one of the first to third aspects, further including a second learning unit configured to generate the nipple bleeding region detection model by performing supervised machine learning using the second training data.

[0012] In a fifth aspect of the embodiment, in any one of the first to fourth aspects, the front image of the fundus is a color front image.

[0013] A sixth aspect of the embodiment, in any of the first to fifth aspects, includes an analysis unit that analyzes a front image of the fundus of the test eye to generate at least one of position information representing the position of the papilla hemorrhage area in the front image of the fundus of the test eye, shape information representing the shape of the papilla hemorrhage area, and occurrence information representing the occurrence status of the papilla hemorrhage area.

[0014] In a seventh aspect of the embodiment, in the sixth aspect, the position information includes information indicating a direction of the representative position of the nipple bleeding region relative to a reference position.

[0015] In an eighth aspect of the embodiment, in the sixth or seventh aspect, the position information includes information indicating whether the representative position of the disc hemorrhage region is inside or outside the optic disc region.

[0016] In a ninth aspect of the embodiment, in the eighth aspect, the position information includes information indicating whether the representative position is within the optic disc peripheral region or outside the optic disc peripheral region.

[0017] In a tenth aspect of the embodiment, in any one of the sixth to ninth aspects, the shape information includes at least one of an ellipticity of the nipple bleeding region and an area of ​​the nipple bleeding region.

[0018] In an eleventh aspect of the embodiment, in any one of the sixth to tenth aspects, the occurrence information includes at least one of an occurrence frequency of the nipple bleeding region and an occurrence interval of the nipple bleeding region.

[0019] A twelfth aspect of the embodiment, in any of the sixth to eleventh aspects, includes an alignment unit that aligns the OCT data of the test eye with a frontal image of the test eye in which the disc hemorrhage area detected by the detector is depicted, and the analysis unit generates at least one of the position information, the shape information, and the generation information using the OCT data that has been aligned with the frontal image by the alignment unit.

[0020] A thirteenth aspect of the embodiment is an ophthalmologic apparatus including an imaging unit that images a fundus of the subject's eye, and the ophthalmologic information-processing device according to any one of the first to twelfth aspects.

[0021] A fourteenth aspect of the embodiment is an ophthalmic device including an imaging unit that images the fundus of the test eye, an OCT unit that acquires the OCT data by performing optical coherence tomography on the test eye, and the ophthalmic information processing device described in the twelfth aspect.

[0022] A fifteenth aspect of the embodiment is an ophthalmologic information processing method including a determination step of determining the presence or absence of nipple hemorrhage in a frontal image of the fundus of a test eye using a nipple hemorrhage determination model obtained by machine learning using multiple frontal images of the fundus labeled with a label indicating the presence or absence of nipple hemorrhage as first teacher data, and a detection step of detecting a nipple hemorrhage area depicted in the frontal image determined to have nipple hemorrhage in the determination step using a nipple hemorrhage area detection model obtained by machine learning using multiple pairs of images, each pair being a frontal image of the fundus and a nipple hemorrhage area image representing the nipple hemorrhage area depicted in the frontal image, as second teacher data.

[0023] In a sixteenth aspect of the present invention, in the fifteenth aspect, the second teacher data includes a front image included in the first teacher data.

[0024] A seventeenth aspect of the embodiment, in the fifteenth or sixteenth aspect, includes a first learning step of generating the nipple bleeding determination model by performing supervised machine learning using the first training data.

[0025] An 18th aspect of the embodiment, in any one of the 15th to 17th aspects, includes a second learning step of generating the nipple bleeding region detection model by performing supervised machine learning using the second training data.

[0026] In a nineteenth aspect of the embodiment, in any one of the fifteenth to eighteenth aspects, the front image of the fundus is a color front image.

[0027] A twentieth aspect of the embodiment, in any of the fifteenth to nineteenth aspects, includes an analysis step of generating at least one of position information representing the position of the papilla hemorrhage area in the front image of the fundus of the test eye, shape information representing the shape of the papilla hemorrhage area, and occurrence information representing the occurrence status of the papilla hemorrhage area by analyzing a front image of the fundus of the test eye.

[0028] In a twenty-first aspect of the embodiment, in the twentieth aspect, the position information includes information indicating a direction of the representative position of the nipple bleeding region relative to a reference position.

[0029] In a twenty-second aspect of the embodiment, in the twentieth or twenty-first aspect, the position information includes information indicating whether the representative position of the disc hemorrhage region is inside or outside the optic disc region.

[0030] In a twenty-third aspect of the embodiment, in the twenty-second aspect, the position information includes information indicating whether the representative position is within the optic disc periphery or outside the optic disc periphery.

[0031] In a twenty-fourth aspect of the embodiment, in any one of the twentieth to twenty-third aspects, the shape information includes at least one of an ellipticity of the nipple bleeding region and an area of ​​the nipple bleeding region.

[0032] In a twenty-fifth aspect of the embodiment, in any one of the twentieth to twenty-fourth aspects, the occurrence information includes at least one of an occurrence frequency of the nipple bleeding region and an occurrence interval of the nipple bleeding region.

[0033] A 26th aspect of the embodiment, in any of the 20th to 25th aspects, includes an alignment step of aligning the OCT data of the test eye with a frontal image of the test eye in which the disc hemorrhage area detected in the detection step is depicted, and the analysis step generates at least one of the position information, the shape information, and the generation information using the OCT data aligned with the frontal image in the alignment step.

[0034] A twenty-seventh aspect of the embodiment is a program for causing a computer to execute each step of the ophthalmologic information processing method according to any one of the fifteenth to twenty-sixth aspects.

[0035] The configurations according to the above-described multiple aspects can be combined in any desired manner. Effect of the Invention

[0036] According to the present invention, a new technique for detecting a nipple bleeding region with high reproducibility and high accuracy can be provided. [Brief description of the drawings]

[0037] [Figure 1] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic apparatus according to a first embodiment. [Diagram 2] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic apparatus according to a first embodiment. [Diagram 3] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic apparatus according to a first embodiment. [Figure 4] 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic apparatus according to a first embodiment. [Diagram 5] 3 is a schematic diagram for explaining an example of an operation of the ophthalmologic apparatus according to the first embodiment. FIG. [Figure 6] 1 is a schematic diagram illustrating an example of the configuration of an ophthalmologic apparatus according to a first embodiment. [Figure 7] FIG. 2 is a schematic diagram for explaining the operation of the ophthalmologic apparatus according to the first embodiment. [Figure 8] 1 is a schematic diagram illustrating an example of the configuration of an ophthalmologic apparatus according to a first embodiment. [Figure 9] FIG. 2 is a schematic diagram for explaining the operation of the ophthalmologic apparatus according to the first embodiment. [Figure 10] FIG. 2 is a schematic diagram for explaining the operation of the ophthalmologic apparatus according to the first embodiment. [Figure 11] FIG. 2 is a schematic diagram for explaining the operation of the ophthalmologic apparatus according to the first embodiment. [Figure 12] 4 is a flowchart illustrating an example of the operation of the ophthalmologic apparatus according to the first embodiment. [Figure 13] FIG. 2 is a schematic diagram for explaining the operation of the ophthalmologic apparatus according to the first embodiment. [Figure 14] FIG. 11 is a schematic diagram illustrating a first configuration example of an ophthalmologic system according to a second embodiment. [Figure 15] FIG. 11 is a schematic diagram illustrating a second configuration example of an ophthalmologic system according to the second embodiment. [Figure 16] FIG. 11 is a schematic diagram illustrating an example of the configuration of an ophthalmologic information processing apparatus according to a second embodiment. [Figure 17] FIG. 13 is a schematic diagram illustrating an example of a configuration of an ophthalmologic system according to a first modified example of the second embodiment. [Figure 18] FIG. 13 is a schematic diagram illustrating an example of a configuration of an ophthalmologic system according to a second modified example of the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] The following describes in detail exemplary embodiments of an ophthalmological information processing device, an ophthalmological device, an ophthalmological information processing method, and a program according to the present invention with reference to the drawings. Note that the contents of documents cited in this specification and any publicly known techniques may be incorporated in the following exemplary embodiments.

[0039] The ophthalmologic information processing device according to the embodiment uses a trained model to determine the presence or absence of a disc hemorrhage in a fundus image of a test eye, and uses another trained model to detect a disc hemorrhage area depicted in a fundus image determined to have a disc hemorrhage. The trained model used to determine the presence or absence of a disc hemorrhage is a disc hemorrhage determination model obtained by machine learning using a plurality of fundus images labeled with an indication of the presence or absence of a disc hemorrhage as first teacher data. The trained model used to detect a disc hemorrhage area is a disc hemorrhage area detection model obtained by machine learning using a plurality of pairs of images, each pair consisting of a fundus image and a disc hemorrhage area image showing a disc hemorrhage area depicted in the fundus image, as second teacher data. The fundus image is preferably a color front image of the fundus obtained by photographing the fundus with a fundus camera or the like.

[0040] In this way, a model for detecting a disc hemorrhage region is generated by performing machine learning using only fundus images in which a disc hemorrhage region exists, and the learning parameters of the model for detecting a disc hemorrhage region are updated specifically for detecting a disc hemorrhage region. As a result, the accuracy of detecting a disc hemorrhage region can be improved. Therefore, compared to a case where a disc hemorrhage region is detected in a fundus image of a test eye using one trained model, it is possible to detect a disc hemorrhage region in a fundus image with higher reproducibility and higher accuracy.

[0041] In some embodiments, the ophthalmologic information processing device generates an analysis result of the nipple hemorrhage region detected with high accuracy and reproducibility by analyzing the fundus image, The analysis result includes at least one of position information indicating the position of the nipple hemorrhage region, shape information indicating the shape of the nipple hemorrhage region, and occurrence information indicating the occurrence status of the nipple hemorrhage region.

[0042] In some embodiments, the ophthalmologic information processing device aligns a fundus image in which a papilla hemorrhage region is detected with OCT (Optical Coherence Tomography) data of the subject's eye, and generates position information and shape information using position information of an OCT coordinate system that defines the OCT data. This makes it possible to obtain position information and shape information of the papilla hemorrhage region with high accuracy from the fundus image mapped to the OCT coordinate system that defines highly reproducible OCT data.

[0043] In some embodiments, an ophthalmic device having the functions of a fundus camera for acquiring a fundus image of the test eye and the functions of an OCT device (optical coherence tomography) capable of performing OCT measurements for acquiring OCT data of the test eye has the functions of the ophthalmic information processing device of the embodiment.

[0044] In some embodiments, an ophthalmic apparatus having a fundus camera function has the function of the ophthalmic processing apparatus according to the embodiment. In this case, the ophthalmic apparatus acquires OCT data from an external OCT device.

[0045] In some embodiments, an ophthalmic apparatus having the functionality of an OCT apparatus has the functionality of the ophthalmic processing apparatus according to the embodiment. In this case, the ophthalmic apparatus acquires a fundus image from an external fundus camera.

[0046] In some embodiments, the ophthalmic processing device acquires fundus images from an external fundus camera and acquires OCT data from an external OCT device.

[0047] In some embodiments, the ophthalmic processing device acquires fundus images and OCT data from an ophthalmic device that has the functionality of a fundus camera and an OCT device.

[0048] In some embodiments, the fundus image is a C-scan image, a projection image, or an en-face image formed by OCT data, or an image acquired by an ophthalmic device such as a Scanning Laser Ophthalmoscope (SLO), a slit lamp ophthalmoscope, or a surgical microscope. When the fundus image is a C-scan image, a projection image, or an en-face image, alignment between the fundus image and the OCT data may not be required.

[0049] The ophthalmological information processing method according to the embodiment includes one or more steps for implementing processing executed by a processor (computer) in the ophthalmological information processing device according to the embodiment. The program according to the embodiment causes the processor to execute each step of the ophthalmological information processing method according to the embodiment. The recording medium according to the embodiment is a computer-readable non-transitory recording medium (storage medium) on which the program according to the embodiment is recorded.

[0050] In this specification, the term "processor" refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA)), etc. The processor realizes the functions according to the embodiment by, for example, reading and executing a program stored in a memory circuit or a storage device.

[0051] In this specification, images acquired by OCT may be collectively referred to as OCT images. Furthermore, the measurement operation for forming OCT images may be referred to as OCT measurement. Furthermore, disc hemorrhage may be simply referred to as DH (Disc Hemorrhage).

[0052] [First embodiment] The ophthalmic device according to the first embodiment has a function of a fundus camera, a function of an OCT device, and a function of an ophthalmic processing device according to the embodiment. The ophthalmic device may further have at least one function of an ophthalmic measurement device and an ophthalmic treatment device. In some embodiments, the ophthalmic measurement device includes at least one of an eye refraction examination device, a tonometer, a specular microscope, a wavefront analyzer, a perimeter, and a microperimeter. In some embodiments, the ophthalmic treatment device includes at least one of a laser treatment device, a surgical device, and a surgical microscope.

[0053] <Configuration> [Optical system] FIG. 1 shows an example of the configuration of an ophthalmic apparatus according to the first embodiment.

[0054] The ophthalmic apparatus 1 according to the first embodiment includes a fundus camera unit 2, an OCT unit 100, and an arithmetic and control unit 200. The fundus camera unit 2 is provided with an optical system and a mechanism for acquiring a front image of the subject's eye E. The OCT unit 100 is provided with a part of the optical system and a mechanism for performing OCT. The other part of the optical system and the mechanism for performing OCT is provided in the fundus camera unit 2. The arithmetic and control unit 200 includes one or more processors for performing various calculations and controls. In addition to these, any element or unit such as a member for supporting the subject's face (such as a chin rest or a forehead rest) or a lens unit for switching the target site of OCT (for example, an attachment for anterior segment OCT) may be provided in the ophthalmic apparatus 1. In some embodiments, the lens unit is configured to be manually inserted and removed between the subject's eye E and an objective lens 22 described later. In some embodiments, the lens unit is configured to be automatically inserted and removed between the subject's eye E and an objective lens 22 described later under the control of a control unit 210 described later.

[0055] [Fundus camera unit] The fundus camera unit 2 is provided with an optical system for photographing the fundus Ef of the subject's eye E. The acquired image of the fundus Ef (called a fundus image, fundus photograph, etc.) is a front image such as an observation image or a capture image. The observation image is obtained by capturing a moving image using near-infrared light. The capture image is a still image (e.g., a color image) using flash light. Furthermore, the fundus camera unit 2 can photograph the anterior segment Ea of the subject's eye E to acquire a front image (anterior segment image).

[0056] The fundus camera unit 2 includes an illumination optical system 10 and an imaging optical system 30. The illumination optical system 10 irradiates illumination light onto the subject's eye E. The imaging optical system 30 detects return light of the illumination light from the subject's eye E. The measurement light from the OCT unit 100 is guided to the subject's eye E through an optical path in the fundus camera unit 2, and the return light is guided to the OCT unit 100 through the same optical path.

[0057] Light (observation illumination light) output from an observation light source 11 of an illumination optical system 10 is reflected by a reflecting mirror 12 having a curved reflecting surface, passes through a condenser lens 13, and passes through a visible light cut filter 14 to become near-infrared light. The observation illumination light is then focused once near a photographing light source 15, reflected by a mirror 16, and passes through relay lenses 17 and 18, an aperture 19, and a relay lens 20. The observation illumination light is then reflected by the peripheral portion (area surrounding the hole) of a holed mirror 21, passes through a dichroic mirror 46, and is refracted by an objective lens 22 to illuminate the subject's eye E (fundus Ef or anterior eye Ea). Return light of the observation illumination light from the subject's eye E is refracted by the objective lens 22, passes through a dichroic mirror 46, passes through a hole formed in the central area of ​​the holed mirror 21, and passes through a dichroic mirror 55. The return light transmitted through the dichroic mirror 55 passes through the photographing focusing lens 31 and is reflected by the mirror 32. The return light further passes through the half mirror 33A and is reflected by the dichroic mirror 33, and is imaged on the light receiving surface of the image sensor 35 by the condenser lens 34. The image sensor 35 detects the return light at a predetermined frame rate. The focus of the photographing optical system 30 is adjusted to match the fundus Ef or the anterior segment Ea.

[0058] The light (imaging illumination light) output from the imaging light source 15 is irradiated onto the fundus Ef through the same path as the observation illumination light. The return light of the imaging illumination light from the subject's eye E is guided to the dichroic mirror 33 through the same path as the return light of the observation illumination light, passes through the dichroic mirror 33, is reflected by a mirror 36, and is imaged on the light receiving surface of an image sensor 38 by a condenser lens 37.

[0059] The LCD (Liquid Crystal Display) 39 displays a fixation target and a visual target for visual acuity measurement. A part of the light beam output from the LCD 39 is reflected by the half mirror 33A, reflected by the mirror 32, passes through the photographing focusing lens 31 and the dichroic mirror 55, and passes through the hole of the aperture mirror 21. The light beam that passes through the hole of the aperture mirror 21 passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef.

[0060] The fixation position of the subject's eye E can be changed by changing the display position of the fixation target on the screen of the LCD 39. Examples of the fixation position include a fixation position for acquiring an image centered on the macula, a fixation position for acquiring an image centered on the optic disc, a fixation position for acquiring an image centered on the fundus center between the macula and the optic disc, and a fixation position for acquiring an image of a site (periphery of the fundus) far away from the macula. The ophthalmic device 1 according to some embodiments includes a GUI (Graphical User Interface) or the like for designating at least one of such fixation positions. The ophthalmic device 1 according to some embodiments includes a GUI or the like for manually moving the fixation position (display position of the fixation target).

[0061] The configuration for presenting a movable fixation target to the subject's eye E is not limited to a display device such as an LCD. For example, a movable fixation target can be generated by selectively turning on a plurality of light sources in a light source array (such as a light emitting diode (LED) array). Also, a movable fixation target can be generated by one or more movable light sources.

[0062] The ophthalmologic apparatus 1 may also be provided with one or more external fixation light sources. One of the one or more external fixation light sources can project fixation light onto the fellow eye of the subject's eye E. The projection position of the fixation light onto the fellow eye can be changed. By changing the projection position of the fixation light onto the fellow eye, the fixation position of the subject's eye E can be changed. The fixation position by the external fixation light source may be the same as the fixation position of the subject's eye E using the LCD 39. For example, a movable fixation target can be generated by selectively turning on a plurality of external fixation light sources. Also, a movable fixation target can be generated by one or more movable external fixation light sources.

[0063] The alignment optical system 50 generates an alignment target used for aligning the optical system with respect to the subject's eye E. Alignment light output from an LED 51 passes through apertures 52 and 53 and a relay lens 54, is reflected by a dichroic mirror 55, and passes through a hole in the aperture mirror 21. The light that passes through the hole in the aperture mirror 21 passes through a dichroic mirror 46 and is projected onto the subject's eye E by the objective lens 22. The corneal reflected light of the alignment light is guided to the image sensor 35 via the same path as the return light of the observation illumination light. Manual alignment or auto alignment can be performed based on the received light image (alignment target image).

[0064] The focusing optical system 60 generates a split target used for focus adjustment of the subject's eye E. The focusing optical system 60 is moved along the optical path (illumination optical path) of the illumination optical system 10 in conjunction with the movement of the photographing focusing lens 31 along the optical path (photographing optical path) of the photographing optical system 30. The reflecting rod 67 can be inserted into and removed from the illumination optical path. When performing focus adjustment, the reflecting surface of the reflecting rod 67 is tilted and disposed in the illumination optical path. The focusing light output from the LED 61 passes through the relay lens 62, is split into two light beams by the split target plate 63, passes through the two-hole diaphragm 64, is reflected by the mirror 65, and is once imaged and reflected on the reflecting surface of the reflecting rod 67 by the condenser lens 66. Furthermore, the focusing light passes through the relay lens 20, is reflected by the aperture mirror 21, passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef. The fundus reflection light of the focusing light is guided to the image sensor 35 through the same path as the cornea reflection light of the alignment light. Manual focusing and auto focusing can be performed based on the received light image (split target image).

[0065] The dichroic mirror 46 combines the imaging optical path and the OCT optical path (optical path of the interference optical system). The dichroic mirror 46 couples both optical systems so that the optical axis of the imaging optical path (imaging optical system 30) and the optical axis of the OCT optical path (interference optical system) are approximately coaxial. The dichroic mirror 46 reflects light in the wavelength band used for OCT and transmits light for fundus imaging. The OCT optical path (optical path of the measurement light) is provided with a collimator lens unit 40, an optical path length changer 41, an optical scanner 42, an OCT focusing lens 43, a mirror 44, and a relay lens 45 in this order from the OCT unit 100 side to the dichroic mirror 46 side.

[0066] The optical path length changing unit 41 is movable in the direction of the arrow shown in Fig. 1 and changes the length of the optical path for OCT. This change in the optical path length is used for correcting the optical path length according to the axial length, adjusting the interference state, etc. The optical path length changing unit 41 includes a corner cube and a mechanism for moving it.

[0067] The optical scanner 42 is disposed at a position optically conjugate with the pupil of the subject's eye E. The optical scanner 42 deflects the measurement light LS passing through the OCT optical path. The optical scanner 42 can deflect the measurement light LS one-dimensionally or two-dimensionally.

[0068] In the case of one-dimensional deflection, the optical scanner 42 includes a galvanometer scanner that deflects the measurement light LS in a predetermined deflection direction within a predetermined deflection angle range. In the case of two-dimensional deflection, the optical scanner 42 includes a first galvanometer scanner and a second galvanometer scanner. The first galvanometer scanner deflects the measurement light LS so as to scan the imaging site (fundus Ef or anterior segment Ea) in a horizontal direction perpendicular to the optical axis of the interference optical system (OCT optical system). The second galvanometer scanner deflects the measurement light LS deflected by the first galvanometer scanner so as to scan the imaging site in a vertical direction perpendicular to the optical axis of the interference optical system. Examples of the scanning mode of the measurement light LS by the optical scanner 42 include horizontal scan, vertical scan, cross scan, radial scan, circular scan, concentric circle scan, and spiral scan.

[0069] The OCT focusing lens 43 is moved along the optical path of the measurement light LS to adjust the focus of the OCT optical system. The OCT focusing lens 43 is movable within a movement range including a first lens position for locating the focal position of the measurement light LS on or near the fundus Ef of the subject's eye E, and a second lens position for making the measurement light LS irradiated to the subject's eye E into a parallel beam. The movement of the photographing focusing lens 31, the movement of the focus optical system 60, and the movement of the OCT focusing lens 43 can be controlled in a coordinated manner.

[0070] [OCT unit] An example of the configuration of the OCT unit 100 is shown in Fig. 2. The OCT unit 100 is provided with an optical system for acquiring an OCT image of the subject's eye E. This optical system is an interference optical system that splits light from a wavelength sweep type (wavelength scanning type) light source into measurement light and reference light, generates interference light by causing return light of the measurement light from the subject's eye E to interfere with the reference light that has passed through a reference light path, and detects this interference light. The detection result (detection signal) of the interference light by the interference optical system is an interference signal indicating the spectrum of the interference light, and is sent to the arithmetic and control unit 200.

[0071] The light source unit 101 includes a wavelength-swept (wavelength-scanning) light source capable of sweeping (scanning) the wavelength of emitted light, similar to a general swept-source type ophthalmic device. The wavelength-swept light source includes a laser light source including a resonator. The light source unit 101 changes the output wavelength over time in the near-infrared wavelength band that is not visible to the human eye.

[0072] Light L0 output from the light source unit 101 is guided by an optical fiber 102 to a polarization controller 103, where the polarization state of the light L0 is adjusted. The polarization controller 103 adjusts the polarization state of the light L0 guided through the optical fiber 102, for example, by applying stress from the outside to the looped optical fiber 102.

[0073] The light L0, whose polarization state has been adjusted by the polarization controller 103, is guided by an optical fiber 104 to a fiber coupler 105, where it is split into a measurement light LS and a reference light LR.

[0074] The reference light LR is guided by an optical fiber 110 to a collimator 111, where it is converted into a parallel light beam, and is guided to an optical path length changer 114 via an optical path length correction member 112 and a dispersion compensation member 113. The optical path length correction member 112 acts to match the optical path length of the reference light LR with the optical path length of the measurement light LS. The dispersion compensation member 113 acts to match the dispersion characteristics between the reference light LR and the measurement light LS. The optical path length changer 114 includes, for example, a corner cube and a moving mechanism for moving the corner cube, and is capable of moving the corner cube by the moving mechanism in the incident direction of the reference light LR. This changes the optical path length of the reference light LR.

[0075] The reference light LR that has passed through the optical path length changing unit 114 passes through the dispersion compensation member 113 and the optical path length correction member 112, is converted from a parallel beam to a focused beam by the collimator 116, and is then incident on the optical fiber 117. The reference light LR that has passed through the optical fiber 117 is guided to a polarization controller 118 where its polarization state is adjusted, is guided by the optical fiber 119 to an attenuator 120 where the amount of light is adjusted, and is then guided by the optical fiber 121 to a fiber coupler 122.

[0076] On the other hand, the measurement light LS generated by the fiber coupler 105 is guided by the optical fiber 127 and converted into a parallel beam by the collimator lens unit 40, and passes through the optical path length changing unit 41, the optical scanner 42, the OCT focusing lens 43, the mirror 44, and the relay lens 45. The measurement light LS passing through the relay lens 45 is reflected by the dichroic mirror 46, refracted by the objective lens 22, and enters the subject's eye E. The measurement light LS is scattered and reflected at various depth positions in the subject's eye E. The return light of the measurement light LS from the subject's eye E travels in the opposite direction along the same path as the outward path and is guided to the fiber coupler 105, and reaches the fiber coupler 122 via the optical fiber 128.

[0077] The fiber coupler 122 generates interference light by combining (causing interference between) the measurement light LS incident via the optical fiber 128 and the reference light LR incident via the optical fiber 121. The fiber coupler 122 splits the interference light at a predetermined splitting ratio (for example, 1:1) to generate a pair of interference lights LC. The pair of interference lights LC are guided to a detector 125 via optical fibers 123 and 124, respectively.

[0078] The detector 125 is, for example, a balanced photodiode having a pair of photodetectors that detect a pair of interference lights LC, respectively, and outputting the difference between the detection results by these. The detector 125 sends the detection result (interference signal) to a DAQ (Data Acquisition System) 130. A clock KC is supplied to the DAQ 130 from the light source unit 101. The clock KC is generated in synchronization with the output timing of each wavelength swept (scanned) in a predetermined wavelength range by the wavelength swept light source in the light source unit 101. For example, the light source unit 101 optically delays one of two branched lights obtained by branching the light L0 of each output wavelength, and then generates the clock KC based on the result of detecting the combined light. The DAQ 130 samples the detection result of the detector 125 based on the clock KC. The DAQ 130 sends the sampled detection result of the detector 125 to the arithmetic and control unit 200. The arithmetic and control unit 200 forms a reflection intensity profile for each A-line by performing a Fourier transform or the like on the spectral distribution based on the detection result obtained by the detector 125, for example, for each series of wavelength scans (for each A-line). Furthermore, the arithmetic and control unit 200 forms image data by imaging the reflection intensity profile of each A-line.

[0079] 1 and 2, both an optical path length changer 41 for changing the length of the optical path (measurement optical path, measurement arm) of the measurement light LS and an optical path length changer 114 for changing the length of the optical path (reference optical path, reference arm) of the reference light LR are provided. However, only one of the optical path length changers 41 and 114 may be provided. In addition, it is also possible to change the difference between the reference optical path length and the measurement optical path length using optical members other than these.

[0080] [Calculation and control unit] The arithmetic and control unit 200 analyzes the detection signal input from the DAQ 130 to form an OCT image of the subject's eye E. The arithmetic and control process for this purpose is similar to that of a conventional swept-source type OCT device.

[0081] In addition, the arithmetic and control unit 200 controls each part of the fundus camera unit 2, the display device 3, and the OCT unit 100.

[0082] To control the fundus camera unit 2, the arithmetic and control unit 200 controls the operation of the observation light source 11, the imaging light source 15, and the LEDs 51 and 61, controls the operation of the LCD 39, controls the movement of the imaging focusing lens 31, controls the movement of the OCT focusing lens 43, controls the movement of the reflecting rod 67, controls the movement of the focus optical system 60, controls the movement of the optical path length changing unit 41, and controls the operation of the optical scanner 42.

[0083] To control the display device 3, the arithmetic and control unit 200 causes the display device 3 to display an OCT image of the eye E to be examined.

[0084] To control the OCT unit 100, the arithmetic and control unit 200 controls the operation of the light source unit 101, the movement of the optical path length changing unit 114, the operation of the attenuator 120, the operation of the polarization controllers 103 and 118, the operation of the detector 125, and the operation of the DAQ 130.

[0085] The arithmetic control unit 200 includes a processor, a RAM, a ROM, a hard disk drive, a communication interface, and the like, similar to a conventional computer. A computer program for controlling the ophthalmic apparatus 1 is stored in a storage device such as a hard disk drive. The arithmetic control unit 200 may include various circuit boards, for example, a circuit board for forming an OCT image. The arithmetic control unit 200 may also include operation devices (input devices) such as a keyboard and a mouse, and a display device such as an LCD.

[0086] The fundus camera unit 2, the display device 3, the OCT unit 100, and the arithmetic and control unit 200 may be configured integrally (i.e., within a single housing) or may be configured separately in two or more housings.

[0087] [Control system] Fig. 3 shows an example of the configuration of a control system of the ophthalmic apparatus 1. In Fig. 3, some of the components included in the ophthalmic apparatus 1 are omitted.

[0088] The arithmetic and control unit 200 includes a control unit 210 , and controls each part of the fundus camera unit 2 , the display device 3 , and the OCT unit 100 .

[0089] (Control unit) The control unit 210 executes various types of control. The control unit 210 includes a main control unit 211 and a storage unit 212.

[0090] (Main control unit) The main control unit 211 includes a processor and controls each unit of the ophthalmologic apparatus 1. For example, the main control unit 211 controls the focusing drive units 31A and 43A, the image sensors 35 and 38, the LCD 39, the optical path length change unit 41, the optical scanner 42, and the movement of the optical system (movement mechanism 150) of the fundus camera unit 2. Furthermore, the main control unit 211 controls the light source unit 101, the optical path length change unit 114, the attenuator 120, the polarization controllers 103 and 118, the detector 125, the DAQ 130, and the like of the OCT unit 100.

[0091] For example, the main control unit 211 displays the fixation target at a position on the screen of the LCD 39 that corresponds to a manually or automatically set fixation position. The main control unit 211 can also change (continuously or stepwise) the display position of the fixation target displayed on the LCD 39. This makes it possible to move the fixation target (i.e., to change the fixation position). The display position and movement manner of the fixation target are set manually or automatically. Manual setting is performed using, for example, a GUI. Automatic setting is performed by, for example, the data processing unit 300.

[0092] The focusing drive unit 31A moves the photographing focusing lens 31 in the optical axis direction of the photographing optical system 30, and moves the focus optical system 60 in the optical axis direction of the illumination optical system 10. This changes the focus position of the photographing optical system 30. In some embodiments, the focusing drive unit 31A has a mechanism for moving the photographing focusing lens 31 and a mechanism for moving the focus optical system 60 separately. The focusing drive unit 31A is controlled when performing focus adjustment, etc.

[0093] The focusing driver 43A moves the OCT focusing lens 43 in the optical axis direction of the measurement light path. This changes the focusing position of the measurement light LS. For example, by moving the OCT focusing lens 43 to a first lens position, the focusing position of the measurement light LS can be located at the fundus Ef or in its vicinity. For example, by moving the OCT focusing lens 43 to a second lens position, the focusing position of the measurement light LS can be located at a far point position to make the measurement light LS a parallel beam. The focusing position of the measurement light LS corresponds to the depth position (z position) of the beam waist of the measurement light LS.

[0094] The moving mechanism 150, for example, moves at least the fundus camera unit 2 (optical system) three-dimensionally. In a typical example, the moving mechanism 150 includes at least a mechanism for moving the fundus camera unit 2 in the x direction (left and right direction), a mechanism for moving in the y direction (up and down direction), and a mechanism for moving in the z direction (depth direction, front and back direction). The mechanism for moving in the x direction includes, for example, an x ​​stage that can move in the x direction, and an x ​​moving mechanism that moves the x stage. The mechanism for moving in the y direction includes, for example, a y stage that can move in the y direction, and a y moving mechanism that moves the y stage. The mechanism for moving in the z direction includes, for example, a z stage that can move in the z direction, and a z moving mechanism that moves the z stage. Each moving mechanism includes an actuator such as a pulse motor, and operates under the control of the main controller 211.

[0095] The control of the moving mechanism 150 is used for alignment and tracking. Tracking refers to moving the device optical system in accordance with the eye movement of the subject's eye E. When tracking is performed, alignment and focus adjustment are performed in advance. Tracking is a function of maintaining a suitable positional relationship where alignment and focus are achieved by making the position of the device optical system follow the eye movement. In some embodiments, the moving mechanism 150 is configured to be controlled in order to change the optical path length of the reference light (hence, the optical path length difference between the optical path of the measurement light and the optical path of the reference light).

[0096] In the case of manual alignment, the user operates a user interface 240 described below to cancel the displacement of the eye E relative to the optical system, thereby moving the optical system and the eye E relatively. For example, the main controller 211 controls the moving mechanism 150 by outputting a control signal corresponding to the operation content on the user interface 240 to the moving mechanism 150, thereby moving the optical system and the eye E relatively.

[0097] In the case of auto-alignment, the main controller 211 controls the moving mechanism 150 so that the displacement of the subject's eye E relative to the optical system is cancelled, thereby moving the optical system and the subject's eye E relatively. In some embodiments, the main controller 211 controls the moving mechanism 150 by outputting a control signal to the moving mechanism 150 so that the optical axis of the optical system approximately coincides with the axis of the subject's eye E and the distance of the optical system to the subject's eye E is a predetermined working distance, thereby moving the optical system and the subject's eye E relatively. Here, the working distance is a preset value also called the working distance of the objective lens 22, and corresponds to the distance between the subject's eye E and the optical system during measurement (photography) using the optical system.

[0098] The main controller 211 controls the fundus camera unit 2 and the like to control fundus photography and anterior eye photography. The main controller 211 also controls the fundus camera unit 2 and the OCT unit 100 and the like to control OCT measurement. The main controller 211 can execute multiple preliminary operations before performing OCT measurement. The preliminary operations include alignment, coarse focus adjustment, polarization adjustment, fine focus adjustment, and the like. The multiple preliminary operations are executed in a predetermined order. In some embodiments, the multiple preliminary operations are executed in the above order.

[0099] The types and order of the preliminary operations are not limited to the above and are arbitrary. For example, a preliminary operation (small pupil determination) for determining whether the subject's eye E is a small pupil eye can be added to the preliminary operations. The small pupil determination is performed, for example, between the coarse focus adjustment and the optical path length difference adjustment. In some embodiments, the small pupil determination includes the following series of processes: a process of acquiring a front image (anterior eye image) of the subject's eye E; a process of identifying an image area corresponding to the pupil; a process of obtaining a size (diameter, circumference, etc.) of the identified pupil area; a process of determining whether the eye has a small pupil based on the obtained size (threshold processing); and a process of controlling the aperture 19 when the eye is determined to have a small pupil. In some embodiments, the process further includes a process of approximating the pupil area to a circle or an ellipse to obtain the pupil size.

[0100] The coarse focus adjustment is a focus adjustment using a split target. Note that the coarse focus adjustment can also be performed by determining the position of the focusing lens 31 based on information relating the eye refractive power and the position of the focusing lens 31 obtained in advance and the measured value of the refractive power of the subject's eye E.

[0101] The focus fine adjustment is performed based on the interference sensitivity of the OCT measurement. For example, by monitoring the interference intensity (interference sensitivity) of the interference signal acquired by the OCT measurement of the subject's eye E, the position of the OCT focusing lens 43 where the interference intensity is maximized is obtained, and the OCT focusing lens 43 is moved to that position, thereby performing the focus fine adjustment.

[0102] In the optical path length difference adjustment, a predetermined position in the subject's eye E is controlled to become a reference position of the measurement range in the depth direction. This control is performed on at least one of the optical path length changing units 41, 114. This adjusts the optical path length difference between the measurement optical path and the reference optical path. By setting the reference position by adjusting the optical path length difference, it becomes possible to perform OCT measurement with high accuracy for a desired measurement range in the depth direction simply by changing the wavelength sweep speed.

[0103] In the polarization adjustment, the polarization state of the reference light LR is adjusted to optimize the interference efficiency between the measurement light LS and the reference light LR.

[0104] (Storage part) The storage unit 212 stores various data. Examples of data stored in the storage unit 212 include image data of an OCT image, image data of a fundus image, image data of an anterior segment image, and information about the subject's eye. The information about the subject's eye includes information about the subject, such as a patient ID and name, and information about the subject's eye, such as identification information of the left eye / right eye.

[0105] Moreover, the storage unit 212 stores various programs and data for operating the ophthalmologic apparatus 1.

[0106] The control unit 210 is capable of controlling the image forming unit 230 and the data processing unit 300 .

[0107] (Image forming section) The image forming unit 230 forms an OCT image (image data) of the subject's eye E based on sampling data obtained by sampling the detection signal from the detector 125 with the DAQ 130. The OCT images formed by the image forming unit 230 include an A-scan (A-mode) image, a B-scan (B-mode) image (tomographic image), and a C-scan (C-mode) image. This processing includes noise removal (noise reduction), filtering, dispersion compensation, FFT (Fast Fourier Transform), and the like, as in conventional swept-source type OCT. In the case of other types of OCT devices, the image forming unit 230 executes known processing according to the type.

[0108] Image forming section 230 is configured to include, for example, the circuit board described above. Note that in this specification, "image data" and an "image" based on the image data may be regarded as being the same thing.

[0109] (Data Processing Unit) The data processing unit 300 processes data acquired by photographing the subject's eye E and performing OCT measurement. For example, the data processing unit 300 performs various image processing and analysis processing on the image formed by the image forming unit 230. Specifically, the data processing unit 300 executes various correction processing such as brightness correction of the image. In addition, the data processing unit 300 can perform various image processing and analysis processing on the images (fundus image, anterior eye image, etc.) obtained by the fundus camera unit 2.

[0110] The data processing unit 300 executes known image processing such as an interpolation process for interpolating pixels between tomographic images to form image data of a three-dimensional image of the fundus oculi Ef or the anterior segment Ea. The image data of the three-dimensional image means image data in which the positions of pixels are defined by a three-dimensional coordinate system. The image data of the three-dimensional image includes image data consisting of three-dimensionally arranged voxels. This image data is called volume data or voxel data. When an image based on the volume data is to be displayed, the data processing unit 300 performs a rendering process (such as volume rendering or MIP (Maximum Intensity Projection)) on the volume data to form image data of a pseudo three-dimensional image as viewed from a specific line of sight. The pseudo three-dimensional image is displayed on a display device such as the display unit 240A.

[0111] It is also possible to form stack data of multiple tomographic images as image data of a three-dimensional image. Stack data is image data obtained by arranging multiple tomographic images obtained along multiple scan lines three-dimensionally based on the positional relationship of the scan lines. In other words, stack data is image data obtained by expressing multiple tomographic images that were originally defined by individual two-dimensional coordinate systems using one three-dimensional coordinate system (i.e., embedding them in one three-dimensional space).

[0112] The data processing unit 300 can perform various renderings on the acquired three-dimensional data set (volume data, stack data, etc.) to form a B-mode image (longitudinal section image, axial section image) in an arbitrary cross section, a C-mode image (transverse section image, horizontal section image) in an arbitrary cross section, a projection image, a shadowgram, etc. An image of an arbitrary cross section such as a B-mode image or a C-mode image is formed by selecting pixels (voxels) on a specified cross section from the three-dimensional data set. A projection image is formed by projecting the three-dimensional data set in a predetermined direction (z direction, depth direction, axial direction). A shadowgram is formed by projecting a part of the three-dimensional data set (for example, partial data corresponding to a specific layer) in a predetermined direction. An image with a viewpoint on the front side of the subject's eye, such as a C-mode image, a projection image, or a shadowgram, is called an en-face image.

[0113] The data processing unit 300 can construct a B-mode image or a front image (vessel-enhanced image, angiogram) in which retinal blood vessels and choroidal blood vessels are emphasized based on data (e.g., B-scan image data) collected in a time series by OCT. For example, time series OCT data can be collected by repeatedly scanning approximately the same part of the subject's eye E.

[0114] In some embodiments, the data processor 300 compares time-series B-scan images obtained by B-scanning approximately the same region, and constructs an enhanced image in which the changed portion is emphasized by converting pixel values ​​of the changed portion of signal intensity into pixel values ​​corresponding to the changed portion. Furthermore, the data processor 300 extracts information of a predetermined thickness of a desired region from the constructed multiple enhanced images, and constructs the information as an en-face image to form an OCTA image.

[0115] Images generated by the data processing unit 300 (for example, a three-dimensional image, a B-mode image, a C-mode image, a projection image, a shadowgram, and an OCTA image) are also included in the OCT image.

[0116] Furthermore, the data processing unit 300 analyzes the detection result of the interference light obtained by the OCT measurement to determine the focus state of the measurement light LS in the focus fine adjustment control. For example, the main control unit 211 performs repetitive OCT measurement while controlling the focus driving unit 43A according to a predetermined algorithm. The data processing unit 300 calculates a predetermined evaluation value related to the image quality of the OCT image by analyzing the detection result of the interference light LC repeatedly obtained by the OCT measurement. The data processing unit 300 determines whether the calculated evaluation value is equal to or less than a threshold value. In some embodiments, the focus fine adjustment is continued until the calculated evaluation value is equal to or less than a threshold value. That is, when the evaluation value is equal to or less than the threshold value, it is determined that the focus state of the measurement light LS is appropriate, and the focus fine adjustment is continued until it is determined that the focus state of the measurement light LS is appropriate.

[0117] In some embodiments, the main controller 211 performs the above-described repetitive OCT measurements to acquire interference signals, while monitoring the intensity (interference intensity, interference sensitivity) of the interference signals acquired sequentially. Furthermore, while performing this monitoring process, the OCT focusing lens 43 is moved to search for the position of the OCT focusing lens 43 where the interference intensity is maximized. By performing such fine focus adjustment, the OCT focusing lens 43 can be guided to a position where the interference intensity is optimized.

[0118] Moreover, the data processing unit 300 analyzes the detection result of the interference light obtained by the OCT measurement to determine the polarization state of at least one of the measurement light LS and the reference light LR. For example, the main control unit 211 performs repetitive OCT measurement while controlling at least one of the polarization controllers 103 and 118 according to a predetermined algorithm. In some embodiments, the main control unit 211 controls the attenuator 120 to change the attenuation of the reference light LR. The data processing unit 300 calculates a predetermined evaluation value regarding the image quality of the OCT image by analyzing the detection result of the interference light LC repeatedly acquired by the OCT measurement. The data processing unit 300 determines whether the calculated evaluation value is equal to or less than a threshold value. This threshold value is set in advance. The polarization adjustment is continued until the calculated evaluation value is equal to or less than the threshold value. That is, when the evaluation value is equal to or less than the threshold value, it is determined that the polarization state of the measurement light LS is appropriate, and the polarization adjustment is continued until it is determined that the polarization state of the measurement light LS is appropriate.

[0119] In some embodiments, the main controller 211 can also monitor the interference strength during polarization adjustment.

[0120] The data processing unit 300 according to the embodiment judges whether or not the image formed by the image forming unit 230 or the data processing unit 300 is an analysis error image including a predetermined analysis error factor. This makes it possible to automatically judge whether or not re-imaging (re-acquisition, re-measurement) is necessary based on a certain judgment criterion.

[0121] 4 to 11 are explanatory diagrams of the data processing unit 300 according to the embodiment. FIG. 4 shows a block diagram of a configuration example of the data processing unit 300 according to the embodiment. FIG. 5 shows a schematic diagram for explaining the operation of the data processing unit 300 of FIG. 4. FIG. 6 shows a configuration example of the nipple bleeding (DH) judgment model 311 of FIG. 5. FIG. 7 shows a schematic diagram for explaining machine learning for the DH judgment model 311 by the first learning unit 410 of FIG. 4. FIG. 8 shows a configuration example of the nipple bleeding (DH) region detection model 321 of FIG. 5. FIG. 9 shows a schematic diagram for explaining machine learning for the DH region detection model 321 by the second learning unit 420 of FIG. 4. FIGS. 10 and 11 show schematic diagrams for explaining the operation of the analysis unit 360 of FIG. 5.

[0122] In addition to the above data processing, the data processing unit 300 is capable of detecting a disc hemorrhage (DH) region in a fundus image (e.g., the above-mentioned captured image) of the subject's eye E acquired using the fundus camera unit 2, and generating analysis information for the detected DH region. Such a data processing unit 300 includes a determiner 310, a detector 320, a projection image forming unit 340, an alignment unit 350, an analysis unit 360, and a learning unit 400. The data processing unit 300 does not necessarily have to include the learning unit 400.

[0123] As shown in Fig. 5, the determiner 310 determines the presence or absence of DH for the fundus image IMG0 acquired by the fundus camera unit 2, using a DH determination model 311 obtained by machine learning. The detector 320 detects the DH region in the fundus image IMG0 determined to have DH by the determiner 310, using a DH region detection model 321 obtained by machine learning. The projection image forming unit 340 forms a projection image based on OCT data obtained by OCT measurement performed using the OCT unit 100 on the subject's eye E from which the fundus image IMG0 was acquired. The alignment unit 350 aligns the fundus image IMG0 from which the DH region is detected with the projection image as OCT data.

[0124] The analysis unit 360 generates position information, shape information, and occurrence information of the DH region by analyzing the fundus image that has been aligned with the projection image (i.e., OCT data defined in the OCT coordinate system) by the alignment unit 350. Such an analysis unit 360 includes a position information generation unit 361, a shape information generation unit 362, and a occurrence information generation unit 363. The position information generation unit 361 generates position information of the DH region. The shape information generation unit 362 generates shape information of the DH region. The occurrence information generation unit 363 generates occurrence information of the DH region.

[0125] The learning unit 400 generates the DH judgment model 311 and the DH area detection model 321 by executing machine learning. The learning unit 400 includes a first learning unit 410 and a second learning unit 420. The first learning unit 410 generates the DH judgment model 311 by executing machine learning. The second learning unit 420 generates the DH area detection model 321 by executing machine learning.

[0126] (judgment device) The determiner 310 determines whether or not a DH is present in the fundus image IMG0 by using a DH determination model 311, which is a trained model obtained in advance by machine learning.

[0127] The trained model according to the embodiment is used in a computer (processor) having a CPU and a memory. The function of the determiner 310 is realized by, for example, a convolutional neural network (CNN). That is, the CPU operates in accordance with an instruction from the trained model stored in the memory to perform calculations based on trained weighting coefficients and response functions in the CNN for pixel values ​​of a fundus image input to a convolutional layer 314 of a feature extraction unit 312 described later, and to output a determination result (classification result) from a classifier 313 described later. The determiner 310 having such a configuration can extract a local correlation pattern while gradually lowering the resolution of the fundus image, and output a determination result based on the extracted correlation pattern.

[0128] 6, the DH judgment model 311 (judgment device 310) includes a feature extraction unit 312 and a classifier 313. The feature extraction unit 312 extracts the feature of the input fundus image IMG0 by repeating feature extraction and downsampling (filtering) for each predetermined image region. The classifier 313 generates output information indicating the presence or absence of a DH based on the feature extracted by the feature extraction unit 312, and outputs information Out1 (DH present / DH absent) indicating whether or not the fundus image IMG0 has a DH based on the generated output information.

[0129] The feature extraction unit 312 includes a plurality of units in which units including a convolution layer and a pooling layer are connected in multiple stages. In each unit, the output of the convolution layer is connected to the input of the pooling layer. The pixel value of the corresponding pixel in the fundus image is input to the input of the first stage convolution layer. The input of the subsequent stage convolution layer is connected to the output of the previous stage pooling layer.

[0130] 6, the feature extraction unit 312 includes two units connected in two stages. That is, in the feature extraction unit 312, a unit including a convolution layer 314 and a pooling layer 315 is connected to a unit including a convolution layer 316 and a pooling layer 317 in the subsequent stage. The output of the pooling layer 315 is connected to the input of the convolution layer 316.

[0131] The classifier 313 includes fully connected layers 318 and 319, and the output of the fully connected layer 318 is connected to the input of the fully connected layer 319.

[0132] In the feature extraction unit 312 and the classifier 313, learned weighting coefficients (learning parameters) are assigned between the neurons of the two connected layers. The weighting coefficients are updated by executing machine learning. Each neuron performs a calculation using a response function on the calculation result that takes into account the weighting coefficients from one or more input neurons, and outputs the obtained calculation result to the neuron in the next stage.

[0133] As shown in FIG. 7, the first learning unit 410 performs known machine learning on an existing initial model using a fundus image LIMG labeled with a label indicating the presence or absence of a DH as training data to generate a DH judgment model 311 in which the weighting coefficient is updated for the initial model. The existing weighting coefficient is updated by machine learning using a fundus image LIMG labeled with a label indicating the presence or absence of a DH as training data. The training data may be a plurality of pairs of data, each pair consisting of a fundus image and label information indicating whether or not a DH is depicted in the fundus image. The presence or absence of a DH is determined in advance for each fundus image based on a doctor's judgment result, and is attached as a label of the fundus image. The first learning unit 410 can further train the DH judgment model 311 by performing machine learning using another training data on the generated DH judgment model 311.

[0134] That is, the first learning unit 410 performs known supervised learning (machine learning) using a plurality of fundus images LIMG as training data to generate a DH determination model 311 that is a trained model for determining (classifying) whether or not a DH is depicted in a fundus image of the subject's eye E. The machine learning may be unsupervised learning or reinforcement learning. In some embodiments, the weighting coefficient is updated by transfer learning.

[0135] The DH judgment model 311 (feature extraction unit 312) may have a known layer structure such as VGG16, VGG19, InceptionV3, ResNet18, ResNet50, Xception, DenseNet201, etc. The classifier 313 may have a known configuration such as Random Forest or Support Vector Machine (SVM). For example, the DH judgment model 311 according to the first embodiment is constructed by DenseNet201.

[0136] (Detector) The detector 320 detects the DH region in the fundus image IMG0 in which the determiner 310 has determined that there is a DH, using a DH region detection model 321, which is a trained model obtained in advance by machine learning.

[0137] The trained model according to the embodiment is used in a computer (processor) having a CPU and a memory. The function of the detector 320 is realized by, for example, a CNN. That is, the CPU operates in accordance with an instruction from the trained model stored in the memory to perform calculations based on trained weighting coefficients and response functions in the CNN for pixel values ​​of a fundus image input to a convolution layer 324 of a feature extraction unit 322 described later, and to output an image in which the detected DH region is depicted from a restoration unit 323 described later. The detector 320 having such a configuration can extract a local correlation pattern while gradually reducing the resolution of the fundus image by a convolution operation, and output a detection image corresponding to the fundus image by a deconvolution operation from the extracted correlation pattern.

[0138] As shown in FIG. 8, the DH region detection model 321 (detector 320) includes a feature extraction unit 322 and a restoration unit 323. The feature extraction unit 322 extracts the feature of the fundus image IMG0 by repeating extraction of the feature and downsampling (filtering) for each predetermined image region. The restoration unit 323 repeats restoration of the image corresponding to the feature extracted by the feature extraction unit 322 for each predetermined image region, and outputs a detection image Out2 (see FIG. 5) corresponding to the feature. The detected DH region is depicted in the detection image Out2. In FIG. 8, the detection image Out2 is an output image (DH region detection image) DIMG in which the DH region DH0 is depicted.

[0139] The feature extraction unit 322, like the feature extraction unit 312, includes a plurality of units in which units including a convolution layer and a pooling layer are connected in multiple stages. In each unit, the output of the convolution layer is connected to the input of the pooling layer. The pixel value of the corresponding pixel in the fundus image is input to the input of the first stage convolution layer. The input of the subsequent stage convolution layer is connected to the output of the previous stage pooling layer.

[0140] 8, feature extraction unit 322 includes two units connected in two stages. That is, in feature extraction unit 322, a unit including convolution layer 324 and pooling layer 325 is connected to a unit including convolution layer 326 and pooling layer 327 in the subsequent stage. The output of pooling layer 325 is connected to the input of convolution layer 326.

[0141] The restoration unit 323 includes one or more units in which units including a deconvolution layer and a pooling layer are connected in one or more stages. In each unit, the input of the pooling layer is connected to the output of the deconvolution layer. The input of the deconvolution layer in the first stage is connected to the output of the pooling layer of the feature extraction unit 322. The input of the deconvolution layer in the subsequent stage is connected to the output of the pooling layer in the previous stage.

[0142] 8, the restoration unit 323 includes a deconvolution layer 328 corresponding to the pooling layer 327 of the feature extraction unit 322, and one unit. That is, in the restoration unit 323, a unit including a pooling layer 329 and a deconvolution layer 330 is connected to the rear stage of the deconvolution layer 328 corresponding to the pooling layer 327. The output of the pooling layer 329 is connected to the input of the deconvolution layer 330.

[0143] As in the DH judgment model 311, learned weighting coefficients (learning parameters) are assigned between the neurons of the two connected layers in the feature extraction unit 322 and the restoration unit 323. Each neuron performs a calculation using a response function on the calculation result that takes into account the weighting coefficients from one or more input neurons, and outputs the obtained calculation result to the neuron in the next stage.

[0144] As shown in FIG. 9, the second learning unit 420 performs known machine learning using a group of images, each of which is a fundus image IMG1 and a DH region image TG1 representing the DH region depicted in the fundus image IMG1, as training data for an existing initial model, to generate a DH region detection model 321 in which the weighting coefficient is updated for the initial model. The existing weighting coefficient is updated by machine learning using a group of images, each of which is a pair of pairs, as training data. The DH region image is generated for each fundus image based on the result of a doctor's judgment in advance. For example, the doctor uses the operation unit 240B to specify the boundary of the DH region in the fundus image IMG1 to generate the DH region image TG1. The second learning unit 420 can further train the DH region detection model 321 by performing machine learning using another training data for the generated DH region detection model 321.

[0145] That is, the second learning unit 420 generates a DH region detection model 321, which is a trained model for detecting the DH region in a fundus image, by performing known supervised learning (machine learning) using a group of multiple pairs of images (fundus image IMG1, DH region image TG1) as training data. The machine learning may be unsupervised learning or reinforcement learning. In some embodiments, the weighting coefficient is updated by transfer learning.

[0146] Here, the teacher data (second teacher data) used by the second learning unit 420 to train the DH region detection model 321 includes at least a part of the teacher data (first teacher data) used by the first learning unit 410 to train the DH judgment model 311. In other words, the teacher data used by the second learning unit 420 to train the DH region detection model 321 includes a fundus image included in the teacher data used by the first learning unit 410 to train the DH judgment model 311. This makes it possible to improve the detection accuracy of the DH region, as described below, without increasing the amount of teacher data, compared to the case where the DH region is detected using one trained model.

[0147] The DH region detection model 321 (detector 320) may have a known layer structure such as VGG16, VGG19, InceptionV3, ResNet18, ResNet50, Xception, U-Net, ResUnet, ResUnet++, etc. For example, the DH region detection model 321 according to the first embodiment is constructed by ResUnet++ (Debesh Jha et al., “ResUnet++: An Advanced Architecture for Medical Image Segmentation”, 21st IEEE International Symposium on Multimedia, December 2019).

[0148] (Projection image forming section) As described above, the projection image forming section 340 forms a projection image of the fundus Ef of the subject's eye E. For example, the projection image forming section 340 forms a projection image of the fundus Ef by projecting volume data of the fundus Ef of the subject's eye E in the z direction.

[0149] (Alignment part) The alignment unit 350 aligns the fundus image in which the DH region is detected with the projection image formed by the projection image forming unit 340. The fundus image aligned by the alignment unit 350 is a fundus image in which the DH region detected by the detector 320 is depicted. This makes it possible to associate the position of each pixel in the fundus image aligned by the alignment unit 350 with a position in the OCT coordinate system.

[0150] In some embodiments, the alignment section 350 performs alignment so that the position corresponding to the optical axis of the OCT unit 100 (interference optical system) in the projection image coincides with the position corresponding to the optical axis of the fundus camera unit 2 (photography optical system) in the fundus image.

[0151] In some embodiments, the alignment unit 350 identifies a characteristic portion in the projection image and also identifies a characteristic portion in the fundus image, and aligns the positions of the characteristic portions in both images so that the positions of the characteristic portions match. In this case, for example, the area corresponding to the optic disc in the fundus image may be automatically or manually corrected so that it matches the area corresponding to the optic disc in the projection image. Since the OCT data makes it possible to identify the characteristic portion from the tomographic structure of the fundus with high accuracy, it becomes possible to detect with high accuracy the boundary of the area corresponding to the optic disc, which is difficult to distinguish from the fundus image alone.

[0152] In some embodiments, the registration unit 350 performs affine transformation on at least one of the fundus image and the projection image when registering the fundus image and the projection image. For example, the projection image is affine transformed together with the coordinate system.

[0153] The OCT data obtained by the OCT measurement is highly reproducible and is capable of identifying the position and shape of the optic disc with high accuracy. As a result, the analysis results of the DH region by the analysis unit 360 described later can also be obtained with high accuracy and high reproducibility.

[0154] (Analysis Department) The analysis unit 360 generates position information, shape information, and occurrence information of the DH region by analyzing the fundus image aligned with the projection image by the alignment unit 350. Specifically, the analysis unit 360 generates position information and the like of the DH region detected in the fundus image to be analyzed, using position information of the coordinate system defined in the projection image. Here, the analysis target may be a fundus image in which the region corresponding to the optic disc and the like has been corrected by alignment with the projection image.

[0155] The analysis unit 360 can generate statistical information on the position information, shape information, and occurrence information of the DH region. The statistical information includes statistical information for a plurality of examinee's eyes, statistical information for the same examinee's eye (or examinee), etc. An example of the statistical information is a histogram.

[0156] In addition, the analysis unit 360 can manage the position information, shape information, and occurrence information of the DH region, or statistical information thereof, in chronological order in association with the date of detection of the DH region or the date of occurrence of the DH region estimated by a doctor.

[0157] The position information generating unit 361 generates position information that indicates the position or direction of the DH region based on a reference position in the fundus image. For example, the reference position is identified by analyzing the fundus image or the projection image. Examples of the reference position include the center of the optic disc and the center of gravity of the optic disc.

[0158] In some embodiments, the position information includes information indicating a representative position of the detected DH region with respect to the center of the optic disc (information indicating a relative position of the DH region with respect to the center of the optic disc). Examples of the representative position of the DH region include the center position of the DH region, the center of gravity position, the position in the DH region closest to the reference position, and the position in the DH region farthest from the reference position.

[0159] In some embodiments, the position information includes information indicating the direction of the detected representative position of the DH region relative to the center of the optic disc (the angle with respect to a reference direction passing through the center of the optic disc).

[0160] In some embodiments, the location information includes information indicating whether the representative location of the detected DH region is within the optic disc region or outside the optic disc region. In some embodiments, the location information includes information indicating whether the representative location of the detected DH region is within the optic disc rim or outside the optic disc rim in the optic disc region.

[0161] The position information generating unit 361 is capable of generating at least one of the above-mentioned multiple pieces of position information.

[0162] The shape information generating unit 362 generates shape information representing the shape of the DH region detected in the fundus image. At this time, the shape information generating unit 362 can quantitatively generate shape information of the DH region in the fundus image using the shape of the DH region in the OCT coordinate system and size information per pixel (pixel spacing) in the OCT coordinate system.

[0163] In some embodiments, the shape information includes information indicating the ellipticity, area, etc. of the DH region. For example, the ellipticity is obtained by approximating the DH region to an ellipse and calculating the ratio of the minor axis to the major axis of the identified ellipse. For example, the area of ​​the DH region is quantitatively obtained by identifying the boundary of the DH region, counting the number of pixels within the identified boundary, and using size information per pixel in the OCT coordinate system.

[0164] The shape information generating unit 362 is capable of generating at least one of the above-mentioned pieces of shape information.

[0165] The occurrence information generating unit 363 generates occurrence information of the detected DH region. For example, the occurrence information generating unit 363 (analysis unit 360) generates the occurrence information of the DH region based on the detection date of the DH region that occurred in the past for a plurality of test eyes or a test eye to be analyzed, or the occurrence date of the DH region estimated by a doctor.

[0166] In some embodiments, the occurrence information includes at least one of a frequency of occurrence of the DH region and an interval between occurrences of the DH region.

[0167] The occurrence frequency includes the occurrence frequency of the DH region in multiple test eyes or the same test eye, the frequency for each occurrence position of the DH region in multiple test eyes or the same test eye, etc. The frequency for each occurrence position includes the frequency for each block radially divided with respect to a reference position, etc. In this case, the occurrence information is obtained for each occurrence position of the DH region based on the above position information, or for each block radially divided with respect to a reference position specified based on the above position information.

[0168] The occurrence interval is calculated as the interval between the date of detection of the current DH region or the occurrence of the current DH region estimated by a doctor and the date of detection of the DH region most recently recorded in the past or the date of occurrence of the DH region estimated by a doctor.

[0169] Fig. 10 shows an example of position information generated by the position information generating unit 361 according to the first embodiment. Fig. 10 shows a histogram of the positions of the DH regions in a plurality of test eyes. In Fig. 10, the T (ear side) direction is 0 degrees and the N (nose side) direction is 180 degrees with respect to the optic disc as the center.

[0170] As shown in Fig. 10, it can be seen that the DH region is likely to occur in the direction of T (ear side) with respect to the center of the optic disc. Although Fig. 10 shows a histogram of the positions of the DH regions for a plurality of test eyes, the position information generating unit 361 may generate a histogram of the positions of the DH region for the same test eye as shown in Fig. 10.

[0171] 11 is a schematic diagram showing an example of shape information generated by the shape information generating unit 362 according to the first embodiment. FIG. 11 shows a histogram of the areas of the DH regions in a plurality of test eyes. In FIG. 11, the vertical axis shows the frequency, and the horizontal axis shows the area [mm 2 ].

[0172] The distribution of the area of ​​the DH region can be grasped as shown in Fig. 11. Although Fig. 11 shows a histogram of the area of ​​the DH region for a plurality of test eyes, the shape information generating unit 362 may generate a histogram of the area of ​​the DH region for the same test eye as shown in Fig. 11.

[0173] In some embodiments, the analysis unit 360 aligns the fundus image in which the DH region is detected by the detector 320 with a layer thickness map of a predetermined layer region of the fundus Ef obtained by a known method using OCT measurement, and generates a composite image by superimposing the fundus image and the layer thickness map. The predetermined layer region includes the ganglion cell layer (GCL), the retinal nerve fiber layer (RNFL), the peripapillary retinal nerve fiber layer (cpRNFL), and the GCC (=NFL+GCL+IPL (inner plexiform layer)). In this case, the control unit 210 (main control unit 211) can display the generated composite image on the display unit 240A described later.

[0174] In some embodiments, the analysis unit 360 estimates the time (or date) of occurrence of the DH region based on the color of the DH region detected by the detector 320. For example, the analysis unit 360 estimates the time of occurrence of the DH region based on the color of the detected DH region with reference to a reference color of the DH region. The color of the DH region is mainly identified by extracting the red component in the fundus image. In some embodiments, the analysis unit 360 estimates the occurrence interval of the DH region based on the difference between the color of the DH region detected last time and this time (e.g., the pixel value of the red component). In some embodiments, the analysis unit 360 estimates the time of occurrence of the DH region based on the color of the DH region detected on the last examination date and the occurrence interval of the DH region.

[0175] The data processing unit 300 that functions as described above includes, for example, the above-mentioned processor, RAM, ROM, a hard disk drive, a circuit board, etc. A computer program that causes the processor to execute the above-mentioned functions is stored in advance in a storage device such as a hard disk drive.

[0176] (User Interface) As shown in FIG. 3, the user interface 240 includes a display unit 240A and an operation unit 240B. The display unit 240A includes the display device of the arithmetic control unit 200 and the display device 3 described above. The operation unit 240B includes the operation device of the arithmetic control unit 200 described above. The operation unit 240B may include various buttons and keys provided on the housing of the ophthalmic apparatus 1 or on the outside. For example, when the fundus camera unit 2 has a housing similar to that of a conventional fundus camera, the operation unit 240B may include a joystick, an operation panel, etc. provided on the housing. In addition, the display unit 240A may include various display devices such as a touch panel provided on the housing of the fundus camera unit 2.

[0177] It is not necessary for the display unit 240A and the operation unit 240B to be configured as separate devices. For example, it is possible to use a device in which the display function and the operation function are integrated, such as a touch panel. In that case, the operation unit 240B is configured to include this touch panel and a computer program. The operation content on the operation unit 240B is input to the control unit 210 as an electric signal. Also, operations and information input may be performed using a graphical user interface (GUI) displayed on the display unit 240A and the operation unit 240B.

[0178] The data processing unit 300 is an example of an "ophthalmologic information processing device" according to the embodiment. The projection image is an example of "OCT data" according to the embodiment. The fundus camera unit 2 (imaging optical system 30) is an example of an "imaging section" according to the embodiment. The OCT unit 100, the image forming section 230 (and / or the data processing unit 300) are an example of an "OCT section" according to the embodiment.

[0179] [Operation] The operation of the ophthalmologic apparatus 1 according to the first embodiment will be described.

[0180] Fig. 12 shows an example of the operation of the ophthalmic apparatus 1 according to the first embodiment. Fig. 12 shows a flowchart of the example of the operation of the ophthalmic apparatus 1 according to the first embodiment. The storage unit 212 stores a computer program for realizing the processing shown in Fig. 12. The main control unit 211 operates according to the computer program to execute the processing shown in Fig. 12.

[0181] In the following, it is assumed that the alignment between the subject's eye E and the optical system has already been completed. Also, it is assumed that the DH judgment model 311 has already been machine-learned by the first learning unit 410, and the DH region detection model 321 has already been machine-learned by the second learning unit 420.

[0182] (S1: Acquire color fundus image) First, the main controller 211 (controller 210) controls the fundus camera unit 2 (imaging optical system 30) to photograph the fundus Ef of the subject's eye E. In this way, the main controller 211 can obtain a color fundus image of the subject's eye E.

[0183] (S2: Determine whether or not DH is present) Next, the main controller 211 controls the determiner 310 to determine whether or not the DH is present in the color fundus image acquired in step S1.

[0184] The determiner 310 uses the DH determination model 311 to determine (classify) whether or not a color fundus image has a DH, as described above.

[0185] (S3:DH available?) When it is determined in step S2 that the color fundus image contains a DH (S3: Y), the operation of the ophthalmologic apparatus 1 proceeds to step S4.

[0186] When it is determined in step S2 that the color fundus image does not contain a DH (S3: N), the operation of the ophthalmologic apparatus 1 ends (END).

[0187] (S4: DH region detected) When it is determined in step S2 that the color fundus image has a DH (S3: Y), the main controller 211 controls the detector 320 to detect the DH region in the color fundus image determined to have a DH in step S2.

[0188] The detector 320 uses the DH region detection model 321 as described above to detect the DH region in the color fundus image determined to have a DH in step S2.

[0189] (S5: Alignment) Next, the main control unit 211 controls the alignment unit 350 to perform alignment between the fundus image in which the DH region was detected in step S4 and the projection image formed based on the OCT data obtained by performing OCT measurement on the test eye E.

[0190] In some embodiments, after the DH region is detected in the fundus image in step S4, the main controller 211 controls the OCT unit 100 to perform OCT measurement, controls the image forming unit 230 to form an OCT image, and controls the projection image forming unit 340 to form a projection image.

[0191] In some embodiments, OCT measurement is performed on the subject's eye E in advance, and a projection image of the subject's eye E is formed by the projection image forming unit 340.

[0192] (S6: Generate location information) Next, the main controller 211 controls the position information generator 361 to generate position information of the DH region in the fundus image aligned in step S5.

[0193] For example, the position information generating unit 361 obtains the relative position between the center of the optic disc and the representative position of the DH region. For example, the position information generating unit 361 obtains the direction of the representative position of the DH region with respect to the center of the optic disc, and generates position information as shown in FIG.

[0194] (S7: Generate shape information) Next, the main controller 211 controls the shape information generator 362 to generate shape information of the DH region in the fundus image aligned in step S5.

[0195] For example, the shape information generating unit 362 obtains the area of ​​the DH region as described above, and generates shape information as shown in FIG.

[0196] (S8: Generate occurrence information) Next, the main control unit 211 controls the generated information generating unit 363 to generate generated information of the DH area detected in step S4.

[0197] For example, the occurrence information generating unit 363 generates occurrence information including the occurrence frequency of the DH area and the occurrence interval of the DH area, as described above.

[0198] This is the end of the operation of the ophthalmologic apparatus 1.

[0199] Here, the detection accuracy of the DH region by the ophthalmologic apparatus 1 according to the first embodiment will be described.

[0200] Fig. 13 is an explanatory diagram of the detection accuracy of the DH region by the ophthalmologic apparatus 1 according to the first embodiment. Fig. 13 is a schematic diagram showing the detection accuracy of the DH region by the ophthalmologic apparatus 1 according to the first embodiment in comparison with a comparative example of the first embodiment. In the comparative example of the first embodiment, one trained model (ResUnet++) is generated by machine learning, and the DH region is directly detected from a fundus image using the generated trained model.

[0201] In FIG. 13, mIOU (mean intersection over union) is used as an index representing the detection accuracy of the DH region. mIOU corresponds to the average value of IOU calculated for each image. Here, as expressed by formula (1), IOU represents the ratio of the number of pixels in the true positive region (TP) to the sum of the number of pixels in the positive region (P) and the number of pixels in the true region (T) in the fundus image, which is the input image, and the detection image (DH region detection image), which is the output image. In other words, IOU is an index that indicates that the closer to 1, the higher the detection accuracy.

[0202]

number

[0203] In this embodiment, the positive area corresponds to the area detected as the DH area in the detection image (output image). The true area corresponds to the true DH area in the fundus image. The true positive area corresponds to the area (correct area) where the DH area detected in the detection image and the true DH area in the fundus image match.

[0204] 13 shows mIOU (average value of IOU for 146×2 images) by the configuration according to the comparative example and mIOU by the configuration according to the first embodiment for 146 images with DH and 146 images without DH. That is, in the comparative example, when a DH area is detected using one trained model as described above, mIOU=0.573. In contrast, when a DH area is detected using the DH judgment model 311 and the DH area detection model 321 in the first embodiment, mIOU=0.611.

[0205] Therefore, FIG. 13 shows that the configuration according to the first embodiment can detect the DH region with high accuracy compared to the comparative example.

[0206] As described above, according to the first embodiment, it is possible to detect the DH region with high reproducibility and high accuracy compared to the case where the DH region is detected from the fundus image using one trained model. In addition, the fundus image in which the DH region is detected is aligned with the projection image, and the position information, shape information, and generation information of the DH region are generated using OCT data, so that the detected DH region can be quantitatively evaluated with high accuracy.

[0207] [Second embodiment] In the first embodiment, a case has been described in which a DH region in a fundus image is detected by an ophthalmologic apparatus to which an ophthalmologic information processing apparatus according to an embodiment has been applied, but the configuration according to an embodiment is not limited to this.

[0208] In the second embodiment, the ophthalmologic information processing apparatus according to the embodiment can execute the above-mentioned DH region detection process on fundus images or OCT data (projection images) acquired by one or more external devices. Hereinafter, the second embodiment will be described, focusing on the differences from the first embodiment.

[0209] FIG. 14 shows a block diagram of a first configuration example of an ophthalmologic system according to the second embodiment.

[0210] The ophthalmologic system 500 according to the first configuration example includes ophthalmologic apparatuses 5101 to 510. N (N is an integer of 2 or more) and an ophthalmologic information-processing device 520. The ophthalmologic information-processing device 520 is connected to the ophthalmologic devices 5101 to 510 via a network 530. N The network 530 may be a wired or wireless network (LAN, WAN).

[0211] The ophthalmological information processing device 520 includes ophthalmological devices 5101 to 510 N It is possible to communicate with either

[0212] In some embodiments, the ophthalmic devices 5101-510 NAny of the above transmits a request to the ophthalmologic information-processing device 520, and the ophthalmologic device that has accepted the request transmits data to the ophthalmologic information-processing device 520. The transmitted data includes image data of the fundus image of the subject's eye E and the above-mentioned OCT data or image data of the projection image of the subject's eye E.

[0213] In some embodiments, the ophthalmological information processing device 520 includes the ophthalmological devices 5101 to 510 N and receives data from an ophthalmic apparatus that has approved the request. The received data includes image data of a fundus image of the subject's eye E and the above-mentioned OCT data or image data of a projection image of the subject's eye E.

[0214] FIG. 15 shows a block diagram of a second configuration example of an ophthalmologic system according to the second embodiment.

[0215] The ophthalmology system 500 according to the second configuration example includes an ophthalmology apparatus 510 and an ophthalmology information processing device 520. The ophthalmology information processing device 520 is connected to the ophthalmology apparatus 510 via a predetermined communication path. In some embodiments, the ophthalmology information processing device 520 is connected peer-to-peer to the ophthalmology apparatus 510 via a network.

[0216] The ophthalmological information-processing device 520 is capable of communicating with the ophthalmological device 510 .

[0217] In some embodiments, the ophthalmic apparatus 510 transmits a request to the ophthalmic information-processing device 520, and when the request is approved, the ophthalmic apparatus 510 transmits data to the ophthalmic information-processing device 520. The transmitted data includes image data of a fundus image of the subject's eye E and the above-mentioned OCT data or image data of a projection image of the subject's eye E.

[0218] In some embodiments, the ophthalmological information processing device 520 transmits a request to the ophthalmological device 510 and receives data from the ophthalmological device 510 that has approved the request. The received data includes image data of a fundus image of the subject's eye E and the above-mentioned OCT data or image data of a projection image of the subject's eye E.

[0219] Ophthalmology equipment 5101~510 N 1 to 3. In the second embodiment, among the functions of the ophthalmological apparatus 1 shown in FIGS. 1 to 3, a part of the functions of the data processing unit 300 is realized by an ophthalmological information-processing device 520.

[0220] Fig. 16 is a block diagram showing an example of the configuration of an ophthalmologic information-processing device 520 according to the second embodiment. In Fig. 16, the same components as those in Fig. 4 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0221] The ophthalmological information-processing device 520 includes a communication unit 521, a determiner 310, a detector 320, a projection image forming unit 340, an analysis unit 360, a learning unit 400, and a control unit 522. The ophthalmological information-processing device 520 does not necessarily include the learning unit 400.

[0222] The communication unit 521 is connected to the ophthalmic devices 5101 to 510 N Or, the communication unit 521 performs a communication interface process with the ophthalmic device 510. N Or, communication is performed between the ophthalmic device 510 and the ophthalmic device 5101 to 510 according to a predetermined communication protocol. N Alternatively, the communication unit 521 receives communication data including image data of a fundus image and the above-mentioned OCT data or image data of a projection image of the subject's eye E from the ophthalmic device 510. The communication unit 521 receives the image data included in the received communication data, thereby acquiring the fundus image and OCT data of the subject's eye obtained by the ophthalmic device.

[0223] The control unit 522, like the control unit 210, includes a processor and controls each unit of the ophthalmologic information-processing device 520. The control unit 522, like the control unit 210, includes a main control unit and a storage unit.

[0224] Furthermore, the control unit 522, as a display control unit, can perform display control of a display unit 540 connected to the outside of the ophthalmologic information-processing device 520. The display unit 540 has the same functions as the display unit 240A.

[0225] The operation of the ophthalmologic information-processing device 520 is similar to the flow shown in FIG. 12 except that the fundus image and the projection image (OCT data) are acquired from the outside, and therefore a detailed description thereof will be omitted. N Alternatively, the ophthalmological information processing apparatus 520 determines the presence or absence of DH for a fundus image acquired from the ophthalmological apparatus 510 in the same manner as in the first embodiment, and detects a DH region for a fundus image determined to have a DH. N Alternatively, the OCT data acquired from the ophthalmologic device 510 is used to perform alignment with the fundus image, and analysis results such as the position information of the DH region are obtained.

[0226] The configuration of the ophthalmologic system 500 according to the second embodiment is not limited to the configurations shown in FIGS.

[0227] FIG. 17 shows an example of the configuration of an ophthalmologic system according to a first modified example of the second embodiment.

[0228] An ophthalmological system 500a according to a first modified example of the second embodiment includes an ophthalmological information-processing device 520, a fundus camera 550, and an OCT device 560. The ophthalmological information-processing device 520 is capable of communicating with each of the fundus camera 550 and the OCT device 560.

[0229] The fundus camera 550 has the function of the fundus camera unit 2 of the ophthalmic apparatus 1 according to the first embodiment (i.e., the function of acquiring a fundus image). The OCT apparatus 560 has some of the functions of the OCT unit 100, the image forming section 230, and the data processing section 300 of the ophthalmic apparatus 1 according to the first embodiment (i.e., the function of acquiring OCT data and forming an OCT image).

[0230] The ophthalmologic information processing device 520 acquires image data of the fundus image of the subject's eye E from the fundus camera 550, and acquires OCT data (or image data of a projection image) of the subject's eye E from the OCT device 560. The ophthalmologic information processing device 520 determines the presence or absence of a DH for the fundus image acquired from the fundus camera 550, as in the first embodiment, and detects a DH region for the fundus image determined to have a DH. Furthermore, the ophthalmologic information processing device 520 aligns the fundus image using the OCT data acquired from the OCT device 560, and obtains analysis results such as position information of the above-mentioned DH region.

[0231] FIG. 18 shows an example of the configuration of an ophthalmologic system according to a second modification of the second embodiment.

[0232] An ophthalmologic system 500b according to a second modification of the second embodiment includes a fundus camera 550 and an OCT device 560, and the OCT device 560 includes an ophthalmologic information processing unit 561. The OCT device 560 is capable of communicating with the fundus camera 550.

[0233] The fundus camera 550 has the function of the fundus camera unit 2 of the ophthalmic apparatus 1 according to the first embodiment (i.e., the function of acquiring a fundus image). The OCT device 560 has some of the functions of the OCT unit 100, the image forming section 230, and the data processing section 300 of the ophthalmic apparatus 1 according to the first embodiment (i.e., the function of acquiring OCT data and forming an OCT image). The ophthalmic information processing section 561 has the function of the ophthalmic information processing device 520.

[0234] The ophthalmologic information processing unit 561 acquires image data of the fundus image of the eye E from the fundus camera 550, and executes the analysis process by the analysis unit 360 using the OCT data (or image data of the projection image) of the eye E acquired by the OCT device 560. The ophthalmologic information processing unit 561 determines the presence or absence of a DH for the fundus image acquired from the fundus camera 550, as in the first embodiment, and detects a DH region for the fundus image determined to have a DH. Furthermore, the ophthalmologic information processing unit 561 performs alignment with the fundus image using the OCT data acquired by the OCT device 560, and obtains analysis results such as position information of the DH region.

[0235] [Effect] An ophthalmological information processing apparatus, an ophthalmological apparatus, an ophthalmological information processing method, and a program according to an embodiment will be described.

[0236] The ophthalmological information processing apparatus (data processing unit 300, ophthalmological information processing apparatus 520, ophthalmological information processing unit 561) according to the embodiment includes a determiner (310) and a detector (320). The determiner determines the presence or absence of a disk hemorrhage for a front image (fundus image IMG0) of the fundus (E) of the subject eye (E) using a disk hemorrhage determination model (311) obtained by machine learning using a plurality of front images (LIMG) of the fundus labeled with a label indicating the presence or absence of a disk hemorrhage as first teacher data. The detector detects a disk hemorrhage region depicted in the front image determined by the determiner to have a disk hemorrhage using a disk hemorrhage region detection model (322) obtained by machine learning using a plurality of pairs of images, each pair being a front image (IMG1) of the fundus and a disk hemorrhage region image (TG1) representing a disk hemorrhage region depicted in the front image, as second teacher data.

[0237] According to this aspect, the determiner determines the presence or absence of a nipple hemorrhage in a front image of the fundus of the test eye, and detects a nipple hemorrhage area in the front image in which the detector determines that a nipple hemorrhage is present. At this time, the nipple hemorrhage area detection model in the detector is machine-learned using only fundus images in which a nipple hemorrhage area is present, and the learning parameters are updated to be specialized for detecting the nipple hemorrhage area. As a result, the detection accuracy of the nipple hemorrhage area can be improved. Therefore, compared to the case where a nipple hemorrhage area is detected in a fundus image of the test eye using one trained model, it is possible to detect a nipple hemorrhage area in a fundus image with higher reproducibility and higher accuracy.

[0238] In the ophthalmologic information processing apparatus according to the embodiment, the second teacher data includes a front image included in the first teacher data.

[0239] According to this aspect, the discriminator's nipple bleeding judgment model and the detector's nipple bleeding area detection model are each machine-learned using the same frontal image, making it possible to detect nipple bleeding areas in fundus images with higher reproducibility and accuracy without increasing the amount of training data.

[0240] The ophthalmologic information processing apparatus according to the embodiment includes a first learning unit (410) that generates a disc bleeding determination model by performing supervised machine learning using first training data.

[0241] According to this aspect, since the first learning unit is configured to learn a disc bleeding judgment model, it is possible to provide an ophthalmologic information processing device that can improve the accuracy of determining the presence or absence of disc bleeding.

[0242] The ophthalmologic information processing apparatus according to the embodiment includes a second learning unit (420) that generates a disc hemorrhage region detection model by performing supervised machine learning using second training data.

[0243] According to this aspect, since the second learning unit is configured to learn a nipple bleeding region detection model, it is possible to provide an ophthalmologic information processing device capable of improving the detection accuracy of a nipple bleeding region.

[0244] In the ophthalmologic information processing apparatus according to the embodiment, the front image of the fundus is a color front image.

[0245] According to this aspect, by using a front image with a larger amount of information, it is possible to further improve the determination accuracy by the determiner and the detection accuracy by the detector.

[0246] The ophthalmologic information processing device of the embodiment includes an analysis unit (360) that generates at least one of position information representing the position of a papilla hemorrhage area in the front image of the fundus of the test eye, shape information representing the shape of the papilla hemorrhage area, and occurrence information representing the occurrence status of the papilla hemorrhage area by analyzing a front image of the fundus of the test eye.

[0247] According to this aspect, the area of ​​nipple bleeding detected with higher reproducibility and accuracy is analyzed, so that it is possible to obtain analysis results of the area of ​​nipple bleeding in the fundus image of the test eye with high reproducibility and accuracy.

[0248] In the ophthalmologic information processing apparatus according to the embodiment, the position information includes information indicating the direction of the representative position of the disc bleeding region relative to the reference position.

[0249] According to this aspect, it becomes easier to grasp the direction of the occurrence position of the nipple hemorrhage region relative to the reference position on the fundus, which contributes to identifying future symptoms of a disease and determining a future treatment plan based on the occurrence direction of the nipple hemorrhage region relative to the reference position.

[0250] In the ophthalmologic information processing apparatus according to the embodiment, the position information includes information indicating whether the representative position of the disc hemorrhage region is inside or outside the optic disc region.

[0251] According to this aspect, it becomes easier to know whether the area of ​​disk hemorrhage is inside or outside the optic disc region, which can contribute to identifying future symptoms of a disease and determining a future treatment plan depending on whether the area of ​​disk hemorrhage is inside or outside the optic disc region.

[0252] In the ophthalmologic information processing apparatus according to the embodiment, the position information includes information indicating whether the representative position is inside the optic disc peripheral portion or outside the optic disc peripheral portion.

[0253] According to this aspect, it becomes easier to know whether the area of ​​disk hemorrhage is inside or outside the optic disc peripheral area, which can contribute to identifying future symptoms of a disease and determining a future treatment plan depending on whether the area of ​​disk hemorrhage is inside or outside the optic disc peripheral area.

[0254] In the ophthalmologic information processing apparatus according to the embodiment, the shape information includes at least one of the ellipticity of the disc hemorrhage region and the area of ​​the disc hemorrhage region.

[0255] According to this embodiment, it becomes easier to grasp the shape and size of the nipple bleeding area, which can contribute to identifying future disease symptoms and determining future treatment plans based on the shape and size of the nipple bleeding area.

[0256] In the ophthalmologic information processing apparatus according to the embodiment, the occurrence information includes at least one of an occurrence frequency of a disc bleeding region and an occurrence interval of a disc bleeding region.

[0257] According to this embodiment, it becomes easier to grasp the occurrence status of the nipple bleeding region, which contributes to identifying future disease symptoms that are predicted based on the occurrence status of the nipple bleeding region and determining future treatment plans.

[0258] The ophthalmologic information processing device of the embodiment includes an alignment unit (350) that aligns OCT data of the test eye with a frontal image of the test eye in which the area of ​​papilla hemorrhage detected by the detector is depicted, and the analysis unit generates at least one of position information, shape information, and generation information using the OCT data that has been aligned with the frontal image by the alignment unit.

[0259] According to this embodiment, the nipple bleeding region in the front image is aligned with the OCT data, so that it is possible to identify the nipple bleeding region in the OCT coordinate system that defines the OCT data acquired with high accuracy and reproducibility. This makes it possible to quantitatively obtain the analysis result of the nipple bleeding region with high accuracy and high reproducibility.

[0260] An ophthalmic apparatus (1) according to the embodiment includes an imaging section (imaging optical system 30) that images the fundus of a subject's eye, and any one of the ophthalmic information-processing devices described above.

[0261] According to this aspect, it is possible to provide an ophthalmic apparatus capable of acquiring a front image of the test eye and detecting an area of ​​disc bleeding in the acquired front image with higher reproducibility and accuracy.

[0262] The ophthalmic apparatus (1) according to the embodiment includes an imaging section (imaging optical system 30) that images the fundus of the subject's eye, an OCT section (OCT unit 100, image forming section 230, part of data processing section 300) that acquires OCT data by performing optical coherence tomography on the subject's eye, and the ophthalmic information processing device described above.

[0263] According to this aspect, it is possible to provide an ophthalmologic apparatus capable of acquiring a front image and OCT data of a subject's eye and detecting a disc hemorrhage region in the acquired front image with high accuracy and reproducibility. In addition, it is possible to quantitatively obtain an analysis result of the detected disc hemorrhage region with high accuracy and high reproducibility by using the acquired OCT data.

[0264] The ophthalmologic information processing method according to the embodiment includes a determination step and a detection step. In the determination step, the presence or absence of a disk hemorrhage is determined for a front image of the fundus (Ef) of the subject's eye (E) using a disk hemorrhage determination model (311) obtained by machine learning using a plurality of front images of the fundus (LIMG) labeled with a label indicating the presence or absence of a disk hemorrhage as first teacher data. In the detection step, a disk hemorrhage region depicted in the front image determined to have a disk hemorrhage in the determination step is detected using a disk hemorrhage region detection model (322) obtained by machine learning using a plurality of pairs of images, each pair being a front image of the fundus (IMG1) and a disk hemorrhage region image (TG1) representing a disk hemorrhage region depicted in the front image, as second teacher data.

[0265] According to this aspect, the presence or absence of a nipple hemorrhage is determined for a front image of the fundus of the test eye in the determination step, and a nipple hemorrhage area is detected for the front image in which it is determined that a nipple hemorrhage is present in the detection step. At this time, the nipple hemorrhage area detection model in the detection step is machine-learned using only fundus images in which a nipple hemorrhage area is present, and the learning parameters are updated specifically for detecting the nipple hemorrhage area. As a result, the detection accuracy of the nipple hemorrhage area can be improved. Therefore, it is possible to detect a nipple hemorrhage area in a fundus image with higher reproducibility and higher accuracy than when a nipple hemorrhage area is detected in a fundus image of the test eye using one trained model.

[0266] In the ophthalmologic information processing method according to the embodiment, the second teacher data includes a front image included in the first teacher data.

[0267] According to this aspect, the nipple bleeding judgment model in the judgment step and the nipple bleeding area detection model in the detection step are each machine-learned using the same frontal image, making it possible to detect nipple bleeding areas in fundus images with higher reproducibility and accuracy without increasing the amount of training data.

[0268] The ophthalmologic information processing method according to the embodiment includes a first learning step of generating a disc bleeding determination model by performing supervised machine learning using first training data.

[0269] According to this aspect, since a disc bleeding judgment model is trained in the first learning step, it is possible to provide an ophthalmologic information processing method that can improve the accuracy of determining whether or not disc bleeding is present.

[0270] The ophthalmologic information processing method according to the embodiment includes a second learning step of generating a disc hemorrhage region detection model by performing supervised machine learning using second training data.

[0271] According to this aspect, since the disc bleeding region detection model is trained in the second learning step, it is possible to provide an ophthalmologic information processing method capable of improving the detection accuracy of the disc bleeding region.

[0272] In the ophthalmologic information processing method according to the embodiment, the front image of the fundus is a color front image.

[0273] According to this aspect, by using a front image with a larger amount of information, it is possible to further improve the determination accuracy in the determination step and the detection accuracy in the detection step.

[0274] An ophthalmologic information processing method according to an embodiment includes an analysis step of generating at least one of position information representing the position of a papilla hemorrhage area in the frontal image of the fundus of the test eye, shape information representing the shape of the papilla hemorrhage area, and occurrence information representing the occurrence status of the papilla hemorrhage area by analyzing a frontal image of the fundus of the test eye.

[0275] According to this aspect, the area of ​​nipple bleeding detected with higher reproducibility and accuracy is analyzed, so that it is possible to obtain analysis results of the area of ​​nipple bleeding in the fundus image of the test eye with high reproducibility and accuracy.

[0276] In the ophthalmologic information processing method according to the embodiment, the position information includes information indicating a direction of the representative position of the disc bleeding region relative to a reference position.

[0277] According to this aspect, it becomes easier to grasp the direction of the occurrence position of the nipple hemorrhage region relative to the reference position on the fundus, which contributes to identifying future symptoms of a disease and determining a future treatment plan based on the occurrence direction of the nipple hemorrhage region relative to the reference position.

[0278] In the ophthalmologic information processing method according to the embodiment, the position information includes information indicating whether the representative position of the disc hemorrhage region is inside or outside the optic disc region.

[0279] According to this aspect, it becomes easier to know whether the area of ​​disk hemorrhage is inside or outside the optic disc region, which can contribute to identifying future symptoms of a disease and determining a future treatment plan depending on whether the area of ​​disk hemorrhage is inside or outside the optic disc region.

[0280] In the ophthalmologic information processing method according to the embodiment, the position information includes information indicating whether the representative position is inside the optic disc periphery or outside the optic disc periphery.

[0281] According to this aspect, it becomes easier to know whether the area of ​​disk hemorrhage is inside or outside the optic disc peripheral area, which can contribute to identifying future symptoms of a disease and determining a future treatment plan depending on whether the area of ​​disk hemorrhage is inside or outside the optic disc peripheral area.

[0282] In the ophthalmologic information processing method according to the embodiment, the shape information includes at least one of the ellipticity of the disc hemorrhage region and the area of ​​the disc hemorrhage region.

[0283] According to this embodiment, it becomes easier to grasp the shape and size of the nipple bleeding area, which can contribute to identifying future disease symptoms and determining future treatment plans based on the shape and size of the nipple bleeding area.

[0284] In the ophthalmologic information processing method according to the embodiment, the occurrence information includes at least one of an occurrence frequency of a disc hemorrhage region and an occurrence interval of a disc hemorrhage region.

[0285] According to this embodiment, it becomes easier to grasp the occurrence status of the nipple bleeding region, which contributes to identifying future disease symptoms that are predicted based on the occurrence status of the nipple bleeding region and determining future treatment plans.

[0286] The ophthalmologic information processing of the embodiment includes an alignment step of aligning OCT data of the test eye with a frontal image of the test eye in which the disc hemorrhage area detected in the detection step is depicted, and the analysis step generates at least one of position information, shape information, and generation information using the OCT data aligned with the frontal image in the alignment step.

[0287] According to this embodiment, the nipple bleeding region in the front image is aligned with the OCT data, so that it is possible to identify the nipple bleeding region in the OCT coordinate system that defines the OCT data acquired with high accuracy and reproducibility. This makes it possible to quantitatively obtain the analysis result of the nipple bleeding region with high accuracy and high reproducibility.

[0288] A program according to the embodiment causes a computer to execute each step of any one of the ophthalmologic information processing methods described above.

[0289] According to this aspect, the presence or absence of a nipple hemorrhage is determined for a front image of the fundus of the test eye in the determination step, and a nipple hemorrhage area is detected for the front image in which it is determined that a nipple hemorrhage is present in the detection step. At this time, the nipple hemorrhage area detection model in the detection step is machine-learned using only fundus images in which a nipple hemorrhage area is present, and the learning parameters are updated specifically for detecting the nipple hemorrhage area. As a result, the detection accuracy of the nipple hemorrhage area can be improved. Therefore, it is possible to detect a nipple hemorrhage area in a fundus image with higher reproducibility and higher accuracy than when a nipple hemorrhage area is detected in a fundus image of the test eye using one trained model.

[0290] <Other> The embodiment described above is merely one example for carrying out the present invention. Anyone who wishes to carry out the present invention may make any modifications, omissions, additions, etc. within the scope of the gist of the present invention.

[0291] In some embodiments, a program is provided for causing a computer to execute the above-mentioned method for controlling an ophthalmic apparatus. Such a program can be stored in any non-transitory recording medium that can be read by a computer. As the recording medium, for example, a semiconductor memory, an optical disk, a magneto-optical disk (CD-ROM / DVD-RAM / DVD-ROM / MO, etc.), a magnetic storage medium (hard disk / floppy disk / ZIP, etc.), etc. can be used. In addition, the program can be transmitted and received via a network such as the Internet or a LAN. [Explanation of symbols]

[0292] 1, 510, 5101~510 N ophthalmology equipment 100 OCT units 210 Control section 211 Main control unit 230 Image forming section 300 Data Processing Unit 310 Judgment device 320 Detector 340 Projection Image Formation Department 350 Alignment section 360 Analysis Department 361 Location information generation unit 362 Shape information generation section 363 Information Generation Unit 400 Learning Department 410 First Study Section 420 Second Learning Section 500, 500a, 500b Ophthalmic Systems 520 Ophthalmology information processing device 561 Ophthalmology Information Processing Department E. Examined eye Ef fundus

Claims

1. a classifier that uses a disk hemorrhage determination model obtained by machine learning using a plurality of disk hemorrhage front images as first teacher data, the disk hemorrhage determination model being obtained by machine learning using a plurality of disk hemorrhage front images as first teacher data; a detector that detects a nipple hemorrhage region depicted in the front image, which is determined to have nipple hemorrhage by the determiner, using a nipple hemorrhage region detection model obtained by machine learning using a group of images, each of which is a pair of a front image of the fundus and a nipple hemorrhage region image showing a nipple hemorrhage region depicted in the front image, as second teacher data; an analysis unit that generates position information representing a position of the optic nerve hemorrhage region in the front image of the fundus of the subject's eye by analyzing the front image of the fundus of the subject's eye; Including, The ophthalmologic information processing apparatus, wherein the analysis unit generates, as the position information, a histogram of positions of disc hemorrhage regions in a plurality of test eyes.

2. The second teacher data includes a front image included in the first teacher data. The ophthalmological information processing apparatus according to claim 1 .

3. A first learning unit that generates the nipple bleeding determination model by performing supervised machine learning using the first training data.

3. An ophthalmologic information processing apparatus according to claim 1 or 2.

4. A second learning unit that generates the nipple bleeding region detection model by performing supervised machine learning using the second training data.

4. The ophthalmologic information processing apparatus according to claim 1, wherein the ophthalmologic information processing apparatus is a computer.

5. The en face image of the fundus is a color en face image.

5. The ophthalmological information processing apparatus according to claim 1, wherein the ophthalmological information processing apparatus is a computer.

6. a registration unit that performs registration between the OCT data of the subject's eye and a front image of the subject's eye in which the disc hemorrhage region is depicted and detected by the detector, The analysis unit generates the position information by using the OCT data that has been aligned with the front image by the alignment unit.

6. The ophthalmological information processing apparatus according to claim 1,

7. an imaging unit that images a fundus of the subject's eye; An ophthalmological information processing device according to any one of claims 1 to 6, 13. An ophthalmic device comprising:

8. an imaging unit that images a fundus of the subject's eye; an OCT unit that acquires OCT data of the subject's eye by performing optical coherence tomography on the subject's eye; The ophthalmological information processing apparatus according to claim 7 ; 13. An ophthalmic device comprising:

9. A determination step in which the processor determines the presence or absence of disc hemorrhage in the front image of the fundus of the subject eye using a disc hemorrhage determination model obtained by machine learning using a plurality of front images of the fundus labeled with a label indicating the presence or absence of disc hemorrhage as first teacher data; a detection step in which the processor detects a nipple hemorrhage region depicted in the front image determined to have nipple hemorrhage in the determination step by using a nipple hemorrhage region detection model obtained by machine learning using a group of images, each of which is a pair of a front image of the fundus and a nipple hemorrhage region image representing a nipple hemorrhage region depicted in the front image, as second teacher data; an analysis step of generating position information representing a position of the optic nerve hemorrhage region in the front image of the fundus of the subject eye by analyzing the front image of the fundus of the subject eye by the processor; Including, A method for operating an ophthalmologic information processing apparatus, wherein the analyzing step generates a histogram of positions of disc hemorrhage regions in a plurality of examinee's eyes as the position information.

10. The second teacher data includes a front image included in the first teacher data.

10. A method for operating an ophthalmologic information processing apparatus according to claim 9.

11. A first learning step in which the processor performs supervised machine learning using the first training data to generate the nipple bleeding determination model.

11. A method for operating an ophthalmologic information processing apparatus according to claim 9 or 10.

12. A second learning step in which the processor generates the nipple bleeding region detection model by performing supervised machine learning using the second training data. A method for operating an ophthalmologic information processing apparatus according to any one of claims 9 to 11.

13. The en face image of the fundus is a color en face image. A method for operating an ophthalmologic information processing apparatus according to any one of claims 9 to 12.

14. a registration step in which the processor performs registration between the OCT data of the subject's eye and a front image of the subject's eye in which the disc hemorrhage region detected in the detection step is depicted, The analyzing step generates the position information by using the OCT data that has been aligned with the front image in the alignment step. A method for operating an ophthalmologic information processing apparatus according to any one of claims 9 to 13.

15. A program for causing a computer to execute each step of the method for operating an ophthalmologic information processing apparatus according to any one of claims 9 to 14.

Images