Ophthalmic information processing apparatus, ophthalmic apparatus, ophthalmic information processing method, and program

Abstract

To provide a novel technology for positioning a spectral image simpler and more accurately.SOLUTION: An ophthalmologic information processing apparatus includes an acquisition part and a positioning part. The acquisition part acquires a plurality of images of an eye to be examined which is acquired by sequentially receiving return light having wavelength ranges different from one another and returning from the eye to be examined which is irradiated with illumination light. The positioning part positions a plurality of images on the basis of OCT data acquired by executing optical coherence tomography with respect to an eye to be examined.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an ophthalmologic information processing device, an ophthalmologic apparatus, an ophthalmologic information processing method, and a program. [Background technology]

[0002] Fundus observation is useful for diagnosing ocular diseases and estimating the state of sclerosis of the whole body (particularly cerebral blood vessels) by observing the retina, blood vessels, optic nerve, etc. through the pupil. For fundus observation, for example, a fundus image obtained by an ophthalmic device (fundus photography device) such as a fundus camera or a scanning light ophthalmoscope (SLO) is used.

[0003] It is known that in fundus observation, by acquiring a plurality of spectral fundus images over a wide analysis wavelength range, it is possible to extract various features of the fundus that are difficult to grasp from a general fundus image. For example, Patent Document 1 and Patent Document 2 disclose an ophthalmic device that acquires a spectral fundus image. For example, Non-Patent Document 1 and Non-Patent Document 2 disclose a method of applying a hyperspectral image as a spectral fundus image to the retina.

[0004] Since it takes a certain time to acquire such a plurality of spectral fundus images, positional deviation of the same part between the spectral fundus images may occur due to fixation deviation and alignment deviation between the subject's eye and the imaging unit while acquiring the plurality of spectral fundus images. For example, Patent Document 3 discloses a method of correcting the position and shape of one of two images so that a plurality of feature points in each of the two images match. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2006-158546 A [Patent Document 2] JP 2010-200916 A [Patent Document 3] JP 2006-158547 A [Non-patent literature]

[0006] [Non-Patent Document 1] Sophie Lemments, et al., “Hyperspectral Imaging and the Retina: Worth the Wave?”, translational vision science & technology, August 5, 2020, Vol. 9, No. 9, Ariticle 9 [Non-Patent Document 2] Edith R. Reshef, et el., “Hyperspectral Imaging of the Retina: A Review”, INTERNATIONAL OPHTHALMOLOGY CLINICS, August 21, 2020, Vol. 60, No. 1, pp.85-96 Summary of the Invention [Problem to be solved by the invention]

[0007] As the resolution of spectral images such as spectral fundus images and spectral anterior segment images increases, a new technique for aligning spectral images more easily and with higher accuracy is desired.

[0008] The present invention has been made in view of the above circumstances, and one of its objects is to provide a new technique for aligning spectroscopic images more easily and with high accuracy. [Means for solving the problem]

[0009] A first aspect of the embodiment is an ophthalmologic information processing device that includes an acquisition unit that acquires multiple images of a test eye obtained by sequentially receiving return light having different wavelength ranges from the test eye illuminated with illumination light, and an alignment unit that aligns the multiple images based on OCT data obtained by performing optical coherence tomography on the test eye.

[0010] In a second aspect of the embodiment, in the first aspect, the OCT data is a detection result of interference light obtained by performing optical coherence tomography on the test eye, a B-scan image, an OCTA (OCT Angiography) image, or an en-face image.

[0011] In a third aspect of the embodiment, in the first aspect, the alignment unit aligns the image of the subject's eye with an OCT front image at a depth position corresponding to a wavelength range of the image.

[0012] A fourth aspect of the embodiment, in any of the first to third aspects, includes a positional deviation information generating unit that acquires positional deviation information between the first image and the second image by analyzing a first image and a second image among the multiple images.

[0013] A fifth aspect of the embodiment is the fourth aspect, which includes a first judgment unit that judges whether or not the positional shift between the first image and the second image is large based on the positional shift information, and the acquisition unit re-acquires the first image or the second image based on the judgment result obtained by the first judgment unit.

[0014] A sixth aspect of the embodiment is based on any one of the first to fifth aspects and further includes a quality evaluation unit that calculates an evaluation value of quality of at least one of the plurality of images.

[0015] A seventh aspect of the embodiment is the sixth aspect, further comprising a second judgment unit that judges the quality of the plurality of images based on the evaluation value, and based on the judgment result obtained by the second judgment unit, the acquisition unit re-acquires at least one of the plurality of images or all of the plurality of images.

[0016] An eighth aspect of the embodiment is an ophthalmic device including an illumination optical system that illuminates the test eye with illumination light, a light receiving optical system that sequentially receives return light of the illumination light from the test eye having different wavelength ranges, an OCT optical system that performs optical coherence tomography on the test eye, and an ophthalmic information processing device described in any of the first to sixth aspects, wherein the acquisition unit sequentially acquires the multiple images based on the light receiving results obtained by the light receiving optical system.

[0017] A ninth aspect of the embodiment is an ophthalmologic information processing method including an acquisition step of acquiring a plurality of images of a test eye obtained by sequentially receiving return light having different wavelength ranges from the test eye illuminated with illumination light, and an alignment step of aligning the plurality of images based on OCT data obtained by performing optical coherence tomography on the test eye.

[0018] In a tenth aspect of the embodiment, in the ninth aspect, the OCT data is a detection result of interference light obtained by performing optical coherence tomography on the test eye, a B-scan image, an OCTA (OCT Angiography) image, or an en-face image.

[0019] In an eleventh aspect, in the ninth aspect, the positioning step performs positioning between the image of the subject's eye and an OCT front image at a depth position corresponding to a wavelength range of the image.

[0020] A twelfth aspect of the embodiment, in any of the ninth to eleventh aspects, includes a positional deviation information generation step of acquiring positional deviation information between the first image and the second image by analyzing a first image and a second image among the multiple images.

[0021] A thirteenth aspect of the embodiment is the twelfth aspect, which includes a first judgment step of judging whether or not the positional shift between the first image and the second image is large based on the positional shift information, and the acquisition step re-acquires the first image or the second image based on the judgment result obtained in the first judgment step.

[0022] A fourteenth aspect of the embodiment is based on any one of the ninth to thirteenth aspects and further includes a quality evaluation step of calculating an evaluation value of quality of at least one of the plurality of images.

[0023] A fifteenth aspect of the embodiment is the same as the fourteenth aspect, and includes a second judgment step of judging the quality of the plurality of images based on the evaluation value, and the acquisition step re-acquires at least one of the plurality of images or all of the plurality of images based on the judgment result obtained in the second judgment step.

[0024] A sixteenth aspect of the embodiment is a program for causing a computer to execute each step of the ophthalmologic information processing method according to any one of the ninth to fifteenth aspects.

[0025] The configurations according to the above-described multiple aspects can be combined in any desired manner. Effect of the Invention

[0026] According to the present invention, it is possible to provide a new technique for aligning spectral images more easily and with higher accuracy. [Brief description of the drawings]

[0027] [Figure 1] 1 is a schematic diagram illustrating an example of a configuration of an optical system of an ophthalmic apparatus according to an embodiment. [Diagram 2] 1 is a schematic diagram illustrating an example of a configuration of an optical system of an ophthalmic apparatus according to an embodiment. [Diagram 3] 2 is a schematic diagram illustrating an example of the configuration of a control system of an ophthalmic apparatus according to an embodiment. FIG. [Figure 4] 2 is a schematic diagram illustrating an example of the configuration of a control system of an ophthalmic apparatus according to an embodiment. FIG. [Diagram 5] 2 is a schematic diagram illustrating an example of the configuration of a control system of an ophthalmic apparatus according to an embodiment. FIG. [Figure 6] 2 is a schematic diagram illustrating an example of the configuration of a control system of an ophthalmic apparatus according to an embodiment. FIG. [Figure 7] 4 is a flowchart illustrating an example of an operation of the ophthalmologic apparatus according to the embodiment. [Figure 8] 4 is a flowchart illustrating an example of an operation of the ophthalmologic apparatus according to the embodiment. [Figure 9] 4 is a flowchart illustrating an example of an operation of the ophthalmologic apparatus according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] An example of an embodiment of an ophthalmological information processing device, an ophthalmological device, an ophthalmological information processing method, and a program according to the present invention will be described in detail with reference to the drawings. Note that in the embodiment, the technology described in the documents cited in this specification can be arbitrarily used.

[0029] An ophthalmologic information processing device according to an embodiment acquires multiple spectroscopic images of a test eye over a predetermined analysis wavelength range, and aligns the acquired multiple spectroscopic images based on OCT data obtained by performing optical coherence tomography (OCT) on the test eye.

[0030] The multiple spectral images are acquired by illuminating the fundus (posterior segment) or anterior segment of the subject's eye with illumination light, and sequentially receiving return light of the illumination light from the subject's eye, which have different wavelength ranges in a predetermined analysis wavelength region. The spectral image is an example of a two-dimensional spectral distribution. For first return light and second return light having adjacent wavelength ranges among the sequentially received return light, a part of the wavelength range of the first return light may overlap the wavelength range of the second return light. Examples of the spectral image include a hyperspectral image, a multispectral image, and an RGB color image.

[0031] In some embodiments, a test eye is sequentially illuminated with illumination light having two or more wavelength components with different wavelength ranges, and multiple spectral images are obtained by sequentially selecting return light having wavelength components in a predetermined wavelength range from the return light from the test eye.

[0032] In some embodiments, illumination light having wavelength components in a predetermined wavelength range is selected in sequence from illumination light having two or more wavelength components with different wavelength ranges, the examined eye is illuminated in sequence with the selected illumination light, and the return light from the examined eye is received in sequence to obtain multiple spectral images.

[0033] In some embodiments, a light source capable of arbitrarily changing the wavelength range is used to sequentially emit illumination light having two or more wavelength components with different wavelength ranges, the emitted illumination light is sequentially used to illuminate the test eye, and the return light from the test eye is sequentially received to obtain multiple spectroscopic images.

[0034] In some embodiments, a plurality of spectroscopic images are obtained by illuminating the test eye with illumination light, sequentially changing the wavelength range in which the light receiving device has high light receiving sensitivity, and sequentially selecting the return light from the test eye.

[0035] The OCT data may be a detection result of interference light obtained by performing OCT on the test eye, a B-scan image, or an en-face image. The interference light is generated by splitting light from an OCT light source into measurement light and reference light, projecting the measurement light on the test eye, and causing return light of the measurement light from the test eye to interfere with the reference light passing through the reference light path. In some embodiments, the ophthalmic information processing device is configured to acquire OCT data acquired by an externally provided OCT device. In some embodiments, the function of the ophthalmic information processing device is realized by an ophthalmic device capable of acquiring OCT data.

[0036] In some embodiments, the alignment is achieved by changing the relative positions of the multiple spectroscopic images. In some embodiments, the alignment is achieved by performing an affine transformation or a Helmert transformation on the spectroscopic images to be aligned with reference to a reference image. Examples of the reference image include an OCT image (e.g., a B-scan image or an en-face image) formed based on OCT data.

[0037] This makes it possible to easily compare the spectral characteristics between the spectral images even if fixation shift and alignment shift between the test eye and the imaging unit occur while acquiring multiple spectral images. In particular, by performing alignment based on OCT data, it becomes possible to align the spectral images with high accuracy based on information on the depth direction (depth direction) of the eye, and it becomes possible to improve the accuracy of the spectral analysis at the observation site of the test eye.

[0038] The ophthalmological information processing method according to the embodiment includes one or more steps executed by the above-mentioned ophthalmological information processing device. The program according to the embodiment causes a computer (processor) to execute each step of the ophthalmological information processing method according to the embodiment. The recording medium according to the embodiment is a non-transitory recording medium (storage medium) on which the program according to the embodiment is recorded.

[0039] In this specification, the processor includes circuits such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA)), etc. The processor realizes the functions of the embodiment by, for example, reading and executing a program stored in a memory circuit or a storage device. The memory circuit or storage device may be included in the processor. Furthermore, the memory circuit or storage device may be provided outside the processor.

[0040] Hereinafter, a case where alignment is performed for a spectral image of the fundus of the subject eye (spectral fundus image) will be described, but the configuration according to the embodiment is not limited thereto. For example, the following embodiment can also be applied to a case where alignment is performed for a spectral image of the anterior segment other than the fundus (spectral anterior segment image).

[0041] In some embodiments, the ophthalmologic information-processing device is configured to acquire a spectroscopic image of the test eye acquired externally through a communication function. In some embodiments, an ophthalmologic device capable of acquiring a spectroscopic image of the test eye has a function of the ophthalmologic information-processing device.

[0042] In the following embodiments, an ophthalmic apparatus including the functions of the ophthalmic information processing apparatus according to the embodiment will be described as an example. The ophthalmic apparatus according to the embodiment includes an ophthalmic imaging apparatus. The ophthalmic imaging apparatus included in the ophthalmic apparatus according to some embodiments is, for example, any one or more of a fundus camera, a scanning optical ophthalmoscope, a slit lamp ophthalmoscope, a surgical microscope, etc. The ophthalmic apparatus according to some embodiments includes, in addition to the ophthalmic imaging apparatus, any one or more of an ophthalmic measurement apparatus and an ophthalmic treatment apparatus. The ophthalmic measurement apparatus included in the ophthalmic apparatus according to some embodiments is, for example, any one or more of an eye refraction examination apparatus, a tonometer, a specular microscope, a wavefront analyzer, a perimeter, a microperimeter, etc. The ophthalmic treatment apparatus included in the ophthalmic apparatus according to some embodiments is, for example, any one or more of a laser treatment apparatus, a surgical apparatus, a surgical microscope, etc.

[0043] In the following embodiment, the ophthalmologic apparatus includes an optical coherence tomography and a fundus camera. Although the optical coherence tomography uses swept-source OCT, the type of OCT is not limited thereto, and other types of OCT (spectral domain OCT, time domain OCT, unfath OCT, etc.) may be used.

[0044] Hereinafter, the x-direction is defined as the direction perpendicular to the optical axis of the objective lens (left-right direction), the y-direction is defined as the direction perpendicular to the optical axis of the objective lens (up-down direction), and the z-direction is defined as the optical axis direction of the objective lens.

[0045] <Configuration> [Optical system] As shown in FIG. 1, the ophthalmic apparatus 1 includes a fundus camera unit 2, an OCT unit 100, and an arithmetic and control unit 200. The fundus camera unit 2 is provided with an optical system and a mechanism for acquiring a front image of the subject's eye E. The OCT unit 100 is provided with a part of the optical system and a mechanism for performing OCT. The other part of the optical system and a mechanism for performing OCT is provided in the fundus camera unit 2. The arithmetic and control unit 200 includes one or more processors for performing various calculations and controls. In addition to these, the ophthalmic apparatus 1 may be provided with any elements or units such as a member for supporting the subject's face (such as a chin rest or a forehead rest) and a lens unit for switching the target site of OCT (for example, an attachment for anterior eye OCT). Furthermore, the ophthalmic apparatus 1 includes a pair of anterior eye cameras 5A and 5B.

[0046] [Fundus camera unit 2] The fundus camera unit 2 is provided with an optical system for photographing the fundus Ef of the subject's eye E. The acquired image of the fundus Ef (called a fundus image, fundus photograph, etc.) is a front image such as an observation image or a photographed image. The observation image is obtained by video shooting using near-infrared light. The photographed image is a still image using flash light, or a spectral image (spectral fundus image, spectral anterior segment image). Furthermore, the fundus camera unit 2 can photograph the anterior segment Ea of the subject's eye E to acquire a front image (anterior segment image).

[0047] The fundus camera unit 2 includes an illumination optical system 10 and an imaging optical system 30. The illumination optical system 10 irradiates illumination light onto the subject's eye E. The imaging optical system 30 detects return light of the illumination light from the subject's eye E. The measurement light from the OCT unit 100 is guided to the subject's eye E through an optical path in the fundus camera unit 2, and the return light is guided to the OCT unit 100 through the same optical path.

[0048] Light (observation illumination light) output from an observation light source 11 of an illumination optical system 10 is reflected by a reflecting mirror 12 having a curved reflecting surface, passes through a condenser lens 13, and passes through a visible light cut filter 14 to become near-infrared light. The observation illumination light is once focused near an imaging light source 15, reflected by a mirror 16, and passes through relay lenses 17 and 18, an aperture 19, and a relay lens 20. The observation illumination light is then reflected at the periphery (area surrounding the hole) of a perforated mirror 21, passes through a dichroic mirror 46, and is refracted by an objective lens 22 to illuminate the subject's eye E (fundus Ef or anterior eye Ea). Return light of the observation illumination light from the subject's eye E is refracted by the objective lens 22, passes through a dichroic mirror 46, passes through a hole formed in the central area of ​​the perforated mirror 21, passes through a photographing focusing lens 31, and is reflected by a mirror 32. Furthermore, this return light passes through the half mirror 33A, is reflected by the dichroic mirror 33, and is imaged on the light receiving surface of the image sensor 35 by the condenser lens 34. The image sensor 35 detects the return light at a predetermined frame rate. The focus of the photographing optical system 30 is adjusted to match the fundus Ef or the anterior segment Ea.

[0049] The light (imaging illumination light) output from the imaging light source 15 is irradiated onto the fundus Ef through the same path as the observation illumination light. The return light of the imaging illumination light from the subject's eye E is guided to the dichroic mirror 33 through the same path as the return light of the observation illumination light, passes through the dichroic mirror 33, is reflected by the mirror 36, and is guided to the tunable filter 80.

[0050] The tunable filter 80 is a filter capable of selecting the wavelength range of transmitted light in a predetermined analysis wavelength region. The wavelength range of light transmitted through the tunable filter 80 can be arbitrarily selected.

[0051] In some embodiments, the tunable filter 80 is similar to the liquid crystal tunable filter disclosed in, for example, JP 2006-158546 A. In this case, the tunable filter 80 can arbitrarily select the wavelength selection range of the transmitted light by changing the voltage applied to the liquid crystal.

[0052] In some embodiments, the tunable filter 80 may include two or more wavelength-selective filters having different wavelength selection ranges for transmitted light, and may be configured so that the two or more wavelength-selective filters can be selectively positioned in the optical path of the return light of the illumination light.

[0053] In some embodiments, the tunable filter 80 is a filter capable of selecting a range of wavelengths of reflected light in a predetermined analysis wavelength region.

[0054] The return light from the mirror 36 that has been transmitted through the tunable filter 80 is focused by the condenser lens 37 onto the light receiving surface of the image sensor 38 .

[0055] In some embodiments, the tunable filter 80 is disposed between the dichroic mirror 33 and the collecting lens 34 .

[0056] In some embodiments, the tunable filter 80 is configured to be insertable into and removable from the optical path between the dichroic mirror 33 or the mirror 36 and the condenser lens 37. For example, when the tunable filter 80 is disposed in the optical path between the dichroic mirror 33 and the condenser lens 37, the ophthalmic device 1 can acquire a plurality of spectral fundus images by sequentially acquiring the light reception results of the return light obtained by the image sensor 38. For example, when the tunable filter 80 is retracted from the optical path between the dichroic mirror 33 and the condenser lens 37, the ophthalmic device 1 can acquire a normal still image (fundus image, anterior eye image) by acquiring the light reception results of the return light obtained by the image sensor 38.

[0057] The display device 3 displays an image (observation image) based on the fundus reflected light detected by the image sensor 35. When the photographing optical system 30 is focused on the anterior segment, an observation image of the anterior segment of the subject's eye E is displayed. The display device 3 also displays an image (photographed image, spectral fundus image) based on the fundus reflected light detected by the image sensor 38. The display device 3 that displays the observation image and the display device 3 that displays the photographed image may be the same or different. When the subject's eye E is illuminated with infrared light and a similar photograph is performed, an infrared photographed image is displayed.

[0058] The LCD (Liquid Crystal Display) 39 displays a fixation target and a visual target for visual acuity measurement. A part of the light beam output from the LCD 39 is reflected by the half mirror 33A, reflected by the mirror 32, passes through the photographing focusing lens 31, and passes through the hole of the aperture mirror 21. The light beam that passes through the hole of the aperture mirror 21 passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef.

[0059] The fixation position of the subject's eye E can be changed by changing the display position of the fixation target on the screen of the LCD 39. Examples of the fixation position include a fixation position for acquiring an image centered on the macula, a fixation position for acquiring an image centered on the optic disc, a fixation position for acquiring an image centered on the fundus center between the macula and the optic disc, and a fixation position for acquiring an image of a site (periphery of the fundus) far away from the macula. The ophthalmic device 1 according to some embodiments includes a GUI (Graphical User Interface) or the like for designating at least one of such fixation positions. The ophthalmic device 1 according to some embodiments includes a GUI or the like for manually moving the fixation position (display position of the fixation target).

[0060] The configuration for presenting a movable fixation target to the subject's eye E is not limited to a display device such as an LCD. For example, a movable fixation target can be generated by selectively turning on a plurality of light sources in a light source array (such as a light emitting diode (LED) array). Also, a movable fixation target can be generated by one or more movable light sources.

[0061] The focusing optical system 60 generates a split index used for focus adjustment of the subject's eye E. The focusing optical system 60 is moved along the optical path (illumination optical path) of the illumination optical system 10 in conjunction with the movement of the photographing focusing lens 31 along the optical path (photographing optical path) of the photographing optical system 30. The reflecting rod 67 can be inserted into and removed from the illumination optical path. When performing focus adjustment, the reflecting surface of the reflecting rod 67 is tilted and disposed in the illumination optical path. The focusing light output from the LED 61 passes through the relay lens 62, is split into two light beams by the split index plate 63, passes through the two-hole diaphragm 64, is reflected by the mirror 65, and is once imaged and reflected on the reflecting surface of the reflecting rod 67 by the condenser lens 66. Furthermore, the focusing light passes through the relay lens 20, is reflected by the aperture mirror 21, passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef. The fundus reflection light of the focusing light is guided to the image sensor 35 through the same path as the return light of the observation illumination light. Manual focusing or autofocusing can be performed based on the received light image (split target image).

[0062] The dichroic mirror 46 combines the optical path for fundus photography and the optical path for OCT. The dichroic mirror 46 reflects light in the wavelength band used for OCT and transmits light for fundus photography. The optical path for OCT (optical path of measurement light) is provided with a collimator lens unit 40, an optical path length changer 41, an optical scanner 42, an OCT focusing lens 43, a mirror 44, and a relay lens 45 in this order from the OCT unit 100 side to the dichroic mirror 46 side.

[0063] The optical path length changing unit 41 is movable in the direction of the arrow shown in Fig. 1 and changes the length of the optical path for OCT. This change in the optical path length is used for correcting the optical path length according to the axial length, adjusting the interference state, etc. The optical path length changing unit 41 includes a corner cube and a mechanism for moving it.

[0064] The optical scanner 42 is disposed at a position optically conjugate with the pupil of the subject's eye E. The optical scanner 42 deflects the measurement light LS passing through the OCT optical path. The optical scanner 42 is, for example, a galvano scanner capable of two-dimensional scanning.

[0065] The OCT focusing lens 43 is moved along the optical path of the measurement light LS to adjust the focus of the OCT optical system. The movement of the photographing focusing lens 31, the movement of the focus optical system 60, and the movement of the OCT focusing lens 43 can be controlled in a coordinated manner.

[0066] [Anterior Eye Cameras 5A and 5B] The anterior eye cameras 5A and 5B are used to obtain the relative position between the optical system of the ophthalmic apparatus 1 and the subject's eye E, similarly to the invention disclosed in JP 2013-248376 A. The anterior eye cameras 5A and 5B are provided on the surface of the subject's eye E side of a housing (fundus camera unit 2, etc.) in which the optical system is stored. The ophthalmic apparatus 1 obtains the three-dimensional relative position between the optical system and the subject's eye E by analyzing two anterior eye images acquired substantially simultaneously from different directions by the anterior eye cameras 5A and 5B. The analysis of the two anterior eye images may be similar to the analysis disclosed in JP 2013-248376 A. The number of anterior eye cameras may be any number equal to or greater than two.

[0067] In this example, the position of the subject's eye E (i.e., the relative position between the subject's eye E and the optical system) is obtained using two or more anterior eye cameras, but the method for obtaining the position of the subject's eye E is not limited to this. For example, the position of the subject's eye E can be obtained by analyzing a front image of the subject's eye E (e.g., an observation image of the anterior eye Ea). Alternatively, a means for projecting an index onto the cornea of ​​the subject's eye E can be provided, and the position of the subject's eye E can be obtained based on the projection position of this index (i.e., the detection state of the corneal reflected light beam of this index).

[0068] [OCT unit 100] 2, the OCT unit 100 is provided with an optical system for performing swept-source OCT. This optical system includes an interference optical system. This interference optical system has a function of splitting light from a wavelength-variable light source (wavelength-swept light source) into measurement light and reference light, a function of superimposing return light of the measurement light from the subject's eye E and the reference light that has passed through a reference light path to generate interference light, and a function of detecting this interference light. A detection result (detection signal) of the interference light obtained by the interference optical system is a signal indicating the spectrum of the interference light, and is sent to the arithmetic and control unit 200.

[0069] The light source unit 101 includes, for example, a near-infrared tunable laser that changes the wavelength of the emitted light at high speed. The light L0 output from the light source unit 101 is guided to a polarization controller 103 by an optical fiber 102, where its polarization state is adjusted. The light L0 whose polarization state has been adjusted is guided to a fiber coupler 105 by an optical fiber 104, where it is split into a measurement light LS and a reference light LR.

[0070] The reference light LR is guided by an optical fiber 110 to a collimator 111 where it is converted into a parallel beam, and is then guided to a corner cube 114 via an optical path length correction member 112 and a dispersion compensation member 113. The optical path length correction member 112 acts to match the optical path length of the reference light LR with the optical path length of the measurement light LS. The dispersion compensation member 113 acts to match the dispersion characteristics between the reference light LR and the measurement light LS. The corner cube 114 is movable in the incident direction of the reference light LR, thereby changing the optical path length of the reference light LR.

[0071] The reference light LR that has passed through the corner cube 114 passes through the dispersion compensation member 113 and the optical path length correction member 112, is converted from a parallel beam to a focused beam by the collimator 116, and enters the optical fiber 117. The reference light LR that has entered the optical fiber 117 is guided to a polarization controller 118 where its polarization state is adjusted, is guided by the optical fiber 119 to an attenuator 120 where the amount of light is adjusted, and is guided by the optical fiber 121 to a fiber coupler 122.

[0072] On the other hand, the measurement light LS generated by the fiber coupler 105 is guided by the optical fiber 127 and converted into a parallel beam by the collimator lens unit 40, and passes through the optical path length change unit 41, the optical scanner 42, the OCT focusing lens 43, the mirror 44, and the relay lens 45. The measurement light LS passing through the relay lens 45 is reflected by the dichroic mirror 46, refracted by the objective lens 22, and enters the subject's eye E. The measurement light LS is scattered and reflected at various depth positions of the subject's eye E. The return light of the measurement light LS from the subject's eye E travels in the opposite direction along the same path as the outward path and is guided to the fiber coupler 105, and reaches the fiber coupler 122 via the optical fiber 128. The incident end of the optical fiber 127 into which the measurement light LS is incident is located at a position approximately conjugate with the fundus Ef of the subject's eye E.

[0073] The fiber coupler 122 generates interference light by combining (causing interference between) the measurement light LS incident via the optical fiber 128 and the reference light LR incident via the optical fiber 121. The fiber coupler 122 splits the interference light at a predetermined splitting ratio (for example, 1:1) to generate a pair of interference lights LC. The pair of interference lights LC are guided to a detector 125 via optical fibers 123 and 124, respectively.

[0074] The detector 125 is, for example, a balanced photodiode. The balanced photodiode includes a pair of photodetectors that detect a pair of interference lights LC, respectively, and outputs a difference between a pair of detection results obtained by these photodetectors. The detector 125 sends this output (detection signal) to a DAQ (Data Acquisition System) 130.

[0075] The DAQ130 is supplied with a clock KC from the light source unit 101. The clock KC is generated in the light source unit 101 in synchronization with the output timing of each wavelength swept within a predetermined wavelength range by the wavelength-tunable light source. For example, the light source unit 101 optically delays one of two branched lights obtained by branching the light L0 of each output wavelength, and generates the clock KC based on the result of detecting the combined light. The DAQ130 samples the detection signal input from the detector 125 based on the clock KC. The DAQ130 sends the sampling result of the detection signal from the detector 125 to the arithmetic and control unit 200.

[0076] In this example, both an optical path length changer 41 for changing the length of the optical path (measurement optical path, measurement arm) of the measurement light LS and a corner cube 114 for changing the length of the optical path (reference optical path, reference arm) of the reference light LR are provided. However, only one of the optical path length changer 41 and the corner cube 114 may be provided. It is also possible to change the difference between the measurement optical path length and the reference optical path length using optical members other than these.

[0077] [Control system] 3 to 6 show an example of the configuration of a control system of the ophthalmic apparatus 1. Some of the components included in the ophthalmic apparatus 1 are omitted in Fig. 3 to Fig. 6. In Fig. 3, the same parts as those in Fig. 1 and Fig. 2 are given the same reference numerals, and the description will be omitted as appropriate. The control unit 210, the image forming unit 220, and the data processing unit 230 are provided in, for example, an arithmetic control unit 200.

[0078] <Control unit 210> The control unit 210 executes various types of control. The control unit 210 includes a main control unit 211 and a storage unit 212.

[0079] <Main control unit 211> The main controller 211 includes a processor (e.g., a control processor) and controls each part (including each element shown in Fig. 1 to Fig. 6) of the ophthalmologic apparatus 1. For example, the main controller 211 controls each part of the optical system of the fundus camera unit 2 shown in Fig. 1 and Fig. 2, each part of the optical system of the OCT unit 100, the anterior eye cameras 5A and 5B, the moving mechanism 150 that moves the above optical systems, the image forming unit 220, the data processing unit 230, and a user interface (UI) 240.

[0080] The control of the fundus camera unit 2 includes control of the focusing drivers 31A and 43A, control of the tunable filter 80, control of the image sensors 35 and 38, control of the optical path length changer 41, and control of the optical scanner 42.

[0081] The control over the focus driver 31A includes control to move the photographing focus lens 31 in the optical axis direction. The control over the focus driver 43A includes control to move the OCT focus lens 43 in the optical axis direction.

[0082] The control of the tunable filter 80 includes selective control of the wavelength range of transmitted light (for example, control of the voltage applied to the liquid crystal).

[0083] The control of the image sensors 35 and 38 includes control of the light receiving sensitivity of the image sensor, control of the frame rate (light receiving timing, exposure time), control of the light receiving area (position, size, dimension), and control of reading out the light receiving result from the image sensor. In some embodiments, the image sensors 35 and 38 are controlled so that the light receiving intensity is uniform in each wavelength range of the analysis wavelength range in which the multiple spectral fundus images are acquired by changing the exposure time according to the wavelength range of the return light. In some embodiments, the main controller 211 controls the light intensity of the wavelength component of each wavelength range of the illumination light so that the light receiving intensity is uniform in each wavelength range of the analysis wavelength range in which the multiple spectral fundus images are acquired.

[0084] The control of the LCD 39 includes control of the fixation position. For example, the main control unit 211 displays a fixation target at a position on the screen of the LCD 39 that corresponds to a fixation position that is set manually or automatically. The main control unit 211 can also change (continuously or stepwise) the display position of the fixation target displayed on the LCD 39. This makes it possible to move the fixation target (i.e., to change the fixation position). The display position and movement manner of the fixation target are set manually or automatically. The manual setting is performed using, for example, a GUI. The automatic setting is performed by, for example, the data processing unit 230.

[0085] The control over the optical path length changing unit 41 includes control to change the optical path length of the measurement light LS. The main control unit 211 controls a drive unit that drives the corner cube of the optical path length changing unit 41 to move the optical path length changing unit 41 along the optical path of the measurement light LS, thereby changing the optical path length of the measurement light LS.

[0086] The control of the optical scanner 42 includes control of the scan mode, the scan range (scan start position, scan end position), the scan speed, etc. The main controller 211 controls the optical scanner 42 to perform an OCT scan with the measurement light LS on a desired region in a measurement region (image capture region).

[0087] The main control unit 211 also controls the observation light source 11, the imaging light source 15, the focus optical system 60, and the like.

[0088] The control over the OCT unit 100 includes control over the light source unit 101, control over the reference driver 114A, control over the detector 125, and control over the DAQ .

[0089] The control of the light source unit 101 includes control of turning the light source on and off, control of the amount of light emitted from the light source, control of the wavelength sweep range, control of the wavelength sweep speed, and control of the emission timing of light of each wavelength component.

[0090] The control over the reference driver 114A includes control to change the optical path length of the reference light LR. The main controller 211 controls the reference driver 114A to move the corner cube 114 along the optical path of the reference light LR, thereby changing the optical path length of the reference light LR.

[0091] Control of the detector 125 includes control of the light receiving sensitivity of the detection elements, control of the frame rate (light receiving timing), control of the light receiving area (position, size, dimension), and readout control of the light receiving results of the detection elements.

[0092] The control over the DAQ 130 includes control over taking in the detection results of the interference light obtained by the detector 125 (taking in timing, sampling timing), and control over reading out an interference signal corresponding to the detection results of the taken in interference light.

[0093] The control of the anterior eye cameras 5A and 5B includes control of the light receiving sensitivity of each camera, control of the frame rate (light receiving timing), and synchronization control of the anterior eye cameras 5A and 5B.

[0094] The moving mechanism 150, for example, moves at least the fundus camera unit 2 (optical system) three-dimensionally. In a typical example, the moving mechanism 150 includes at least a mechanism for moving the fundus camera unit 2 in the x direction (left and right direction), a mechanism for moving in the y direction (up and down direction), and a mechanism for moving in the z direction (depth direction, front and back direction). The mechanism for moving in the x direction includes, for example, an x ​​stage that can move in the x direction, and an x ​​moving mechanism that moves the x stage. The mechanism for moving in the y direction includes, for example, a y stage that can move in the y direction, and a y moving mechanism that moves the y stage. The mechanism for moving in the z direction includes, for example, a z stage that can move in the z direction, and a z moving mechanism that moves the z stage. Each moving mechanism includes a pulse motor as an actuator, and operates under the control of the main controller 211.

[0095] The control of the moving mechanism 150 is used for alignment and tracking. Tracking refers to moving the device optical system in accordance with the eye movement of the subject's eye E. When tracking is performed, alignment and focus adjustment are performed in advance. Tracking is a function of maintaining a suitable positional relationship where alignment and focus are achieved by making the position of the device optical system follow the eye movement. In some embodiments, the moving mechanism 150 is configured to be controlled in order to change the optical path length of the reference light (hence, the optical path length difference between the optical path of the measurement light and the optical path of the reference light).

[0096] In the case of manual alignment, the user operates the user interface 240 to cancel the displacement of the subject's eye E relative to the optical system, thereby moving the optical system relative to the subject's eye E. For example, the main controller 211 controls the moving mechanism 150 by outputting a control signal corresponding to the operation content on the user interface 240 to the moving mechanism 150, thereby moving the optical system relative to the subject's eye E.

[0097] In the case of auto-alignment, the main controller 211 controls the moving mechanism 150 so that the displacement of the eye E relative to the optical system is cancelled, thereby moving the optical system relative to the eye E. Specifically, as described in JP 2013-248376 A, calculation processing is performed using trigonometry based on the positional relationship between the pair of anterior eye cameras 5A and 5B and the eye E, and the main controller 211 controls the moving mechanism 150 so that the positional relationship of the eye E relative to the optical system becomes a predetermined positional relationship. In some embodiments, the main controller 211 controls the moving mechanism 150 by outputting a control signal to the moving mechanism 150 so that the optical axis of the optical system approximately coincides with the axis of the eye E and the distance of the optical system to the eye E is a predetermined working distance, thereby moving the optical system relative to the eye E. Here, the working distance is a preset value also called the working distance of the objective lens 22, and corresponds to the distance between the eye E and the optical system during measurement (photography) using the optical system.

[0098] The main controller 211, as a display controller, can display various information on the display unit 240A. For example, the main controller 211 causes the display unit 240A to display a plurality of spectral fundus images in association with wavelength ranges. For example, the main controller 211 causes the display unit 240A to display the analysis processing results obtained by the analysis unit 232.

[0099] <Storage section 212> The storage unit 212 stores various data. The function of the storage unit 212 is realized by a storage device such as a memory or a storage device. Examples of the data stored in the storage unit 212 include control parameters, image data of a fundus image, image data of an anterior eye image, OCT data (including an OCT image), spectral image data of a fundus image, spectral image data of an anterior eye image, and information on the subject's eye. Examples of the control parameters include hyperspectral imaging control data. The hyperspectral imaging control data is control data for acquiring a plurality of fundus images based on return light having different central wavelengths within a predetermined analysis wavelength range. Examples of the hyperspectral imaging control data include an analysis wavelength range in which a plurality of spectral fundus images are acquired, a wavelength range in which each spectral fundus image is acquired, a central wavelength, a central wavelength step, and control data of a wavelength-variable filter 80 corresponding to the central wavelength. The information on the subject's eye includes information on the subject, such as a patient ID and name, identification information of the left eye / right eye, and information on the subject's eye, such as electronic medical record information. The storage unit 212 stores programs for executing various processors (control processor, image formation processor, data processing processor).

[0100] <Image forming unit 220> The image forming unit 220 includes a processor (e.g., an image forming processor) and forms an OCT image (image data) of the subject's eye E based on the output (sampling result of the detection signal) from the DAQ 130. For example, the image forming unit 220 performs signal processing on the spectral distribution based on the sampling result for each A-line to form a reflection intensity profile for each A-line, as in the conventional swept-source OCT, and images these A-line profiles and arranges them along the scan line. The above signal processing includes noise removal (noise reduction), filter processing, FFT (Fast Fourier Transform), etc. When performing other types of OCT, the image forming unit 220 performs known processing according to the type.

[0101] Data Processing Unit 230 Data processing unit 230 includes a processor (e.g., a data processor) and performs image processing and analysis processing on the image formed by image forming unit 220. At least two of the processor included in main control unit 211, the processor included in data processing unit 230, and the processor included in image forming unit 220 may be configured by a single processor.

[0102] The data processing unit 230 executes known image processing such as an interpolation process for interpolating pixels between tomographic images to form image data of a three-dimensional image of the fundus oculi Ef or the anterior eye part Ea. The image data of the three-dimensional image means image data in which the positions of pixels are defined by a three-dimensional coordinate system. The image data of the three-dimensional image includes image data consisting of three-dimensionally arranged voxels. This image data is called volume data or voxel data. When an image based on the volume data is to be displayed, the data processing unit 230 performs a rendering process (such as volume rendering or MIP (Maximum Intensity Projection)) on the volume data to form image data of a pseudo three-dimensional image as viewed from a specific line of sight. The pseudo three-dimensional image is displayed on a display device such as the display unit 240A.

[0103] It is also possible to form stack data of multiple tomographic images as image data of a three-dimensional image. Stack data is image data obtained by arranging multiple tomographic images obtained along multiple scan lines three-dimensionally based on the positional relationship of the scan lines. In other words, stack data is image data obtained by expressing multiple tomographic images that were originally defined by individual two-dimensional coordinate systems using one three-dimensional coordinate system (i.e., embedding them in one three-dimensional space).

[0104] In some embodiments, the data processor 230 generates a B-scan image by arranging the A-scan images in the B-scan direction. In some embodiments, the data processor 230 can perform various renderings on the acquired three-dimensional data set (volume data, stack data, etc.) to form a B-mode image (B-scan image) (longitudinal section image, axial section image) in an arbitrary section, a C-mode image (C-scan image) (transverse section image, horizontal section image) in an arbitrary section, a projection image, a shadowgram, etc. An image of an arbitrary section such as a B-scan image or a C-scan image is formed by selecting pixels (pixels, voxels) on a specified section from the three-dimensional data set. A projection image is formed by projecting a three-dimensional data set in a predetermined direction (z direction, depth direction, axial direction). A shadowgram is formed by projecting a part of a three-dimensional data set (for example, partial data corresponding to a specific layer) in a predetermined direction. It is possible to form two or more shadowgrams that are different from each other by changing the depth range in the layer direction to be integrated. An image with the front side of the subject's eye as a viewpoint, such as a C-scan image, a projection image, or a shadowgram, is called an en-face image.

[0105] The data processing unit 230 can construct a B-scan image or a front image (vessel-enhanced image, angiogram) in which retinal blood vessels and choroidal blood vessels are emphasized based on data (e.g., B-scan image data) collected in a time series by OCT. For example, time-series OCT data can be collected by repeatedly scanning approximately the same part of the subject's eye E.

[0106] In some embodiments, the data processor 230 compares time-series B-scan images obtained by B-scanning substantially the same region, and constructs an enhanced image in which the changed portion is emphasized by converting pixel values ​​of the changed portion of the signal intensity into pixel values ​​corresponding to the changed portion. Furthermore, the data processor 230 extracts information of a predetermined thickness in a desired region from the constructed multiple enhanced images, and constructs the information as an en-face image to form an OCTA image.

[0107] Such a data processing unit 230 includes an alignment processing unit 231 and an analysis unit 232 .

[0108] <Alignment Processing Unit 231> The alignment processing unit 231 performs alignment of multiple spectral fundus images of the subject's eye E corresponding to multiple wavelength ranges of a predetermined analysis wavelength range, based on OCT data obtained by performing OCT on the subject's eye E.

[0109] The OCT data may be a detection result of interference light obtained by performing OCT on the subject's eye, a B-scan image, or an en-face image.

[0110] Since the relative position of the optical axis of the OCT unit 100 with respect to the optical axis of the imaging optical system 30 is known, it is possible to identify the OCT scan position (or the position in the OCT image) using the OCT unit 100 from the position in the image obtained by the imaging optical system 30.

[0111] In addition, the fundus Ef is composed of a plurality of layer regions having different characteristics with respect to the wavelength components of the incident light. That is, the plurality of layer regions have different proportions of reflected light, absorbed light, and transmitted light according to the wavelength components of the incident light. Specifically, it is known that in the analysis wavelength range, the shorter the wavelength, the stronger the intensity of reflected light in the superficial layer (nerve fiber layer), the intermediate wavelength, the stronger the intensity of reflected light in the retinal pigment epithelium layer, and the longer the wavelength, the stronger the intensity of reflected light in the deep layer (choroid layer). Therefore, the layer regions of the fundus Ef can be previously associated with the wavelength range of light incident on the fundus Ef.

[0112] Therefore, the alignment processing unit 231 aligns a plurality of spectral fundus images using the detection result of interference light obtained by OCT scanning positions corresponding to characteristic sites (characteristic regions, characteristic points) in the spectral fundus images. The characteristic sites are identified, for example, by the analysis unit 232 described below. For example, the alignment processing unit 231 identifies the first layer region and the second layer region of the fundus Ef from the first depth position (z position) and the second depth position where the intensity of the interference light is high based on the detection result of the interference light. The alignment processing unit 231 aligns the first spectral fundus image corresponding to the identified first layer region and the second spectral fundus image corresponding to the identified second layer region in the xy directions.

[0113] In some embodiments, the registration processing unit 231 aligns a plurality of spectral fundus images using a B-scan image passing through a position corresponding to a characteristic portion in the spectral fundus image. For example, the registration processing unit 231 identifies a first layer region and a second layer region of the fundus Ef at a first depth position and a second depth position by performing a known segmentation process on the B-scan image. The registration processing unit 231 aligns the first spectral fundus image corresponding to the identified first layer region and the second spectral fundus image corresponding to the identified second layer region in the xy directions.

[0114] In some embodiments, the registration processor 231 aligns the spectral fundus images using an en-face image in which one or more characteristic parts in the spectral fundus images are depicted. For example, the registration processor 231 aligns the positions of one or more characteristic parts in the en-face image with the positions of the respective characteristic parts of the spectral fundus images. For example, the registration processor 231 aligns the two images by least squares matching (LSM) so that the sum of squares of the difference in shading between the en-face image and the spectral fundus image to be aligned is minimized for one or more characteristic parts in each of the en-face image and the spectral fundus image to be aligned. In some embodiments, the registration processor 231 aligns the spectral fundus image to the en-face image by performing an affine transformation or a Helmert transformation on the spectral fundus image to be aligned.

[0115] In some embodiments, the alignment processor 231 aligns the spectral fundus image using a reference image that differs for each wavelength range (for each depth position in the fundus). The reference image that differs for each wavelength range may be, for example, a shadowgram (en-face image or C-scan image with different depth positions) with a different integration range depending on the wavelength range.

[0116] In some embodiments, the registration processor 231 performs registration with a different OCT en face image (en-face image, C-scan image) for each spectral fundus image. The OCT en face image may be an image with the highest correlation with the spectral fundus image to be registered.

[0117] Here, the spectral fundus image is formed based on return light from various depth positions. As a result, each of the multiple spectral fundus images has a characteristic of change in intensity of return light from different depth positions for each wavelength range. Therefore, the spectral fundus image can be easily aligned with the OCT front image at the depth position (layer region) corresponding to the spectral fundus image. For example, in the wavelength range up to 600 nm, the retinal blood vessels become clearer, in the wavelength range from 620 nm to 690 nm, the optic disc becomes clearer, and in the wavelength range of 660 nm or more, the choroidal blood vessels become clearer. Therefore, the alignment processing unit 231 aligns the spectral fundus image based on the OCT front image at the depth position according to the wavelength range of the spectral fundus image, thereby enabling highly accurate alignment of multiple spectral fundus images. For example, in the spectral fundus image, the parts depicted are different depending on the wavelength range, so compared to the method of the comparative example in which alignment is performed using a spectral fundus image with a close wavelength range, alignment is performed using an OCT front image formed based on OCT data with a higher resolution in the depth direction than the spectral fundus image, so that the spectral fundus image can be aligned with high accuracy.

[0118] <Analysis section 232> 4, the analysis unit 232 includes a characteristic part identification unit 232A, a three-dimensional position calculation unit 232B, a position shift processing unit 232C, and an image quality processing unit 232D. Either the position shift processing unit 232C or the image quality processing unit 232D may be omitted.

[0119] The analysis unit 232 can analyze an image (including a spectral fundus image) of the subject's eye E to identify a characteristic portion depicted in the image. For example, the analysis unit 232 determines a three-dimensional position of the subject's eye E based on the positions of the anterior eye cameras 5A and 5B and the positions of the identified characteristic portions. The main controller 211 aligns the optical system with the subject's eye E by moving the optical system relative to the subject's eye E based on the determined three-dimensional position.

[0120] Furthermore, the analysis unit 232 can identify characteristic parts used in the registration process executed by the registration processing unit 231 described above.

[0121] The analysis unit 232 can execute a predetermined analysis process on the multiple spectral fundus images that have been subjected to the alignment process by the alignment processing unit 231. Examples of the predetermined analysis process include a comparison process of any two of the multiple spectral fundus images, an extraction process of a common area or a difference area identified by the comparison process, a process of identifying a site of interest or a characteristic area in at least one of the multiple spectral fundus images, a process of identifying and displaying the above-mentioned common area, the above-mentioned difference area, the above-mentioned site of interest, or the above-mentioned characteristic area in the spectral fundus images, a synthesis process of at least two images of the multiple spectral fundus images, and the like.

[0122] <Characteristic part identification unit 232A> The characteristic site identifying unit 232A identifies a position in the captured image corresponding to the characteristic site of the anterior eye Ea (called a characteristic site) by analyzing each captured image obtained by the anterior eye cameras 5A and 5B. As the characteristic site, for example, the pupil area of ​​the subject's eye E, the pupil center position of the subject's eye E, the pupil centroid position, the corneal center position, the corneal apex position, the subject's eye center position, or the iris is used. A specific example of a process for identifying the pupil center position of the subject's eye E will be described below.

[0123] First, the characteristic part identifying unit 232A identifies an image area (pupil area) corresponding to the pupil of the subject's eye E based on the distribution of pixel values ​​(such as brightness values) of the captured image. Since the pupil is generally depicted with lower brightness than other parts, the pupil area can be identified by searching for an image area with low brightness. At this time, the pupil area may be identified taking into consideration the shape of the pupil. In other words, the characteristic part identifying unit 232A can be configured to identify the pupil area by searching for an image area with a substantially circular shape and low brightness.

[0124] Next, characteristic site specifying unit 232A specifies the center position of the specified pupil region. Since the pupil is approximately circular as described above, the outline of the pupil region is specified, and the center position of this outline (approximate circle or approximate ellipse) is specified, and this can be set as the pupil center position. Alternatively, the center of gravity of the pupil region may be found, and this center position may be specified as the pupil center position.

[0125] It should be noted that even when identifying a feature position corresponding to another feature portion, it is possible to identify the feature position based on the distribution of pixel values ​​of the captured image in the same manner as described above.

[0126] The characteristic site specifying unit 232A can sequentially specify characteristic positions corresponding to characteristic sites in the captured images sequentially obtained by the anterior eye cameras 5A and 5B. In addition, the characteristic site specifying unit 232A may specify characteristic positions every arbitrary number of frames (one or more) in the captured images sequentially obtained by the anterior eye cameras 5A and 5B.

[0127] Similarly, the characteristic part specifying part 232A can specify a characteristic part of the spectral fundus image. In this case, the characteristic part may be a blood vessel region, a diseased part, a part with a characteristic change in pixel brightness, etc. The characteristic part of the spectral fundus image may be specified by using the operation part 240B of the user interface 240.

[0128] <3D position calculation unit 232B> The three-dimensional position calculation unit 232B specifies the three-dimensional position of the characteristic portion as the three-dimensional position of the subject's eye E based on the positions of the anterior eye cameras 5A and 5B and the characteristic position corresponding to the characteristic portion specified by the characteristic portion specification unit 232A. As disclosed in JP 2013-248376 A, the three-dimensional position calculation unit 232B calculates the three-dimensional position of the subject's eye E by applying a known trigonometric method to the positions (known) of the two anterior eye cameras 5A and 5B and the positions corresponding to the characteristic portions in the two captured images. The three-dimensional position calculated by the three-dimensional position calculation unit 232B is sent to the main controller 211. The main controller 211 controls the moving mechanism 150 based on the three-dimensional position so that the positions of the optical axis of the optical system in the x and y directions coincide with the positions of the three-dimensional position in the x and y directions and the distance in the z direction is a predetermined working distance.

[0129] <Positional Misalignment Processing Unit 232C> The misalignment processing unit 232C performs evaluation processing of misalignment of the acquired multiple spectral fundus images. Specifically, the misalignment processing unit 232C obtains misalignment information of the acquired multiple spectral fundus images. The misalignment information includes a displacement relative to a reference position and a direction of the displacement. The misalignment processing unit 232C determines whether or not the misalignment of the spectral fundus images (part or all) is large based on the obtained misalignment information. When it is determined that the misalignment is large, the control unit 210 causes a part or all of the multiple spectral fundus images to be reacquired.

[0130] As shown in FIG. 5, the position shift processing unit 232C includes a position shift information generating unit 2321C and a position shift determining unit 2322C.

[0131] <Displacement information generating unit 2321C> The positional deviation information generating unit 2321C obtains positional deviation information of the spectral fundus image relative to the reference image. The reference image may be any of the multiple spectral fundus images or the above-mentioned en-face image. When any of the multiple spectral fundus images is adopted as the reference image, examples of the reference image include a spectral fundus image having a wavelength range adjacent to the spectral fundus image for which the positional deviation information is calculated, a spectral fundus image having the shortest wavelength range among the multiple spectral fundus images, a spectral fundus image having the longest wavelength range among the multiple spectral fundus images, a spectral fundus image having a wavelength range approximately in the middle of the analysis wavelength range among the multiple spectral fundus images, and a spectral fundus image in which a predetermined characteristic portion is most clearly depicted (highly contrasted) among the multiple spectral fundus images.

[0132] When the reference image is a spectral fundus image having an adjacent wavelength range to the spectral fundus image for which the misalignment information is to be calculated, the misalignment information generating unit 2321C obtains misalignment information between the first and second spectral fundus images by analyzing the first and second spectral fundus images among the acquired multiple spectral fundus images. For example, the misalignment information generating unit 2321C identifies displacements (amount of displacement, direction of displacement) of positions corresponding to the characteristic sites (blood vessels, diseased sites, optic disc, macula, etc.) identified by the characteristic site identifying unit 232A in the first and second spectral fundus images, and generates misalignment information based on the identified displacements. The misalignment information generating unit 2321C can generate misalignment information for all of the multiple spectral fundus images.

[0133] <Position deviation determination unit 2322C> The positional deviation determining part 2322C determines whether or not the positional deviation is large for at least one of the spectral fundus images based on the positional deviation information calculated by the positional deviation information generating part 2321C.

[0134] For example, the positional deviation determination unit 2322C determines whether or not the positional deviation is large for each spectral fundus image. In this case, the positional deviation determination unit 2322C determines that the positional deviation of the spectral fundus image is large when the positional deviation is larger than a predetermined threshold. In some embodiments, the threshold is determined according to the wavelength range. For example, the threshold may be smaller (or larger) as the wavelength range becomes shorter, and the threshold may be larger (or smaller) as the wavelength range becomes longer. For example, when the wavelength range in which the desired feature is most clearly depicted is known, the threshold determined according to the wavelength range may be set to be the smallest.

[0135] For example, when a determination result of whether or not the positional deviation is large is obtained for each spectral fundus image, if it is determined that the positional deviation is large for all of the multiple spectral fundus images, the positional deviation determination unit 2322C determines that the positional deviation of the multiple spectral fundus images (image group) is large. On the other hand, if it is determined that the positional deviation is not large for any of the multiple spectral fundus images, the positional deviation determination unit 2322C determines that the positional deviation of the multiple spectral fundus images (image group) is not large.

[0136] For example, when a determination result of whether or not the positional deviation is large for each spectral fundus image is obtained, the positional deviation determination unit 2322C compares the minimum (or maximum) of the multiple positional deviations with a threshold value. If the minimum (or maximum) positional deviation is equal to or greater than the threshold value, the positional deviation determination unit 2322C determines that the positional deviation of the multiple spectral fundus images (image group) is large. On the other hand, if the minimum (or maximum) positional deviation is less than the threshold value, the positional deviation determination unit 2322C determines that the positional deviation of the multiple spectral fundus images (image group) is not large.

[0137] For example, when a determination result of whether or not the positional deviation is large for each spectral fundus image is obtained, the positional deviation determination unit 2322C applies a predetermined statistical operation to the positional deviation obtained for the multiple spectral fundus images to calculate a statistical value. The type of this statistical value may be any type, such as an average value, a minimum value, a maximum value, a mode value, or a median value. Note that the case where the statistical value is the minimum value or the maximum value corresponds to the above example. The positional deviation determination unit 2322C compares the calculated statistical value with a predetermined threshold value. If the statistical value is equal to or greater than the threshold value, the positional deviation determination unit 2322C determines that the positional deviation of the multiple spectral fundus images (image group) is large. On the other hand, if the statistical value is less than the threshold value, the positional deviation determination unit 2322C determines that the positional deviation of the multiple spectral fundus images (image group) is not large.

[0138] For example, when a determination result of whether or not the positional deviation is large for each spectral fundus image is obtained, the positional deviation determination unit 2322C compares each of one or more positional deviations obtained for a representative image of the multiple spectral fundus images with a predetermined threshold. Examples of the representative image include a spectral fundus image having the shortest wavelength range among the multiple spectral fundus images, a spectral fundus image having the longest wavelength range among the multiple spectral fundus images, a spectral fundus image having a wavelength range approximately in the middle of the analysis wavelength range among the multiple spectral fundus images, and a spectral fundus image in which a predetermined characteristic part is most clearly depicted (with high contrast) among the multiple spectral fundus images. When all of the one or more positional deviations are equal to or greater than the threshold, the positional deviation determination unit 2322C determines that the positional deviation of the multiple spectral fundus images (image group) is large. On the other hand, when any of the one or more positional deviations is less than the threshold, the positional deviation determination unit 2322C determines that the positional deviation of the multiple spectral fundus images (image group) is not large.

[0139] In some embodiments, the misalignment information generating unit 2321C generates misalignment information for some of the spectral fundus images selected from the plurality of spectral fundus images. In this case, the misalignment determining unit 2322C performs the above-mentioned determination process on the generated misalignment information. That is, the misalignment is evaluated using the plurality of spectral fundus images for which the misalignment information is generated.

[0140] The misalignment determination process according to the embodiment is not limited to the above example, and the process executed by the misalignment processor 232C is not limited to the process based on the misalignment information generated by the misalignment information generator 2321C.

[0141] The control unit 210 (main control unit 211) executes reacquisition of some or all of the multiple spectral fundus images based on the determination result obtained by the positional deviation determination unit 2322C. For example, the control unit 210 executes reacquisition of all of the multiple spectral fundus images when it is determined that the positional deviation of the multiple spectral fundus images is large.

[0142] In addition, when a determination result of whether or not the positional deviation is large for each spectral fundus image is obtained, for example, the control unit 210 can cause only the spectral fundus image determined to have a large positional deviation to be reacquired. For example, the control unit 210 can cause a plurality of spectral fundus images (including adjacent spectral fundus images) corresponding to a second wavelength range including the first wavelength range of the spectral fundus image determined to have a large positional deviation to be reacquired.

[0143] In some embodiments, when the number of spectral fundus images determined to have a large positional deviation is equal to or greater than a predetermined threshold, the controller 210 executes reacquisition of some or all of the multiple spectral fundus images.

[0144] <Image quality processing unit 232D> The image quality processing unit 232D performs evaluation processing of the image quality of the acquired spectral fundus images. Specifically, the image quality processing unit 232D calculates an evaluation value of the image quality of the acquired spectral fundus images. The image quality processing unit 232D judges whether the image quality is good or not (whether the image quality is high quality or not) based on the calculated evaluation value. When it is determined that the image quality is not good, the control unit 210 causes some or all of the spectral fundus images to be reacquired.

[0145] As shown in FIG. 6, the image quality processing section 232D includes an image quality evaluation value calculation section 2321D and an image quality determination section 2322D.

[0146] <Image quality evaluation value calculation unit 2321D> The image quality evaluation value calculation unit 2321D calculates an evaluation value of the image quality of the spectral fundus images. In some embodiments, the image quality evaluation value calculation unit 2321D calculates an evaluation value that quantitatively represents the image quality of the multiple spectral fundus images.

[0147] The image quality evaluation value calculation unit 2321D may perform any image quality evaluation value calculation process, such as signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), root-mean-square (RMS) granularity, Wiener spectrum, modulation transfer function (MTF), quality index (QI), or other known techniques.

[0148] The evaluation value may be any parameter that quantitatively expresses the quality of an image, and typically, the higher the quality of an image, the larger the evaluation value.

[0149] For example, the image quality evaluation value calculation unit 2321D calculates an image quality value (IQ value) as an evaluation value for the spectral fundus image. In this case, the image quality evaluation value calculation unit 2321D detects an image area (tissue image area) corresponding to a predetermined tissue (site) and other image areas (background area, non-tissue image area) by applying a predetermined analysis process to an evaluation area set in the image to be evaluated. Next, the image quality evaluation value calculation unit 2321D generates a histogram of brightness in the tissue image area and generates a histogram of brightness in the background area. Then, the image quality evaluation value calculation unit 2321D calculates an image quality evaluation value (IQ value) from the degree of overlap of these two histograms. For example, the IQ value is defined in the range of "0" to "100" so that the IQ value is 0 when both histograms completely overlap, and the IQ value is 100 when both histograms are completely separated. This image quality evaluation calculation may include, for example, normalization of two histograms, generation of a probability distribution function, calculation of an IQ value using a predetermined calculation formula, and so on.

[0150] In this manner, the image quality evaluation value calculation unit 2321D is configured to execute the processes of identifying a tissue image region corresponding to a specific tissue and a background region in the image to be evaluated, creating a first histogram showing the frequency distribution of luminance in the tissue image region, creating a second histogram showing the frequency distribution of luminance in the background region, and calculating an image quality evaluation value (IQ value) as image quality evaluation data based on the first and second histograms.

[0151] The process of calculating the evaluation value of image quality according to the embodiment is not limited to the above example.

[0152] <Image quality determination unit 2322D> The image quality determination unit 2322D determines whether the image quality of at least one of the plurality of spectral fundus images is good or bad based on the evaluation value calculated by the image quality evaluation value calculation unit 2321 D. In some embodiments, the image quality determination unit 2322D determines whether the plurality of spectral fundus images are a group of images, based on the calculated evaluation value.

[0153] For example, when an IQ value is obtained for each spectral fundus image, the image quality determination unit 2322D compares each of the multiple IQ values ​​obtained for the multiple spectral fundus images with a predetermined threshold. If all of the multiple IQ values ​​are equal to or greater than the threshold, the image quality determination unit 2322D determines that the quality of the multiple spectral fundus images (image group) is good. On the other hand, if any of the multiple IQ values ​​is less than the threshold, the image quality determination unit 2322D determines that the quality of the multiple spectral fundus images (image group) is not good.

[0154] For example, when an IQ value is obtained for each spectral fundus image, the image quality determination unit 2322D selects the lowest IQ value among the multiple IQ values ​​obtained for the multiple spectral fundus images and compares this minimum IQ value with a predetermined threshold. If the minimum IQ value is equal to or greater than the threshold, the image quality determination unit 2322D determines that the quality of the multiple spectral fundus images (image group) is good. On the other hand, if the minimum IQ value is less than the threshold, the image quality determination unit 2322D determines that the quality of the multiple spectral fundus images (image group) is not good.

[0155] For example, when an IQ value is obtained for each spectral fundus image, the image quality determination unit 2322D applies a predetermined statistical operation to the multiple IQ values ​​obtained for the multiple spectral fundus images to calculate a statistical value. The type of this statistical value may be any, for example, an average value, a minimum value, a maximum value, a mode value, a median value, etc. Note that the case where the statistical value is the minimum value corresponds to the above example. The image quality determination unit 2322D compares the calculated statistical value with a predetermined threshold value. If the statistical value is equal to or greater than the threshold value, the image quality determination unit 2322D determines that the quality of the multiple spectral fundus images (image group) is good. On the other hand, if the statistical value is less than the threshold value, the image quality determination unit 2322D determines that the quality of the multiple spectral fundus images (image group) is not good.

[0156] For example, when an IQ value is obtained for each spectral fundus image, the image quality determination unit 2322D compares each of the one or more IQ values ​​obtained for the representative image of the multiple spectral fundus images with a predetermined threshold. Examples of the representative image include a spectral fundus image having the shortest wavelength range among the multiple spectral fundus images, a spectral fundus image having the longest wavelength range among the multiple spectral fundus images, a spectral fundus image having a wavelength range approximately in the middle of the analysis wavelength range among the multiple spectral fundus images, and a spectral fundus image in which a predetermined characteristic part is most clearly depicted (with high contrast) among the multiple spectral fundus images. When all of the IQ values ​​of one or more are equal to or greater than the threshold, the image quality determination unit 2322D determines that the quality of the multiple spectral fundus images (image group) is good. On the other hand, when any of the IQ values ​​of one or more is less than the threshold, the image quality determination unit 2322D determines that the quality of the multiple spectral fundus images (image group) is not good.

[0157] For example, when an IQ value is obtained for each spectral fundus image, the image quality determination unit 2322D applies a predetermined statistical operation to two or more IQ values ​​obtained for a representative image of the multiple spectral fundus images to calculate a statistical value. The type of the statistical value may be any type, such as an average value, a minimum value, a maximum value, a mode value, or a median value. The image quality determination unit 2322D compares the calculated statistical value with a predetermined threshold value. If the statistical value is equal to or greater than the threshold value, the image quality determination unit 2322D determines that the quality of the multiple spectral fundus images (image group) is good. On the other hand, if the statistical value is less than the threshold value, the image quality determination unit 2322D determines that the quality of the multiple spectral fundus images (image group) is not good.

[0158] In some embodiments, the image quality evaluation value calculation unit 2321D calculates an evaluation value for a part of the spectral fundus images selected from the plurality of spectral fundus images. In this case, the image quality determination unit 2322D performs the above-mentioned determination process on the calculated evaluation value. That is, the image quality is evaluated using the plurality of spectral fundus images for which the evaluation values ​​are calculated.

[0159] The image quality determination process according to the embodiment is not limited to the above example, and the process executed by the image quality processor 232D is not limited to the process based on the evaluation value calculated by the image quality evaluation value calculator 2321D.

[0160] The control unit 210 (main control unit 211) executes reacquisition of some or all of the multiple spectral fundus images based on the determination result obtained by the image quality determination unit 2322D. For example, the control unit 210 causes all of the multiple spectral fundus images to be reacquired when it is determined that the quality of the multiple spectral fundus images is not good.

[0161] Furthermore, when an IQ value is obtained for each spectral fundus image, for example, the control unit 210 can cause only the spectral fundus image whose IQ value is less than the threshold (the spectral fundus image whose image quality is determined to be poor based on the IQ value) to be reacquired. For example, the control unit 210 can cause a plurality of spectral fundus images (including adjacent spectral fundus images) corresponding to a second wavelength range including the first wavelength range of the spectral fundus image whose IQ value is less than the threshold to be reacquired.

[0162] In some embodiments, when the number of spectral fundus images with IQ values ​​less than the threshold is equal to or greater than a predetermined threshold, the controller 210 causes reacquisition of some or all of the plurality of spectral fundus images.

[0163] <User Interface 240> The user interface 240 includes a display unit 240A and an operation unit 240B. The display unit 240A includes the display device 3. The operation unit 240B includes various operation devices and input devices.

[0164] The user interface 240 may include a device that combines a display function and an operation function, such as a touch panel. In other embodiments, at least a part of the user interface may not be included in the ophthalmic device. For example, the display device may be an external device connected to the ophthalmic device.

[0165] <Communications Division 250> The communication unit 250 has a function for communicating with an external device (not shown). The communication unit 250 has a communication interface according to the connection form with the external device. Examples of the external device include a server device, an OCT device, a scanning laser ophthalmoscope, a slit lamp ophthalmoscope, an ophthalmic measurement device, and an ophthalmic treatment device. Examples of the ophthalmic measurement device include an eye refraction examination device, a tonometer, a specular microscope, a wavefront analyzer, a perimeter, and a microperimeter. Examples of the ophthalmic treatment device include a laser treatment device, a surgical device, and a surgical microscope. In addition, the external device may be a device (reader) that reads information from a recording medium, or a device (writer) that writes information to a recording medium. Furthermore, the external device may be a hospital information system (HIS) server, a DICOM (Digital Imaging and COmmunication in Medicine) server, a doctor terminal, a mobile terminal, a personal terminal, a cloud server, and the like.

[0166] The arithmetic control unit 200 (the control unit 210, the image forming unit 220, and the data processing unit 230) are an example of an "ophthalmologic information processing device" according to the embodiment. The multiple spectral fundus images (spectral images) are an example of "multiple images of the test eye obtained by sequentially receiving return light from the test eye having different wavelength ranges" according to the embodiment. A configuration for acquiring multiple spectral images of the test eye by a communication function not shown, or the fundus camera unit 2 (the illumination optical system 10, the photographing optical system 30) are an example of an "acquisition unit" according to the embodiment. The alignment processing unit 231 is an example of an "alignment unit" according to the embodiment. The en-face image or the C-scan image is an example of an "OCT front image" according to the embodiment. The position shift determination unit 2322C is an example of a "first determination unit" according to the embodiment. The image quality evaluation value calculation unit 2321D is an example of a "quality evaluation unit" according to the embodiment. The image quality determination unit 2322D is an example of a "second determination unit" according to the embodiment. The photographing optical system 30 is an example of a "light receiving optical system" according to the embodiment. The optical system from the OCT unit 100 to the objective lens 22 is an example of the "OCT optical system" according to the embodiment.

[0167] <Operation> An example of the operation of the ophthalmologic apparatus 1 will be described.

[0168] 7 to 9 show an operation example of the ophthalmic apparatus 1 according to the embodiment. Fig. 7 shows a flow diagram of the operation example of the ophthalmic apparatus 1. Fig. 8 shows a flow diagram of the operation example of step S8 in Fig. 7. Fig. 9 shows a flow diagram of another operation example of step S8 in Fig. 7.

[0169] The storage unit 212 stores a computer program for implementing the processes shown in Figures 7 to 9. The main control unit 211 operates in accordance with this computer program to execute the processes shown in Figures 7 to 9.

[0170] (S1: Alignment) First, the main controller 211 executes alignment.

[0171] For example, the main controller 211 controls the anterior eye cameras 5A and 5B to capture images of the anterior eye Ea of the subject's eye E substantially simultaneously. The characteristic site identification unit 232A, under the control of the main controller 211, analyzes a pair of anterior eye images captured substantially simultaneously by the anterior eye cameras 5A and 5B to identify the pupil center position of the subject's eye E as a characteristic site. The three-dimensional position calculation unit 232B obtains the three-dimensional position of the subject's eye E. This process includes, for example, calculation processing using trigonometry based on the positional relationship between the pair of anterior eye cameras 5A and 5B and the subject's eye E, as described in JP 2013-248376 A.

[0172] The main controller 211 controls the moving mechanism 150 based on the three-dimensional position of the subject's eye E obtained by the three-dimensional position calculator 232B so that the optical system (e.g., the fundus camera unit 2) and the subject's eye E have a predetermined positional relationship. Here, the predetermined positional relationship is a positional relationship in which the optical system can be used to capture and inspect the subject's eye E. As a typical example, when the three-dimensional position (x coordinate, y coordinate, z coordinate) of the subject's eye E is obtained by the three-dimensional position calculator 232B, a position where the x coordinate and y coordinate of the optical axis of the objective lens 22 respectively match the x coordinate and y coordinate of the subject's eye E and where the difference between the z coordinate of the objective lens 22 (front lens surface) and the z coordinate of the subject's eye E (corneal surface) is equal to a predetermined distance (working distance) is set as the moving destination of the optical system.

[0173] (S2: Autofocus) Next, the main control unit 211 starts autofocus.

[0174] For example, the main controller 211 controls the focus optical system 60 to project a split index onto the subject's eye E. The analysis unit 232, under the control of the main controller 211, extracts a pair of split index images by analyzing the observation image of the fundus Ef onto which the split index is projected, and calculates the relative deviation between the pair of split indexes. The main controller 211 controls the focusing driver 31A and the focusing driver 43A based on the calculated deviation (deviation direction, deviation amount).

[0175] (S3: OCT measurement) Next, the main controller 211 executes the OCT measurement.

[0176] For example, the main controller 211 controls the LCD 39 to present a fixation target to the subject's eye E. Then, the main controller 211 controls the optical scanner 42 and the OCT unit 100 to start OCT measurement. When the OCT measurement starts, the OCT unit 100 sends data collected in each scan to the image forming unit 220. The image forming unit 220 forms a plurality of B-scan images from the data collected in each scan and sends them to the controller 210. The controller 210 sends a plurality of B-scan images corresponding to each scan to the data processor 230. For example, the data processor 230 forms a three-dimensional image from the plurality of B-scan images corresponding to each scan.

[0177] (S4: Set wavelength range) Next, the main controller 211 controls the tunable filter 80 to set the wavelength selection range of the transmitted light to a predetermined wavelength range. An example of the predetermined wavelength range is an initial wavelength range when the selection of the wavelength range is sequentially repeated so as to cover the analysis wavelength range.

[0178] (S5: Acquire image data) Next, the main controller 211 causes image data of the spectral fundus image to be acquired.

[0179] For example, the main controller 211 controls the illumination optical system 10 to illuminate the subject's eye E with illumination light, captures the reception result of the reflected light of the illumination light obtained by the image sensor 38, and acquires image data of the spectral fundus image.

[0180] (S6:Next?) Next, the main controller 211 determines whether to acquire a spectral fundus image in the next wavelength range. For example, when the wavelength selection is changed sequentially in a predetermined wavelength range step within the analysis wavelength range, the main controller 211 can determine whether to acquire a next spectral fundus image based on the number of times the wavelength range is changed. For example, the main controller 211 can determine whether to acquire a next spectral fundus image by determining whether all of a plurality of predetermined wavelength ranges have been selected.

[0181] In step S6, when it is determined that the next spectral fundus image is to be acquired (step S6: Y), the operation of the ophthalmic apparatus 1 proceeds to step S7. In step S6, when it is determined that the next spectral fundus image is not to be acquired (step S6: N), the operation of the ophthalmic apparatus 1 proceeds to step S8.

[0182] (S7: Change wavelength range) When it is determined in step S6 that the next spectral fundus image is to be acquired (step S6: Y), the main controller 211 controls the tunable filter 80 to change the selection range of the transmitted light to be selected next. Then, the operation of the ophthalmologic apparatus 1 proceeds to step S5.

[0183] (S8: Re-shooting process) When it is determined in step S6 that the next spectral fundus image is not to be acquired (step S6: N), the main controller 211 executes a re-photographing process and re-acquires a spectral fundus image as necessary. Details of step S8 will be described later.

[0184] Following step S8, the operation of the ophthalmologic apparatus 1 ends (END).

[0185] For example, in step S8, a process is performed to determine whether or not to re-acquire spectral fundus images by evaluating the positional shift of multiple spectral fundus images (Figure 8), or a process is performed to determine whether or not to re-acquire spectral fundus images by evaluating the image quality of multiple spectral fundus images (Figure 9).

[0186] First, as shown in FIG. 8, a case in which positional deviations of a plurality of spectral fundus images are evaluated will be described.

[0187] (S11: Alignment) When multiple spectral fundus images within a predetermined analysis wavelength range are acquired according to the flow shown in FIG. 7, the main controller 211 controls the alignment processor 231 to execute alignment processing of the multiple spectral fundus images based on the OCT data acquired in step S3.

[0188] The OCT data may be the detection result of the interference light acquired in step S3, a B-scan image, or an en-face image.

[0189] (S12: Generate position deviation information) Next, the main controller 211 controls the positional deviation information generator 2321C to generate positional deviation information of the multiple spectral fundus images that have been subjected to the position alignment process in step S11.

[0190] (S13:Reacquisition?) Next, the main controller 211 controls the positional deviation determination unit 2322C to determine whether or not the positional deviation of the multiple spectral fundus images is large based on the positional deviation information generated in step S12. When the positional deviation determination unit 2322C determines that the positional deviation of the multiple spectral fundus images is large, the main controller 211 causes the multiple spectral fundus images to be reacquired part or all of the multiple spectral fundus images. The reacquired spectral fundus image may be the spectral fundus image determined to have a large positional deviation, or two or more spectral fundus images including the spectral fundus image determined to have a large positional deviation.

[0191] When it is determined in step S13 that the positional deviation is large (step S13: Y), the process of step S8 proceeds to step S14. When it is determined in step S13 that the positional deviation is not large (step S13: N), the process of step S8 ends (END).

[0192] (S14: Set wavelength range) When it is determined in step S13 that the positional deviation is large (step S13: Y), the main controller 211 controls the tunable filter 80 to set the wavelength selection range of the transmitted light to a wavelength range corresponding to the spectral fundus image to be reacquired.

[0193] (S15: Acquire image data) Next, the main controller 211 causes image data of the spectral fundus image to be acquired, similarly to step S5.

[0194] (S16:Next?) Next, the main controller 211 determines whether or not to reacquire a spectral fundus image in the next wavelength range.

[0195] In step S16, when it is determined that the next spectral fundus image is to be reacquired (step S16: Y), the process of step S8 proceeds to step S17. In step S16, when it is determined that the next spectral fundus image is not to be reacquired (step S16: N), the process of step S8 ends (END).

[0196] (S17: Change wavelength range) When it is determined in step S16 that the next spectral fundus image is to be reacquired (step S16: Y), the main controller 211 controls the tunable filter 80 to change the selection range of the transmitted light to be selected next. Then, the process of step S8 proceeds to step S15.

[0197] Next, a case where image quality deviations of a plurality of spectral fundus images are evaluated as shown in FIG. 9 will be described.

[0198] (S21: Alignment) When multiple spectral fundus images within a predetermined analysis wavelength range are acquired according to the flow shown in FIG. 7, the main controller 211 controls the alignment processor 231 to perform alignment processing of the multiple spectral fundus images based on the OCT data acquired in step S3, as in step S11.

[0199] (S22: Calculate image quality evaluation value) Next, the main controller 211 controls the image quality evaluation value calculator 2321D to calculate the evaluation values ​​of the image quality of the multiple spectral fundus images that have been subjected to the alignment process in step S11.

[0200] (S23:Reacquisition?) Next, the main controller 211 controls the image quality determination unit 2322D to determine whether the image quality of the multiple spectral fundus images is good or bad based on the evaluation value calculated in step S22. When the image quality determination unit 2322D determines that the image quality of the multiple spectral fundus images is not good, the main controller 211 causes some or all of the multiple spectral fundus images to be reacquired. The reacquired spectral fundus image may be the spectral fundus image determined to have a poor image quality, or two or more spectral fundus images including the spectral fundus image determined to have a poor image quality.

[0201] When it is determined in step S23 that the image quality is not good (step S23: Y), the process of step S8 proceeds to step S24. When it is determined in step S23 that the image quality is not good (step S23: N), the process of step S8 ends (END).

[0202] (S24: Set wavelength range) When it is determined in step S23 that the image quality is not good (step S23: Y), the main controller 211 controls the tunable filter 80 to set the wavelength selection range of the transmitted light to a wavelength range corresponding to the spectral fundus image to be reacquired.

[0203] (S25: Acquire image data) Next, the main controller 211 causes image data of the spectral fundus image to be acquired, similarly to step S15.

[0204] (S26:Next?) Next, the main controller 211 determines whether or not to reacquire a spectral fundus image in the next wavelength range.

[0205] In step S26, when it is determined that the next spectral fundus image is to be reacquired (step S26: Y), the process of step S8 proceeds to step S27. In step S26, when it is determined that the next spectral fundus image is not to be reacquired (step S26: N), the process of step S8 ends (END).

[0206] (S27: Change wavelength range) When it is determined in step S26 that the next spectral fundus image is to be reacquired (step S26: Y), the main controller 211 controls the tunable filter 80 to change the selection range of the transmitted light to be selected next. Then, the process of step S8 proceeds to step S25.

[0207] <Effect> An ophthalmological information processing apparatus, an ophthalmological apparatus, an ophthalmological information processing method, and a program according to an embodiment will be described.

[0208] An ophthalmologic information processing device (control unit 210, image forming unit 220, and data processing unit 230) according to some embodiments includes an acquisition unit (a configuration for acquiring multiple spectral images of the subject's eye by a communication function not shown, or a fundus camera unit 2 (illumination optical system 10, photographing optical system 30)) and an alignment unit (alignment processing unit 231). The acquisition unit acquires multiple images of the subject's eye (spectral images, spectral fundus images, and spectral anterior segment images) obtained by sequentially receiving return light having different wavelength ranges from the subject's eye (E) illuminated with illumination light. The alignment unit aligns the multiple images based on OCT data obtained by performing optical coherence tomography on the subject's eye.

[0209] According to this configuration, even if fixation shift and alignment shift between the test eye and the imaging unit occur while acquiring a plurality of images, the spectral characteristics between the images can be easily compared. In particular, by performing alignment based on OCT data, it becomes possible to align the images with high accuracy based on information on the depth direction (depth direction) of the eye, and it becomes possible to improve the accuracy of the spectral analysis at the observation site of the test eye.

[0210] In some embodiments, the OCT data is a detection result of interference light obtained by performing optical coherence tomography on the test eye, a B-scan image, an OCTA (OCT Angiography) image, or an en-face image.

[0211] According to this configuration, it becomes possible to easily align a plurality of images with high accuracy by using OCT data that can be acquired by an existing optical system.

[0212] In some embodiments, the alignment unit aligns the image of the subject's eye with the OCT front image at a depth position corresponding to the wavelength range of the image.

[0213] According to this configuration, since the image to be aligned is aligned based on the OCT front image at the depth position corresponding to the wavelength range of the image to be aligned, highly accurate alignment is possible.

[0214] Some embodiments include a misalignment information generating unit (2321C) that acquires misalignment information between a first image and a second image by analyzing a first image and a second image of the plurality of images.

[0215] With this configuration, if fixation shift and alignment shift between the test eye and the imaging unit occur while acquiring multiple images, the quality of the multiple acquired images can be quantitatively evaluated by determining the positional shift between the multiple images.

[0216] Some embodiments include a first judgment unit (positional deviation judgment unit 2322C) that judges whether or not the positional deviation between the first image and the second image is large based on the positional deviation information, and the acquisition unit re-acquires the first image or the second image based on the judgment result obtained by the first judgment unit.

[0217] According to such a configuration, it is possible to acquire a plurality of images with small positional deviation, and it is possible to improve the accuracy of the spectroscopic analysis of the observation site of the subject's eye.

[0218] Some embodiments include a quality assessment unit (image quality assessment value calculation unit 2321D) that calculates an assessment value of the quality of at least one of the multiple images.

[0219] According to this configuration, if fixation misalignment and alignment misalignment between the test eye and the imaging unit occur while acquiring multiple images, an evaluation value of the quality of at least one of the multiple images can be calculated, making it possible to quantitatively evaluate the quality of the multiple acquired images.

[0220] Some embodiments include a second judgment unit (image quality judgment unit 2322D) that judges the quality of the multiple images based on the evaluation value, and based on the judgment result obtained by the second judgment unit, the acquisition unit re-acquires at least one of the multiple images or all of the multiple images.

[0221] According to such a configuration, it is possible to acquire a plurality of images with good image quality, and it is possible to improve the accuracy of the spectroscopic analysis of the observation site of the subject's eye.

[0222] An ophthalmic apparatus (1) according to some embodiments includes an illumination optical system (10), a light receiving optical system (imaging optical system 30), an OCT optical system (an optical system from an OCT unit 100 to an objective lens 22), and any one of the above-mentioned ophthalmic information processing devices. The illumination optical system illuminates the subject's eye with illumination light. The light receiving optical system sequentially receives return light of the illumination light from the subject's eye, which have different wavelength ranges. The OCT optical system performs optical coherence tomography on the subject's eye. The acquisition unit sequentially acquires a plurality of images based on the light receiving results obtained by the light receiving optical system.

[0223] With this configuration, even if fixation shift or alignment shift between the test eye and the imaging unit occurs while acquiring multiple images, it is possible to align the images with high precision based on information about the depth direction of the eye, making it possible to provide an ophthalmic device that can improve the accuracy of spectroscopic analysis at the observation site of the test eye.

[0224] An ophthalmologic information processing method according to some embodiments includes an acquisition step and an alignment step. The alignment step acquires a plurality of images of the test eye obtained by sequentially receiving return light having different wavelength ranges from the test eye illuminated with illumination light. The alignment step aligns the plurality of images based on OCT data obtained by performing optical coherence tomography on the test eye.

[0225] According to this method, even if fixation shift and alignment shift between the test eye and the imaging unit occur while acquiring multiple images, the spectral characteristics between the images can be easily compared. In particular, by performing alignment based on OCT data, it becomes possible to align the images with high accuracy based on information on the depth direction (depth direction) of the eye, and it becomes possible to improve the accuracy of the spectral analysis at the observation site of the test eye.

[0226] In some embodiments, the OCT data is a detection result of interference light obtained by performing optical coherence tomography on the test eye, a B-scan image, an OCTA (OCT Angiography) image, or an en-face image.

[0227] According to this method, it becomes possible to align a plurality of images simply and with high accuracy by using OCT data that can be acquired by an existing optical system.

[0228] In some embodiments, the registration step includes registering the image of the test eye with the OCT en face image at a depth position corresponding to the wavelength range of the image.

[0229] According to this method, the image to be aligned is aligned based on the OCT front image at the depth position corresponding to the wavelength range of the image to be aligned, so that highly accurate alignment is possible.

[0230] Some embodiments include a misalignment information generating step of obtaining misalignment information between a first image and a second image by analyzing a first image and a second image of the plurality of images.

[0231] According to this method, if fixation shift and alignment shift between the test eye and the imaging unit occur while acquiring multiple images, the quality of the multiple acquired images can be quantitatively evaluated by determining the positional shift between the multiple images.

[0232] Some embodiments include a first determination step of determining whether the misalignment between the first image and the second image is large based on the misalignment information, and based on the determination result obtained in the first determination step, the acquisition step re-acquires the first image or the second image.

[0233] According to such a method, it is possible to acquire a plurality of images with small positional deviation, and it is possible to improve the accuracy of the spectroscopic analysis of the observation site of the subject's eye.

[0234] Some embodiments include a quality assessment step of calculating an assessment value of the quality of at least one of the plurality of images.

[0235] According to this method, if fixation misalignment and alignment misalignment between the test eye and the imaging unit occur while acquiring multiple images, an evaluation value of the quality of at least one of the multiple images can be calculated, making it possible to quantitatively evaluate the quality of the multiple acquired images.

[0236] Some embodiments include a second determination step of determining whether the quality of the multiple images is good or bad based on the evaluation value, and based on the determination result obtained in the second determination step, the acquisition step re-acquires at least one of the multiple images or all of the multiple images.

[0237] According to such a method, it is possible to obtain a plurality of images with good image quality, and it is possible to improve the accuracy of the spectroscopic analysis of the observation site of the examinee's eye.

[0238] A program according to some embodiments causes a computer to execute each step of any of the above-described ophthalmologic information processing methods.

[0239] According to such a program, even if fixation shift or alignment shift between the test eye and the imaging unit occurs while acquiring multiple images, it becomes possible to align the images with high precision based on information about the depth direction of the eye, making it possible to provide a computer program that can improve the accuracy of spectroscopic analysis at the observation site of the test eye.

[0240] The embodiment described above is merely one example of the present invention. Those who wish to implement the present invention may make any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention.

[0241] In some embodiments, a program for causing a computer to execute the control method for an ophthalmic apparatus is stored in the storage unit 212. Such a program may be stored in any computer-readable recording medium. The recording medium may be an electronic medium that utilizes magnetism, light, magneto-optical, semiconductors, or the like. Typically, the recording medium is a magnetic tape, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, a solid-state drive, or the like. [Explanation of symbols]

[0242] 1 Ophthalmology equipment 2 Fundus camera unit 10 Illumination optical system 22 Objective Lens 30. Photographing Optical System 80 Tunable Filter 100 OCT units 210 Control section 211 Main control unit 220 Image forming section 230 Data Processing Unit 231 Alignment processing unit 232 Analysis Department 232A Characteristic part identification part 232B 3D position calculation unit 232C Position shift processing unit 2321C Position deviation information generator 2322C Position deviation judgement unit 232D Image Processing Unit 2321D Image quality evaluation value calculation unit 2322D Image quality judgment unit E. Examined eye Ef fundus LS measurement light

Claims

1. An acquisition unit that acquires a plurality of images of the eye to be examined obtained by sequentially receiving return light from the eye to be examined illuminated with illumination light and having different wavelength ranges; An alignment unit that aligns the plurality of images based on OCT data obtained by performing optical coherence tomography on the eye to be examined; An ophthalmic information processing apparatus comprising:

2. The OCT data is a detection result of interference light, a B-scan image, an OCTA (OCT Angiography) image, or an en-face image obtained by performing optical coherence tomography on the eye to be examined The ophthalmic information processing apparatus according to claim 1, characterized in that:

3. The alignment unit performs alignment between an image of the eye to be examined and an OCT frontal image at a depth position corresponding to the wavelength range of the image The ophthalmic information processing apparatus according to claim 1, characterized in that:

4. Including a misalignment information generation unit that acquires misalignment information between a first image and a second image by analyzing the first image and the second image among the plurality of images The ophthalmic information processing apparatus according to any one of claims 1 to 3, characterized in that:

5. Including a first determination unit that determines whether or not the misalignment between the first image and the second image is large based on the misalignment information, Based on the determination result obtained by the first determination unit, the acquisition unit re-acquires the first image or the second image The ophthalmic information processing apparatus according to claim 4, characterized in that:

6. Including a quality evaluation unit that calculates an evaluation value of the quality of at least one of the plurality of images The ophthalmic information processing apparatus according to any one of claims 1 to 5, characterized in that:

7. Including a second determination unit that determines whether the quality of the plurality of images is good or bad based on the evaluation value, Based on the determination result obtained by the second determination unit, the acquisition unit re-acquires at least one of the plurality of images or all of the plurality of images The ophthalmic information processing apparatus according to claim 6, characterized in that:

8. An illumination optical system that illuminates the eye to be examined with illumination light; A light receiving optical system that sequentially receives the return light of the illumination light from the eye to be examined having different wavelength ranges; An OCT optical system that performs optical coherence tomography on the eye to be examined; The ophthalmic information processing apparatus according to any one of claims 1 to 7; Including: An ophthalmic apparatus, wherein the acquisition unit sequentially acquires the plurality of images based on a light reception result obtained by the light reception optical system.

9. An acquisition step of acquiring a plurality of images of the eye to be examined, which are sequentially obtained by receiving return light having different wavelength ranges from the eye to be examined illuminated with illumination light; An alignment step of aligning the plurality of images based on OCT data obtained by performing optical coherence tomography on the eye to be examined; An ophthalmic information processing method comprising:

10. The OCT data is a detection result of interference light, a B-scan image, an OCTA (OCT Angiography) image, or an en-face image obtained by performing optical coherence tomography on the eye to be examined. The ophthalmic information processing method according to claim 9, characterized in that:

11. The alignment step performs alignment between an image of the eye to be examined and an OCT frontal image at a depth position corresponding to the wavelength range of the image. The ophthalmic information processing method according to claim 9, characterized in that:

12. The method includes a misregistration information generation step of analyzing a first image and a second image among the plurality of images to obtain misregistration information between the first image and the second image. The ophthalmic information processing method according to any one of claims 9 to 11, characterized in that:

13. The method includes a first determination step of determining whether or not there is a large misregistration between the first image and the second image based on the misregistration information. Based on the determination result obtained in the first determination step, the acquisition step re-acquires the first image or the second image. The ophthalmic information processing method according to claim 12, characterized in that:

14. The method includes a quality evaluation step of calculating an evaluation value of the quality of at least one of the plurality of images. The ophthalmic information processing method according to any one of claims 9 to 13, characterized in that:

15. The method includes a second determination step of determining whether the quality of the plurality of images is good or bad based on the evaluation value. Based on the determination result obtained in the second determination step, the acquisition step re-acquires at least one of the plurality of images or all of the plurality of images. The ophthalmic information processing method according to claim 14, characterized in that:

16. A program for causing a computer to execute each step of the ophthalmic information processing method according to any one of claims 9 to 15.

Images