Ophthalmological data processing method and ophthalmological data processing device

Abstract

To provide a novel method for measuring movement of an eye by using a scanning imaging technique.SOLUTION: An ophthalmological data processing method receives a data set and time-series data. The data set is acquired by applying optical scanning to a subject's eye following a two-dimensional pattern including a series of cycles that intersect each other. The time-series data includes data representing time-series changes in the subject's head position. Alternatively, the method generates position history data representing time-series changes in the position of the subject's eye based on the received data set. Further alternatively, the method generates evaluation data based on the generated position history data and the received time-series data.SELECTED DRAWING: Figure 4B

Description

[Technical Field]

[0001] The present invention relates to an ophthalmic data processing method and an ophthalmic data processing device. [Background technology]

[0002] Various imaging techniques are applied in the field of ophthalmology, one of which is scanning imaging, which involves projecting a beam of light sequentially onto multiple locations on a sample, collecting data, and then constructing an image of the sample from the collected data.

[0003] Optical coherence tomography (OCT) is an example of optical scanning imaging. OCT is a technique that can measure and image light-scattering media with micrometer-level or higher resolution, and is used in medical imaging and non-destructive testing. OCT is based on low-coherence interferometry and typically uses a probe light (measurement light) in the near-infrared region to ensure deep penetration into the sample of the light-scattering medium. In the field of ophthalmology, the use of near-infrared light has the advantage of preventing the eye from tracking the movement of the measurement light and preventing miosis.

[0004] OCT devices are becoming increasingly popular in ophthalmic imaging diagnostics, and not only 2D imaging but also 3D imaging, rendering, structural analysis, and functional analysis have been put into practical use, making them widely used as a powerful diagnostic tool. In addition, scanning imaging methods other than OCT, such as scanning laser ophthalmoscopy (SLO), are also used in the field of ophthalmology. Additionally, scanning imaging using light (electromagnetic waves) and ultrasound in wavelength bands other than the near-infrared region is also known.

[0005] There are various scanning modes used in OCT and SLO, but the so-called "Lissajous scan" for the purpose of correcting motion artifacts has been attracting attention in recent years (see, for example, Patent Documents 1 to 4 and Non-Patent Documents 1 and 2).

[0006] In a typical Lissajous scan, the measurement light is scanned at high speed to sequentially draw multiple loops (cycles) of a certain size, so the difference in data acquisition time from multiple positions on one cycle can be substantially ignored, and since cycles can be aligned by referencing the intersection areas of different cycles, it is possible to correct artifacts caused by sample movement. Focusing on these characteristics of the Lissajous scan, the field of ophthalmology is working to address motion artifacts caused by eye movement.

[0007] In the technique described in Non-Patent Document 1, a data set acquired by a Lissajous scan is divided into multiple sub-volumes that do not involve relatively large motion, and a front projection image (en face projection) of each sub-volume is constructed. Such front projection images are called strips. Images with motion artifacts corrected can be obtained by performing registration between these strips.

[0008] A characteristic of Lissajous scanning is that two strips may overlap (intersect) at four locations. In the image processing described in Non-Patent Document 1, the strip with the largest dimensions is adopted as the initial reference strip, and strips ordered by size are sequentially (recursively) registered with the initial reference strip and then combined. In the registration, a cross-correlation function is used to determine the relative position between the reference strip and other strips. Here, since the shapes of the strips vary, a mask is used to treat each strip as an image of a specific shape (e.g., square), and correlation calculations are performed on the overlapping areas between the strips (see Appendix A of Non-Patent Document 1). [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-17915 [Patent Document 2] Japanese Patent Application Publication No. 2018-68578 [Patent Document 3] Japanese Patent Application Laid-Open No. 2018-140004 [Patent Document 4] Japanese Patent Application Laid-Open No. 2018-140049 [Non-patent literature]

[0010] [Non-Patent Document 1] Yiwei Chen, Young-Joo Hong, Shuichi Makita, and Yoshiaki Yasuno, “Three-dimensional eye motion correction by Lissajous scan optical coherence tomography”, Biomedical Optics EXPRESS, Vol. 8, No. 3, 1 Mar 2017, PP. 1783-1802 [Non-patent document 2] Yiwei Chen, Young-Joo Hong, Shuichi Makita, and Yoshiaki Yasuno, “Eye-motion-corrected optical coherence tomography angiography using Lissajous scanning”, Biomedical Optics EXPRESS, Vol. 9, No. 3, 1 Mar 2018, PP. 1111-1129 Summary of the Invention [Problem to be solved by the invention]

[0011] One object of the present invention is to provide a novel method for measuring eye movements using scanning imaging technology. [Means for solving the problem]

[0012] Some embodiments of the ophthalmic data processing method accept a data set acquired by applying an optical scan to a test eye according to a two-dimensional pattern including a series of cycles that intersect with each other, and generate position history data representing changes in the position of the test eye over time based on the data set. [Effects of the Invention]

[0013] According to an exemplary embodiment, a novel application of scanning imaging in the field of ophthalmology is provided. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a schematic diagram illustrating an example of the configuration of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 2] 1 is a schematic diagram illustrating an example of the configuration of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 3] 1 is a schematic diagram illustrating an example of the configuration of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 4A] 1 is a schematic diagram illustrating an example of the configuration of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 4B] 1 is a schematic diagram illustrating an example of the configuration of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 4C] 1 is a schematic diagram illustrating an example of the configuration of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 4D] 1 is a schematic diagram illustrating an example of the configuration of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 5] 1 is a schematic diagram illustrating an example of a Lissajous scan pattern performed by an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 6] 1 is a schematic diagram illustrating an example of a process that can be performed by an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 7A] 10 is a flowchart illustrating an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 7B] 10 is a flowchart illustrating an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 7C]10 is a flowchart illustrating an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 7D] 10 is a flowchart illustrating an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 7E] 10 is a flowchart illustrating an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 7F] 10 is a flowchart illustrating an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 8A] 10A and 10B are diagrams for explaining an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 8B] 10A and 10B are diagrams for explaining an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 8C] 10A and 10B are diagrams for explaining an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 9] 10A and 10B are diagrams for explaining an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 10A] 10A and 10B are diagrams for explaining an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 10B] 10A and 10B are diagrams for explaining an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 10C] 10A and 10B are diagrams for explaining an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. [Figure 10D] 10A and 10B are diagrams for explaining an example of the operation of an ophthalmic examination apparatus according to an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Some exemplary aspects of the embodiments will be described with reference to the drawings. The exemplary aspects described below relate to an ophthalmic data processing method, an ophthalmic data processing device, a method for controlling an ophthalmic data processing device, an ophthalmic examination device, a method for controlling an ophthalmic examination device, a program, and a recording medium. However, the embodiments are not limited to these aspects and may be applied to any field, such as a medical field other than ophthalmology, a medical method such as a diagnostic method, or a field other than the medical field (biology, non-destructive testing, etc.).

[0016] The matters disclosed in the documents cited in this specification and any other matters related to publicly known techniques can be combined with the exemplary embodiments. Furthermore, unless otherwise specified, no distinction is made between "image data" and an "image" based on the image data, and no distinction is made between a "region" of the subject's eye and an "image" thereof.

[0017] The ophthalmic examination apparatus according to the exemplary embodiment described below is configured to be capable of measuring and imaging the fundus of a living eye using Fourier-domain OCT (particularly, swept-source OCT). The type of OCT that can be employed in the embodiment is not limited to swept-source OCT, and may be, for example, spectral-domain OCT or time-domain OCT.

[0018] Some exemplary embodiments of the ophthalmic examination apparatus may be capable of utilizing scanning modalities other than OCT, for example, some exemplary embodiments may employ any optical scanning modality, such as SLO.

[0019] Furthermore, the scanning modality applicable to the ophthalmic examination apparatus according to some exemplary embodiments does not have to be an optical scanning modality. For example, some exemplary embodiments may employ a scanning modality that uses electromagnetic waves other than light, or a scanning modality that uses ultrasound.

[0020] Some exemplary embodiments may be capable of processing data acquired by other modalities in addition to processing data acquired by OCT scans and / or SLO. The other modalities may be any ophthalmic modality, such as a fundus camera, a slit lamp microscope, or an ophthalmic surgical microscope. The ophthalmic examination apparatus according to some exemplary embodiments may have the functionality of such modalities. Furthermore, the ophthalmic examination apparatus according to some exemplary embodiments may have a stereo alignment function using two or more anterior eye cameras, as disclosed in Japanese Patent Application Laid-Open No. 2013-248376 and Japanese Patent Application Laid-Open No. 2016-49243.

[0021] The object (sample) to which OCT is applied is not limited to the fundus, but may be any part of the eye, such as the anterior segment or the vitreous body. Note that the configurations and functions according to some exemplary embodiments may be applied to measurement and imaging of biological parts (tissues) other than the eye, and may also be applied to measurement and imaging of non-biological objects (moving objects) or parts thereof. In other words, the industrial application fields of some exemplary embodiments are not limited to ophthalmology-related fields, but may also include medical, veterinary, and biology-related fields, and more generally, may include fields related to any object (sample) with local and / or global motion.

[0022] The exemplary embodiments described below provide some aspects of an ophthalmic examination apparatus, some aspects of a method for controlling an ophthalmic examination apparatus, some aspects of an ophthalmic data processing apparatus, some aspects of a method for controlling an ophthalmic data processing apparatus, some aspects of an ophthalmic data processing method, some aspects of a program, and some aspects of a recording medium, all of which are merely examples and are not intended to limit the invention.

[0023] <Configuration of ophthalmic examination device> The ophthalmic examination apparatus 1 shown in FIG. 1 includes a fundus camera unit 2, an OCT unit 100, and an arithmetic and control unit 200. The fundus camera unit 2 is provided with a group of elements (optical elements, mechanisms, etc.) for photographing the subject's eye E from the front. The OCT unit 100 is provided with some of the group of elements (optical elements, mechanisms, etc.) for applying an OCT scan to the subject's eye E. Another part of the group of elements for OCT scanning is provided in the fundus camera unit 2. The arithmetic and control unit 200 includes one or more processors that perform various calculations and controls. In addition to these, the ophthalmic examination apparatus 1 may also include elements for supporting the subject's face and elements for switching the region to which the OCT scan is applied. Examples of the former elements include a chin rest and a forehead rest. An example of the latter element is a lens unit used to switch the region to which the OCT scan is applied from the fundus to the anterior segment.

[0024] The functionality of some elements disclosed herein is implemented using circuitry or processing circuitry. Circuitry or processing circuitry includes any of a general-purpose processor, a special-purpose processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA), conventional circuitry, and any combination thereof, configured and / or configured to perform the disclosed functions. A processor is considered to be processing circuitry or circuitry that includes transistors and / or other circuitry. In this disclosure, circuitry, unit, means, or similar terms is hardware that performs the disclosed functions or hardware that is programmed to perform the disclosed functions. The hardware may be hardware disclosed herein or may be known hardware that is programmed and / or configured to perform the described functions. In the case of a processor, where the hardware can be considered to be a type of circuitry, the circuitry, unit, means, or similar terms is a combination of hardware and software, and the software is used to configure the hardware and / or processor.

[0025] <Fundus camera unit 2> The fundus camera unit 2 is provided with an optical system for photographing the fundus Ef of the subject's eye E. The acquired image of the fundus Ef (called a fundus image, fundus photograph, etc.) is a front image such as an observed image or a photographed image. The observed image is obtained by, for example, video shooting using near-infrared light, and is used for alignment, focusing, tracking, etc. The photographed image is a still image obtained using, for example, flash light in the visible or infrared range.

[0026] The fundus camera unit 2 includes an illumination optical system 10 and an imaging optical system 30. The illumination optical system 10 irradiates illumination light onto the subject's eye E. The imaging optical system 30 detects return light of the illumination light from the subject's eye E. The measurement light from the OCT unit 100 is guided to the subject's eye E through an optical path within the fundus camera unit 2, and the return light is guided to the OCT unit 100 through the same optical path.

[0027] Light (observation illumination light) output from an observation light source 11 of an illumination optical system 10 is reflected by a concave mirror 12, passes through a condenser lens 13, and is transmitted through a visible light cut filter 14 to become near-infrared light. The observation illumination light is then focused near an imaging light source 15, reflected by a mirror 16, and passes through a relay lens system 17, a relay lens 18, an aperture 19, and a relay lens system 20. The observation illumination light is then reflected by the peripheral portion (the area surrounding the hole) of a perforated mirror 21, passes through a dichroic mirror 46, and is refracted by an objective lens 22 to illuminate the subject's eye E (fundus oculi Ef). Return light of the observation illumination light from the subject's eye E is refracted by the objective lens 22, passes through the dichroic mirror 46, passes through a hole formed in the central region of the perforated mirror 21, passes through a dichroic mirror 55, passes through an imaging focusing lens 31, and is reflected by a mirror 32. Furthermore, this returned light passes through the half mirror 33A, is reflected by the dichroic mirror 33, and is imaged on the light receiving surface of the image sensor 35 by the imaging lens 34. The image sensor 35 detects the returned light at a predetermined frame rate. The focus (focal position) of the photographing optical system 30 is adjusted to match the fundus Ef or the anterior segment of the eye.

[0028] Light (photography illumination light) output from the photography light source 15 is irradiated onto the fundus oculi Ef through the same path as the observation illumination light. Return light of the photography illumination light from the subject's eye E is guided to the dichroic mirror 33 through the same path as the return light of the observation illumination light, passes through the dichroic mirror 33, is reflected by a mirror 36, and is imaged by an imaging lens 37 on the light-receiving surface of an image sensor 38.

[0029] The liquid crystal display (LCD) 39 displays a fixation target (fixation target image). A portion of the light beam output from the LCD 39 is reflected by the half mirror 33A, reflected by the mirror 32, passes through the photographing focusing lens 31 and the dichroic mirror 55, and then passes through the hole in the aperture mirror 21. The light beam that passes through the hole in the aperture mirror 21 passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef. By changing the display position of the fixation target image on the LCD 39, the direction (fixation direction, fixation position) in which the gaze of the subject's eye E is guided can be changed. Instead of a display device such as an LCD, for example, a light-emitting element array, or a combination of a light-emitting element and a mechanism for moving it, may be used.

[0030] The alignment optical system 50 generates an alignment index used to align the optical system with respect to the subject's eye E. Alignment light output from a light-emitting diode (LED) 51 passes through an aperture 52, an aperture 53, and a relay lens 54, is reflected by a dichroic mirror 55, passes through the hole in the aperture mirror 21, transmits through the dichroic mirror 46, and is projected onto the subject's eye E via the objective lens 22. Return light of the alignment light from the subject's eye E (corneal reflected light, etc.) is guided to the image sensor 35 via the same path as the return light of the observation illumination light. Manual alignment or automatic alignment can be performed based on the received light image (alignment index image).

[0031] As in the conventional example, the alignment index image in this example consists of two bright spot images whose positions change depending on the alignment state. When the relative position between the subject's eye E and the optical system changes in the x and y directions, the two bright spot images displace together in the x and y directions. When the relative position between the subject's eye E and the optical system changes in the z direction, the relative position (distance) between the two bright spot images changes. When the distance between the subject's eye E and the optical system in the z direction matches the predetermined working distance, the two bright spot images overlap. When the position of the subject's eye E matches the position of the optical system in the x and y directions, two bright spot images are presented within or near a specified alignment target. When the distance between the subject's eye E and the optical system in the z direction matches the working distance and the position of the subject's eye E matches the position of the optical system in the x and y directions, the two bright spot images overlap and are presented within the alignment target.

[0032] In auto-alignment, the data processing unit 230 detects the positions of the two bright spot images, and the main control unit 211 controls the moving mechanism 150 (described later) based on the positional relationship between the two bright spot images and the alignment target. In manual alignment, the main control unit 211 displays the two bright spot images together with the observed image of the subject's eye E on the display unit 241, and the user operates the moving mechanism 150 using the operation unit 242 while referring to the two displayed bright spot images.

[0033] The focusing optical system 60 generates a split index used for focus adjustment of the subject's eye E. The focusing optical system 60 moves along the optical path (illumination optical path) of the illumination optical system 10 in conjunction with movement of the photographing focusing lens 31 along the optical path (photographing optical path) of the photographing optical system 30. The reflecting rod 67 is inserted into and removed from the illumination optical path. When performing focus adjustment, the reflecting surface of the reflecting rod 67 is tilted and positioned in the illumination optical path. Focusing light output from the LED 61 passes through the relay lens 62, is split into two beams by the split index plate 63, passes through the two-hole diaphragm 64, is reflected by the mirror 65, and is first imaged and reflected on the reflecting surface of the reflecting rod 67 by the condenser lens 66. The focusing light then passes through the relay lens 20, is reflected by the aperture mirror 21, passes through the dichroic mirror 46, and is projected onto the subject's eye E via the objective lens 22. The return light (fundus reflected light, etc.) of the focusing light from the subject's eye E passes through the same path as the return light of the alignment light and is guided to the image sensor 35. Manual focusing or autofocusing can be performed based on the received light image (split target image).

[0034] Diopter correction lenses 70 and 71 can be selectively inserted into the photographing optical path between the aperture mirror 21 and the dichroic mirror 55. The diopter correction lens 70 is a plus lens (convex lens) for correcting severe hyperopia. The diopter correction lens 71 is a minus lens (concave lens) for correcting severe myopia.

[0035] The dichroic mirror 46 combines the optical path for fundus imaging and the optical path for OCT (measurement arm). The dichroic mirror 46 reflects light in the wavelength band used for OCT and transmits light for fundus imaging. The measurement arm is provided with, in order from the OCT unit 100 side, a collimator lens unit 40, a retroreflector 41, a dispersion compensation member 42, an OCT focusing lens 43, an optical scanner 44, and a relay lens 45.

[0036] The retroreflector 41 is movable in the direction of the arrow shown in Figure 1, thereby changing the length of the measurement arm. Changing the length of the measurement arm is used, for example, to correct the optical path length according to the axial length of the eye, adjust the interference state, and so on.

[0037] The dispersion compensation member 42, together with a dispersion compensation member 113 (described later) disposed in the reference arm, acts to match the dispersion characteristics of the measurement light LS with the dispersion characteristics of the reference light LR.

[0038] The OCT focusing lens 43 is moved along the measurement arm to adjust the focus of the measurement arm. The movement of the imaging focusing lens 31, the movement of the focus optical system 60, and the movement of the OCT focusing lens 43 can be controlled in a coordinated manner.

[0039] The optical scanner 44 is disposed at a position that is substantially optically conjugate with the pupil of the subject's eye E. The optical scanner 44 deflects the measurement light LS guided by the measurement arm. The optical scanner 44 is, for example, a galvanometer scanner capable of two-dimensional scanning, including a galvanometer mirror for scanning in the x direction and a galvanometer mirror for scanning in the y direction.

[0040] <OCTユニット100> As shown in FIG. 2, the OCT unit 100 is provided with an optical system for applying swept-source OCT. This optical system includes an interference optical system. This interference optical system splits light from a wavelength-tunable light source (swept-wavelength light source) into measurement light and reference light, and generates interference light by superimposing the return light of the measurement light guided to the subject's eye E by the measurement arm on the reference light guided by the reference arm, and detects this interference light. Data (detection signal) obtained by the interference optical system is a signal representing the spectrum of the interference light, and is sent to the arithmetic and control unit 200.

[0041] The light source unit 101 includes, for example, a near-infrared tunable laser that changes the output wavelength at high speed at least in the near-infrared wavelength band. Light L0 output from the light source unit 101 is guided by an optical fiber 102 to a polarization controller 103, where its polarization state is adjusted. The light L0 is further guided by an optical fiber 104 to a fiber coupler 105, where it is split into measurement light LS and reference light LR. The optical path of the measurement light LS is called a measurement arm, and the optical path of the reference light LR is called a reference arm.

[0042] The reference light LR is guided by an optical fiber 110 to a collimator 111, where it is converted into a parallel beam, and then guided to a retroreflector 114 via an optical path length correction element 112 and a dispersion compensation element 113. The optical path length correction element 112 acts to match the optical path length of the reference light LR with that of the measurement light LS. The dispersion compensation element 113, together with a dispersion compensation element 42 arranged in the measurement arm, acts to match the dispersion characteristics between the reference light LR and the measurement light LS. The retroreflector 114 is movable along the optical path of the reference light LR incident thereon, thereby changing the length of the reference arm. Changing the reference arm length is used, for example, to correct the optical path length according to the axial length of the eye, adjust the interference state, and so on.

[0043] The reference light LR that has passed through the retroreflector 114 passes through the dispersion compensation member 113 and the optical path length correction member 112, is converted from a parallel beam into a convergent beam by the collimator 116, and enters an optical fiber 117. The reference light LR that has entered the optical fiber 117 is guided to a polarization controller 118 where its polarization state is adjusted, is guided through an optical fiber 119 to an attenuator 120 where its light amount is adjusted, and is guided through an optical fiber 121 to a fiber coupler 122.

[0044] On the other hand, the measurement light LS generated by the fiber coupler 105 is guided by the optical fiber 127 and converted into a parallel beam by the collimator lens unit 40, passes through the retroreflector 41, the dispersion compensation member 42, the OCT focusing lens 43, the optical scanner 44, and the relay lens 45, is reflected by the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the subject's eye E. The measurement light LS is scattered and reflected at various depth positions in the subject's eye E. The returning light of the measurement light LS from the subject's eye E travels the same path as the outward path in the opposite direction and is guided to the fiber coupler 105, and reaches the fiber coupler 122 via the optical fiber 128.

[0045] The fiber coupler 122 generates interference light by superimposing the measurement light LS incident via the optical fiber 128 and the reference light LR incident via the optical fiber 121. The fiber coupler 122 splits the generated interference light at a predetermined splitting ratio (for example, 1:1) to generate a pair of interference lights LC. The pair of interference lights LC are guided to the detector 125 via optical fibers 123 and 124, respectively.

[0046] The detector 125 includes, for example, a balanced photodiode. The balanced photodiode has a pair of photodetectors that respectively detect a pair of interference light LC, and outputs the difference between the pair of detection results obtained by these. The detector 125 sends this output (detection signal) to a data acquisition system (DAS) 130.

[0047] A clock KC is supplied to the data collection system 130 from the light source unit 101. The clock KC is generated in the light source unit 101 in synchronization with the output timing of each wavelength swept within a predetermined wavelength range by the wavelength-tunable light source. The light source unit 101, for example, branches the light L0 of each output wavelength to generate two branched lights, optically delays one of the branched lights, combines the branched lights, detects the resulting combined light, and generates the clock KC based on the detection result. The data collection system 130 samples the detection signal input from the detector 125 based on the clock KC. The data collection system 130 sends the sampling result to the arithmetic and control unit 200.

[0048] In this example, both an element for changing the measurement arm length (e.g., retroreflector 41) and an element for changing the reference arm length (e.g., retroreflector 114 or reference mirror) are provided, but only one of these elements may be provided. Furthermore, the element for changing the difference between the measurement arm length and the reference arm length (optical path length difference) is not limited to these and may be any element (optical member, mechanism, etc.).

[0049] <Control system / processing system> 3, 4A, 4B, 4C, and 4D show configuration examples of the control system and processing system of the ophthalmic examination apparatus 1. The control unit 210, image data creation unit 220, and data processing unit 230 are provided in, for example, the arithmetic control unit 200. The ophthalmic examination apparatus 1 may include a communication device for performing data communication with an external device. The ophthalmic examination apparatus 1 may also include a drive device (reader / writer) for reading data from a recording medium and writing data to the recording medium.

[0050] <Control unit 210> The control unit 210 executes various types of control. The control unit 210 includes a main control unit 211 and a storage unit 212. In this embodiment, the main control unit 211 includes a scanning control unit 2111, and the storage unit 212 stores a scanning protocol 2121, as shown in FIG. 4A.

[0051] <Main control unit 211> The main control unit 211 includes a processor and controls each element (including the elements shown in FIGS. 1 to 4D) of the ophthalmic examination apparatus 1. The main control unit 211 is realized by cooperation between hardware including the processor and control software. The scan control unit 2111 controls OCT scanning of a scan area of ​​a predetermined shape and size.

[0052] The imaging focusing driver 31A moves the imaging focusing lens 31 arranged in the imaging optical path and the focus optical system 60 arranged in the illumination optical path under the control of the main controller 211. The retroreflector (RR) driver 41A moves the retroreflector 41 provided in the measurement arm under the control of the main controller 211. The OCT focusing driver 43A moves the OCT focusing lens 43 arranged in the measurement arm under the control of the main controller 211. The retroreflector (RR) driver 114A provided in the measurement arm moves the retroreflector 114 arranged in the reference arm under the control of the main controller 211. Each driver includes an actuator such as a pulse motor that operates under the control of the main controller 211. The optical scanner 44 operates under the control of the main controller 211 (scan controller 2111).

[0053] The movement mechanism 150 is typically configured to move the fundus camera unit 2 three-dimensionally. Such a movement mechanism 150 includes, for example, an x-stage configured to be movable in ±x directions (left and right directions), an x-movement mechanism configured to move the x-stage, a y-stage configured to be movable in ±y directions (up and down directions), a y-movement mechanism configured to move the y-stage, a z-stage configured to be movable in ±z directions (depth direction), and a z-movement mechanism configured to move the z-stage. Each of the x-movement mechanism, y-movement mechanism, and z-movement mechanism includes an actuator such as a pulse motor that operates under the control of the main controller 211.

[0054] <Storage section 212> The storage unit 212 stores various types of data. The data stored in the storage unit 212 includes OCT images, fundus images, subject eye information, and control information. The subject eye information includes subject information such as patient ID and name, left / right eye identification information, and electronic medical record information. The control information is information related to specific controls. The control information in this embodiment includes a scanning protocol 2121.

[0055] The scanning protocol 2121 is an agreement regarding the control content for an OCT scan of a scanning area of ​​a predetermined shape and a predetermined size, and includes a set of various control parameters (scanning control parameters). The scanning protocol 2121 includes a protocol for each scanning mode. The scanning protocol 2121 of this embodiment includes at least a protocol for Lissajous scanning, and may further include protocols for, for example, B-scan (line scanning), cross scanning, radial scanning, and raster scanning.

[0056] The scanning control parameters of this embodiment include at least a parameter indicating the content of control over the optical scanner 44. These parameters include, for example, a parameter indicating the scan pattern, a parameter indicating the scan speed, and a parameter indicating the scan interval. The scan pattern indicates the shape of the scan path, and examples thereof include a Lissajous pattern, a line pattern, a cross pattern, a radial pattern, and a raster pattern. The scan speed is defined, for example, as the repetition rate of A-scans. The scan interval is defined, for example, as the interval between adjacent A-scans, that is, the array interval of scan points.

[0057] As with the conventional techniques disclosed in Patent Documents 1 to 4 and Non-Patent Documents 1 and 2, the "Lissajous scan" of this embodiment may be not only a "narrowly defined" Lissajous scan whose path is a pattern (Lissajous pattern, Lissajous figure, Lissajous curve, Lissajous function, Bowditch curve) drawn by the locus of points obtained as an ordered pair of two mutually perpendicular simple harmonic motions, but also a "broadly defined" Lissajous scan that follows a predetermined two-dimensional pattern including a series of cycles.

[0058] The optical scanner 44 of this embodiment includes, for example, a first galvanometer mirror that deflects the measurement light LS in the x direction and a second galvanometer mirror that deflects the measurement light LS in the y direction. Lissajous scanning is achieved by controlling the first galvanometer mirror so that the deflection direction along the x direction repeatedly changes in a first period, while controlling the second galvanometer mirror so that the deflection direction along the y direction repeatedly changes in a second period. Here, the first period and the second period are different from each other.

[0059] For example, the Lissajous scan of this embodiment may not only be a scan of a narrowly defined Lissajous pattern obtained from a combination of two sine waves, but also a scan of a pattern obtained by adding a specific term (e.g., an odd-order polynomial) to a sine wave, or a scan of a pattern based on a triangular wave.

[0060] A "cycle" generally refers to an object consisting of a plurality of sampling points having a certain length. The cycle in this embodiment may be, for example, a closed curve or an almost closed curve (a substantially closed curve, an approximately closed curve). In other words, the start point and end point of the cycle in this embodiment may coincide or nearly coincide.

[0061] Typically, the scanning protocol 2121 is set based on the Lissajous function. As shown in equation (9) of Non-Patent Document 1, the Lissajous function is expressed by, for example, the following parametric equation system: x(t i )=A·cos(2π·(f A / n)·t i ), y(t i )=A·cos(2π·(f A (n-2) / n 2 )·t i ).

[0062] Here, "x" is the horizontal axis of the two-dimensional coordinate system in which the Lissajous curve is defined, "y" is the vertical axis, and "t i ” is the acquisition time of the i-th A-line in the Lissajous scan, “A” is the scan range (amplitude), and “f A" indicates the A-line acquisition rate (scan speed, A-scan repetition rate), and n indicates the number of A-lines in each cycle in the x-direction (horizontal axis).

[0063] An example of the distribution of scan lines (scan pattern) in such an exemplary Lissajous scan is shown in FIG.

[0064] Any pair of cycles included in a Lissajous scan (in the narrow or broad sense) intersects with each other at at least one point (particularly two or more points). Utilizing such intersections makes it possible to perform registration between data pairs collected from any pair of cycles in a Lissajous scan, and makes it possible to implement the image construction method and motion artifact correction method disclosed in Non-Patent Document 1 (or 2). Hereinafter, unless otherwise specified, a case where the method described in Non-Patent Document 1 is applied will be described.

[0065] The control information stored in the storage unit 212 is not limited to the above example. For example, the control information may include information for performing focus control (focus control parameters).

[0066] The focus control parameters are parameters that indicate the content of control over the OCT focusing drive unit 43A. Examples of focus control parameters include a parameter that indicates the focal position of the measurement arm, a parameter that indicates the moving speed of the focal position, and a parameter that indicates the moving acceleration of the focal position. The parameter that indicates the focal position is, for example, a parameter that indicates the position of the OCT focusing lens 43. The parameter that indicates the moving speed of the focal position is, for example, a parameter that indicates the moving speed of the OCT focusing lens 43. The parameter that indicates the moving acceleration of the focal position is, for example, a parameter that indicates the moving acceleration of the OCT focusing lens 43. The moving speed may or may not be constant. The same applies to the moving acceleration.

[0067] Such focus control parameters enable focus adjustment according to the shape of the fundus oculi Ef (typically a concave shape with a deep center and a shallow periphery) and aberration distribution. Focus control is performed, for example, in conjunction with scan control (repeated control of Lissajous scan). This allows motion artifacts to be corrected and high-quality images that are in focus across the entire scan range to be obtained.

[0068] <Scanning control unit 2111> The scanning control unit 2111 controls at least the optical scanner 44 based on the scanning protocol 2121. The scanning control unit 2111 may further control the light source unit 101 in cooperation with the control of the optical scanner 44 based on the scanning protocol 2121. The scanning control unit 2111 is realized by cooperation between hardware including a processor and scanning control software including the scanning protocol 2121.

[0069] <Image data construction unit 220> The image data constructing unit 220 includes a processor and constructs OCT image data of the fundus Ef based on signals (sampling data) input from the data acquisition system 130. This OCT image data construction includes noise removal (noise reduction), filtering, fast Fourier transform (FFT), and the like, similar to conventional Fourier domain OCT (swept-source OCT). When another type of OCT method is adopted, the image data constructing unit 220 constructs OCT image data by performing known processing corresponding to that OCT type.

[0070] For example, the image data constructing unit 220 is configured to at least execute a process of constructing image data (A-scan image data) corresponding to each scan point (each A-line) from the sampling data. The image data constructed by the image data constructing unit 220 may be data before being imaged, and may be, for example, a signal profile (reflection profile, scattering profile) along the depth direction (z direction, optical axis direction, A-line direction).

[0071] As described above, in this embodiment, a Lissajous scan is applied to the fundus Ef. The image data constructing unit 220, together with the data processing unit 230, can construct three-dimensional image data of the fundus Ef by applying, for example, the image construction method and motion artifact correction method disclosed in Non-Patent Document 1 to the data set acquired through collection by the Lissajous scan and sampling by the data acquisition system 130.

[0072] The image data constructor 220 and / or the data processor 230 can apply rendering to the 3D image data to generate a display image. Examples of applicable rendering methods include volume rendering, surface rendering, maximum intensity projection (MIP), minimum intensity projection (MIP), and multiplanar reconstruction (MPR).

[0073] The image data constructing unit 220 and / or the data processing unit 230 can construct an en face OCT image based on the three-dimensional image data. For example, the image data constructing unit 220 and / or the data processing unit 230 can construct projection data by projecting the three-dimensional image data in the z direction (A-line direction, optical axis direction, depth direction). The image data constructing unit 220 and / or the data processing unit 230 can also construct a shadowgram by projecting partial three-dimensional image data, which is a part of the three-dimensional image data, in the z direction. This partial three-dimensional image data may be set using, for example, any segmentation method. Segmentation is a process of identifying a partial region in an image. In this example, segmentation can be performed to identify an image region corresponding to one or more tissues (regions) of the fundus oculi Ef.

[0074] The segmentation in the embodiment is not limited to the shadowgram construction process, and any known technique such as threshold processing, edge detection, filtering, machine learning (e.g., semantic segmentation) can be used.

[0075] The ophthalmic examination apparatus 1 may be capable of performing OCT angiography (OCT-angiography). OCT angiography is an imaging technique that constructs an image in which blood vessels are emphasized (see, for example, Non-Patent Document 2 and JP2015-515894A). Generally, fundus tissue (structure) does not change over a short period of time, but blood flow inside blood vessels also changes over a short period of time. OCT angiography generates an image by emphasizing areas where such temporal changes exist (blood flow signals). Note that OCT angiography is also called OCT motion contrast imaging. Furthermore, images obtained by OCT angiography are called angiographic images, angiograms, motion contrast images, etc.

[0076] When OCT angiography is possible, the ophthalmic examination apparatus 1 repeatedly scans the same region of the fundus Ef a predetermined number of times. For example, the ophthalmic examination apparatus 1 repeatedly executes the above-described scanning control (repeated control of Lissajous scanning) a predetermined number of times. As a result, multiple pieces of three-dimensional data (time-series three-dimensional data sets) are collected by the data acquisition system 130 from the application region of the Lissajous scan. The image data construction unit 220 and / or the data processing unit 230 can construct a motion contrast image from this three-dimensional data set. This motion contrast image is an angiographic image in which the temporal change in the interference signal caused by the blood flow in the fundus Ef is emphasized. This angiographic image is three-dimensional angiographic image data that represents the three-dimensional distribution of blood vessels in the fundus Ef.

[0077] The image data constructing unit 220 and / or the data processing unit 230 can construct any two-dimensional angiographic image data and / or any pseudo three-dimensional angiographic image data from this three-dimensional angiographic image data. For example, the image data constructing unit 220 and / or the data processing unit 230 can construct two-dimensional angiographic image data representing any cross section of the fundus Ef by applying multiplanar reconstruction to the three-dimensional angiographic image data. Furthermore, the image data constructing unit 220 and / or the data processing unit 230 can construct frontal angiographic image data of the fundus Ef by applying projection imaging or shadowgramming to the three-dimensional angiographic image data.

[0078] In this embodiment, the image data constructor 220 and / or the data processor 230 construct multiple strips from data collected by the data acquisition system 130. As described in Non-Patent Document 1, the image data constructor 220 and / or the data processor 230 divide the volume (three-dimensional data) collected by Lissajous scanning into multiple subvolumes without relatively large intervening motion, and construct a front projection image of each subvolume. These front projection images are strips. By applying registration and merging processes to the multiple strips obtained in this manner, an image in which motion artifacts have been corrected can be obtained. As will be described later, in this example, the registration and merging processes are performed by the data processor 230.

[0079] The image data constructing unit 220 is realized by cooperation between hardware including a processor and image constructing software. In some embodiments, the image data constructing unit 220 and the data processing unit 230 may be integrally configured.

[0080] <Data processing unit 230> The data processing unit 230 includes a processor and applies various types of data processing to the image of the subject's eye E. For example, the data processing unit 230 is realized by cooperation between hardware including a processor and data processing software.

[0081] The data processing unit 230 may be configured to perform registration between two images acquired of the fundus oculi Ef. For example, the data processing unit 230 may be configured to perform registration between three-dimensional image data acquired by OCT and a front image acquired by the fundus camera unit 2. The data processing unit 230 may also be configured to perform registration between two OCT images acquired by OCT. The data processing unit 230 may also be configured to perform registration between two front images acquired by the fundus camera unit 2. The data processing unit 230 may also be configured to apply registration to the analysis results of the OCT images and the analysis results of the front images. These registrations can be performed by known methods, and include, for example, feature point extraction and affine transformation.

[0082] Furthermore, the data processing unit 230 is configured to process a data set obtained using a Lissajous scan. As previously described, the image data constructor 220 is configured to construct strips from data collected by the Lissajous scan, for example. The data processing unit 230 is configured to construct a motion artifact corrected image by applying registration and merging operations to the strips.

[0083] A characteristic of Lissajous scanning is that any two strips have an overlapping area. Similar to the technique described in Non-Patent Document 1, the data processing unit 230 is configured to perform strip-to-strip registration using the overlapping area. The data processing unit 230 first orders the multiple strips constructed by the image data construction unit 220 according to size (e.g., area) (first to Nth strips) and designates the largest strip, the first strip, as the initial reference strip. Next, the data processing unit 230 registers the second strip using the first strip as a reference and merges (combines) the first and second strips. The data processing unit 230 registers the third strip using the resulting merged strip as a reference and merges the merged strip with the third strip. By sequentially performing these registration and merging processes in the above order, the first to Nth strips are aligned and stitched together, resulting in an image with motion artifacts corrected.

[0084] In such registration, a cross-correlation function is used to determine the relative position between a reference strip and other strips. However, since the strips can have arbitrary shapes, the correlation between the strips is calculated using a mask that treats each strip as an image of a specific shape (e.g., a square) (see Appendix A of Non-Patent Document 1).

[0085] As mentioned above, the strip is an intensity image of a predetermined gradation, and the mask image is a binary image, so the difference between the absolute values ​​of the pixel values ​​of the strip and the mask becomes large, and there is a risk that the effect of the mask in the correlation calculation using the strip and the mask will be ignored and an accurate correlation coefficient will not be obtained. In particular, when single-precision floating-point (float type) calculations are used to calculate the correlation coefficient between strips from the perspective of cost, etc., this problem becomes more pronounced due to the influence of rounding errors caused by the small number of significant digits.

[0086] In this embodiment, in order to deal with the problem of rounding errors and further to detect and evaluate the movement of the subject's eye E, a data processing unit 230 having the configuration shown in FIGS. 4B to 4D is employed.

[0087] As shown in Figure 4B, the data processing unit 230 in this example includes an image data processing unit 231, an xy position history data generation unit 232, a z shift amount calculation unit 233, a z position history data generation unit 234, a visualization data generation unit 235, an image data correction unit 236, an evaluation data generation unit 237, and a position history data correction unit 238.

[0088] <Image data processing unit 231> The image data processing unit 231 is configured to process the image data constructed by the image data construction unit 220. As shown in Fig. 4C, the image data processing unit 231 includes a mask image generation unit 2311, a range adjustment unit 2312, a composite image generation unit 2313, a cross-correlation function calculation unit 2314, a correlation coefficient calculation unit 2315, an xy shift amount calculation unit 2316, a registration unit 2317, and a merge processing unit 2318. In the example described below, any two strips are considered, one of which is designated as a reference strip, and the other strip (registering strip) is registered to this reference strip.

[0089] <Mask image generation unit 2311> The mask image generator 2311 generates a reference mask image corresponding to the reference strip and a target mask image corresponding to the target strip. Some examples of mask images are described below, but the present invention is not limited to these.

[0090] The contour shape of the mask image is, for example, rectangular, typically square. The shape of the reference mask image and the shape of the target mask image may be the same. Furthermore, the dimensions of the reference mask image and the dimensions of the target mask image may be the same.

[0091] The range of pixel values ​​of the mask image is set to be included in the closed interval [0,1], for example. Typically, the mask image may be a binary image with pixel values ​​of 0 or 1. As a specific example, the mask image is a binary image in which the pixel value in the area corresponding to the domain of the strip is 1 and the other pixels are 0. In other words, as shown in equation (20) of Non-Patent Document 1, the pixel value of the exemplary mask image is 1 in the image area of ​​the corresponding strip and 0 in the other areas.

[0092] <Range Adjustment Unit 2312> The range adjuster 2312 is configured to adjust the pixel value range of the reference strip and the pixel value range of the target strip based on the pixel value range of the reference mask image and the pixel value range of the target mask image. Typically, the range adjuster 2312 adjusts the pixel value range of the reference strip and the pixel value range of the target strip so as to reduce the difference between the pixel value range of the strip and the pixel value range of the mask image.

[0093] In general, the range adjuster 2312 adjusts the pixel value range of the reference strip, the pixel value range of the target strip, and the pixel value range of the mask image relatively. In a typical example, the reference mask image applied to the reference strip and the target mask image applied to the target strip are different, and the range adjuster 2312 adjusts the pixel value range of the reference strip, the pixel value range of the target strip, the pixel value range of the reference mask image, and the pixel value range of the target strip relatively.

[0094] The range adjustment unit 2312 is configured to adjust the range of pixel values ​​of the reference strip and the range of pixel values ​​of the reference mask image so as to reduce the range of pixel values ​​of the reference strip and the range of pixel values ​​of the reference mask image, and to relatively adjust the range of pixel values ​​of the target strip and the range of pixel values ​​of the target strip so as to reduce the range of pixel values ​​of the target strip and the range of pixel values ​​of the target mask image.

[0095] The range adjustment unit 2312 may be configured to adjust the pixel value range of the strip and the pixel value range of the mask image so that the pixel value range of one of the strip and the mask image matches the pixel value range of the other. For example, the range adjustment unit 2312 may be configured to adjust the pixel value range of the reference strip and the pixel value range of the reference mask image so that the pixel value range of the reference strip and the pixel value range of the reference mask image match, and to relatively adjust the pixel value range of the target strip and the pixel value range of the target strip so that the pixel value range of the target strip and the pixel value range of the target mask image match.

[0096] For example, the range adjuster 2312 is configured to normalize the range of pixel values ​​of the strip according to the range of pixel values ​​of the mask image, some examples of this normalization (standardization) are described below, but are not limited to these.

[0097] A first example of normalization will be described. When the range of pixel values ​​of the mask image is included in the closed interval [0, 1], the range adjustment unit 2312 divides the value of each pixel in the strip by the maximum pixel value in the strip. In this example, the range adjustment unit 2312 first compares the values ​​of all pixels in the strip to identify the maximum value (maximum pixel value), and then divides the value of each pixel in the strip by the maximum pixel value. This causes the range of pixel values ​​of the strip to match the closed interval [0, 1], which is the same as the range of pixel values ​​of the mask image.

[0098] A second example of normalization will be described. When the range of pixel values ​​of the mask image is included in the closed interval [0, 1], the range adjustment unit 2312 divides the value of each pixel of the strip by the maximum value of the range of pixel values ​​of this strip. The range of pixel values ​​of the strip is set in advance, and the range adjustment unit 2312 divides the value of each pixel of the strip by the upper limit (maximum value) of this range. In this example, the range of pixel values ​​of the strip is also matched to the same closed interval [0, 1] as the range of pixel values ​​of the mask image.

[0099] The processing performed by the range adjustment unit 2312 can reduce the difference between the absolute values ​​of the pixel values ​​of the strip and the absolute values ​​of the pixel values ​​of the mask, so that the effect of the mask is not ignored in the correlation calculation using the strip and the mask, and it becomes possible to accurately calculate the correlation coefficient. In particular, even when single-precision floating-point (float) calculation is used, it is possible to eliminate (reduce) the influence of rounding errors caused by a small number of significant digits.

[0100] Although this specification mainly describes an example of changing only the pixel value range of the strip, it is also possible to change only the pixel value range of the mask image, or to change both the pixel value range of the strip and the pixel value range of the mask image.

[0101] <Synthetic image generation unit 2313> For the strips and mask images to which the pixel value range adjustment by the range adjustment unit 2312 has been applied, the composite image generation unit 2313 generates two composite images by combining the mask image with each of the reference strip and the target strip. Typically, the composite image generation unit 2313 combines the reference mask image with the reference strip to generate a reference composite image, and combines the target mask image with the target strip to generate a target composite image.

[0102] The process of combining the strip and the mask image may be performed in the same manner as in Non-Patent Document 1. However, unlike the method in Non-Patent Document 1, in this embodiment, the difference between the pixel value range of the strip and the pixel value range of the mask image is adjusted to be small. Typically, the pixel value ranges of the reference strip and the target strip are normalized to match the pixel value ranges of the reference mask image and the target mask image.

[0103] For example, in the same manner as equation (19) in Non-Patent Document 1, the composite image generation unit 2313 may be configured to embed a strip with a normalized pixel value range into an image of the same size and shape as the mask image.

[0104] Furthermore, the composite image generation unit 2313 generates a composite image of the embedded image of the strip and the mask image. This composite image corresponds to "f´(r)m f (r)" (the second line on page 1800, etc.) in Non-Patent Document 1. However, as described above, the value of the embedded image of the strip is different from that in Equation (19).

[0105] <Cross-correlation function calculation unit 2314> The cross-correlation function calculation unit 2314 obtains a plurality of cross-correlation functions based on two composite images generated by the composite image generation unit 2313. The calculation of the cross-correlation function is performed in the same manner as in Non-Patent Document 1. For example, the cross-correlation function calculation unit 2314 calculates six cross-correlation functions (image cross-correlation) included in Equation (33) of Non-Patent Document 1 based on a reference composite image generated from a reference strip and a reference mask image and a target composite image generated from a target strip and a target mask image.

[0106] <Correlation coefficient calculation unit 2315> The correlation coefficient calculation unit 2315 calculates a correlation coefficient based on the plurality of cross-correlation functions calculated by the cross-correlation function calculation unit 2314. This calculation follows Equation (33) of Non-Patent Document 1.

[0107] <XY shift amount calculation unit 2316> The XY shift amount calculation unit 2316 calculates the shift amount in the XY direction (lateral direction, horizontal direction) between the reference strip and the target strip based on the correlation coefficient calculated by the correlation coefficient calculation unit 2315.

[0108] For example, the calculation for calculating the XY direction shift amount includes an operation corresponding to "rough lateral motion correction" (page 1787) in Non-Patent Document 1, and the XY shift amount calculation unit 2316 is configured to estimate the XY direction shift amount by obtaining the maximum value of the cross-correlation function.

[0109] Furthermore, the xy shift amount calculation unit 2316 may be configured to perform a calculation equivalent to "fine lateral motion correction" (page 1789) in Non-Patent Document 1 in order to calculate small lateral shift amounts caused by eye movements such as slow drift and tremor.

[0110] <Registration Department 2317> The registration unit 2317 performs lateral registration based on the lateral shift amount calculated by the xy shift amount calculation unit 2316. For example, this registration includes processing equivalent to "rough lateral motion correction" (page 1787) in Non-Patent Document 1. The registration unit 2317 can perform registration between the reference strip and the target strip so as to cancel out the lateral shift amount calculated by the xy shift amount calculation unit 2316.

[0111] When the xy shift amount calculation unit 2316 performs a calculation equivalent to "fine lateral motion correction" (page 1789) in Non-Patent Document 1, the registration unit 2317 can remove small lateral motion artifacts between the reference strip and the target strip by performing a registration equivalent to "fine lateral motion correction" (page 1789) in Non-Patent Document 1.

[0112] The amount of shift in the lateral direction calculated by the xy shift amount calculation unit 2316 is sent to the xy position history data generation unit 232 .

[0113] <Merge processing section 2318> The merge processing unit 2318 constructs a merged image of the reference strip and the target strip whose relative positions have been adjusted by the registration unit 2317. This processing is also performed in the same manner as the method described in Non-Patent Document 1.

[0114] As described above, the image data processing unit 231 sequentially executes the above-described series of processes on the plurality of strips constructed by the image data construction unit 220 in accordance with the order according to the size. As a result, a merged image in which lateral motion artifacts are corrected is obtained from the plurality of strips constructed by the image data construction unit 220. This merged image is typically an image representing the entire range to which the resampling scan is applied.

[0115] Note that, as described above, the plurality of strips are a plurality of frontal projection images based on a plurality of sub-volumes obtained by dividing the volume (3D data) collected by the resampling scan. Therefore, the merged image constructed from the plurality of strips by the image data processing unit 231 provides a plurality of sub-volumes (and their merged images) with lateral position adjustment. In this example, the registration and merging processes in the depth direction orthogonal to the lateral direction are executed by the z-shift amount calculation unit 233 and the image data correction unit 236 (described later).

[0116] <xy position history data generation unit 232> When the above-described series of processes are sequentially executed in accordance with the order according to the size of the plurality of strips constructed by the image data construction unit 220, the xy shift amount calculation unit 2316 sequentially calculates the shift amount in the lateral direction between the sequentially set strip pairs (reference strip and target strip). The xy position history data generation unit 232 collects the shift amount data sequentially calculated by the xy shift amount calculation unit 2316. As a result, the xy position history data generation unit 232 acquires a set of shift amounts in the lateral direction between the plurality of strips.

[0117] The xy position history data generation unit 232 arranges the collected set of shift amounts in the scan order corresponding to a plurality of strips. That is, the xy position history data generation unit 232 arranges the collected set of shift amounts in a time series (time progression, time axis, scan order) (ordered set). Thereby, data indicating the lateral movement of the subject eye E during the period when the resampling scan was applied, that is, data indicating the history of the position of the subject eye E in the lateral direction is obtained. This data is called xy position history data.

[0118] The "time" in this example is not limited to being expressed by parameters or units related to time (for example, microseconds), and may be expressed by any parameter or unit equivalent (mutually convertible) to the parameters or units related to time. For example, the order of the resampling scan (scan order corresponding to each cycle) is equivalent to the parameters or units related to time when the scan interval is known (for example, at a constant interval).

[0119] <z shift amount calculation unit 233> The z shift amount calculation unit 233 calculates the shift amount in the z direction (A-line direction, optical axis direction, depth direction) between a plurality of sub-volumes obtained by dividing the volume collected by the resampling scan (that is, a plurality of sub-volumes that are the basis of a plurality of strips).

[0120] In this example, after the image data processing unit 231 performs position adjustment in the xy direction (lateral direction, horizontal direction), the z shift amount calculation unit 233 and the image data correction unit 236 perform position adjustment in the z direction. That is, the z shift amount calculation unit 233 in this example is configured to calculate the shift amount in the z direction using the data (for example, xy direction shift amount data, merged image) obtained by the image data processing unit 231.

[0121] An example of processing executed by the z-shift amount calculation unit 233 will be described. Similar to the pair of strips (reference strip and target strip) considered in calculating the shift amounts in the x and y directions, a pair of sub-volumes is also considered in calculating the shift amounts in the z direction. The pair of sub-volumes may be two sub-volumes corresponding to the reference strip and target strip considered in calculating the shift amounts in the x and y directions, and these sub-volumes are referred to as the reference sub-volume and the target sub-volume, respectively.

[0122] Based on the result of the xy-direction registration between the corresponding reference strips and target strips, the reference sub-volumes and target sub-volumes are aligned in the xy-direction. Also, as mentioned above, any two strips in a Lissajous scan overlap (intersect) at four points, and therefore any two sub-volumes also overlap (intersect) at four points.

[0123] The z-shift amount calculation unit 233 first identifies an intersection area (common area) between the reference sub-volume and the target sub-volume whose positions have been adjusted in the x and y directions. Each intersection area is three-dimensional image data.

[0124] Next, the z-shift amount calculation unit 233 sets a cross section in the identified intersection area. This cross section is, for example, a plane formed by an arbitrary axis (for example, the x-axis, the y-axis, or an axis obliquely intersecting both the x-axis and the y-axis) in the xy plane and the z-axis. Note that the cross section is not limited to a plane, and may be a curved surface, etc.

[0125] Next, the z-shift amount calculation unit 233 constructs an image of the set cross section from the reference sub-volume, and also constructs an image of the same cross section from the target sub-volume. The cross-sectional image constructed from the reference sub-volume is called the reference cross-sectional image, and the cross-sectional image constructed from the target sub-volume is called the target cross-sectional image. The reference cross-sectional image and the target cross-sectional image are images that represent the same cross section in the intersection region between the reference sub-volume and the target sub-volume that have been registered in the x- and y-directions.

[0126] Next, the z-shift amount calculation unit 233 analyzes the reference cross-sectional image to identify an image of a predetermined portion of the subject's eye E, and analyzes the target cross-sectional image to identify an image of the same portion. This portion may be any portion, and may be, for example, the surface of the fundus Ef (the retinal surface, the internal limiting membrane, or the boundary between the retina and the vitreous body). Furthermore, if an artificial object has been implanted in the subject's eye E, an image of this artificial object may be identified. The analysis for identifying the image of the predetermined portion may include, for example, segmentation.

[0127] Next, the z-shift amount calculation unit 233 calculates the z-coordinate of an image (reference image) of a predetermined region identified from the reference cross-sectional image, and calculates the z-coordinate of an image (target image) of a predetermined region identified from the target cross-sectional image. Typically, the image of the predetermined region consists of multiple pixels, and the z-coordinates of these pixels are not constant. For example, the global shape of the retinal surface is generally a curved shape convex in the +z direction, and images of local areas (examples of the reference image and target image) are at least partially tilted with respect to the z-axis. The z-shift amount calculation unit 233 can calculate the z-coordinate of the predetermined region based on at least one pixel of a group of pixels constituting the image of the predetermined region. For example, a statistic of the z-coordinates can be calculated from the group of pixels, and this statistic can be used as the z-coordinate of the image. This statistic can be any representative value (summary statistic), such as a maximum value, minimum value, mean value, median, mode, or quantile. On the other hand, if the image of the predetermined region consists of a single pixel, the z-coordinate of this pixel can be used as the z-coordinate of the predetermined region.

[0128] Next, the z-shift amount calculation unit 233 calculates the difference between the z-coordinate of the reference image and the z-coordinate of the target image (the shift amount in the z direction). In some aspects, processing control can be performed so that correction processing is performed by the image data correction unit 236 when the z-direction shift amount exceeds a threshold. This threshold is, for example, zero or a positive value, and may be a preset fixed value or a variable value.

[0129] In the above example, the two-dimensional image data (cross-sectional image) is constructed from the three-dimensional image data (sub-volume) to obtain the z-direction shift amount. However, in some aspects, the z-direction shift amount may be obtained from the three-dimensional image data without constructing the two-dimensional image data. In this case, the same processing as that performed on the pair of the reference cross-sectional image and the target cross-sectional image can be performed on the pair of the reference sub-volume and the target sub-volume.

[0130] In the above example, the two-dimensional image data (cross-sectional image) is constructed from the three-dimensional image data (sub-volume), and the image of a predetermined part is specified from this two-dimensional image data to obtain the z-direction shift amount. However, in some aspects, the image of a predetermined part may be specified from the three-dimensional image data, and the two-dimensional image data of the image of this predetermined part may be constructed to obtain the z-direction shift amount.

[0131] Further, the z-shift amount calculation unit 233 may be configured to obtain the z-direction shift amount between the reference sub-volume and the target sub-volume based on the correlation coefficient calculated based on the pair of the reference sub-volume and the target sub-volume (for example, the correlation coefficient calculated by the correlation coefficient calculation unit 2315). The method for calculating the z-direction shift amount is not limited to the method exemplified above, and may be any method applicable to the pair of the reference sub-volume and the target sub-volume, or may be any method applicable to other object pairs.

[0132] <z-position history data generation unit 234> The z-shift amount calculation unit 233 sequentially calculates the z-direction shift amount between pairs of sub-volumes (pairs of the reference sub-volume and the target sub-volume) sequentially set from the plurality of sub-volumes obtained by dividing the volume collected by the resuscitation scan. The z-position history data generation unit 234 collects the shift amount data sequentially calculated by the z-shift amount calculation unit 233. Thereby, the z-position history data generation unit 234 obtains a set of z-direction shift amounts between the plurality of sub-volumes obtained by dividing the volume collected by the resuscitation scan.

[0133] The z-position history data generation unit 234 arranges the collected set of z-direction shift amounts in accordance with the scan order corresponding to the multiple sub-volumes. That is, the z-position history data generation unit 234 arranges the collected set of z-direction shift amounts in chronological order (time passage, time axis) (ordered set). This provides data indicating the movement of the subject's eye E in the z direction during the period when the Lissajous scan was applied, that is, data indicating the position history of the subject's eye E in the z direction. This data is called z-position history data.

[0134] In this embodiment, by combining the xy position history data acquired by the xy position history data generation unit 232 and the z position history data acquired by the z position history data generation unit 234, data indicating the three-dimensional movement of the subject's eye E during the period when the Lissajous scan was applied, that is, data indicating the position history of the subject's eye E in three-dimensional directions (xyz directions), is obtained.

[0135] The position history data generated by the xy position history data generation unit 232 and the z position history data generation unit 234 is data in which time information and position information are associated with each other. As can be seen from the above-mentioned method of generating position history data, the position information in the position history data may represent displacement based on a certain point in time. The reference point in time of the position information may be, for example, the initial point in time during which the Lissajous scan was applied.

[0136] In some embodiments, when multiple sub-volumes (multiple strips based on the sub-volumes) obtained by dividing a volume acquired by Lissajous scanning are assigned ordinal numbers m=1, 2, . . . , M (M is an integer equal to or greater than 2) according to the scan order (i.e., chronological order), the position history data is expressed as {(Δx m , Δy m , Δz m , t m ):m=1, 2, . . . , M}.

[0137] where t m represents time (point in time), and Δxm , Δy m and Δz m is the time t m represent the displacement in the x direction, the y direction, and the z direction of the eye E at the time of measurement. For example, Δx1, Δy1, and Δz1 are all zero, and Δx m , Δy m and Δz m (m=2~M) is the time t m-1 From time t m 10A and 10B respectively represent the displacement of the eye E in the x direction, the y direction, and the z direction during the period from

[0138] The position history data according to another aspect may be the above-described exemplary position history data {(Δx m , Δy m , Δz m , t m ): m = 1, 2, . . . , M}, which associates time information with three-dimensional coordinates. m x coordinate in m is x m =Δx1+···+Δx m (m=1 to M). Similarly, at time t m y coordinate in m is y m =Δy1+···+Δy m is defined by, and at time t m z coordinate in z m is z m =Δz1+ +Δz m The resulting ordered set {(x m , y m , z m , t m ): m=1, 2, . . . , M} is an example of location history data.

[0139] The form of the position history data is not limited to these examples. For example, the position information is not limited to three-dimensional coordinates, but may be one-dimensional coordinates or two-dimensional coordinates. Furthermore, the position history data is not limited to discrete data, but may be continuous data. The visualization data generation unit 235, which will be described below, provides several forms of position history data.

[0140] <Visualization data generation unit 235> The visualization data generation unit 235 generates visualization data of time-series changes in the position of the subject's eye E based on the position history data generated by the xy position history data generation unit 232 and the z position history data generation unit 234. The visualization data is data that can be used to present (visualize) time-series changes in the position of the subject's eye E as visual information. Note that visual information presented based on the visualization data may also be referred to as visualization data.

[0141] An example of the visualization data generated by the visualization data generating unit 235 is graph data. The graph data is data that can be used to present, as visual information, a graph that represents a time-series change in the position of the subject's eye E. In some embodiments, the graph may be any mathematical diagram or statistical chart, such as a line graph, a regression curve graph, a bar graph, or a heat map. Some examples of graph data are described below.

[0142] First, the first example of graph data (first graph data) will be described. The first graph data represents a time series change in a predetermined directional component of the position of the subject's eye E defined in a predetermined coordinate system.

[0143] The first graph data in some embodiments may be graph data generated based on the xy position history data generated by the xy position history data generation unit 232 and the z position history data generated by the z position history data generation unit 234, and may include, for example, any of graph data representing the time series change in the x-direction component of the position of the subject eye E defined in the xyz coordinate system, graph data representing the time series change in the y-direction component, and graph data representing the time series change in the z-direction component.

[0144] In addition, the first graph data in some embodiments may be graph data generated based on the xy position history data generated by the xy position history data generation unit 232, and may include, for example, either graph data representing the time series change in the x-direction component of the position of the subject's eye E defined in the xy coordinate system, or graph data representing the time series change in the y-direction component.

[0145] In addition, the first graph data in some embodiments may be graph data generated based on the z position history data generated by the z position history data generation unit 234, and may include, for example, graph data representing the time series change in the z direction component of the position of the test eye E defined in the z coordinate system (z coordinate axis).

[0146] The graph data (first graph data) representing the time series change of the x-direction component is defined by, for example, a two-dimensional orthogonal coordinate system consisting of a time axis (t) and an x-axis (x). Using the above coordinate notation method, the ordered set {(t m , Δx m ):m=1, 2, ,M}, or an ordered set {(t m , x m ): m=1, 2, . . . , M}. The same applies to the graph data representing the time series change in the y-direction component and the graph data representing the time series change in the z-direction component.

[0147] Such first graph data is graph data obtained by projecting in a predetermined direction (x direction, y direction, or z direction) time-series changes in the three-dimensional position (x coordinate, y coordinate, and z coordinate) of the subject's eye E. Note that the projection direction is not limited to the x direction, y direction, or z direction, and may be any direction obliquely intersecting all of the x direction, y direction, and z direction.

[0148] Next, a second example of graph data (second graph data) will be described. The second graph data represents a time-series change in the two-dimensional position of the subject's eye E defined in a predetermined two-dimensional coordinate system.

[0149] The second graph data in some embodiments may be graph data generated based on the xy position history data generated by the xy position history data generating unit 232 and the z position history data generated by the z position history data generating unit 234. For example, the second graph data may include any of graph data representing time series changes in the two-dimensional position (x coordinate and y coordinate) of the subject's eye E defined in an xy coordinate system, graph data representing time series changes in the two-dimensional position (y coordinate and z coordinate) of the subject's eye E defined in a yz coordinate system, and graph data representing time series changes in the two-dimensional position (z coordinate and x coordinate) of the subject's eye E defined in a zx coordinate system.

[0150] In addition, the second graph data in some embodiments may be graph data generated based on the xy position history data generated by the xy position history data generation unit 232, and may include, for example, graph data representing the time series changes in the two-dimensional position (x coordinate and y coordinate) of the subject's eye E defined in the xy coordinate system.

[0151] Such second graph data is graph data obtained by projecting onto a predetermined plane (xy plane, yz plane, or zx plane) time-series changes in the three-dimensional position (x coordinate, y coordinate, and z coordinate) of the subject's eye E. Note that the projection plane is not limited to the xy plane, yz plane, or zx plane, and may be a plane of any orientation that obliquely intersects all of the xy plane, yz plane, and zx plane.

[0152] Next, a third example of graph data (third graph data) will be described. The third graph data represents a time-series change in the three-dimensional position of the subject's eye E defined in a predetermined three-dimensional coordinate system.

[0153] The third graph data in some embodiments may be graph data generated based on the xy position history data generated by the xy position history data generating unit 232 and the z position history data generated by the z position history data generating unit 234. For example, the third graph data may include graph data representing time-series changes in the three-dimensional position (x coordinate, y coordinate, and z coordinate) of the subject's eye E defined in an xyz coordinate system.

[0154] Next, a fourth example of graph data (fourth graph data) will be described. The fourth graph data represents a time-series change in the speed of movement of the subject's eye E.

[0155] The velocity of the movement of the subject's eye E may include any of a velocity in a three-dimensional space defined by a three-dimensional coordinate system, a velocity in a two-dimensional space, and a velocity in a one-dimensional space. m velocity v at m is the formula (Δx m 2 +Δy m 2 +Δz m 2 ) 1 / 2 / (t m -t m-1 ) or the formula (Δx m+1 2 +Δy m+1 2 +Δz m+1 2 ) 1 / 2 / (t m+1 -t m ) may include a three-dimensional velocity calculated at time t m velocity v at m is the formula (Δx m 2 +Δy m 2 ) 1 / 2 / (t m-t m-1 ) or the formula (Δx m+1 2 +Δy m+1 2 ) 1 / 2 / (t m+1 -t m ) is the two-dimensional velocity (velocity in the xy plane) calculated by the formula (Δy m 2 +Δz m 2 ) 1 / 2 / (t m -t m-1 ) or the formula (Δy m+1 2 +Δz m+1 2 ) 1 / 2 / (t m+1 -t m ) and the two-dimensional velocity (velocity in the yz plane) calculated by the formula (Δz m 2 +Δx m 2 ) 1 / 2 / (t m -t m-1 ) or the formula (Δz m+1 2 +Δx m+1 2 ) 1 / 2 / (t m+1 -t m ) may include either a two-dimensional velocity (velocity in the zx plane) calculated at time t m velocity v at m is the formula Δx m / (t m -t m-1 ) or the formula Δx m+1 / (t m+1 -t m ) is the one-dimensional velocity (velocity in the x direction), calculated by the formula Δy m / (t m -t m-1 ) or the formula Δy m+1 / (t m+1 -t m ) and the one-dimensional velocity (velocity in the y direction) calculated by the formula Δz m / (t m -tm-1 ) or the formula Δz m+1 / (t m+1 -t m ) or one-dimensional velocity (velocity in the z direction) calculated by

[0156] The fourth graph data representing the time series change in one-dimensional velocity is not limited to graph data corresponding to the x direction, y direction, or z direction, but may be graph data representing the time series change in the velocity of movement of the subject's eye E in any direction obliquely intersecting all of the x direction, y direction, and z direction.

[0157] Similarly, the fourth graph data representing the time series change in two-dimensional velocity is not limited to graph data corresponding to the xy plane, yz plane, or zx plane, but may be graph data representing the time series change in the velocity of movement of the subject's eye E in a plane of any orientation that intersects obliquely with all of the xy plane, yz plane, and zx plane.

[0158] Furthermore, the coordinate system used to generate the graph data is not limited to the xyz coordinate system or a coordinate system consisting of a part of it, but may be any coordinate system that can be used to represent the position of the subject's eye E.

[0159] A specific example of the graph data will be described later.

[0160] The visualization data is not limited to the graph data in the above-described manner, but may be any data for visualization based on xy position history data and / or z position history data.

[0161] For example, the visualization data generating unit 235 may be configured to generate visualization data representing frequency components of the movement (eye movement) of the subject's eye E. As described in Japanese Patent Application Laid-Open No. 2007-130403 (U.S. Patent Application Publication No. 2010 / 0142780) and "Heartbeat-Induced Axial Motion Artifacts in Optical Coherence Tomography Measurements of the Retina" (Roy de Kinkelder, et al., IOVS, May 2011, Vol. 52, No. 6, pp. 3908-3913), it is known that pulse-like (almost periodic) motion artifacts in the depth direction (z direction) resulting from heartbeats exist. The visualization data of this pulse-like motion artifact may be data representing a time-series change in the amount of shift in the z direction, and may be, for example, graph data defined by a time axis and a z axis.

[0162] The ophthalmic examination apparatus 1 can extract and evaluate heartbeat-induced motion artifacts from z position history data (or three-dimensional position history data obtained by combining xy position history data and z position history data). This makes it possible to obtain data on the movement of the subject's eye E caused by heartbeat with high accuracy. The ophthalmic examination apparatus 1 can also remove the extracted heartbeat-induced motion artifacts from the z position history data or three-dimensional position history data. This makes it possible to obtain data on the movement of the subject's eye E excluding the influence of heartbeat. By generating this data using a Lissajous scan, it is believed that it will be possible to evaluate heartbeat-induced motion artifacts and eye movements such as nystagmus with higher accuracy than before.

[0163] <Image data correction unit 236> The image data correction unit 236 is configured to perform z-direction position adjustment for multiple sub-volumes whose positions have been adjusted in the x- and y-directions by the image data processing unit 231, based on the z-direction shift amount obtained by the z-shift amount calculation unit 233 (a collection of z-direction shift amounts collected by the z position history data generation unit 234).

[0164] Similar to the registration unit 2317 that performs registration based on the xy direction shift amount, the image data correction unit 236 performs registration between the reference subvolume and the target subvolume so as to cancel out the corresponding z direction shift amount obtained by the z shift amount calculation unit 233.

[0165] As previously mentioned, in some aspects, the image data corrector 236 may be controlled to perform registration between the reference sub-volume and the target sub-volume only if the corresponding z-shift exceeds a threshold.

[0166] The image data obtained as a result of the position adjustment by the image data corrector 236 is a group of subvolumes that have been registered in all of the x-, y-, and z-directions, i.e., a group of subvolumes that have been three-dimensionally registered. The group of subvolumes before application of such registration is a volume acquired by Lissajous scanning. The data processor 230 recombines the group of subvolumes obtained by three-dimensional registration to obtain a volume to which three-dimensional registration has been applied.

[0167] The 3D registered dataset can then be used for any processing (image processing, post-processing, analysis, evaluation, visualization, rendering, etc.) This allows the desired processing to be performed using a 3D dataset that has been corrected for motion artifacts with higher accuracy and precision than ever before.

[0168] In addition, the z-shift amount calculation unit 233 and the image data correction unit 236 may be configured to perform registration equivalent to "axial motion correction (rough axial motion correction and / or fine axial motion correction)" (pages 1789 to 1790) of Non-Patent Document 1 in order to remove motion artifacts in the z direction.

[0169] <Evaluation data generation unit 237> The evaluation data generation unit 237 is configured to generate evaluation data based on the xy position history data generated by the xy position history data generation unit 232 and / or the z position history data generated by the z position history data generation unit 234.

[0170] The evaluation data may be data relating to any evaluation item that can be referenced in the field of ophthalmology and / or other fields. Some examples of evaluation data and some examples of processing that can be executed by the evaluation data generation unit 237 are described below. Note that the types of evaluation data and the contents of processing executed by the evaluation data generation unit 237 are not limited to the following examples.

[0171] The evaluation data generation unit 237 may be configured to generate evaluation data (reliability evaluation data) regarding the reliability of a predetermined process as the evaluation data. The reliability evaluation data may include, for example, any of evaluation data regarding the reliability of the process itself to be evaluated, evaluation data regarding the reliability of a step (sub-process) included in the process to be evaluated, evaluation data regarding the reliability of a process that includes the process to be evaluated as a sub-process, evaluation data regarding the reliability of a result (output) of the process to be evaluated, and evaluation data regarding the reliability of an input to the process to be evaluated.

[0172] Reliability refers to the property (ability) or probability that the process being evaluated can perform the required function (role) under given conditions, and is also called reliability. The reliability evaluation data may include information indicating whether or not the process is reliable, information indicating the degree of reliability, or information that quantifies reliability.

[0173] In some embodiments, the reliability evaluation data may include reliability evaluation data of motion artifact correction. In this embodiment, the reliability evaluation data may include reliability evaluation data of motion artifact correction in the x and y directions performed by the image data processing unit 231, or may include reliability evaluation data of motion artifact correction in the z direction performed by the z-shift amount calculation unit 233 and the image data correction unit 236.

[0174] As described above, the image data constructing unit 220 constructs multiple image data (e.g., multiple sub-volumes, multiple strips) based on the data set collected by the Lissajous scan. The image data processing unit 231 functions as a position correcting unit and performs relative position correction (motion artifact correction in the x and y directions) for the multiple image data. The z-shift amount calculating unit 233 and the image data correcting unit 236 also function as position correcting units and perform relative position correction (motion artifact correction in the z direction) for the multiple image data. The evaluation data generating unit 237 in this example is configured to generate, as reliability evaluation data, reliability evaluation data for motion artifact correction in the x and y directions and / or reliability evaluation data for motion artifact correction in the z direction.

[0175] Machine learning can be used to generate the reliability evaluation data. For example, as shown in FIG. 4D , the evaluation data generation unit 237 may include an inference model 2371. The inference model 2371 is trained in advance by machine learning using training data. Any machine learning method may be used, and may include at least one of supervised learning, unsupervised learning, and reinforcement learning.

[0176] The training data may include, for example, a set of position history data (data representing time-series changes in eye position) collected by processing similar to that performed by the data processing unit 230. The training data may also include a set of processed data generated by processing the position history data. The processed data may be of a type that can be generated by the data processing unit 230, and may include, for example, any of the various types of visualization data described above. In the case of supervised learning, the training data includes, for example, labels assigned to each piece of position history data and / or each piece of processed data. The label is data indicating reliability (confidence level), and may be, for example, a label indicating "success" of motion artifact correction (Lissajous scan, photography), a label indicating "failure," or a label indicating "probability of success (numeric value)."

[0177] Input to the inference model 2371 includes position history data and / or processed data (such as visualization data) generated by the data processing unit 230. Output from the inference model 2371 corresponding to such data input includes a confidence level. The output confidence level is information indicating, for example, the "success," "failure," or "probability of success (numeric value)" of the motion artifact correction (Lissajous scan, photography).

[0178] In this embodiment, for example, at least one or more of the xy position history data of the subject's eye E generated by the xy position history data generation unit 232, the z position history data generated by the z position history data generation unit 234, and the visualization data generated by the visualization data generation unit 235 are input to the inference model 2371. The inference model 2371 can generate reliability evaluation data of the motion artifact correction (relative position correction) based on the input data.

[0179] In this way, the evaluation data generating unit 237 of this aspect can generate reliability evaluation data of motion artifact correction based on the x-direction shift characteristics, y-direction shift characteristics, z-direction shift characteristics, etc. of the subject's eye E during the period when the Lissajous scan was being performed. The shift characteristics may be any characteristics related to the movement (shift) of the subject's eye E, and may be, for example, the shift amount, statistical values ​​of the shift amount (maximum value, minimum value, average value, standard deviation, etc.), time-series change in the shift amount, parameter values ​​of a graph of the time-series change in the shift amount (frequency, amplitude, etc.), shift speed, statistical values ​​of the shift speed, time-series change in the shift speed, parameter values ​​of a graph of the time-series change in the shift speed, etc.

[0180] The data input to the inference model 2371 is not limited to the shift characteristics of the subject's eye E. For example, blink data of the subject's eye E can be input to the inference model 2371 together with the shift characteristics. The blink data includes, for example, whether or not the subject's eye E blinked during the period in which the Lissajous scan was applied, the number of times, the frequency, and the time point. The blink data may also include integrated data covering at least a portion of the period in which the Lissajous scan was applied.

[0181] If the training data also includes data on specific eye-related items such as blink data, the inference model 2371 trained using this training data is constructed to take as input at least one of the position history data and its processed data, and at least one of the data on the item and its processed data, and to output reliability.

[0182] The ophthalmic examination apparatus 1 of this example has a function of acquiring data on a particular item related to the subject's eye E. The ophthalmic examination apparatus 1 may be configured to generate data on the particular item by examining the subject's eye E. For example, the ophthalmic examination apparatus 1 can generate blink data by analyzing an observation image acquired by the fundus camera unit 2. Alternatively, the blink data can be generated by analyzing video images acquired by two or more anterior eye cameras used for the stereo alignment described above. Similarly, when generating data on an item other than blinking, the data can be generated by acquiring and analyzing an image of the subject's eye E. Meanwhile, the ophthalmic examination apparatus 1 may be configured to externally accept data on a particular item related to the subject's eye E. For example, the ophthalmic examination apparatus 1 may have a component (e.g., a data accepting unit 250 described below) for acquiring data on a particular item related to the subject's eye E from an external device (e.g., a computer, a storage device, a recording medium, an information system, etc.). The ophthalmic examination apparatus 1 of this example may be configured to process the generated or accepted data on a particular item of the subject's eye E to generate new data (processed data).

[0183] The evaluation data generation unit 237 of this example inputs at least one of data of predetermined items of the subject's eye E and its processed data, and at least one of position history data of the subject's eye E and its processed data (visualization data, etc.) generated by the data processing unit 230 to the inference model 2371. The inference model 2371 can generate reliability evaluation data of the motion artifact correction based on the input data.

[0184] We will now explain some examples of the inference model 2371. A first example of the inference model 2371 includes a feature extractor that extracts features from input data, and a support vector machine (SVM) that receives the extracted features as input and outputs reliability evaluation data.

[0185] The feature extractor may be constructed using, for example, any feature engineering technique. A support vector machine is a pattern recognition model that uses supervised learning. It uses linear input elements to construct a two-class pattern classifier, and learns the parameters of the linear input elements based on the criterion (separation hyperplane theorem) of finding a margin-maximizing hyperplane that maximizes the distance from each data point in the training data (training sample). The support vector machine in this example performs inference (classification, regression, etc.) based on the features extracted by the feature extractor.

[0186] The evaluation data generation unit 237 having the inference model 2371 configured as described above can, for example, perform frequency analysis on the position history data and / or visualization data generated by the data processing unit 230 to determine the amplitude and frequency of the movement of the subject's eye E as feature quantities. Furthermore, the evaluation data generation unit 237 of this example can determine the total blink time of the subject's eye E during the period when the Lissajous scan was being performed as feature quantity based on the blink data. Furthermore, the evaluation data generation unit 237 of this example can input these feature quantities into a support vector machine to classify (identify) whether the motion artifact correction (Lissajous scan, photography) was successful or unsuccessful, and can also determine the success probability (reliability).

[0187] A second example of the inference model 2371 includes a neural network (NN) that generates reliability evaluation data from input data. The neural network in this example may typically be a learned convolutional neural network (CNN) that has been trained using a dataset (with labels such as success, failure, and probability of success (reliability) of imaging) of the same type as the input data (position history data, visualization data, blink data, etc.).

[0188] An example of a convolutional neural network will be described. Predetermined data (such as position history data, visualization data, and blink data) is input to the input layer of a convolutional neural network. The input data is typically an image. Multiple pairs of convolutional layers and pooling layers are arranged behind the input layer. The number of pairs is arbitrary. The convolutional layer performs convolution operations to extract features (such as contours) from an image. The convolutional operation is a multiplication-and-accumulation operation on the input image, using a filter function (weighting coefficient, filter kernel) of the same dimension as the image. The convolutional layer applies the convolution operation to multiple portions of the input image. More specifically, the convolutional layer multiplies the value of each pixel in the partial image to which the filter function has been applied by the value (weight) of the filter function corresponding to that pixel to calculate the product, and then calculates the sum of the products across the multiple pixels in this partial image. The resulting product-and-accumulation value is assigned to the corresponding pixel in the output image. By performing the product-and-accumulation operation while moving the location (partial image) to which the filter function is applied, the convolutional operation result for the entire input image can be obtained. Such convolutional operations produce a large number of images in which various features are extracted using a large number of weighting coefficients. In other words, a large number of filtered images, such as smoothed images and edge images, are produced. The large number of images produced by the convolutional layer are called feature maps. The pooling layer compresses (e.g., thins out) the feature map produced by the previous convolutional layer. More specifically, the pooling layer calculates statistical values ​​for a predetermined number of neighboring pixels of a pixel of interest in the feature map at predetermined pixel intervals, and outputs an image with dimensions smaller than the input feature map. The statistical values ​​applied to the pooling operation are, for example, maximum values ​​(max pooling) or average values ​​(average pooling). The pixel interval applied to the pooling operation is called the stride. In a convolutional neural network, multiple pairs of convolutional layers and pooling layers are provided for processing, allowing for the extraction of many features from an input image. A fully connected layer is provided after the last pair of convolutional layers and pooling layers. The number of fully connected layers is arbitrary.In the fully connected layer, inference (classification, regression, etc.) is performed using features compressed by convolution and pooling. An output layer that provides output results is provided after the fully connected layer. Note that in some exemplary embodiments, the convolutional neural network may not include a fully connected layer, or may include a support vector machine or a recurrent neural network (RNN).

[0189] The evaluation data generation unit 237 having the inference model 2371 configured in this manner can classify (identify) whether the motion artifact correction (Lissajous scan, photography) was successful or unsuccessful based on, for example, the position history data, visualization data, blink data, etc. generated by the data processing unit 230, and can also calculate the probability of success (reliability).

[0190] The evaluation data generating unit 237 may be configured to obtain evaluation data for each of a plurality of portions of the image data. This image data may be any image data constructed from a data set collected by a Lissajous scan, such as image data constructed by the image data constructing unit 220, image data constructed by the image data processing unit 231 (e.g., the merge processing unit 2318), or image data constructed by the image data correcting unit 236.

[0191] For example, the evaluation data generation unit 237 may be configured to execute a process of dividing image data into a plurality of partial regions and a process of generating evaluation data for each partial region. The ophthalmic examination apparatus 1 of this example (e.g., the main control unit 211) can assign display parameters to each partial region according to its evaluation data. The display parameters may be, for example, color parameters corresponding to the magnitude of the reliability value. The main control unit 211 can display a heat map relating to the reliability based on the color parameters assigned to each partial region. It is possible to display the heat map superimposed on the image of the subject's eye E or to display the heat map synthesized with the image of the subject's eye E. This makes it possible to visualize the reliability of each part of the image (i.e., the distribution of reliability).

[0192] Heat map display is also effective in wide-angle imaging (panoramic imaging, montage imaging). Wide-angle imaging is a technique in which multiple regions of the subject's eye E are sequentially scanned (Lissajous scan), multiple image data are constructed based on multiple data sets collected from each region, and these image data are then combined into a panoramic image. Wide-angle imaging is achieved, for example, by moving the scan application region by moving the fixation position. The evaluation data generation unit 237 can generate evaluation data for each of the multiple image data corresponding to the multiple regions. The main control unit 211 can assign display parameters (color parameters) to each image data according to the evaluation data. The main control unit 211 can display a heat map indicating the reliability of each image data based on the color parameters assigned to each of the multiple image data. It is possible to display the heat map superimposed on a panoramic composite image or to combine the heat map with the panoramic composite image. This makes it possible to visualize the reliability of each of the multiple images taken in wide-angle imaging.

[0193] The evaluation data generation method that takes into account data other than the position history data is not limited to a method using machine learning. For example, in some embodiments, the evaluation data generation unit 237 may be configured to generate the evaluation data using any method based on the position history data and data of predetermined items. The data of the predetermined items includes data generated by the ophthalmic examination apparatus 1 and / or data accepted by the data accepting unit 250.

[0194] Several application examples of evaluation using the ophthalmic examination device 1 will be described. By using the position history data and visualization data as biomarkers of eye movement, it is possible to detect and evaluate neurological disorders (such as nystagmus) caused by Alzheimer's disease or Parkinson's disease. For example, by using the position history data and visualization data as biomarkers for amyloid beta (Aβ) and combining them with information on thinning of the retinal nerve fiber layer (RNFL) (RNFL thickness distribution) obtained from fundus OCT image data (B-scan image data, 3D image data, etc.), it is possible to evaluate Alzheimer's disease.

[0195] There are other diseases for which position history data and visualization data can be used as biomarkers of eye movement. For example, internal carotid-cavernous fistula, a disease caused by a skull base fracture or a ruptured aneurysm, presents with pulsatile exophthalmos, pulsatile noise in the orbit, conjunctival congestion, external ophthalmoplegia, visual impairment, diplopia, elevated intraocular pressure, retinal hemorrhage, and papilledema. However, because the symptoms are similar to those of conjunctivitis, physicians must carefully differentiate the condition. When an ophthalmologist suspects this disease, they typically listen to the eyeballs and orbital noise with a bell-shaped stethoscope and refer the patient to a neurosurgeon. In addition to these conventional diagnostic procedures, some applications of the present embodiment make it possible to evaluate pulsatile exophthalmos and external ophthalmoplegia by using position history data and visualization data as biomarkers of eye movement.

[0196] <Position history data correction unit 238> The position history data correcting unit 238 corrects the position history data based on the time-series data acquired from the subject. As shown in Figures 4B and 4C, the xy position history data generated by the xy position history data generating unit 232 and the z position history data generated by the z position history data generating unit 234 are input to the position history data correcting unit 238. The position history data correcting unit 238 can correct at least one of the xy position history data and the z position history data.

[0197] An example of correction of position history data will be described. When the ophthalmic examination apparatus 1 is equipped with a stereo camera (not shown), the pupil center is detected from a pair of anterior eye segment moving images obtained by the stereo camera, and the movement of the subject's eye E (including eye movement and head movement) can be evaluated by detecting the change in its position over time. In addition, by detecting a part other than the pupil center (for example, the outer corner of the eye, the inner corner of the eye, or the eyelid) as a feature point and subtracting the displacement of the feature point from the displacement of the pupil center, it is possible to separate and evaluate eye movement alone from head movement.

[0198] The xy position history data generated by the xy position history data generating unit 232 and the z position history data generated by the z position history data generating unit 234 (and further, the visualization data and evaluation data based on these) contain information on not only eye movement but also head movement, etc. In order to accurately detect and evaluate eye movement, it is desirable to remove movement of the subject's eye E (head movement, etc.) that is not due to eye movement from the position history data (visualization data, evaluation data).

[0199] Therefore, the position history data correction unit 238 of this embodiment is configured to remove movements of the subject's eye E (such as head movements) that are not due to eye movements from the position history data by utilizing the above information obtained using the stereo camera (separation of movements of the subject's eye E that are due to eye movements and movements of the subject's eye E that are due to other factors).

[0200] For example, the position history data correction unit 238 first performs a process of aligning the time axes of time series data (correction time series data) of the movement of the subject's eye E derived from factors other than eye movement, obtained using a stereo camera, with the position history data (time series data). Next, the position history data correction unit 238 extracts eye movement from the position history data by subtracting the correction time series data from the position history data. Note that detection of the movement of the subject's eye E using a stereo camera can detect three-dimensional movement of the subject's eye E (movement in the x direction, y direction, and z direction), and the position history data correction unit 238 can subtract the correction time series data from the position history data for each of the x direction, y direction, and z direction. The processing flow when such position history data correction is performed is shown in FIG. 6.

[0201] The visualization data generation unit 235 is capable of generating visualization data from the position history data corrected by the position history data correction unit 238. This makes it possible to obtain visualization data corresponding to eye movement. Furthermore, the evaluation data generation unit 237 is capable of generating evaluation data based on the position history data corrected by the position history data correction unit 238 and / or visualization data generated from the corrected position history data. This makes it possible to obtain evaluation data corresponding to eye movement.

[0202] The visualization data generation unit 235 can apply data correction similar to that performed by the position history data correction unit 238 to the visualization data generated by the visualization data generation unit 235 based on the xy position history data generated by the xy position history data generation unit 232 and / or the z position history data generated by the z position history data generation unit 234. This makes it possible to obtain visualization data corresponding to eye movement. Such correction of the visualization data is performed by the data processing unit 230 (a visualization data correction unit not shown).

[0203] The evaluation data generated by the evaluation data generation unit 237 based on the xy position history data generated by the xy position history data generation unit 232 and / or the z position history data generated by the z position history data generation unit 234 can be subjected to data correction similar to that performed by the position history data correction unit 238. This makes it possible to obtain evaluation data corresponding to eye movement. Such correction of the evaluation data is performed by the data processing unit 230 (an evaluation data correction unit not shown).

[0204] It should be noted that technologies that can be combined with the detection, analysis, and evaluation of the movement of the subject's eye E using a Lissajous scan are not limited to a stereo camera (detection and analysis of the movement of the subject's eye E using the stereo camera). In some embodiments, any measurement method can be combined with the generation of position history data and / or evaluation data. This measurement method is typically a medical method for acquiring data related to eye movement (such as time-series data). Such medical methods include Doppler measurement, infrared photography, a pulse meter, an optical coherence elastography (OCE) device, and the like.

[0205] For example, a method for measuring tissue stiffness using optical coherence elastography is known. In this conventional technique, OCE imaging is performed using phase information to evaluate minute movements, but this embodiment can provide a novel measurement method for estimating tissue stiffness from measurement data of the depth-direction pulse-like vibration (motion artifact) caused by the heartbeat.

[0206] <User Interface 240> The user interface 240 includes a display unit 241 and an operation unit 242. The display unit 241 includes the display device 3. The operation unit 242 includes various operation devices and input devices. The user interface 240 may include a device that combines a display function and an operation function, such as a touch panel. It is also possible to construct an embodiment that does not include at least a part of the user interface 240. For example, the display device may be an external device connected to the ophthalmic examination apparatus 1.

[0207] <Data Receiving Unit 250> The data receiving unit 250 acquires data from an external device. For example, the data receiving unit 250 receives data on predetermined items related to the eye E from the external device.

[0208] The external device may be, for example, a computer, a storage device, a recording medium, an information system (for example, a hospital information system, an electronic medical record system, an image archiving system), etc. The external device may include, for example, a device directly connected to the ophthalmic examination apparatus 1, a device connected to the ophthalmic examination apparatus 1 via a local area network (LAN), a device connected to the ophthalmic examination apparatus 1 via a wide area network (WAN), etc.

[0209] The data receiving unit 250 may include, for example, a communication interface, a drive device, etc. The data receiving unit 250 may also include a scanner for reading data recorded on paper sheets, etc. The data receiving unit 250 may also include an input device for a user to input data. The input device may be a keyboard, a pen tablet, etc.

[0210] <Operation> The operation of the ophthalmic examination apparatus 1 will now be described. Examples of the operation of the ophthalmic examination apparatus 1 are shown in Figs. 7A to 7F. Note that in some embodiments, only some of the steps in the operation examples may be executed. Also, in some embodiments, some of the steps in the operation examples may be replaced with other steps (for example, similar steps). Also, in some embodiments, any step may be combined with the operation examples.

[0211] Prior to step S1, preparatory processing similar to that of the conventional method is performed, such as inputting the patient ID, setting the scanning mode (specifying Lissajous scan), presenting the fixation target, alignment, focus adjustment, and OCT optical path length adjustment.

[0212] (S1: Lissajous Scan) Upon receiving a predetermined scan start trigger signal, the scan control unit 2111 starts applying an OCT scan (Lissajous scan) to the subject's eye E (fundus oculi Ef).

[0213] The scanning start trigger signal is generated, for example, in response to the completion of a predetermined preparatory operation (alignment, focus adjustment, OCT optical path length adjustment, etc.), or in response to a scanning start instruction operation being performed using the operation unit 242.

[0214] The scan control unit 2111 applies a Lissajous scan to the fundus Ef by controlling the optical scanner 44, the OCT unit 100, etc. based on a scan protocol 2121 (a protocol corresponding to the Lissajous scan). The data collected by the Lissajous scan is sent to the image data construction unit 220.

[0215] (S2: Build multiple strips) The image data constructing unit 220 constructs a plurality of strips based on the data collected in step S1 in the manner described above. The constructed plurality of strips are sent to the data processing unit 230.

[0216] (S3: Set the reference strip and target strip) The data processing unit 230 orders the multiple strips constructed in step S2 according to their dimensions (area, etc.). In this example, the number of multiple strips constructed in step S2 is assumed to be N (N is an integer equal to or greater than 2). Furthermore, according to the order specified by the ordering, these N strips are referred to as the first strip, second strip, ..., Nth strip. Furthermore, any strip from the first to Nth strips may be referred to as the nth strip (n = 1, 2, ..., N). In this way, the first strip is the largest strip, the second strip is the second largest strip, the nth strip is the nth largest strip, and the Nth strip is the smallest strip.

[0217] In this example, the data processing unit 230 sets the first strip as the initial reference strip and the second strip as the initial target strip.

[0218] The reference strip and the target strip correspond to the arbitrary shaped images f(r) and g(r) in Appendix A of Non-Patent Document 1, respectively.

[0219] (S4: Set the reference mask image and the target mask image) The mask image generation unit 2311 generates a mask image (reference mask image) corresponding to the reference strip and a target mask image corresponding to the target strip.

[0220] At this stage, the mask image generation unit 2311 generates two mask images corresponding to the first strip and the second strip, respectively. The data processing unit 230 sets the first mask image corresponding to the first strip as an initial reference mask image, and sets the second mask image corresponding to the second strip as an initial target mask image.

[0221] The reference mask image and the target mask image are rectangular shaped binary image masks m in Appendix A of Non-Patent Document 1, respectively. f (r) and m g Equivalent to (r).

[0222] (S5: Normalize the reference strip and target strip) The range adjustment unit 2312 normalizes the reference strip and the target strip set in step S3 in the manner described above. The strips are normalized according to the range of pixel values ​​of the mask image. At this stage, the range adjustment unit 2312 applies normalization to the first strip (reference strip) and the second strip (target strip). Note that the process performed by the range adjustment unit 2312 is not limited to normalization, and may be any of the range adjustment processes described above or a process similar thereto.

[0223] In this example, the pixel value range of each mask image is assumed to be in the closed interval [0, 1]. In particular, each mask image in this example is assumed to be a binary image in which the pixel values ​​within the region corresponding to the domain of the corresponding strip are set to 1 and the other pixel values ​​are set to 0.

[0224] The range adjustment unit 2312 normalizes the strip, for example, by dividing the value of each pixel in the strip by the maximum pixel value in the strip, or by dividing the value of each pixel in the strip by the maximum value of the range of pixel values ​​(tone range) of the strip.

[0225] Furthermore, the data processing unit 230 (synthetic image generation unit 2313) embeds the strip with the normalized pixel value range into an image with the same dimensions and shape as the corresponding mask image. The embedded image of the normalized first strip f(r) is similar to the rectangular shaped image f'(r) in Appendix A of Non-Patent Document 1, but the absolute value (|f(r)|) of the value range within the image area of ​​the first strip f(r) is normalized to 1 or less. This embedded image is also represented as f'(r).

[0226] Similarly, the embedded image of the normalized second strip g(r) is an image similar to the rectangular image g'(r), but the absolute value of the range within the image area of ​​the second strip g(r) (|g(r)|) is normalized to be less than or equal to 1. This embedded image is also denoted as g'(r).

[0227] (S6: Generate a reference composite image and a target composite image) The composite image generator 2313 generates a composite image of the embedded image of the normalized strip and the corresponding mask image.

[0228] At this stage, the composite image generation unit 2313 generates a composite image of the embedded image of the normalized first strip (reference strip) and the first mask image (reference mask image), and also generates a composite image of the embedded image of the normalized second strip (target strip) and the second mask image (target mask image). The former composite image is called the reference composite image, and the latter is called the target composite image.

[0229] Here, the reference composite image and the target composite image are the combination of two rectangle shaped images f'(r)m in Appendix A of Non-Patent Document 1, respectively. f (r) and g´(r)m g(r), where, as mentioned above, the absolute value of the range of the first strip f(r) in the image area (|f(r)|) is less than or equal to 1, and the absolute value of the range of the second strip g(r) in the image area (|g(r)|) is less than or equal to 1.

[0230] Image 311 in Figure 8A is an example of an embedded image f'(r) of the normalized first strip f(r), and image 321 is an example of a first mask image. Combining these two images 311 and 321 produces a reference composite image f'(r)m f (r) is obtained.

[0231] Similarly, image 312 in Figure 8B is an example of an embedded image g'(r) of the normalized second strip g(r), and image 322 is an example of a second mask image. Combining these two images 312 and 322 produces a reference composite image g'(r)m g (r) is obtained.

[0232] (S7: Calculate multiple cross-correlation functions) The cross-correlation function calculation unit 2314 calculates a plurality of cross-correlation functions based on the reference composite image and the target composite image generated in step S6. In this example, the cross-correlation function calculation unit 2314 calculates six cross-correlation functions included in equation (33) of Non-Patent Document 1 based on the reference composite image and the target composite image generated in step S6.

[0233] (S8: Calculate the correlation coefficient) The correlation coefficient calculation unit 2315 calculates a correlation coefficient based on the multiple cross-correlation functions calculated in step S7. In this example, the correlation coefficient calculation unit 2315 calculates the correlation coefficient (ρ(r')) from the multiple cross-correlation functions calculated in step S7 according to equation (33) in Non-Patent Document 1.

[0234] (S9: Calculate the shift amount in the x and y directions between the reference strip and the target strip) The xy shift amount calculation unit 2316 calculates the amount of shift in the xy directions between the first strip (reference strip) f(r) and the second strip (target strip) g(r) based on the correlation coefficient calculated in step S8. In this example, the relative amount of shift in the xy directions (Δx, Δy) between the reference strip f(r) and the target strip g(r) is found by detecting the peak of the correlation coefficient ρ(r') calculated in step S8.

[0235] (S10: Saves the shift amount in the x and y directions) The x and y direction shift amounts calculated for the strip pair in step S9 are sent to and stored in the x and y position history data generation unit 232. The x and y position history data generation unit 232 records the x and y direction shift amounts together with time information corresponding to the strip pair, for example.

[0236] The time information may be a time parameter corresponding to a Lissajous scan or information based thereon, or may be order information regarding a Lissajous scan or information based thereon. The order information regarding a Lissajous scan may be, for example, a scan order of multiple cycles, an order of multiple strips based thereon, etc.

[0237] The position history data generation unit 234 generates xy position history data by recording the xy direction shift amounts sequentially acquired by repeating steps S3 to S12 together with time information.

[0238] (S11: Registration between the reference strip and the target strip) The registration unit 2317 applies rough lateral motion correction to the reference strip and the target strip based on the x and y shift amounts calculated in step S9. At this stage, registration between the first strip f(r) and the second strip is performed.

[0239] Furthermore, the registration unit 2317 may apply any xy registration to the reference strip and the target strip, such as the "fine lateral motion correction" described above.

[0240] (S12: Construct a merged image of the reference strip and the target strip) The merge processing unit 2318 constructs a merged image of the reference strip and the target strip whose relative positions have been adjusted by the registration in step S11. Image 330 in Figure 8C shows an example of a merged image of the first strip and the second strip.

[0241] Here, an example of implementing the processes in steps S7 to S12 will be described. First, f'(r)m f (r), g´(r)m g (r), m f (r), m g (r), (f´(r)m f (r)) 2 , and (g´(r)m g (r)) 2 Set the imaginary part of each of the above to 0, leaving only the real part.

[0242] Next, f´(r)m f (r), g´(r)m g (r), m f (r), m g (r), (f´(r)m f (r)) 2 , and (g´(r)m g (r)) 2 A fast Fourier transform (FFT) is applied to each real part of

[0243] Next, based on the group of functions obtained by these fast Fourier transforms, the six cross-correlation functions shown in step S7 of FIG. 6 are calculated.

[0244] Next, an inverse fast Fourier transform (IFFT) is applied to each of the six calculated cross-correlation functions.

[0245] Next, the correlation coefficient ρ(r′) of equation (33) in Non-Patent Document 1 is calculated.

[0246] Then, the relative shift amount (Δx, Δy) between the reference strip f(r) and the target strip g(r) is calculated by identifying the peak position of the correlation coefficient ρ(r'), and registration and merging processes between the reference strip f(r) and the target strip g(r) are performed based on this shift amount.

[0247] By applying this series of processes to the N strips in turn, merged images of all N strips are obtained. In addition, the xy direction shift amount (Δx, Δy) calculated for the strip pair is saved together with the time information (i.e., associated with the time information).

[0248] (S13: Have you processed all the strips?) A series of processes from steps S3 to S12 are executed for all of the N strips obtained in step S2, in the order described above.

[0249] When N is 3 or more, if a merged image of the first strip and the second strip is created in step S12 (S13: No), the process returns to step S3. In step S3, the merged image of the first strip and the second strip is set as a new reference strip, and the third strip is set as a new target strip. By performing the processes of steps S4 to S12 based on the new reference strip and the new target strip, a merged image of the new reference strip and the new target strip is obtained. This new merged image is a merged image of the first to third strips. By sequentially applying this series of processes to the N strips, merged images of all N strips are obtained (S13: Yes).

[0250] (S14: Save the image with motion artifacts corrected in the xy direction) The merged image finally obtained by the above-described repetitive processing is an image in which motion artifacts in the x and y directions have been corrected, and is an image that represents the entire application range of the Lissajous scan in step S1. The main control unit 211 can store this final merged image in the storage unit 212 (and / or other storage device).

[0251] The data processing unit 230 can use the registration results based on the N strips to register the data (three-dimensional data) collected in step S1 and / or the three-dimensional image data constructed by the image data construction unit 220 from this three-dimensional data. In other words, the data processing unit 230 can use the final merged image to register the three-dimensional data or the three-dimensional image data. This registration includes converting the coordinate system defining the Lissajous scan into a three-dimensional Cartesian coordinate system (x-y-z coordinate system). This coordinate conversion corresponds to "remapping" in the method described in Non-Patent Document 1. In this way, three-dimensional image data (volume) in which x- and y-direction motion artifacts have been corrected is obtained. The main control unit 211 can store this x- and y-direction motion artifact-corrected volume in the storage unit 212 (and / or another storage device) together with or instead of the final merged image.

[0252] (S15: Select the reference strip) The process then moves on to processing for correcting z-direction motion artifacts. In z-direction motion artifact correction, first, the z-shift amount calculation unit 233 selects one strip (reference strip) from multiple strips corresponding to the volume in which x- and y-direction motion artifacts have been corrected. The reference strip selected here may be the first strip selected in step S3 the first time, or may be a strip other than this.

[0253] (S16: Select the target strip) Next, the z-shift amount calculation unit 233 selects one strip (target strip) other than the reference strip selected in step S15 from the multiple strips corresponding to the volume in which the xy-direction motion artifacts have been corrected. The reference strip selected here may be the second strip selected in step S3 the first time, or may be a strip other than this.

[0254] (S17: Set the reference subvolume corresponding to the reference strip) Next, the z-shift amount calculation unit 233 sets the sub-volume corresponding to the reference strip selected in step S15 as the reference sub-volume. Here, the front projection image of the reference sub-volume is the reference strip.

[0255] (S18: Set the target subvolume corresponding to the target strip) Similarly, the z-shift amount calculation unit 233 sets the sub-volume corresponding to the target strip selected in step S16 as the target sub-volume. Here, the front projection image of the target sub-volume is the target strip.

[0256] (S19: Identify the intersection area between the reference subvolume and the target subvolume) Next, the z-shift amount calculation section 233 identifies an intersection area (common area) between the reference sub-volume set in step S17 and the target sub-volume set in step S18.

[0257] As described above, there are four intersection regions between the two sub-volumes. The z-shift amount calculation unit 233 identifies one or more intersection regions. When two or more intersection regions are identified, the following step S20 is executed for each intersection region.

[0258] (S20: Calculate the z-direction shift amount between the reference subvolume and the target subvolume) Next, the z-shift amount calculation unit 233 calculates the amount of shift in the z direction between the reference subvolume and the target subvolume based on the intersecting region between the subvolumes identified in step S19.

[0259] For example, the z-shift amount calculation unit 233 sets a cross section (cross section of interest) in the intersection region identified in step S19. The cross section of interest may be any cross section in the intersection region. Next, the z-shift amount calculation unit 233 constructs an image of the cross section of interest (reference cross section image) from the reference sub-volume, and constructs an image of the cross section of interest (target cross section image) from the target sub-volume. Next, the z-shift amount calculation unit 233 analyzes the reference cross section image to identify an image of a predetermined portion of the subject's eye E (reference image), and analyzes the target cross section image to identify an image of the same portion (target image). Next, the z-shift amount calculation unit 233 obtains the z coordinate of the reference image and the z coordinate of the target image, and calculates the difference between these two z coordinates. In this example, the calculated difference is used as the z-direction shift amount between the reference sub-volume and the target sub-volume.

[0260] A specific example is shown in FIG. 9. Reference numeral 401 denotes the intersection state between a strip 402 corresponding to the reference subvolume and a strip 403 corresponding to the target subvolume. In this example, attention is focused on the intersection region surrounded by a white circle. The diameter of the white circle along the x direction (indicated by a dotted line) is the cross section of interest. The intersection region and the cross section of interest are also similarly indicated for each of the two strips 402 and 403. The z-shift amount calculation unit 233 constructs an image of this cross section of interest (reference cross section image) 412 from the reference subvolume corresponding to the strip 402, and constructs an image of this cross section of interest (target cross section image) 413 from the target subvolume corresponding to the strip 403. Furthermore, the z-shift amount calculation unit 233 analyzes the reference cross section image 412 to identify the z position (z coordinate) 422 of the image of the retinal surface, and analyzes the target cross section image 413 to identify the z position (z coordinate) 423 of the image of the retinal surface. In addition, the z-shift amount calculation unit 233 calculates the difference Δz between two z positions (two z coordinates) 422 and 423 identified from the two cross-sectional images 412 and 413, respectively, as the z-direction shift amount between the reference subvolume and the target subvolume.

[0261] As described above, the method for calculating the amount of shift in the z direction between the reference sub-volume and the target sub-volume is not limited to this, and any method that uses the intersection area between them may be used.

[0262] (S21: Saves the z-direction shift amount) The z-direction shift amount calculated for the subvolume pair in step S20 is sent to and stored in the z-position history data generation unit 234. As with storing the xy-direction shift amount (step S10), the z-position history data generation unit 234 records the z-direction shift amount together with time information corresponding to the subvolume pair, for example.

[0263] (S22: Registration between the reference subvolume and the target subvolume) The image data correcting unit 236 applies registration in the z direction to the reference sub-volume and the target sub-volume based on the z-direction shift amount calculated in step S20.

[0264] This registration is a process of adjusting the z-direction positions of the reference sub-volume and the target sub-volume so as to cancel out the z-direction shift amount Δz in Figure 9, that is, so that the image of the retinal surface in the reference cross-sectional image 412 and the image of the retinal surface in the target cross-sectional image 413 are located at the same z position (equal z coordinate).

[0265] (S23: Construct a merged image of the reference subvolume and the target subvolume) The image data corrector 236 constructs a merged image of the reference subvolume and the target subvolume whose relative positions have been adjusted in the z direction by the registration in step S22, thereby obtaining a merged image of the reference subvolume and the target subvolume whose relative positions have been adjusted in the xy and z directions.

[0266] (S24: Have all subvolumes been processed?) A series of processes from steps S16 to S23 are executed for all sub-volumes in a predetermined order (for example, the order assigned to the first to Nth strips) (S24: No). In this example, the merged image constructed in step S23 is set as the reference strip in the next routine. By applying this series of processes to all sub-volumes in a sequential manner, merged images of all sub-volumes are obtained (S24: Yes).

[0267] (S25: Save the volume with all subvolumes merged) The merged image finally obtained by repeating steps S16 to S23 is an image in which motion artifacts in the x, y, and z directions have been corrected, and is an image that represents the entire application range of the Lissajous scan in step S1. The main control unit 211 can store this final merged image in the storage unit 212 (and / or other storage device).

[0268] The following processing is for more precise z-direction motion artifact correction. In steps S15 to S24, z-direction motion artifact correction is performed in subvolume units (strip units). In contrast, in steps S26 to S31 described below, z-direction motion artifact correction is performed in cycle units. This cycle-unit correction is performed by the data processing unit 230 (for example, the image data correction unit 236).

[0269] Note that precise z-direction motion artifact correction does not need to be performed on a cycle-by-cycle basis, but may be performed on a cycle group that constitutes a part of a subvolume basis.Furthermore, precise z-direction motion artifact correction does not need to be performed for all cycles (all cycle groups), but may be performed for only some cycles (some cycle groups).

[0270] (S26: Select cycle from merged volume) The image data corrector 236 selects one cycle (partial data corresponding to one cycle) from the volume in which all sub-volumes are merged.

[0271] The cycle selected first may be any cycle. For example, the cycles may be selected sequentially according to the scan order (chronological order) of the multiple cycles in the Lissajous scan in step S1. Alternatively, the cycle selection order may be set based on the order assigned to multiple strips (multiple sub-volumes).

[0272] (S27: Remove the cycle from the volume) Next, the image data corrector 236 extracts the partial data corresponding to the cycle selected in step S26 from the volume.

[0273] (S28: Calculate the z-direction shift between the volume after cycle removal and the cycle) Next, the image data correcting unit 236 calculates the amount of shift in the z direction between the partial data corresponding to the cycle extracted in step S27 and the volume from which the cycle was extracted.

[0274] The method for calculating the z-direction shift amount in this step may be the same as the method in step S20. For example, the image data correction unit 236 analyzes the partial data (cross-sectional image of interest) of the cycle removed from the volume in step S27 to identify an image (first image) of a predetermined portion of the subject's eye E, and analyzes the volume from which the cross-sectional image of interest has been extracted to identify an image (second image) of the same portion. The analysis for identifying the second image does not need to be applied to the entire volume from which the cross-sectional image of interest has been extracted; for example, it may be applied only to a portion adjacent to the cross-sectional image of interest (adjacent cycle). Furthermore, the image data correction unit 236 calculates the z coordinate of the first image and the z coordinate of the second image, and calculates the difference between these two z coordinates. In this example, the calculated difference is used as the z-direction shift amount between the cycle and the volume after the cycle has been removed.

[0275] (S29: Update z-direction shift amount) Next, the z-position history data generator 234 replaces the z-direction shift amount calculated in step S20 with the new z-direction shift amount calculated in step S28. Here, the z-direction shift amount of the subvolume including the partial data corresponding to the cycle extracted in step S27 is updated.

[0276] (S30: Registration and Merging) Next, the image data correcting unit 236 performs registration between the partial data corresponding to the cycle removed in step S27 and the volume after the cycle removal so as to cancel the z-direction shift amount calculated in step S28. Furthermore, the image data correcting unit 236 constructs a merged image of the partial data whose relative position in the z direction has been adjusted by this registration and the volume after the cycle removal. This results in a volume whose relative position has been adjusted in the x and y directions and whose relative position has been precisely adjusted in the z direction.

[0277] (S31: Have you processed all the cycles?) A series of processes from steps S26 to S30 is executed for all cycles in a predetermined order (S31: No). By applying this series of processes to all cycles in a sequential manner, a volume in which the relative positions in the z direction between all cycles have been adjusted is obtained (S31: Yes). The main controller 211 can store this final volume in the memory 212 (and / or another memory device).

[0278] (S32: Generate visualization data) Next, the visualization data generation unit 235 generates visualization data based on the volume constructed in step S30. Note that in an aspect that does not include steps S26 to S31, the visualization data can be generated based on the volume saved in step S25. Alternatively, the visualization data may be generated based on the volume saved in step S14.

[0279] Specific examples of the visualization data generated in this step are shown in Figures 10A to 10D, but the visualization data generated in this step is not limited to these examples.

[0280] FIG. 10A shows a time profile of the shift of the subject's eye E that occurred during the period when the Lissajous scan of step S1 was being performed. The top graph shows the change in the shift amount in the horizontal direction (x direction) over time. The middle graph shows the change in the shift amount in the vertical direction (y direction) over time. The bottom graph shows the change in the shift amount in the axial direction (z direction) over time. In each graph, the shift amount at the start of measurement is used as the reference (shift amount zero). Each graph in FIG. 10A is an example of the first graph data described above.

[0281] FIG. 10B shows the trajectory of shifts (movements) in the x and y directions of the subject's eye E that occurred during the period when the Lissajous scan in step S1 was being performed. This graph is generated, for example, by combining the graph data in the upper row and the graph data in the middle row of FIG. 10A. The horizontal axis indicates the horizontal position (x coordinate), and the vertical axis indicates the vertical position (y coordinate). In this example, time information is expressed by the color of each point on the graph. A color code indicating the correspondence between time and color is attached to the graph. Note that the method of expressing time information is not limited to using color, and may be, for example, a pattern, thickness, shading, etc. The graph in FIG. 10B is an example of the second graph data described above.

[0282] FIG. 10C shows a three-dimensional trajectory of the shift (movement) of the subject's eye E that occurred during the period when the Lissajous scan of step S1 was being performed. This graph is generated, for example, by combining the three graph data of FIG. 10A. In this example, time information is also expressed by the color of each point on the graph. The expression of the time information may be similar to that of the graph data of FIG. 10B. The graph of FIG. 10C is an example of the third graph data described above.

[0283] 10D shows a time profile of the shift speed of the subject's eye E that occurs during the period when the Lissajous scan in step S1 is being performed. This graph is an example of the aforementioned fourth graph data.

[0284] (S33: Save the visualization data) The main control unit 211 stores the visualization data generated in step S32 in the storage unit 212 (and / or another storage device).

[0285] (S34: Generate evaluation data) Next, the evaluation data generation unit 237 generates evaluation data based on the volume constructed in step S30 and / or the visualization data generated in step S32. Note that instead of the volume constructed in step S30, the volume saved in step S25 or the volume saved in step S14 may be used.

[0286] For example, when blink detection is performed while performing the Lissajous scan in step S1, the evaluation data generation unit 237 can estimate the success or failure or the probability of success (reliability) of the motion artifact correction performed in this example, that is, the success or failure or the probability of success (reliability) of the imaging using the Lissajous scan, by analyzing the blink data acquired by the blink detection, the three graph data in Figure 10A, and the graph data in Figure 10D.

[0287] Furthermore, when blink detection is performed while the Lissajous scan of step S1 is being performed, the evaluation data generation unit 237 can integrate the blink data acquired by the blink detection to calculate the total blink time during the Lissajous scan, and apply frequency analysis to the three graph data in Fig. 10A to obtain values ​​of predetermined parameters (amplitude, frequency, etc.). Furthermore, the evaluation data generation unit 237 can estimate the success or failure or success probability (reliability) of imaging by the Lissajous scan by inputting feature amounts such as the total blink time, amplitude, and frequency into a support vector machine.

[0288] In addition, the evaluation data generation unit 237 can estimate the success or failure or success probability (reliability) of imaging using a Lissajous scan by inputting the graph data of Figure 10C into a neural network (e.g., a convolutional neural network) that has been subjected to machine learning using training data labeled with the success or failure of imaging.

[0289] (S35: Save evaluation data) The main control unit 211 stores the evaluation data generated in step S34 in the storage unit 212 (and / or another storage device).

[0290] (S36: Display information) The main controller 211 causes the display unit 241 to display the information acquired in this operation example (END). The displayed information may include, for example, an image created from the volume saved in step S14, an image created from the volume saved in step S25, an image created from the volume constructed in step S30, information (such as a graph) based on the visualization data generated in step S32, information (such as success or failure of imaging, success probability, reliability, and confidence level) based on the evaluation data generated in step S34, an image acquired by the fundus camera unit 2, and the like.

[0291] In this way, the ophthalmic examination apparatus 1 acquires a data set by applying an optical scan (e.g., a Lissajous scan) to the subject's eye E according to a two-dimensional pattern including a series of cycles that intersect with each other. This data set may include data collected by the optical scan and / or any data generated from this collected data. Furthermore, the ophthalmic examination apparatus 1 can generate position history data that represents changes in the position of the subject's eye E over time based at least on the acquired data set.

[0292] Conventional 3D OCT scans (raster scans) cannot measure eye movement. Conventional methods for measuring eye movement include the use of Purkinje images, the search coil method, and fundus images obtained with a fundus camera or SLO, but these methods suffer from low accuracy.

[0293] In addition, eye movement evaluation devices (eye trackers), which have been attracting attention in recent years, use stereo cameras to photograph the subject's eyes and evaluate eye movement based on the pupil position in the resulting images, but they also have the problem of low accuracy.

[0294] In contrast, the ophthalmic examination apparatus 1 makes it possible to measure and evaluate eye movement with extremely high precision (on the order of microns) by using optical scanning (for example, Lissajous scanning in OCT) that follows a two-dimensional pattern including a series of cycles that intersect with each other. Furthermore, in addition to measuring and evaluating eye movement, the ophthalmic examination apparatus 1 can also provide images in which motion artifacts have been corrected.

[0295] Some example features of the ophthalmic examination apparatus 1 are described below.

[0296] The ophthalmic examination apparatus 1 can generate evaluation data based at least on the position history data. The evaluation data may include reliability evaluation data of a predetermined process. For example, the ophthalmic examination apparatus 1 can construct multiple image data (collection data, volumes, strips, etc.) based on a data set acquired by optical scanning, perform relative position correction (motion artifact correction) on the multiple image data, and generate reliability evaluation data of the relative position correction.

[0297] The ophthalmic examination apparatus 1 may be equipped with an inference model. The inference model is trained by machine learning using training data including at least one of position history data representing time-series changes in eye position and processed data (e.g., frequency analysis data) generated by processing the position history data. The inference model receives as input at least one of the position history data and the processed data, and outputs a reliability. The ophthalmic examination apparatus 1 inputs at least one of the position history data of the subject's eye E and the processed data to such an inference model. The inference model to which the data is input outputs a reliability. In this case, the reliability evaluation data of the relative position correction may include the reliability output from the inference model.

[0298] The training data used for machine learning of the inference model may further include data on predetermined items related to the eyes (e.g., blink data). In this case, the inference model receives as input at least one of position history data and its processed data, and at least one of data on the predetermined items and processed data generated by processing the data (e.g., integrated data of blink data), and outputs a reliability. The ophthalmic examination apparatus 1 inputs at least one of data on predetermined items related to the subject's eye E and its processed data, and at least one of position history data of the subject's eye E and its processed data, to the inference model. The inference model to which the data has been input outputs a reliability. The reliability evaluation data of the relative position correction in this example may include the reliability output from the inference model.

[0299] The ophthalmic examination apparatus 1 may be configured to generate evaluation data based on data of predetermined items related to the subject's eye E and position history data, without using machine learning.

[0300] The ophthalmic examination apparatus 1 may be configured to generate evaluation data based on time-series data and position history data acquired from the subject. The time-series data may include data representing changes in the position of the subject's head over time.

[0301] The ophthalmic examination apparatus 1 may be configured to correct the position history data based on time-series data acquired from the subject. The time-series data may include data representing changes in the position of the subject's head over time.

[0302] The ophthalmic examination apparatus 1 may be configured to generate visualized data of time-series changes in the position of the subject's eye E. The visualized data is data that can be used for visualization such as display and printing.

[0303] The visualization data may include graph data representing a time series change in the position of the subject's eye E. The graph data may include graph data (first graph data) representing a time series change in a predetermined directional component (e.g., x-direction component, y-direction component, z-direction component) of the position of the subject's eye E defined in a predetermined coordinate system. The graph data may also include graph data (second graph data) representing a time series change in a two-dimensional position of the subject's eye E defined in a predetermined two-dimensional coordinate system (e.g., xy coordinate system, yz coordinate system, zx coordinate system). The graph data may also include graph data (third graph data) representing a time series change in a three-dimensional position of the subject's eye E defined in a predetermined three-dimensional coordinate system (e.g., xyz coordinate system). The graph data may also include graph data (fourth graph data) representing a time series change in the velocity of the subject's eye E.

[0304] The ophthalmic examination apparatus 1 may be configured to construct, based on a data set acquired by optical scanning, first image data corresponding to a first cycle group consisting of a plurality of consecutive cycles among the series of cycles, and second image data corresponding to a second cycle group consisting of another plurality of consecutive cycles. The first image data and the second image data may be, for example, sub-volumes used in constructing (projecting) a strip, or data used in constructing the sub-volumes. Furthermore, the ophthalmic examination apparatus 1 may be configured to determine a displacement of the subject's eye E between a first time point corresponding to the first cycle group and a second time point corresponding to the second cycle group based on an intersection area between the first image data and the second image data, and to generate position history data based on the displacement.

[0305] Each of the first image data and the second image data may be three-dimensional image data (subvolume) defined in a predetermined three-dimensional coordinate system (e.g., an xyz coordinate system). In this case, the ophthalmic examination apparatus 1 may be configured to construct first projection image data and second projection image data by projecting the first image data and the second image data, respectively, in a first direction (e.g., the z direction) along a first coordinate axis of the predetermined three-dimensional coordinate system, and to determine the displacement of the subject's eye E in a second direction (e.g., the xy direction) perpendicular to the first direction based on the first projection image data and the second projection image data. Here, each of the first projection image data and the second projection image data is a strip or data that is the basis of the strip.

[0306] The ophthalmic examination apparatus 1 can register the first image data and the second image data so as to cancel out displacement of the subject's eye E in a second direction (e.g., x-y direction). After this registration, the ophthalmic examination apparatus 1 can compare the first image data or image data (e.g., cross-sectional image, partial three-dimensional image, A-scan image, and data thereon) along the first direction (e.g., z-direction) based on the first image data with the second image data or image data (e.g., cross-sectional image, partial three-dimensional image, A-scan image, and data thereon) along the first direction based on the second image data to determine the displacement of the subject's eye E in the first direction. Furthermore, the ophthalmic examination apparatus 1 can generate position history data based on the displacement in the first direction and the displacement in the second direction.

[0307] Each of the first image data and the second image data may be three-dimensional image data (subvolume) defined in a predetermined three-dimensional coordinate system (e.g., an xyz coordinate system). A first coordinate axis of this three-dimensional coordinate system may define a first direction (e.g., a z direction). The ophthalmic examination apparatus 1 may be configured to compare the first image data or image data along the first direction based on the first image data with the second image data or image data along the first direction based on the second image data, thereby determining the displacement of the subject's eye E in the first direction.

[0308] The three-dimensional coordinate system may be an orthogonal coordinate system defined by a first coordinate axis, a second coordinate axis, and a third coordinate axis. The first coordinate axis may correspond to the direction (z direction) in which the probe light of the optical scan is incident on the subject's eye E, and the two-dimensional pattern may be defined by the second coordinate axis and the third coordinate axis (x axis and y axis).

[0309] The above-described exemplary embodiment provides an ophthalmic data processing device including a memory unit (memory unit 212) that stores a data set acquired by applying optical scanning according to a two-dimensional pattern including a series of cycles that intersect with each other to the test eye, and a position history data generation unit (data processing unit 230) that generates position history data representing time-series changes in the position of the test eye based on the data set.

[0310] The above-described exemplary embodiment provides a control method for an ophthalmic data processing device. The ophthalmic data processing device includes a processor (main control units 211, 230) and a storage device (storage unit 212). The method according to this aspect controls the processor to store a data set acquired by applying optical scanning according to a two-dimensional pattern including a series of mutually intersecting cycles to the subject's eye, and controls the processor to generate position history data representing a time-series change in the position of the subject's eye based on the data set.

[0311] The above-described exemplary embodiment provides an ophthalmic examination apparatus including a dataset acquisition unit (fundus camera unit 2, OCT unit 100, image data construction unit 220) that acquires a dataset by applying optical scanning according to a two-dimensional pattern including a series of cycles that intersect with each other to the subject's eye, and a position history data generation unit (data processing unit 230) that generates position history data representing changes in the position of the subject's eye over time based on this dataset.

[0312] The data set acquisition unit may include a scanning unit (fundus camera unit 2, OCT unit 100) that performs optical scanning. The scanning unit may include a deflector (optical scanner 44) that can deflect light in two mutually different directions (e.g., x direction and y direction), and may be configured to apply optical scanning to the subject's eye according to a two-dimensional pattern by repeatedly changing the deflection direction along one of the two directions in a first period while repeatedly changing the deflection direction along the other direction in a second period different from the first period.

[0313] The above-described exemplary embodiment provides a method for controlling an ophthalmic examination apparatus including a scanning unit (fundus camera unit 2, OCT unit 100) that applies optical scanning to the subject's eye, and a processor (image data construction unit 220, data processing unit 230). The method according to this aspect controls the scanning unit to apply optical scanning to the subject's eye according to a two-dimensional pattern including a series of cycles that intersect with each other to collect data, and controls the processor to generate position history data that represents time-series changes in the position of the subject's eye based on the collected data.

[0314] The scanning unit may include a deflector (optical scanner 44) capable of deflecting light in two mutually different directions (e.g., x direction and y direction). In this case, the method for controlling the ophthalmic examination apparatus can control the scanning unit to apply optical scanning according to a two-dimensional pattern to the subject's eye by repeatedly changing the deflection direction along one of the two directions in a first period while repeatedly changing the deflection direction along the other direction in a second period different from the first period.

[0315] The exemplary embodiments described above can provide a program that causes a computer to execute a method for controlling an ophthalmic examination apparatus. The exemplary embodiments described above can also provide a program that causes a computer to execute a method for controlling an ophthalmic data processing apparatus. The exemplary embodiments described above can also provide a program that causes a computer to execute an ophthalmic data processing method.

[0316] Furthermore, the exemplary embodiments described above can create a computer-readable non-transitory recording medium having such a program recorded thereon. This non-transitory recording medium may be in any form, examples of which include a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

[0317] The present disclosure merely exemplifies some aspects and is not intended to limit the invention. Those who intend to implement the present invention can make any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention. [Explanation of symbols]

[0318] 1. Ophthalmic examination equipment 44 Optical Scanner 100 OCT units 211 Main control unit 2111 Scanning control unit 212 Storage section 2121 Scanning Protocol 220 Image Data Construction Department 230 Data Processing Unit 231 Image data processing unit 232 xy position history data generation unit 233 z-shift amount calculation unit 234 z Position history data generation unit 235 Visualization Data Generation Unit 236 Image data correction unit 237 Evaluation Data Generation Unit 238 Location history data correction unit 250 Data Reception Department

Claims

1. receiving a data set acquired by applying an optical scan according to a two-dimensional pattern including a series of mutually intersecting cycles to the subject's eye, and time series data including data representing a time-series change in the position of the subject's head; generating position history data representing a time-series change in the position of the subject's eye based on the data set; generating evaluation data based on the location history data and the time series data; Ophthalmology data processing methods.

2. receiving a data set acquired by applying an optical scan according to a two-dimensional pattern including a series of mutually intersecting cycles to the subject's eye, and time-series data indicating movement of the subject's eye caused by something other than eye movement; generating position history data representing a time-series change in the position of the subject's eye based on the data set; correcting the position history data based on the time series data; Ophthalmology data processing methods.

3. Further, generating evaluation data based on the corrected position history data. The method of claim 2.

4. The evaluation data includes reliability evaluation data of a predetermined process. The method of claim 1 or 3.

5. Further, constructing a plurality of image data based on the data set; Correcting the relative positions of the plurality of image data; the reliability evaluation data includes reliability evaluation data of the relative position correction; The method of claim 4.

6. the position history data includes visualization data of the time-series change in the position of the subject's eye, The method of any one of claims 1 to 5.

7. a storage unit that stores a data set acquired by applying optical scanning according to a two-dimensional pattern including a series of mutually intersecting cycles to the subject's eye, and time-series data including data representing a time-series change in the position of the subject's head; a position history data generating unit that generates position history data representing a time-series change in the position of the subject's eye based on the data set; an evaluation data generation unit that generates evaluation data based on the position history data and the time series data; 1. An ophthalmic data processing device comprising:

8. a storage unit that stores a data set acquired by applying optical scanning according to a two-dimensional pattern including a series of mutually intersecting cycles to the subject's eye, and time-series data indicating movement of the subject's eye caused by factors other than eye movement; a position history data generating unit that generates position history data representing a time-series change in the position of the subject's eye based on the data set; a position history data correction unit that corrects the position history data based on the time series data; 1. An ophthalmic data processing device comprising:

Images