Scanning imaging device, information processing device, control method for scanning imaging device, control method for information processing device, information processing method, program, and recording medium

Abstract

To provide a new utilization method of data acquired by scanning type imaging.SOLUTION: An ophthalmologic examination device 1 is a scanning type imaging device for constructing image data using optical scanning, and collects a data set by applying optical scanning in accordance with a two-dimensional pattern including a series of cycles intersecting with each other to a sample. The ophthalmologic examination device 1 calculates a correlation coefficient among cycles on the basis of the data set. Furthermore, the ophthalmologic examination device 1 records scanning position information corresponding to a subset with a correlation coefficient that meets a predetermined condition of a set of the correlation coefficients calculated from the data set, as an element.SELECTED DRAWING: Figure 4B

Description

[Technical Field]

[0001] The present invention relates to a scanning imaging device, an information processing device, a control method for a scanning imaging device, a control method for an information processing device, an information processing method, a program, and a recording medium. [Background technology]

[0002] Scanning imaging is one type of imaging technology. Scanning imaging involves sequentially irradiating a beam at multiple locations on a sample, collecting data, and then constructing an image of the sample from the collected data. Scanning imaging techniques include, for example, spot-type scanning techniques that use a beam that projects a spot-shaped image, line-type scanning techniques that use a beam that projects a line-shaped image, and area-type scanning techniques that use a beam that projects an area-shaped image. Representative examples of these techniques include Fourier-domain optical coherence tomography (OCT), line-scan cameras, and full-field OCT, respectively.

[0003] Optical Coherence Tomography (OCT) is an example of a scanning imaging modality that uses light. OCT is a technology that can image light-scattering media with a resolution of micrometers or less, and is used in medical imaging and non-destructive testing.

[0004] OCT is a technology based on low-coherence interferometry. In medical imaging, a probe light (measurement light) in the near-infrared region is used, taking into account its depth and invasiveness. However, scanning imaging using light (electromagnetic waves) in wavelength bands other than near-infrared, ultrasound, and radiation are also known.

[0005] Ophthalmology is one of the most advanced fields in the application of OCT, with products equipped with Fourier-domain OCT functionality now widespread in clinics. Ophthalmic imaging diagnostics not only provides two-dimensional imaging, but also three-dimensional imaging, structural analysis, and functional analysis, making it a widely used, powerful diagnostic tool. In addition to Fourier-domain OCT, scanning laser ophthalmoscopes (SLOs), line scan cameras, and full-field OCT are also used in ophthalmology.

[0006] There are various scanning modes used in OCT and SLO, but the so-called "Lissajous scan" for the purpose of correcting motion artifacts has been attracting attention in recent years (see, for example, Patent Documents 1 to 4 and Non-Patent Documents 1 and 2).

[0007] A Lissajous scan is a scan performed according to a two-dimensional pattern generated as a Lissajous curve, obtained by synthesizing two mutually orthogonal simple harmonic motions. In a Lissajous scan, the measurement light is scanned at high speed to trace multiple loops (multiple cycles that intersect with each other) of a certain size, so the difference in data acquisition time from one cycle can be virtually ignored. Furthermore, because the intersection areas of different cycles can be referenced to align the cycles, it is possible to correct motion artifacts caused by sample movement. Focusing on these characteristics of the Lissajous scan, the field of ophthalmology is attempting to address motion artifacts caused by eye movement.

[0008] In the technique described in Non-Patent Document 1, a data set acquired by a Lissajous scan is divided into multiple sub-volumes that do not involve relatively large motion, and a front projection image (en face projection) of each sub-volume is constructed. Such front projection images are called strips. Images with motion artifacts corrected can be obtained by performing registration between these strips. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-17915 [Patent Document 2] Japanese Patent Application Publication No. 2018-68578 [Patent Document 3] Japanese Patent Application Laid-Open No. 2018-140004 [Patent Document 4] Japanese Patent Application Laid-Open No. 2018-140049 [Non-patent literature]

[0010] [Non-Patent Document 1] Yiwei Chen, Young-Joo Hong, Shuichi Makita, and Yoshiaki Yasuno, “Three-dimensional eye motion correction by Lissajous scan optical coherence tomography”, Biomedical Optics EXPRESS, Vol. 8, No. 3, 1 Mar 2017, PP. 1783-1802 [Non-patent document 2] Yiwei Chen, Young-Joo Hong, Shuichi Makita, and Yoshiaki Yasuno, “Eye-motion-corrected optical coherence tomography angiography using Lissajous scanning”, Biomedical Optics EXPRESS, Vol. 9, No. 3, 1 Mar 2018, PP. 1111-1129 Summary of the Invention [Problem to be solved by the invention]

[0011] One object of the present invention is to provide a novel method for utilizing data acquired by scanning imaging. [Means for solving the problem]

[0012] An exemplary aspect of the embodiment is a scanning imaging device that constructs image data using optical scanning. The scanning imaging device includes a dataset collection unit, a correlation coefficient calculation unit, and an information recording unit. The dataset collection unit is configured to collect a dataset by applying an optical scan to a sample according to a two-dimensional pattern including a series of cycles that intersect with each other. The correlation coefficient calculation unit is configured to calculate a correlation coefficient between the cycles based on the dataset collected by the dataset collection unit. The information recording unit is configured to record scanning position information corresponding to a subset of a set of correlation coefficients calculated by the correlation coefficient calculation unit from the dataset collected by the dataset collection unit, the subset having correlation coefficients that satisfy a predetermined condition as elements. [Effects of the Invention]

[0013] Exemplary aspects of the embodiment allow for novel uses of data acquired through scanning imaging. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 2] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 3] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 4A] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 4B]1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 5] 1 is a schematic diagram showing an example of a Lissajous scan pattern performed by a scanning imaging device (ophthalmic examination device) according to an exemplary aspect of the embodiment. [Figure 6A] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 6B] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 6C] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 6D] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 6E] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 6F] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 7A] 1A to 1C are diagrams for explaining an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 7B] 1A to 1C are diagrams for explaining an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 7C] 1A to 1C are diagrams for explaining an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 8] 1A to 1C are diagrams for explaining an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 9] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 10] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 11] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 12] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 13] 1A to 1C are diagrams for explaining an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 14] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 15] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 16] 1 is a schematic diagram illustrating an example of a configuration of an information processing device according to an exemplary aspect of an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Some exemplary aspects of the embodiments will be described with reference to the drawings. The exemplary aspects described below relate to a scanning imaging device, an information processing device, a control method for a scanning imaging device, a control method for an information processing device, an information processing method, a program, and a recording medium, but the embodiments are not limited to these aspects.

[0016] Furthermore, while the exemplary aspects described below provide some applications in the field of ophthalmology, embodiments are not limited to these aspects and may be applied to any field, such as applications in medical departments other than ophthalmology, applications in medical methods such as diagnostic methods, and applications in fields other than the medical field (biology, non-destructive testing, etc.).

[0017] The matters disclosed in the documents cited in this specification and any other matters related to known techniques can be combined with the exemplary embodiments. Furthermore, unless otherwise specified, no distinction is made between "image data" and an "image" based thereon, and no distinction is made between a "site" of a subject (especially the subject's eye) and an "image" thereof.

[0018] A scanning imaging apparatus according to an exemplary embodiment described below is an apparatus used in ophthalmic examinations (referred to as an ophthalmic examination apparatus), and is configured to be capable of measuring and imaging the fundus of a living eye using Fourier-domain OCT (particularly, swept-source OCT). The type of OCT that can be employed in the embodiments is not limited to swept-source OCT, and may be, for example, spectral-domain OCT or time-domain OCT. Furthermore, although the scanning imaging apparatus according to the exemplary embodiment described below uses a spot-type scanning technique, it will be understood by those skilled in the art that any aspect of the present disclosure can be applied to a scanning imaging apparatus that uses other types of scanning techniques (e.g., a line-type scanning technique or an area-type scanning technique).

[0019] A scanning imaging device according to some exemplary embodiments may be capable of using a scanning imaging modality other than OCT. For example, some exemplary embodiments may employ any optical scanning imaging modality such as SLO. The scanning imaging modality applicable to some exemplary embodiments may not be an optical scanning imaging modality, but may be, for example, a scanning imaging modality using electromagnetic waves other than light, a scanning imaging modality using ultrasound, or the like.

[0020] In addition to processing data collected by OCT scans and / or SLO scans, the ophthalmic examination apparatus according to some exemplary embodiments may be capable of processing data acquired by other imaging modalities. The other imaging modalities may be any ophthalmic imaging modality, such as a fundus camera, a slit lamp microscope, or an ophthalmic surgical microscope. The ophthalmic examination apparatus according to some exemplary embodiments may have the functionality of imaging modalities other than OCT and SLO. Furthermore, the ophthalmic examination apparatus according to some exemplary embodiments may have a stereo alignment function using two or more anterior eye cameras, as disclosed in Japanese Patent Application Laid-Open No. 2013-248376 and Japanese Patent Application Laid-Open No. 2016-49243.

[0021] The object (sample) to which scanning imaging is applied may be any object. In the ophthalmology application described in the exemplary embodiments, scanning imaging is applied to the fundus, but it may also be applied to any part of the eye, such as the anterior segment or the vitreous body. The configurations and functions of some exemplary embodiments may be applied to measurement and imaging of biological parts (tissues) other than the eye, and may also be applied to measurement and imaging of non-biological objects (e.g., objects that move) or parts thereof. That is, the industrial application fields of some exemplary embodiments are not limited to ophthalmology-related fields, but may also include medical, veterinary, and biological fields, and may more generally include fields related to any object (sample) that moves locally and / or globally. Note that the industrial application fields of some exemplary embodiments may also include fields related to objects that do not move.

[0022] Any aspects of the embodiments described below are merely illustrative and are not intended to limit the invention.

[0023] A first aspect of the embodiment is a scanning imaging device that constructs image data using optical scanning, and includes a dataset collection unit that applies a first optical scan to a sample according to a two-dimensional pattern including a series of cycles that intersect with each other to collect a first dataset, a correlation coefficient calculation unit that calculates a correlation coefficient between the cycles based on the first dataset, and an information recording unit that records scanning position information corresponding to a subset of the set of correlation coefficients calculated from the first dataset by the correlation coefficient calculation unit, the subset having correlation coefficients that satisfy a predetermined condition as elements.

[0024] A second aspect of the embodiment is the scanning imaging apparatus of the first aspect, wherein the elements of the subset have a correlation coefficient below a predetermined threshold.

[0025] A third aspect of the embodiment is a scanning imaging device of the second aspect, further including a first pixel value determination unit that determines the value of a pixel corresponding to the scanning position information recorded by the information recording unit based on the values ​​of one or more neighboring pixels.

[0026] A fourth aspect of the embodiment is a scanning imaging device of the second aspect, further including a map creation unit that creates a correlation coefficient map in the application area of ​​the first optical scanning based on the set of correlation coefficients acquired by the correlation coefficient calculation unit.

[0027] A fifth aspect of the embodiment is a scanning imaging device of the fourth aspect, further including a second pixel value determination unit that determines the value of a pixel corresponding to the scanning position information recorded by the information recording unit based on the correlation coefficient map.

[0028] A sixth aspect of the embodiment is the scanning imaging device of the second aspect, wherein the dataset collection unit applies a second optical scan to the sample based on the scanning position information recorded by the information recording unit.

[0029] A seventh aspect of the embodiment is a scanning imaging device of the sixth aspect, wherein the dataset collection unit applies the second optical scan to an area set based on the scanning position information recorded by the information recording unit.

[0030] An eighth aspect of the embodiment is a scanning imaging device of the seventh aspect, further including a processing unit that processes the first data set collected by the first optical scan and the second data set collected by the second optical scan to construct image data.

[0031] A ninth aspect of the embodiment is a scanning type imaging device of the eighth aspect, wherein the processing unit processes the first data set to construct first image data of an area obtained by excluding at least a portion of the area corresponding to the scanning position information recorded by the information recording unit from the application area of ​​the first optical scanning, processes the second data set to construct second image data of the area set based on the scanning position information, and combines the first image data and the second image data.

[0032] A tenth aspect of the embodiment is a scanning imaging device according to any one of the first to ninth aspects, wherein the data set collection unit includes a deflector capable of deflecting light for optical scanning in two directions different from each other, and applies the first optical scan to the sample by repeatedly changing the deflection direction along one of the two directions in a first period while repeatedly changing the deflection direction along the other direction in a second period different from the first period.

[0033] An eleventh aspect of the embodiment is an information processing device that includes a receiving unit that receives a data set collected by applying an optical scan to a sample according to a two-dimensional pattern including a series of cycles that intersect with each other, a correlation coefficient calculation unit that calculates a correlation coefficient between the cycles based on the data set, and an information recording unit that records scanning position information corresponding to a subset of the set of correlation coefficients calculated from the data set by the correlation coefficient calculation unit, the subset having correlation coefficients that satisfy predetermined conditions as elements.

[0034] A twelfth aspect of the embodiment is a method for controlling a scanning imaging device including a scanner that performs optical scanning and a processor that constructs image data from data collected by the optical scanning, the method comprising: controlling the scanner to apply an optical scan to a sample according to a two-dimensional pattern including a series of cycles that intersect with each other to collect a data set; controlling the processor to calculate a correlation coefficient between cycles based on the data set; and controlling the processor to record scanning position information corresponding to a subset of a set of correlation coefficients calculated from the data set, the subset having correlation coefficients that satisfy a predetermined condition as elements.

[0035] A thirteenth aspect of the embodiment is a method for controlling an information processing device including a processor, the method controlling the processor to accept a data set collected by applying an optical scan to a sample according to a two-dimensional pattern including a series of cycles that intersect with each other, controlling the processor to calculate a correlation coefficient between cycles based on the data set, and controlling the processor to record scanning position information corresponding to a subset of a set of correlation coefficients calculated from the data set, the subset having correlation coefficients that satisfy a predetermined condition as elements.

[0036] A fourteenth aspect of the embodiment is an information processing method, which acquires a data set collected from a sample by optical scanning according to a two-dimensional pattern including a series of cycles that intersect with each other, calculates a correlation coefficient between the cycles based on the data set, and records scanning position information corresponding to a subset of the set of correlation coefficients calculated from the data set, the subset having correlation coefficients that satisfy a predetermined condition as elements.

[0037] A fifteenth aspect of the embodiment is a program for causing a computer to execute the method according to any one of the twelfth to fourteenth aspects.

[0038] A sixteenth aspect of the embodiment is a computer-readable non-transitory recording medium having the program of the fifteenth aspect recorded thereon.

[0039] The aspects of the embodiment are not limited to the above-described aspects 1 to 16. For example, features that can be combined with aspect 1 (for example, features related to aspects 2 to 10, any feature in the embodiments described below, any known technology, etc.) can be combined with aspects 11 to 16.

[0040] First Embodiment A first embodiment will be described.

[0041] <Configuration of ophthalmic examination device> The ophthalmic examination apparatus 1 shown in FIG. 1 is an exemplary embodiment of a scanning imaging apparatus according to the present embodiment. The ophthalmic examination apparatus 1 includes a fundus camera unit 2, an OCT unit 100, and an arithmetic and control unit 200. The fundus camera unit 2 includes a group of elements (optical elements, mechanisms, etc.) for capturing an image of the subject's eye E from the front. The OCT unit 100 includes some of the group of elements (optical elements, mechanisms, etc.) for performing an OCT scan on the subject's eye E. Another group of elements for the OCT scan is provided in the fundus camera unit 2. The arithmetic and control unit 200 includes one or more processors that perform various calculations and controls. In addition to these, the ophthalmic examination apparatus 1 may also include elements for supporting the subject's face and elements for switching the region to which the OCT scan is applied. Examples of the former elements include a chin rest and a forehead rest. An example of the latter element is a lens unit used to switch the region to which the OCT scan is applied from the fundus to the anterior segment.

[0042] The functionality of some elements disclosed herein is implemented using circuitry or processing circuitry. Circuitry or processing circuitry includes any of a general-purpose processor, a special-purpose processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA), conventional circuitry, and any combination thereof, configured and / or configured to perform the disclosed functions. A processor is considered to be processing circuitry or circuitry that includes transistors and / or other circuitry. In this disclosure, circuitry, unit, means, or similar terms is hardware that performs the disclosed functions or hardware that is programmed to perform the disclosed functions. The hardware may be hardware disclosed herein or may be known hardware that is programmed and / or configured to perform the described functions. In the case of a processor, where the hardware can be considered to be a type of circuitry, the circuitry, unit, means, or similar terms is a combination of hardware and software, and the software is used to configure the hardware and / or processor.

[0043] <Fundus camera unit 2> The fundus camera unit 2 is provided with an optical system for photographing the fundus Ef of the subject's eye E. The acquired image of the fundus Ef (called a fundus image, fundus photograph, etc.) is a front image such as an observed image or a photographed image. The observed image is obtained by, for example, video shooting using near-infrared light, and is used for alignment, focusing, tracking, etc. The photographed image is a still image obtained using, for example, flash light in the visible or infrared range.

[0044] The fundus camera unit 2 includes an illumination optical system 10 and an imaging optical system 30. The illumination optical system 10 irradiates illumination light onto the subject's eye E. The imaging optical system 30 detects return light of the illumination light from the subject's eye E. The measurement light from the OCT unit 100 is guided to the subject's eye E through an optical path within the fundus camera unit 2, and the return light is guided to the OCT unit 100 through the same optical path.

[0045] Light (observation illumination light) output from an observation light source 11 of an illumination optical system 10 is reflected by a concave mirror 12, passes through a condenser lens 13, and is transmitted through a visible light cut filter 14 to become near-infrared light. The observation illumination light is then focused near an imaging light source 15, reflected by a mirror 16, and passes through a relay lens system 17, a relay lens 18, an aperture 19, and a relay lens system 20. The observation illumination light is then reflected by the peripheral portion (the area surrounding the hole) of a perforated mirror 21, passes through a dichroic mirror 46, and is refracted by an objective lens 22 to illuminate the subject's eye E (fundus oculi Ef). Return light of the observation illumination light from the subject's eye E is refracted by the objective lens 22, passes through the dichroic mirror 46, passes through a hole formed in the central region of the perforated mirror 21, passes through a dichroic mirror 55, passes through an imaging focusing lens 31, and is reflected by a mirror 32. Furthermore, this returned light passes through the half mirror 33A, is reflected by the dichroic mirror 33, and is imaged on the light receiving surface of the image sensor 35 by the imaging lens 34. The image sensor 35 detects the returned light at a predetermined frame rate. The focus (focal position) of the photographing optical system 30 is adjusted to match the fundus Ef or the anterior segment of the eye.

[0046] Light (photography illumination light) output from the photography light source 15 is irradiated onto the fundus oculi Ef through the same path as the observation illumination light. Return light of the photography illumination light from the subject's eye E is guided to the dichroic mirror 33 through the same path as the return light of the observation illumination light, passes through the dichroic mirror 33, is reflected by a mirror 36, and is imaged by an imaging lens 37 on the light-receiving surface of an image sensor 38.

[0047] The liquid crystal display (LCD) 39 displays a fixation target (fixation target image). A portion of the light beam output from the LCD 39 is reflected by the half mirror 33A, reflected by the mirror 32, passes through the photographing focusing lens 31 and the dichroic mirror 55, and passes through the hole in the aperture mirror 21. The light beam that passes through the hole in the aperture mirror 21 passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef. By changing the display position of the fixation target image on the LCD 39, the direction in which the gaze of the subject's eye E is guided (fixation direction, fixation position) can be changed. Instead of a display device such as an LCD, for example, a light-emitting element array, or a combination of a light-emitting element and a mechanism for moving it, may be used.

[0048] The alignment optical system 50 generates an alignment index used to align the optical system with the subject's eye E. Alignment light output from a light-emitting diode (LED) 51 passes through an aperture 52, an aperture 53, and a relay lens 54, is reflected by a dichroic mirror 55, passes through the hole in the aperture mirror 21, transmits through the dichroic mirror 46, and is projected onto the subject's eye E via the objective lens 22. Return light of the alignment light from the subject's eye E (such as corneal reflection light) is guided to the image sensor 35 via the same path as the return light of the observation illumination light. Manual alignment or automatic alignment can be performed based on the received light image (alignment index image).

[0049] As in the conventional example, the alignment target image in this example consists of two bright spot images whose positions change depending on the alignment state. When the relative position between the subject's eye E and the optical system changes in the x and y directions (directions expressed by the x and y coordinates), the two bright spot images displace together in the x and y directions. When the relative position between the subject's eye E and the optical system changes in the z direction (direction expressed by the z coordinate), the relative position (distance) between the two bright spot images changes. When the distance between the subject's eye E and the optical system in the z direction matches the predetermined working distance, the two bright spot images overlap. When the position of the subject's eye E matches the position of the optical system in the x and y directions, the two bright spot images are presented within or near a specified alignment target. When the distance between the subject's eye E and the optical system in the z direction matches the working distance and the position of the subject's eye E matches the position of the optical system in the x and y directions, the two bright spot images overlap and are presented within the alignment target.

[0050] In auto-alignment, the data processing unit 230 detects the positions of the two bright spot images, and the main control unit 211 controls the moving mechanism 150 (described later) based on the positional relationship between the two bright spot images and the alignment target. In manual alignment, the main control unit 211 displays the two bright spot images together with the observed image of the subject's eye E on the display unit 241, and the user operates the moving mechanism 150 using the operation unit 242 while referring to the two displayed bright spot images.

[0051] The focusing optical system 60 generates a split index used for focus adjustment of the subject's eye E. The focusing optical system 60 moves along the optical path (illumination optical path) of the illumination optical system 10 in conjunction with movement of the photographing focusing lens 31 along the optical path (photographing optical path) of the photographing optical system 30. The reflecting rod 67 is inserted into and removed from the illumination optical path. When performing focus adjustment, the reflecting surface of the reflecting rod 67 is tilted and positioned in the illumination optical path. Focusing light output from the LED 61 passes through the relay lens 62, is split into two beams by the split index plate 63, passes through the two-hole diaphragm 64, is reflected by the mirror 65, and is first imaged and reflected on the reflecting surface of the reflecting rod 67 by the condenser lens 66. The focusing light then passes through the relay lens 20, is reflected by the aperture mirror 21, passes through the dichroic mirror 46, and is projected onto the subject's eye E via the objective lens 22. The return light (fundus reflected light, etc.) of the focusing light from the subject's eye E passes through the same path as the return light of the alignment light and is guided to the image sensor 35. Manual focusing or autofocusing can be performed based on the received light image (split target image).

[0052] Diopter correction lenses 70 and 71 can be selectively inserted into the photographing optical path between the aperture mirror 21 and the dichroic mirror 55. The diopter correction lens 70 is a plus lens (convex lens) for correcting severe hyperopia. The diopter correction lens 71 is a minus lens (concave lens) for correcting severe myopia.

[0053] The dichroic mirror 46 combines the optical path for fundus imaging and the optical path for OCT (measurement arm). The dichroic mirror 46 reflects light in the wavelength band used for OCT and transmits light for fundus imaging. The measurement arm is provided with, in order from the OCT unit 100 side, a collimator lens unit 40, a retroreflector 41, a dispersion compensation member 42, an OCT focusing lens 43, an optical scanner 44, and a relay lens 45.

[0054] The retroreflector 41 is movable in the direction of the arrow shown in Figure 1, thereby changing the length of the measurement arm. Changing the length of the measurement arm is used, for example, to correct the optical path length according to the axial length of the eye, adjust the interference state, and so on.

[0055] The dispersion compensation member 42, together with a dispersion compensation member 113 (described later) disposed in the reference arm, acts to match the dispersion characteristics of the measurement light LS with the dispersion characteristics of the reference light LR.

[0056] The OCT focusing lens 43 is moved along the measurement arm to adjust the focus of the measurement arm. The movement of the imaging focusing lens 31, the movement of the focus optical system 60, and the movement of the OCT focusing lens 43 can be controlled in a coordinated manner.

[0057] The optical scanner 44 is disposed at a position that is substantially optically conjugate with the pupil of the subject's eye E. The optical scanner 44 deflects the measurement light LS guided by the measurement arm. The optical scanner 44 is, for example, a galvanometer scanner capable of two-dimensional scanning, including a galvanometer mirror for scanning in the x direction and a galvanometer mirror for scanning in the y direction.

[0058] <OCTユニット100> As shown in FIG. 2, the OCT unit 100 is provided with an optical system for applying swept-source OCT. This optical system includes an interference optical system. This interference optical system splits light from a wavelength-tunable light source (swept-wavelength light source) into measurement light and reference light, and generates interference light by superimposing the return light of the measurement light guided to the subject's eye E by the measurement arm on the reference light guided by the reference arm, and detects this interference light. Data (detection signal) obtained by the interference optical system is a signal representing the spectrum of the interference light, and is sent to the arithmetic and control unit 200.

[0059] The light source unit 101 includes, for example, a near-infrared tunable laser that changes the output wavelength at high speed at least in the near-infrared wavelength band. Light L0 output from the light source unit 101 is guided by an optical fiber 102 to a polarization controller 103, where its polarization state is adjusted. The light L0 is further guided by an optical fiber 104 to a fiber coupler 105, where it is split into measurement light LS and reference light LR. The optical path of the measurement light LS is called a measurement arm, and the optical path of the reference light LR is called a reference arm.

[0060] The reference light LR is guided by an optical fiber 110 to a collimator 111, where it is converted into a parallel beam, and then guided to a retroreflector 114 via an optical path length correction element 112 and a dispersion compensation element 113. The optical path length correction element 112 acts to match the optical path length of the reference light LR with that of the measurement light LS. The dispersion compensation element 113, together with a dispersion compensation element 42 arranged in the measurement arm, acts to match the dispersion characteristics between the reference light LR and the measurement light LS. The retroreflector 114 is movable along the optical path of the reference light LR incident thereon, thereby changing the length of the reference arm. Changing the reference arm length is used, for example, to correct the optical path length according to the axial length of the eye, adjust the interference state, and so on.

[0061] The reference light LR that has passed through the retroreflector 114 passes through the dispersion compensation member 113 and the optical path length correction member 112, is converted from a parallel beam into a convergent beam by the collimator 116, and enters an optical fiber 117. The reference light LR that has entered the optical fiber 117 is guided to a polarization controller 118 where its polarization state is adjusted, is guided through an optical fiber 119 to an attenuator 120 where its light amount is adjusted, and is guided through an optical fiber 121 to a fiber coupler 122.

[0062] On the other hand, the measurement light LS generated by the fiber coupler 105 is guided by the optical fiber 127 and converted into a parallel beam by the collimator lens unit 40, passes through the retroreflector 41, the dispersion compensation member 42, the OCT focusing lens 43, the optical scanner 44, and the relay lens 45, is reflected by the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the subject's eye E. The measurement light LS is scattered and reflected at various depth positions in the subject's eye E. The returning light of the measurement light LS from the subject's eye E travels the same path as the outward path in the opposite direction and is guided to the fiber coupler 105, and reaches the fiber coupler 122 via the optical fiber 128.

[0063] The fiber coupler 122 generates interference light by superimposing the measurement light LS incident via the optical fiber 128 and the reference light LR incident via the optical fiber 121. The fiber coupler 122 splits the generated interference light at a predetermined splitting ratio (for example, 1:1) to generate a pair of interference lights LC. The pair of interference lights LC are guided to the detector 125 via optical fibers 123 and 124, respectively.

[0064] The detector 125 includes, for example, a balanced photodiode. The balanced photodiode has a pair of photodetectors that respectively detect a pair of interference light LC, and outputs the difference between the pair of detection results obtained by these. The detector 125 sends this output (detection signal) to a data acquisition system (DAS) 130.

[0065] A clock KC is supplied to the data collection system 130 from the light source unit 101. The clock KC is generated in the light source unit 101 in synchronization with the output timing of each wavelength swept within a predetermined wavelength range by the wavelength-tunable light source. The light source unit 101, for example, branches the light L0 of each output wavelength to generate two branched lights, optically delays one of the branched lights, combines the branched lights, detects the resulting combined light, and generates the clock KC based on the detection result. The data collection system 130 samples the detection signal input from the detector 125 based on the clock KC. The data collection system 130 sends the sampling result to the arithmetic and control unit 200.

[0066] In this example, both an element for changing the measurement arm length (e.g., retroreflector 41) and an element for changing the reference arm length (e.g., retroreflector 114 or reference mirror) are provided, but only one of these elements may be provided. Furthermore, the element for changing the difference between the measurement arm length and the reference arm length (optical path length difference) is not limited to these and may be any element (optical member, mechanism, etc.).

[0067] <Control system / processing system> 3, 4A, and 4B show configuration examples of the control system and processing system of the ophthalmic examination apparatus 1. The control unit 210, image data creation unit 220, and data processing unit 230 are provided in, for example, the arithmetic control unit 200. The ophthalmic examination apparatus 1 may include a communication device for performing data communication with an external device. The ophthalmic examination apparatus 1 may also include a drive device (reader / writer) for reading data from a recording medium and writing data to the recording medium.

[0068] <Control unit 210> The control unit 210 executes various types of control. The control unit 210 includes a main control unit 211 and a storage unit 212. In this embodiment, the main control unit 211 includes a scanning control unit 2111, and the storage unit 212 stores a scanning protocol 2121, as shown in FIG. 4A.

[0069] <Main control unit 211> The main control unit 211 includes a processor and controls each element (including the elements shown in FIGS. 1 to 4B) of the ophthalmic examination apparatus 1. The main control unit 211 is realized by cooperation between hardware including the processor and control software. The scan control unit 2111 controls OCT scanning of a scan area of ​​a predetermined shape and size.

[0070] The imaging focusing driver 31A moves the imaging focusing lens 31 arranged in the imaging optical path and the focus optical system 60 arranged in the illumination optical path under the control of the main controller 211. The retroreflector (RR) driver 41A moves the retroreflector 41 provided in the measurement arm under the control of the main controller 211. The OCT focusing driver 43A moves the OCT focusing lens 43 arranged in the measurement arm under the control of the main controller 211. The retroreflector (RR) driver 114A moves the retroreflector 114 arranged in the reference arm under the control of the main controller 211. Each driver includes an actuator such as a pulse motor that operates under the control of the main controller 211. The optical scanner 44 operates under the control of the main controller 211 (scanning controller 2111).

[0071] The movement mechanism 150 is typically configured to move the fundus camera unit 2 three-dimensionally. Such a movement mechanism 150 includes, for example, an x-stage configured to be movable in ±x directions (left and right directions), an x-movement mechanism configured to move the x-stage, a y-stage configured to be movable in ±y directions (up and down directions), a y-movement mechanism configured to move the y-stage, a z-stage configured to be movable in ±z directions (depth direction), and a z-movement mechanism configured to move the z-stage. Each of the x-movement mechanism, y-movement mechanism, and z-movement mechanism includes an actuator such as a pulse motor that operates under the control of the main controller 211.

[0072] <Storage section 212> The storage unit 212 stores various types of data. The data stored in the storage unit 212 includes OCT images, fundus images, subject eye information, and control information. The subject eye information includes subject information such as patient ID and name, left / right eye identification information, and electronic medical record information. The control information is information related to specific controls. The control information in this embodiment includes a scanning protocol 2121.

[0073] The scanning protocol 2121 is an agreement regarding the control content for an OCT scan of a scanning area of ​​a predetermined shape and a predetermined size, and includes a set of various control parameters (scanning control parameters). The scanning protocol 2121 includes a protocol for each scanning mode. The scanning protocol 2121 of this embodiment includes at least a protocol for Lissajous scanning, and may further include protocols for, for example, B-scan (line scanning), cross scanning, radial scanning, and raster scanning.

[0074] The scanning control parameters of this embodiment include at least a parameter indicating the content of control over the optical scanner 44. These parameters include, for example, a parameter indicating the scan pattern, a parameter indicating the scan speed, and a parameter indicating the scan interval. The scan pattern indicates the shape of the scan path, and examples thereof include a Lissajous pattern, a line pattern, a cross pattern, a radial pattern, and a raster pattern. The scan speed is defined, for example, as the repetition rate of A-scans. The scan interval is defined, for example, as the interval between adjacent A-scans, that is, the array interval of scan points.

[0075] As with the conventional technologies disclosed in Patent Documents 1 to 4 and Non-Patent Documents 1 and 2, the "Lissajous scan" of this embodiment may be not only a "narrowly defined" Lissajous scan whose path is a pattern (Lissajous pattern, Lissajous figure, Lissajous curve, Lissajous function, Bowditch curve) drawn by the locus of points obtained by an ordered pair of two mutually perpendicular simple harmonic motions, but also a "broadly defined" Lissajous scan that follows a predetermined two-dimensional pattern including a series of cycles.

[0076] The optical scanner 44 of this embodiment includes, for example, a first galvanometer mirror that deflects the measurement light LS in the x direction and a second galvanometer mirror that deflects the measurement light LS in the y direction. Lissajous scanning is achieved by controlling the first galvanometer mirror so that the deflection direction along the x direction repeatedly changes in a first period, while controlling the second galvanometer mirror so that the deflection direction along the y direction repeatedly changes in a second period. Here, the first period and the second period are different from each other.

[0077] For example, the Lissajous scan of this embodiment may not only be a scan of a narrowly defined Lissajous pattern obtained by combining two sine waves, but also a scan of a pattern obtained by adding a specific term (e.g., an odd-order polynomial) to a sine wave, or a scan of a pattern based on a triangular wave.

[0078] A "cycle" generally refers to an object consisting of a plurality of sampling points having a certain length. The cycle in this embodiment may be, for example, a closed curve or an almost closed curve (a substantially closed curve, an approximately closed curve). In other words, the start point and end point of the cycle in this embodiment may coincide or nearly coincide.

[0079] Typically, the scanning protocol 2121 is set based on the Lissajous function. As shown in equation (9) of Non-Patent Document 1, the Lissajous function is expressed by the following parametric equation system: x(t i )=A·cos(2π·(f A / n)·t i ), y(t i )=A·cos(2π·(f A (n-2) / n 2 )·t i ).

[0080] Here, "x" is the horizontal axis of the two-dimensional coordinate system in which the Lissajous curve is defined, "y" is the vertical axis, and "t i ” is the acquisition time of the i-th A-line in the Lissajous scan, “A” is the scan range (amplitude), and “f A" indicates the A-line acquisition rate (scan speed, A-scan repetition rate), and n indicates the number of A-lines in each cycle in the x-direction (horizontal axis).

[0081] An example of the distribution of scan lines (scan pattern) in such an exemplary Lissajous scan is shown in FIG.

[0082] Any pair of cycles included in a Lissajous scan (in the narrow or broad sense) intersects with each other at at least one point (particularly, two or more points). Utilizing such intersections makes it possible to perform registration between pairs of data collected from any pair of cycles in a Lissajous scan, and makes it possible to implement the image construction method and motion artifact correction method disclosed in Non-Patent Document 1 (or Non-Patent Document 2). Hereinafter, unless otherwise specified, a case where the method described in Non-Patent Document 1 is applied will be described. However, it is also possible to apply the method described in Non-Patent Document 2, or a method equivalent and / or similar to the method described in Non-Patent Document 1 or 2.

[0083] The control information stored in the storage unit 212 is not limited to the above example. For example, the control information may include information for performing focus control (focus control parameters).

[0084] The focus control parameters are parameters that indicate the content of control over the OCT focusing drive unit 43A. Examples of focus control parameters include a parameter that indicates the focal position of the measurement arm, a parameter that indicates the moving speed of the focal position, and a parameter that indicates the moving acceleration of the focal position. The parameter that indicates the focal position is, for example, a parameter that indicates the position of the OCT focusing lens 43. The parameter that indicates the moving speed of the focal position is, for example, a parameter that indicates the moving speed of the OCT focusing lens 43. The parameter that indicates the moving acceleration of the focal position is, for example, a parameter that indicates the moving acceleration of the OCT focusing lens 43. The moving speed may or may not be constant. The same applies to the moving acceleration.

[0085] Such focus control parameters enable focus adjustment according to the shape of the fundus oculi Ef (typically a concave shape with a deep center and a shallow periphery) and aberration distribution. Focus control is performed, for example, in conjunction with scan control (repeated control of Lissajous scan). This allows motion artifacts to be corrected and high-quality images that are in focus across the entire scan range to be obtained.

[0086] <Scanning control unit 2111> The scanning control unit 2111 controls at least the optical scanner 44 based on the scanning protocol 2121. The scanning control unit 2111 may further control the light source unit 101 in cooperation with the control of the optical scanner 44 based on the scanning protocol 2121. The scanning control unit 2111 is realized by cooperation between hardware including a processor and scanning control software including the scanning protocol 2121.

[0087] <Image Data Construction Unit 220> The image data constructing unit 220 includes a processor and constructs OCT image data of the fundus Ef based on signals (sampling data) input from the data acquisition system 130. This OCT image data construction includes noise removal (noise reduction), filtering, fast Fourier transform (FFT), and the like, similar to conventional Fourier domain OCT (swept-source OCT). When another type of OCT method is adopted, the image data constructing unit 220 constructs OCT image data by performing known processing corresponding to that OCT type.

[0088] For example, the image data constructing unit 220 is configured to at least execute a process of constructing image data (A-scan image data) corresponding to each scan point (each A-line) from the sampling data. The image data constructed by the image data constructing unit 220 may be data before being imaged, and may be, for example, a signal profile (reflection profile, scattering profile) along the depth direction (z direction, optical axis direction, A-line direction).

[0089] As described above, in this embodiment, a Lissajous scan is applied to the fundus Ef. The image data constructing unit 220, together with the data processing unit 230, can construct three-dimensional image data of the fundus Ef by applying, for example, the image construction method and motion artifact correction method disclosed in Non-Patent Document 1 to the data set acquired through collection by the Lissajous scan and sampling by the data acquisition system 130.

[0090] The image data constructor 220 and / or the data processor 230 can apply rendering to the 3D image data to generate a display image. Examples of applicable rendering methods include volume rendering, surface rendering, maximum intensity projection (MIP), minimum intensity projection (MIP), and multiplanar reconstruction (MPR).

[0091] The image data constructing unit 220 and / or the data processing unit 230 can construct an en face OCT image based on the three-dimensional image data. For example, the image data constructing unit 220 and / or the data processing unit 230 can construct projection data by projecting the three-dimensional image data in the z direction (A-line direction, optical axis direction, depth direction). The image data constructing unit 220 and / or the data processing unit 230 can also construct a shadowgram by projecting partial three-dimensional image data, which is a part of the three-dimensional image data, in the z direction. This partial three-dimensional image data may be set using, for example, any segmentation method. Segmentation is a process of identifying a partial region in an image. In this example, segmentation can be performed to identify an image region corresponding to one or more tissues (regions) of the fundus oculi Ef.

[0092] The segmentation in the embodiment is not limited to the shadowgram construction process, and any known technique such as threshold processing, edge detection, filtering, machine learning (e.g., semantic segmentation) can be used.

[0093] The ophthalmic examination apparatus 1 may be capable of performing OCT angiography (OCT-angiography). OCT angiography is an imaging technique that constructs an image in which blood vessels are emphasized (see, for example, Non-Patent Document 2 and JP2015-515894A). Generally, fundus tissue (structure) does not change over a short period of time, but blood flow inside blood vessels also changes over a short period of time. OCT angiography generates an image by emphasizing areas where such temporal changes exist (blood flow signals). Note that OCT angiography is also called OCT motion contrast imaging. Furthermore, images obtained by OCT angiography are called angiographic images, angiograms, motion contrast images, etc.

[0094] When OCT angiography is possible, the ophthalmic examination apparatus 1 repeatedly scans the same region of the fundus Ef a predetermined number of times. For example, the ophthalmic examination apparatus 1 repeatedly executes the above-described scanning control (repeated control of Lissajous scanning) a predetermined number of times. As a result, multiple pieces of three-dimensional data (time-series three-dimensional data sets) are collected by the data acquisition system 130 from the application region of the Lissajous scan. The image data construction unit 220 and / or the data processing unit 230 can construct a motion contrast image from this three-dimensional data set. This motion contrast image is an angiographic image in which the temporal change in the interference signal caused by the blood flow in the fundus Ef is emphasized. This angiographic image is three-dimensional angiographic image data that represents the three-dimensional distribution of blood vessels in the fundus Ef.

[0095] The image data constructing unit 220 and / or the data processing unit 230 can construct any two-dimensional angiographic image data and / or any pseudo three-dimensional angiographic image data from this three-dimensional angiographic image data. For example, the image data constructing unit 220 and / or the data processing unit 230 can construct two-dimensional angiographic image data representing any cross section of the fundus Ef by applying multiplanar reconstruction to the three-dimensional angiographic image data. Furthermore, the image data constructing unit 220 and / or the data processing unit 230 can construct frontal angiographic image data of the fundus Ef by applying projection imaging or shadowgramming to the three-dimensional angiographic image data.

[0096] In this embodiment, the image data constructor 220 and / or the data processor 230 construct multiple strips from data collected by the data acquisition system 130. As described in Non-Patent Document 1, the image data constructor 220 and / or the data processor 230 divide the volume (three-dimensional data) collected by Lissajous scanning into multiple subvolumes without relatively large intervening motion, and construct a front projection image of each subvolume. These front projection images are strips. By applying registration and merging processes to the multiple strips obtained in this manner, an image in which motion artifacts have been corrected can be obtained. As will be described later, in this example, the registration and merging processes are performed by the data processor 230.

[0097] The image data constructing unit 220 is realized by cooperation between hardware including a processor and image constructing software. Note that, in some exemplary embodiments, the image data constructing unit 220 and the data processing unit 230 may be integrally configured.

[0098] <Data processing unit 230> The data processing unit 230 includes a processor and applies various types of data processing to the image of the subject's eye E. For example, the data processing unit 230 is realized by cooperation between hardware including a processor and data processing software. The same is true for each functional element included in the data processing unit 230.

[0099] The data processing unit 230 may be configured to perform registration between two images acquired of the fundus oculi Ef. For example, the data processing unit 230 may be configured to perform registration between three-dimensional image data acquired by OCT and a front image acquired by the fundus camera unit 2. The data processing unit 230 may also be configured to perform registration between two OCT images acquired by OCT. The data processing unit 230 may also be configured to perform registration between two front images acquired by the fundus camera unit 2. The data processing unit 230 may also be configured to apply registration to the analysis results of the OCT images and the analysis results of the front images. These registrations can be performed by known methods, and include, for example, feature point extraction and affine transformation.

[0100] Furthermore, the data processing unit 230 is configured to process a data set obtained using a Lissajous scan. As previously described, the image data constructor 220 is configured to construct strips from data collected by the Lissajous scan, for example. The data processing unit 230 is configured to construct a motion artifact corrected image by applying registration and merging operations to the strips.

[0101] A characteristic of Lissajous scanning is that any two strips have an overlapping area. Similar to the technique described in Non-Patent Document 1, the data processing unit 230 is configured to perform strip-to-strip registration using the overlapping area. The data processing unit 230 first orders the multiple strips constructed by the image data construction unit 220 according to size (e.g., area) (first to Nth strips) and designates the largest strip, the first strip, as the initial reference strip. Next, the data processing unit 230 registers the second strip using the first strip as a reference and merges (combines) the first and second strips. The data processing unit 230 registers the third strip using the resulting merged strip as a reference strip and merges the merged strip with the third strip. By sequentially performing these registration and merging processes in the above order, the first to Nth strips are aligned and stitched together, resulting in an image with motion artifacts corrected.

[0102] In such registration, a cross-correlation function is used to determine the relative position between a reference strip and other strips. However, since the strips can have arbitrary shapes, the correlation between the strips is calculated using a mask that treats each strip as an image of a specific shape (e.g., a square) (see Appendix A of Non-Patent Document 1).

[0103] As mentioned above, the strip is an intensity image of a predetermined gradation, and the mask image is a binary image, so the difference between the absolute values ​​of the pixel values ​​of the strip and the mask becomes large, and there is a risk that the effect of the mask in the correlation calculation using the strip and the mask will be ignored and an accurate correlation coefficient will not be obtained. In particular, when single-precision floating-point (float type) calculations are used to calculate the correlation coefficient between strips from the perspective of cost, etc., this problem becomes more pronounced due to the influence of rounding errors caused by the small number of significant digits.

[0104] <Image data processing unit 231> In addition to addressing the problem of such rounding errors, in order to realize the process of recording information representing scanning positions that satisfy predetermined conditions (scanning position information, which will be described later), this embodiment employs a data processing unit 230 equipped with an image data processing unit 231 shown in FIG. 4B.

[0105] 4B , the image data processing unit 231 includes a mask image generation unit 2311, a range adjustment unit 2312, a composite image generation unit 2313, a cross-correlation function calculation unit 2314, a correlation coefficient calculation unit 2315, an xy shift amount calculation unit 2316, a registration unit 2317, a merge processing unit 2318, a z shift amount calculation unit 232, an image data correction unit 233, and an information recording unit 234.

[0106] In the example described below, the image data processing unit 231 is configured to designate one of any two strips as a reference strip and to register the other strip (registering strip) with respect to this reference strip.

[0107] <Mask image generation unit 2311> The mask image generator 2311 generates a reference mask image corresponding to the reference strip and a target mask image corresponding to the target strip. Some examples of mask images are described below, but the present invention is not limited to these.

[0108] The contour shape of the mask image is, for example, rectangular, typically square. The shape of the reference mask image and the shape of the target mask image may be the same. Furthermore, the dimensions of the reference mask image and the dimensions of the target mask image may be the same.

[0109] The range of pixel values ​​of the mask image is set to be included in the closed interval [0,1], for example. Typically, the mask image may be a binary image with pixel values ​​of 0 or 1. As a specific example, the mask image is a binary image in which the pixel value in the area corresponding to the domain of the strip is 1 and the other pixels are 0. In other words, as shown in equation (20) of Non-Patent Document 1, the pixel value of the exemplary mask image is 1 in the image area of ​​the corresponding strip and 0 in the other areas.

[0110] <Range Adjustment Unit 2312> The range adjuster 2312 is configured to adjust the pixel value range of the reference strip and the pixel value range of the target strip based on the pixel value range of the reference mask image and the pixel value range of the target mask image. Typically, the range adjuster 2312 adjusts the pixel value range of the reference strip and the pixel value range of the target strip so as to reduce the difference between the pixel value range of the strip and the pixel value range of the mask image.

[0111] In general, the range adjuster 2312 adjusts the pixel value range of the reference strip, the pixel value range of the target strip, and the pixel value range of the mask image relatively. In a typical example, the reference mask image applied to the reference strip and the target mask image applied to the target strip are different, and the range adjuster 2312 adjusts the pixel value range of the reference strip, the pixel value range of the target strip, the pixel value range of the reference mask image, and the pixel value range of the target strip relatively.

[0112] The range adjustment unit 2312 is configured to adjust the range of pixel values ​​of the reference strip and the range of pixel values ​​of the reference mask image so as to reduce the range of pixel values ​​of the reference strip and the range of pixel values ​​of the reference mask image, and to relatively adjust the range of pixel values ​​of the target strip and the range of pixel values ​​of the target strip so as to reduce the range of pixel values ​​of the target strip and the range of pixel values ​​of the target mask image.

[0113] The range adjustment unit 2312 may be configured to adjust the pixel value range of the strip and the pixel value range of the mask image so that the pixel value range of one of the strip and the mask image matches the pixel value range of the other. For example, the range adjustment unit 2312 may be configured to adjust the pixel value range of the reference strip and the pixel value range of the reference mask image so that the pixel value range of the reference strip and the pixel value range of the reference mask image match, and to relatively adjust the pixel value range of the target strip and the pixel value range of the target strip so that the pixel value range of the target strip and the pixel value range of the target mask image match.

[0114] For example, the range adjuster 2312 is configured to normalize the range of pixel values ​​of the strip according to the range of pixel values ​​of the mask image, some examples of this normalization (standardization) are described below, but are not limited to these.

[0115] A first example of normalization will be described. When the range of pixel values ​​of the mask image is included in the closed interval [0, 1], the range adjustment unit 2312 divides the value of each pixel in the strip by the maximum pixel value in the strip. In this example, the range adjustment unit 2312 first compares the values ​​of all pixels in the strip to identify the maximum value (maximum pixel value), and then divides the value of each pixel in the strip by the maximum pixel value. This causes the range of pixel values ​​of the strip to match the closed interval [0, 1], which is the same as the range of pixel values ​​of the mask image.

[0116] A second example of normalization will be described. When the range of pixel values ​​of the mask image is included in the closed interval [0, 1], the range adjustment unit 2312 divides the value of each pixel of the strip by the maximum value of the range of pixel values ​​of this strip. The range of pixel values ​​of the strip is set in advance, and the range adjustment unit 2312 divides the value of each pixel of the strip by the upper limit (maximum value) of this range. In this example, the range of pixel values ​​of the strip is also matched to the same closed interval [0, 1] as the range of pixel values ​​of the mask image.

[0117] The processing performed by the range adjustment unit 2312 can reduce the difference between the absolute values ​​of the pixel values ​​of the strip and the absolute values ​​of the pixel values ​​of the mask, so that the effect of the mask is not ignored in the correlation calculation using the strip and the mask, and it becomes possible to accurately calculate the correlation coefficient. In particular, even when single-precision floating-point (float) calculation is used, it is possible to eliminate (reduce) the influence of rounding errors caused by a small number of significant digits.

[0118] Although this specification mainly describes an example of changing only the range of pixel values ​​of the strip, it is also possible to change only the range of pixel values ​​of the mask image, or to change both the range of pixel values ​​of the strip and the range of pixel values ​​of the mask image.

[0119] <Synthetic image generation unit 2313> For the strips and mask images to which the pixel value range adjustment by the range adjustment unit 2312 has been applied, the composite image generation unit 2313 generates two composite images by combining the mask image with each of the reference strip and the target strip. Typically, the composite image generation unit 2313 combines the reference mask image with the reference strip to generate a reference composite image, and combines the target mask image with the target strip to generate a target composite image.

[0120] The process of combining the strip and the mask image may be performed in the same manner as in Non-Patent Document 1. However, unlike the method in Non-Patent Document 1, in this embodiment, the difference between the pixel value range of the strip and the pixel value range of the mask image is adjusted to be small. Typically, the pixel value ranges of the reference strip and the target strip are normalized to match the pixel value ranges of the reference mask image and the target mask image.

[0121] For example, in the same manner as equation (19) in Non-Patent Document 1, the composite image generation unit 2313 may be configured to embed a strip with a normalized pixel value range into an image of the same size and shape as the mask image.

[0122] Furthermore, the composite image generation unit 2313 generates a composite image of the embedded image of the strip and the mask image. This composite image corresponds to "f´(r)m f (r)" (such as the second line on page 1800) in Non-Patent Document 1. However, as described above, the value of the embedded image of the strip is different from that in Equation (19).

[0123] <Cross-correlation function calculation unit 2314> The cross-correlation function calculation unit 2314 obtains a plurality of cross-correlation functions based on two composite images generated by the composite image generation unit 2313. The calculation of the cross-correlation function is performed in the same manner as in Non-Patent Document 1. For example, the cross-correlation function calculation unit 2314 calculates six cross-correlation functions (image cross-correlation) included in Equation (33) of Non-Patent Document 1 based on a reference composite image generated from a reference strip and a reference mask image and a target composite image generated from a target strip and a target mask image.

[0124] <Correlation coefficient calculation unit 2315> The correlation coefficient calculation unit 2315 calculates a correlation coefficient based on the plurality of cross-correlation functions calculated by the cross-correlation function calculation unit 2314. This calculation follows Equation (33) of Non-Patent Document 1.

[0125] <XY shift amount calculation unit 2316> The XY shift amount calculation unit 2316 calculates the shift amount in the XY direction (lateral direction, horizontal direction) between the reference strip and the target strip based on the correlation coefficient calculated by the correlation coefficient calculation unit 2315.

[0126] For example, the calculation for calculating the XY direction shift amount includes an operation corresponding to "rough lateral motion correction" (page 1787) in Non-Patent Document 1, and the XY shift amount calculation unit 2316 is configured to estimate the XY direction shift amount by obtaining the maximum value of the cross-correlation function.

[0127] Furthermore, the xy shift amount calculation unit 2316 may be configured to perform a calculation equivalent to "fine lateral motion correction" (page 1789) in Non-Patent Document 1 in order to calculate small lateral shift amounts caused by eye movements such as slow drift and tremor.

[0128] <Registration Department 2317> The registration unit 2317 performs lateral registration based on the lateral shift amount calculated by the xy shift amount calculation unit 2316. For example, this registration includes processing equivalent to "rough lateral motion correction" (page 1787) in Non-Patent Document 1. The registration unit 2317 can perform registration between the reference strip and the target strip so as to cancel out the lateral shift amount calculated by the xy shift amount calculation unit 2316.

[0129] When the xy shift amount calculation unit 2316 performs a calculation equivalent to "fine lateral motion correction" (page 1789) in Non-Patent Document 1, the registration unit 2317 can remove small lateral motion artifacts between the reference strip and the target strip by performing a registration equivalent to "fine lateral motion correction" (page 1789) in Non-Patent Document 1.

[0130] <Merge processing section 2318> The merge processing unit 2318 constructs a merged image of the reference strip and the target strip whose relative positions have been adjusted by the registration unit 2317. This processing is also performed in the same manner as the method described in Non-Patent Document 1.

[0131] As described above, the image data processing unit 231 sequentially executes the above-described series of processes on the plurality of strips constructed by the image data construction unit 220 in accordance with the order according to the size. As a result, a merged image in which lateral motion artifacts are corrected is obtained from the plurality of strips constructed by the image data construction unit 220. This merged image is typically an image representing the entire range to which the resampling scan is applied.

[0132] As described above, the plurality of strips are a plurality of front projection images based on a plurality of sub-volumes obtained by dividing the volume (3D data) collected by the resampling scan. Therefore, the merged image constructed from the plurality of strips by the image data processing unit 231 provides a plurality of sub-volumes (and their merged images) whose positions are adjusted in the lateral direction. In this example, the registration and merging processes in the depth direction orthogonal to the lateral direction are executed by the z-shift amount calculation unit 232 and the image data correction unit 233 (described later).

[0133] <z-shift amount calculation unit 232> The z-shift amount calculation unit 232 calculates the shift amount in the z direction (A-line direction, optical axis direction, depth direction) between a plurality of sub-volumes obtained by dividing the volume collected by the resampling scan (that is, the plurality of sub-volumes that are the basis of the plurality of strips).

[0134] This example is configured such that after the image data processing unit 231 executes position adjustment in the xy direction (lateral direction, horizontal direction), the z-shift amount calculation unit 232 and the image data correction unit 233 execute position adjustment in the z direction. That is, the z-shift amount calculation unit 232 of this example is configured to calculate the shift amount in the z direction using the data (for example, xy direction shift amount data, merged image) obtained by the image data processing unit 231.

[0135] An example of processing executed by the z-shift amount calculation unit 232 will be described. Similar to the pair of strips (reference strip and target strip) considered in calculating the shift amounts in the x and y directions, a pair of sub-volumes is also considered in calculating the shift amounts in the z direction. The pair of sub-volumes may be two sub-volumes corresponding to the reference strip and target strip considered in calculating the shift amounts in the x and y directions, and these sub-volumes are referred to as the reference sub-volume and the target sub-volume, respectively.

[0136] Based on the result of the xy-direction registration between the corresponding reference strips and target strips, the reference sub-volumes and target sub-volumes are aligned in the xy-direction. Also, as mentioned above, any two strips in a Lissajous scan overlap (intersect) at four points, and therefore any two sub-volumes also overlap (intersect) at four points.

[0137] The z-shift amount calculation unit 232 first identifies an intersection area (common area) between the reference sub-volume and the target sub-volume whose positions have been adjusted in the x and y directions. Each intersection area is three-dimensional image data.

[0138] Next, the z-shift amount calculation unit 232 sets a cross section in the identified intersection area. This cross section is, for example, a plane formed by an arbitrary axis (for example, the x-axis, the y-axis, or an axis obliquely intersecting both the x-axis and the y-axis) in the xy plane and the z-axis. Note that the cross section is not limited to a plane, and may be a curved surface, etc.

[0139] Next, the z-shift amount calculation unit 232 constructs an image of the set cross section from the reference sub-volume, and also constructs an image of the same cross section from the target sub-volume. The cross-sectional image constructed from the reference sub-volume is called the reference cross-sectional image, and the cross-sectional image constructed from the target sub-volume is called the target cross-sectional image. The reference cross-sectional image and the target cross-sectional image are images that represent the same cross section in the intersection region between the reference sub-volume and the target sub-volume that have been registered in the x- and y-directions.

[0140] Next, the z-shift amount calculation unit 232 analyzes the reference cross-sectional image to identify an image of a predetermined portion of the subject's eye E, and analyzes the target cross-sectional image to identify an image of the same portion. This portion may be any portion, and may be, for example, the surface of the fundus Ef (the retinal surface, the internal limiting membrane, or the boundary between the retina and the vitreous body). Furthermore, if an artificial object has been implanted in the subject's eye E, an image of this artificial object may be identified. The analysis for identifying the image of the predetermined portion may include, for example, segmentation.

[0141] Next, the z-shift amount calculation unit 232 calculates the z-coordinate of an image (reference image) of a predetermined region identified from the reference cross-sectional image, and calculates the z-coordinate of an image (target image) of a predetermined region identified from the target cross-sectional image. Typically, the image of the predetermined region consists of multiple pixels, and the z-coordinates of these pixels are not constant. For example, the global shape of the retinal surface is generally a curved shape convex in the +z direction, and images of local areas (examples of the reference image and target image) are at least partially tilted with respect to the z-axis. The z-shift amount calculation unit 232 can calculate the z-coordinate of the image of the predetermined region based on at least one pixel of a group of pixels constituting the image of the predetermined region. For example, a statistic of the z-coordinates can be calculated from the group of pixels, and this statistic can be used as the z-coordinate of the image. This statistic can be any representative value (summary statistic), such as a maximum value, minimum value, mean value, median, mode, or quantile. On the other hand, if the image of the predetermined region consists of a single pixel, the z-coordinate of this pixel can be used as the z-coordinate of the predetermined region.

[0142] Next, the z-shift amount calculation unit 232 calculates the difference between the z-coordinate of the reference image and the z-coordinate of the target image (the shift amount in the z direction). In some exemplary aspects, processing control can be performed so that correction processing is performed by the image data correction unit 233 when the z-direction shift amount exceeds a threshold. This threshold is, for example, zero or a positive value, and may be a preset fixed value or a variable value.

[0143] In the above example, the z-direction shift amount is calculated by constructing two-dimensional image data (cross-sectional images) from three-dimensional image data (sub-volumes), but in some exemplary aspects, the z-direction shift amount may be calculated from three-dimensional image data without constructing two-dimensional image data. In this case, the same processing as that performed on the pair of reference cross-sectional images and target cross-sectional images can be performed on the pair of reference sub-volumes and target sub-volumes.

[0144] In the above example, two-dimensional image data (cross-sectional image) is constructed from three-dimensional image data (subvolume), and an image of a specified area is identified from this two-dimensional image data to determine the amount of shift in the z direction. However, in some exemplary embodiments, an image of a specified area may be identified from the three-dimensional image data, and two-dimensional image data of the image of this specified area may be constructed to determine the amount of shift in the z direction.

[0145] The z-shift amount calculation unit 232 may be configured to calculate the z-direction shift amount between the reference subvolume and the target subvolume based on a correlation coefficient calculated based on a pair of the reference subvolume and the target subvolume. The method for calculating the correlation coefficient between the subvolumes may be the same as the correlation coefficient calculation method executed by the correlation coefficient calculation unit 2315, for example.

[0146] The method for calculating the z-direction shift amount is not limited to the above-mentioned methods, and may be any method applicable to a pair of a reference subvolume and a target subvolume, or may be any method applicable to a pair of an object other than a subvolume. An example of an object other than a subvolume is a rendered image of the subvolume (an image having a dimension in the z direction, such as an image projected in the x direction or y direction).

[0147] <Image data correction unit 233> The image data correction unit 233 is configured to perform z-direction position adjustment on the multiple sub-volumes whose positions have been adjusted in the x- and y-directions by the image data processing unit 231, based on the z-direction shift amount obtained by the z-shift amount calculation unit 232.

[0148] Similar to the registration unit 2317 that performs registration based on the xy direction shift amount, the image data correction unit 233 performs registration between the reference subvolume and the target subvolume so as to cancel out the corresponding z direction shift amount obtained by the z shift amount calculation unit 232.

[0149] As mentioned above, in some exemplary embodiments, the image data corrector 233 may be controlled to perform registration between the reference subvolume and the target subvolume only if the corresponding z-direction shift exceeds a threshold.

[0150] The image data obtained as a result of the position adjustment by the image data corrector 233 is a group of subvolumes that have been registered in all of the x-, y-, and z-directions, i.e., a group of subvolumes that have been three-dimensionally registered. The group of subvolumes before application of such registration is a volume acquired by Lissajous scanning. The data processor 230 recombines the group of subvolumes obtained by three-dimensional registration to obtain a volume to which three-dimensional registration has been applied.

[0151] The 3D registered dataset can then be used for any processing (image processing, post-processing, analysis, evaluation, visualization, rendering, etc.) This allows the desired processing to be performed using a 3D dataset that has been corrected for motion artifacts with higher accuracy and precision than ever before.

[0152] In addition, the z-shift amount calculation unit 232 and the image data correction unit 233 may be configured to perform registration equivalent to "axial motion correction (rough axial motion correction and / or fine axial motion correction)" (pages 1789 to 1790) of Non-Patent Document 1 in order to remove motion artifacts in the z direction.

[0153] <Information recording unit 234> First, the premise of the processing executed by the information recording unit 234 will be described. As described above, the correlation coefficient calculation unit 2315 calculates the correlation coefficient between strips, and the z-shift amount calculation unit 232 calculates the correlation coefficient between sub-volumes. Both the strips and the sub-volumes are defined based on cycles. Therefore, the correlation coefficient between strips calculated by the correlation coefficient calculation unit 2315 can be considered as the correlation coefficient between cycles, and the correlation coefficient between sub-volumes calculated by the z-shift amount calculation unit 232 can be considered as the correlation coefficient between cycles.

[0154] Thus, in the present disclosure, the correlation coefficient between cycles includes the correlation coefficient between strips and / or the correlation coefficient between sub-volumes. That is, some exemplary embodiments may handle only the correlation coefficient between strips, some exemplary embodiments may handle only the correlation coefficient between sub-volumes, and some exemplary embodiments may handle both the correlation coefficient between strips and the correlation coefficient between sub-volumes. Furthermore, some exemplary embodiments may handle the correlation coefficient between objects that are different from both strips and sub-volumes. In any case, it is possible to perform processing similar to that of this embodiment.

[0155] Furthermore, the correlation coefficient between strips may be the correlation coefficient between two strips, the correlation coefficient between one strip and two or more strips (strip group), or the correlation coefficient between two strip groups. The strip group may be, for example, a merged strip (described above) based on two or more strips. In this embodiment, in either case, the correlation coefficient calculation unit 2315 performs the calculation, and the content of the calculation may be the same as the processing described above.

[0156] Similarly, the correlation coefficient between subvolumes may be the correlation coefficient between two subvolumes, the correlation coefficient between one subvolume and two or more subvolumes (subvolume group), or the correlation coefficient between two subvolume groups. In any case, the z-shift amount calculation unit 232 performs the calculation, and the content of the calculation may be the same as the processing described above. The same applies when the correlation coefficient between objects other than strips and subvolumes is handled.

[0157] As described above, the ophthalmic examination apparatus 1 applies optical scanning (e.g., Lissajous scanning) according to a two-dimensional pattern including a series of cycles that intersect with each other to the subject's eye E to collect a data set (data set collection unit), and calculates a correlation coefficient between the cycles based on this data set (correlation coefficient calculation unit: correlation coefficient calculation unit 2315, z-shift amount calculation unit 232).

[0158] In this embodiment, multiple correlation coefficients are calculated. For example, when a correlation coefficient between two strips is calculated, if the number of strips (number of cycles) is N, the maximum number of correlation coefficients calculated from the N strips is N×(N−1) (if overlapping combinations are combined, the maximum number is N×(N−1) / 2). The number of correlation coefficients calculated in this case is determined by the combination of the strips (i.e., the number of strip pairs considered). Similarly, when a correlation coefficient between one strip and a strip group is calculated or when a correlation coefficient between two strip groups is calculated, multiple correlation coefficients are calculated. In these cases, the number of correlation coefficients calculated is determined by the number of strip groups considered, the combination of the strips and strip groups, the combination of the strip groups, and the like. For example, when the aforementioned sequential calculation performed by the correlation coefficient calculation unit 2315 is applied, N−1 correlation coefficients are calculated. In this way, a set of correlation coefficients (referred to as a correlation coefficient set) is generated from the collected data set.

[0159] The information recording unit 234 identifies a subset whose elements are correlation coefficients that satisfy predetermined conditions among the correlation coefficients belonging to this correlation coefficient set, and records information (called scanning position information) that indicates the scanning position corresponding to this subset (the correlation coefficients belonging to this subset).

[0160] Recording the scanning position information includes a process of saving the scanning position information in a predetermined storage device, and may further include a process of associating the scanning position information with predetermined data (for example, image data constructed by the image data processing unit 231). The storage device in which the scanning position information (and the predetermined data) is saved may be any predetermined storage device, and examples thereof include a storage device (not shown) included in the information recording unit 234, the storage unit 212, a storage device accessible by the ophthalmic examination apparatus 1 (such as an in-facility database system or a cloud-based database system), a recording medium to which data can be written by the ophthalmic examination apparatus 1, etc.

[0161] The range of information included in the scanning position information may be arbitrary. The scanning position information may include at least information indicating scanning positions corresponding to a subset identified from the set of correlation coefficients. For example, the scanning position information in some exemplary embodiments may include only information indicating scanning positions corresponding to each element of the subset identified from the set of correlation coefficients. Furthermore, the scanning position information in some exemplary embodiments may include information indicating scanning positions corresponding to each element of a set including the subset identified from the set of correlation coefficients (e.g., the entire or part of the set of correlation coefficients).

[0162] The content of the information included in the scanning position information may be arbitrary. The scanning position information only needs to include at least information indicating the scanning position. In some exemplary aspects, the scanning position information may include, in addition to the information indicating the scanning position, information regarding the judgment conditions and judgment results of the correlation coefficient, which will be described below. For example, in some exemplary aspects, the scanning position information may include information indicating the content of the judgment conditions and / or information indicating the content of the judgment results. Examples of the information indicating the content of the judgment conditions include a first threshold, a second threshold, and a specific range, which will be described later. Examples of the information indicating the content of the judgment results include a judgment result of whether the judgment conditions are satisfied, the degree of satisfaction of the judgment conditions, and the degree of deviation from the judgment conditions. Note that the information included in the scanning position information is not limited to these examples.

[0163] To identify the subset of the set of correlation coefficients, the information recording unit 234 determines whether each correlation coefficient belonging to the set of correlation coefficients satisfies a predetermined condition. This determination condition may be any predetermined condition, such as any of the following exemplary conditions:

[0164] A first example of the determination condition is that the correlation coefficient value is small. When this example condition is applied, the information recording unit 234 is configured to, for example, compare each correlation coefficient included in the correlation coefficient set with a predetermined threshold (referred to as a first threshold) to obtain a subset of correlation coefficients whose values ​​are smaller than the first threshold, and record scanning position information corresponding to this subset (a set whose elements are low-value correlation coefficients). This scanning position information indicates portions of an image with low quality or portions that were not imaged, and can be used for, for example, image completion or rescanning. The scanning position information in this example can be used in any process (such as information presentation, information provision, statistical processing, image analysis, image processing, image evaluation, machine learning, and generation of training data for machine learning).

[0165] A second example of the determination condition is that the correlation coefficient has a large value. When this example condition is applied, the information recording unit 234 is configured to, for example, compare each correlation coefficient included in the correlation coefficient set with a predetermined threshold (referred to as a second threshold) to obtain a subset of correlation coefficients whose values ​​are greater than the second threshold, and record scanning position information corresponding to this subset (a set whose elements are high-value correlation coefficients). This scanning position information indicates parts of an image that are of high quality, and can be used for any processing (such as information presentation, information provision, statistical processing, image analysis, image processing, image evaluation, machine learning, and generation of training data for machine learning).

[0166] A third example of the determination condition is that the value of the correlation coefficient falls within a predetermined range. When this example condition is applied, the information recording unit 234 is configured to, for example, compare each correlation coefficient included in the set of correlation coefficients with a predetermined range (referred to as a specific range) to obtain a subset of correlation coefficients whose values ​​fall within this specific range, and record scanning position information corresponding to this subset. This scanning position information indicates parts of an image whose quality falls within a certain range, and can be used for any processing (such as information presentation, information provision, statistical processing, image analysis, image processing, image evaluation, machine learning, and generation of training data for machine learning).

[0167] The first threshold value, the second threshold value, and the specific range (upper and lower limits) may all be set by any method. These values ​​may be, for example, default values ​​determined based on actual measurements and / or simulation values, individual values ​​determined based on image data to which the correlation coefficient calculation has been applied, or individual values ​​determined based on image data acquired in preparation for acquiring the image data (e.g., image data acquired during a preparatory operation such as alignment). The default values ​​and individual values ​​may typically be determined by applying statistical processing to the image data. For example, one or more of the first threshold value, the second threshold value, and the specific range may be determined based on the results of statistical calculations (e.g., histograms, maps, statistics, etc.) of predetermined parameters (e.g., correlation coefficients, pixel values, etc.) generated from the image data.

[0168] The judgment conditions may be determined depending on the application of the scanning position information. Furthermore, any number of judgment conditions may be prepared. When two or more judgment conditions are provided, these judgment conditions may be selectively used. Furthermore, two or more judgment conditions may be usable in processing related to the same Lissajous scan. In this case, two or more pieces of scanning position information corresponding to two or more judgment conditions may be generated.

[0169] The method of expressing the scanning position information may be any predetermined expression method, and for example, the scanning position information may be expressed using any of the following exemplary identifiers.

[0170] A first example of the identifier is the coordinates of the definition coordinate system of the image data constructed by the image data processing unit 231. When this example of the identifier is applied, for example, the scanning position information may be defined for each pixel (in other words, in units of pixels), or the scanning position information may be defined for a preset image region (in other words, in units of pixel groups), or the scanning position information may be defined using both of these.

[0171] A second example of the identifier is a number based on the ordering of cycles (called a cycle number). When this example identifier is applied, for example, the scan position information may be defined for each cycle (in other words, in units of cycles (i.e., cycle numbers)), or for several cycles (in other words, in units of cycle groups (i.e., several cycle numbers)), or the scan position information may be defined using both of these.

[0172] A third example of the identifier is a number based on the scan order (called a scan number). When this example identifier is applied, for example, scan position information may be defined for the scan of each cycle, or for the scans of several cycles, or scan position information may be defined using both of these. In this embodiment, a scan is performed for each cycle, so the scan number of this example identifier is substantially the same value as the cycle number of the second example.

[0173] A fourth example of the identifier is a number (called a strip number) based on the ordering of strips. When this example identifier is applied, for example, scan position information may be defined for each strip (in other words, in units of strips (i.e., strip numbers)), or scan position information may be defined for several strips (in other words, in units of strip groups (i.e., several strip numbers)), or scan position information may be defined using both of these. Since there is a one-to-one correspondence between cycles and strips in the processing performed by the image data processing unit 231 described above, the strip number in this example identifier is substantially the same value as each of the cycle number in the second example and the scan number in the third example.

[0174] A fifth example of the identifier is a number (called a subvolume number) based on the ordering of the subvolumes. When this example identifier is applied, for example, scan position information may be defined for each subvolume (in other words, in units of subvolumes (i.e., subvolume numbers)), or scan position information may be defined for several subvolumes (in other words, in units of subvolume groups (i.e., several subvolume numbers)), or scan position information may be defined using both of these. Since there is a one-to-one correspondence between strips and subvolumes in the processing performed by the image data processing unit 231 described above, the subvolume number in this example identifier is substantially the same value as each of the cycle number in the second example, the scan number in the third example, and the strip number in the fourth example.

[0175] The sixth example of the identifier is a number (called a merge process number) based on the order in which the merge processes are applied. Since the merge process is performed for each strip or each subvolume in the process executed by the image data processing unit 231 described above, the merge process number in this example of the identifier is substantially the same value as each of the cycle number in the second example, the scan number in the third example, the strip number in the fourth example, and the subvolume number in the fifth example.

[0176] The information recording unit 234 can record the image data (3D image data in which motion artifacts have been corrected) created by the image data correcting unit 233, in addition to the scanning position information. Furthermore, the information recording unit 234 can record the scanning position information and the image data in association with each other. This makes it possible to perform image data processing (e.g., interpolation, correction, adjustment, etc.) based on the scanning position information, perform an operation for processing the image data (e.g., rescanning) based on the scanning position information, display an image generated from the image data and information generated from the scanning position information in association with each other, and perform processing (e.g., analysis processing, statistical processing, image processing, machine learning, generation of training data for machine learning, etc.) based on the image data (and / or the image generated from the image data) and the scanning position information (and / or the information generated from the scanning position information).

[0177] <User Interface 240> The user interface 240 includes a display unit 241 and an operation unit 242. The display unit 241 includes the display device 3. The operation unit 242 includes various operation devices and input devices. The user interface 240 may include a device that combines a display function and an operation function, such as a touch panel. It is also possible to construct an embodiment that does not include at least a part of the user interface 240. For example, the display device may be an external device connected to the ophthalmic examination apparatus 1.

[0178] <Data Receiving Unit 250> The data receiving unit 250 acquires data from an external device. This external device may be, for example, a computer, a storage device, a recording medium, or an information system (for example, a hospital information system, an electronic medical record system, or an image archiving system). The external device may be, for example, a device directly connected to the ophthalmic examination apparatus 1, a device connected to the ophthalmic examination apparatus 1 via a local area network (LAN), or a device connected to the ophthalmic examination apparatus 1 via a wide area network (WAN).

[0179] The data receiving unit 250 may include, for example, a communication interface, a drive device, etc. The data receiving unit 250 may also include a scanner for reading data recorded on paper sheets, etc. The data receiving unit 250 may also include an input device for a user to input data. The input device may be a keyboard, a pen tablet, etc.

[0180] <Operation> The operation of the ophthalmic examination apparatus 1 will now be described. An example of the operation of the ophthalmic examination apparatus 1 is shown in FIGS. 6A to 6F. In this example of operation, image data construction (which includes motion artifact correction) in a Lissajous scan will be described in detail. In some exemplary embodiments, only some of the steps in this example of operation may be executed. In some exemplary embodiments, some of the steps in this example of operation may be replaced with other steps (for example, similar steps). In some exemplary embodiments, any steps may be combined with this example of operation.

[0181] Prior to step S1, the same preparatory operations as in the conventional method are performed, such as inputting the patient ID, setting the scanning mode (specifying Lissajous scan), presenting the fixation target, alignment, focus adjustment, and OCT optical path length adjustment.

[0182] In some exemplary embodiments, the ophthalmic examination apparatus 1 may be configured to determine a judgment condition (e.g., at least one of the first threshold, the second threshold, and the specific range) for identifying a subset of the set of correlation coefficients based on the image data acquired in this preparation operation. Note that a default judgment condition may be used, or a judgment condition determined from the data set collected in the Lissajous scan in step S1 may be used.

[0183] (S1: Lissajous Scan) Upon receiving a predetermined scan start trigger signal, the scan control unit 2111 starts applying an OCT scan (Lissajous scan) to the subject's eye E (fundus oculi Ef).

[0184] The scanning start trigger signal is generated, for example, in response to the completion of a predetermined preparatory operation (alignment, focus adjustment, OCT optical path length adjustment, etc.), or in response to a scanning start instruction operation being performed using the operation unit 242.

[0185] The scan control unit 2111 applies a Lissajous scan to the fundus Ef by controlling the optical scanner 44, the OCT unit 100, etc. based on a scan protocol 2121 (a protocol corresponding to the Lissajous scan). The data collected by the Lissajous scan is sent to the image data construction unit 220.

[0186] (S2: Building multiple strips) The image data constructing unit 220 constructs a plurality of strips based on the data collected in step S1 in the manner described above. The constructed plurality of strips are sent to the data processing unit 230.

[0187] (S3: Set the reference strip and target strip) The data processing unit 230 orders the multiple strips constructed in step S2 according to their dimensions (such as area). In this example, the number of multiple strips constructed in step S2 is assumed to be N (N is an integer equal to or greater than 2). Furthermore, these N strips are referred to as the first strip, second strip, ..., Nth strip according to the order specified by the ordering. Furthermore, any strip from the first to Nth strips may be referred to as the nth strip (n = 1, 2, ..., N). In this way, the first strip is the largest strip, the second strip is the second largest strip, the nth strip is the nth largest strip, and the Nth strip is the smallest strip.

[0188] In this example, the data processing unit 230 sets the first strip as the initial reference strip and the second strip as the initial target strip.

[0189] The reference strip and the target strip correspond to the arbitrary shaped images f(r) and g(r) in Appendix A of Non-Patent Document 1, respectively.

[0190] (S4: Set the reference mask image and the target mask image) The mask image generation unit 2311 generates a mask image (reference mask image) corresponding to the reference strip and a target mask image corresponding to the target strip.

[0191] At this stage, the mask image generation unit 2311 generates two mask images corresponding to the first strip and the second strip, respectively. The data processing unit 230 sets the first mask image corresponding to the first strip as an initial reference mask image, and sets the second mask image corresponding to the second strip as an initial target mask image.

[0192] The reference mask image and the target mask image are rectangular shaped binary image masks m in Appendix A of Non-Patent Document 1, respectively. f (r) and m g Equivalent to (r).

[0193] (S5: Normalize the reference strip and the target strip) The range adjustment unit 2312 normalizes the reference strip and the target strip set in step S3 in the manner described above. The strips are normalized according to the range of pixel values ​​of the mask image. At this stage, the range adjustment unit 2312 applies normalization to the first strip (reference strip) and the second strip (target strip). Note that the process performed by the range adjustment unit 2312 is not limited to normalization, and may be any of the range adjustment processes described above or a process similar thereto.

[0194] In this example, the pixel value range of each mask image is assumed to be in the closed interval [0, 1]. In particular, each mask image in this example is assumed to be a binary image in which the pixel values ​​within the region corresponding to the domain of the corresponding strip are set to 1 and the other pixel values ​​are set to 0.

[0195] The range adjustment unit 2312 normalizes the strip, for example, by dividing the value of each pixel in the strip by the maximum pixel value in the strip, or by dividing the value of each pixel in the strip by the maximum value of the range of pixel values ​​(tone range) of the strip.

[0196] Furthermore, the data processing unit 230 (synthetic image generation unit 2313) embeds the strip with the normalized pixel value range into an image with the same dimensions and shape as the corresponding mask image. The embedded image of the normalized first strip f(r) is similar to the rectangular shaped image f'(r) in Appendix A of Non-Patent Document 1, but the absolute value (|f(r)|) of the value range within the image area of ​​the first strip f(r) is normalized to 1 or less. This embedded image is also represented as f'(r).

[0197] Similarly, the embedded image of the normalized second strip g(r) is an image similar to the rectangular image g'(r), but the absolute value of the range within the image area of ​​the second strip g(r) (|g(r)|) is normalized to be less than or equal to 1. This embedded image is also denoted as g'(r).

[0198] (S6: Generate a reference composite image and a target composite image) The composite image generator 2313 generates a composite image of the embedded image of the normalized strip and the corresponding mask image.

[0199] At this stage, the composite image generation unit 2313 generates a composite image of the embedded image of the normalized first strip (reference strip) and the first mask image (reference mask image), and also generates a composite image of the embedded image of the normalized second strip (target strip) and the second mask image (target mask image). The former composite image is called the reference composite image, and the latter is called the target composite image.

[0200] Here, the reference composite image and the target composite image are the combination of two rectangle shaped images f'(r)m in Appendix A of Non-Patent Document 1, respectively. f (r) and g´(r)m g (r), where, as mentioned above, the absolute value of the range of the first strip f(r) in the image area (|f(r)|) is less than or equal to 1, and the absolute value of the range of the second strip g(r) in the image area (|g(r)|) is less than or equal to 1.

[0201] Image 311 in Figure 7A is an example of an embedded image f'(r) of the normalized first strip f(r), and image 321 is an example of a first mask image. Combining these two images 311 and 321 produces a reference composite image f'(r)m f (r) is obtained.

[0202] Similarly, image 312 in Figure 7B is an example of an embedded image g'(r) of the normalized second strip g(r), and image 322 is an example of a second mask image. Combining these two images 312 and 322 produces a reference composite image g'(r)m g (r) is obtained.

[0203] (S7: Calculate multiple cross-correlation functions) The cross-correlation function calculation unit 2314 calculates a plurality of cross-correlation functions based on the reference composite image and the target composite image generated in step S6. In this example, the cross-correlation function calculation unit 2314 calculates six cross-correlation functions included in equation (33) of Non-Patent Document 1 based on the reference composite image and the target composite image generated in step S6.

[0204] (S8: Calculate the correlation coefficient) The correlation coefficient calculation unit 2315 calculates a correlation coefficient based on the multiple cross-correlation functions calculated in step S7. In this example, the correlation coefficient calculation unit 2315 calculates the correlation coefficient (ρ(r')) from the multiple cross-correlation functions calculated in step S7 according to equation (33) in Non-Patent Document 1.

[0205] (S9: Record the correlation coefficient) The information recording unit 234 records the correlation coefficient (ρ(r')) calculated in step S8. For example, the information recording unit 234 records this correlation coefficient and a corresponding identifier in association with each other. As described above, this identifier may be any of the coordinates in the definition coordinate system of the image data, the cycle number, the scan number, the strip number, and an identifier different from these. By repeating steps S3 to S14, the correlation coefficients included in the correlation coefficient set of this Lissajous scan are accumulated.

[0206] (S10: Calculate the shift amount in the x and y directions between the reference strip and the target strip) The xy shift amount calculation unit 2316 calculates the amount of shift in the xy directions between the first strip (reference strip) f(r) and the second strip (target strip) g(r) based on the correlation coefficient calculated in step S8. In this example, the relative amount of shift in the xy directions (Δx, Δy) between the reference strip f(r) and the target strip g(r) is found by detecting the peak of the correlation coefficient ρ(r') calculated in step S8.

[0207] (S11: Save the shift amount in the x and y directions) The x and y direction shift amounts calculated for the strip pair in step S10 are stored in, for example, the storage unit 212. The x and y direction shift amounts may be recorded together with time information corresponding to the strip pair. This time information may be time parameters corresponding to the Lissajous scan or information based thereon, or sequence information regarding the Lissajous scan or information based thereon. Examples of time information include a cycle number, a scan number, a strip number, etc. The x and y direction shift amounts sequentially acquired by repeating steps S3 to S13 are recorded together with the time information to generate x and y position history data.

[0208] (S12: Perform registration between the reference strip and the target strip) The registration unit 2317 applies x- and y-direction registration (rough lateral motion correction) to the reference strip and the target strip based on the x- and y-direction shift amounts calculated in step S10. At this stage, registration between the first strip f(r) and the second strip g(r) is performed.

[0209] Furthermore, the registration unit 2317 may apply any xy registration to the reference strip and the target strip, such as the "fine lateral motion correction" described above.

[0210] (S13: Construct a merged image of the reference strip and the target strip) The merge processing unit 2318 constructs a merged image of the reference strip and the target strip whose relative positions have been adjusted by the registration in step S12. Image 330 in Figure 7C shows an example of a merged image of the first strip f(r) and the second strip g(r).

[0211] Here, an example of implementing the processes in steps S7 to S13 will be described. First, f'(r)m f (r), g´(r)m g(r), m f (r), m g (r), (f´(r)m f (r)) 2 , and (g´(r)m g (r)) 2 By setting the imaginary part of each of the above to 0, we get only the real part.

[0212] Next, f´(r)m f (r), g´(r)m g (r), m f (r), m g (r), (f´(r)m f (r)) 2 , and (g´(r)m g (r)) 2 A fast Fourier transform (FFT) is applied to each real part of

[0213] Next, based on the group of functions obtained by these fast Fourier transforms, the six cross-correlation functions shown in step S7 of FIG. 6A are derived.

[0214] An inverse fast Fourier transform (IFFT) is then applied to each of the six derived cross-correlation functions.

[0215] Next, the correlation coefficient ρ(r′) of equation (33) in Non-Patent Document 1 is calculated.

[0216] Then, the relative shift amount (Δx, Δy) between the reference strip f(r) and the target strip g(r) is calculated by identifying the peak position of the correlation coefficient ρ(r'), and registration and merging processes between the reference strip f(r) and the target strip g(r) are performed based on this shift amount.

[0217] By sequentially applying this series of processes to the N strips, merged images of all N strips can be obtained. In addition, the xy direction shift amount (Δx, Δy) calculated for the strip pair can be saved together with the time information (i.e., associated with the time information).

[0218] (S14: Have you processed all the strips?) A series of processes from steps S3 to S13 are executed for all of the N strips obtained in step S2, in the order described above.

[0219] When N is 3 or more, if a merged image of the first strip and the second strip is created in step S13 (S14: No), the process returns to step S3. In step S3, the merged image of the first strip and the second strip is set as a new reference strip, and the third strip is set as a new target strip. By performing the processes of steps S4 to S13 based on the new reference strip and the new target strip, a merged image of the new reference strip and the new target strip is obtained. This new merged image is a merged image of the first to third strips. By sequentially applying this series of processes to the N strips, merged images of all N strips are obtained (S14: Yes).

[0220] (S15: Save the image with the xy motion artifacts corrected) The merged image finally obtained by the above-described repetitive processing is an image in which motion artifacts in the x and y directions have been corrected, and is an image that represents the entire application range of the Lissajous scan in step S1. The main control unit 211 can store this final merged image in the storage unit 212 (and / or other storage device).

[0221] The data processing unit 230 can use the registration results based on the N strips to register the data (three-dimensional data) collected in step S1 and / or the three-dimensional image data constructed by the image data construction unit 220 from this three-dimensional data. In other words, the data processing unit 230 can use the final merged image to register the three-dimensional data or the three-dimensional image data. This registration includes converting the coordinate system defining the Lissajous scan into a three-dimensional Cartesian coordinate system (x-y-z coordinate system). This coordinate conversion corresponds to "remapping" in the method described in Non-Patent Document 1. In this way, three-dimensional image data (volume) in which x- and y-direction motion artifacts have been corrected is obtained. The main control unit 211 can store this x- and y-direction motion artifact-corrected volume in the storage unit 212 (and / or another storage device) together with or instead of the final merged image.

[0222] (S16: Select the reference strip) The process then moves on to processing for correcting z-direction motion artifacts. In z-direction motion artifact correction, first, the z-shift amount calculation unit 232 selects one strip (reference strip) from multiple strips corresponding to the volume for which x- and y-direction motion artifact correction has been performed. The reference strip selected here may be the first strip selected in step S3 of the first iteration, or a strip different from this.

[0223] (S17: Select the target strip) Next, the z-shift amount calculation unit 232 selects one strip (target strip) other than the reference strip selected in step S16 from the multiple strips corresponding to the volume in which the x- and y-direction motion artifacts have been corrected. The reference strip selected here may be the second strip selected in step S3 of the first iteration, or may be a strip other than this.

[0224] (S18: Set a reference sub-volume corresponding to the reference strip) Next, the z-shift amount calculation unit 232 sets the sub-volume corresponding to the reference strip selected in step S16 as the reference sub-volume. Here, the front projection image of the reference sub-volume is the reference strip.

[0225] (S19: Set the target subvolume corresponding to the target strip) Similarly, the z-shift amount calculation unit 232 sets the sub-volume corresponding to the target strip selected in step S17 as the target sub-volume. Here, the front projection image of the target sub-volume is the target strip.

[0226] (S20: Identifying the intersection region between the reference subvolume and the target subvolume) Next, the z-shift amount calculation section 232 identifies an intersection area (common area) between the reference sub-volume set in step S18 and the target sub-volume set in step S19.

[0227] As described above, there are four intersection regions between the two sub-volumes. The z-shift amount calculation unit 232 identifies one or more intersection regions. When two or more intersection regions are identified, the following step S21 is executed for each of the identified intersection regions.

[0228] (S21: Calculate the z-direction shift amount between the reference subvolume and the target subvolume) Next, the z-shift amount calculation unit 232 calculates the amount of shift in the z direction between the reference subvolume and the target subvolume based on the intersecting region between the subvolumes identified in step S20.

[0229] For example, the z-shift amount calculation unit 232 sets a cross section (cross section of interest) in the intersection region identified in step S20. The cross section of interest may be any cross section in the intersection region. Next, the z-shift amount calculation unit 232 constructs an image of the cross section of interest (reference cross section image) from the reference sub-volume, and constructs an image of the cross section of interest (target cross section image) from the target sub-volume. Next, the z-shift amount calculation unit 232 analyzes the reference cross section image to identify an image of a predetermined portion of the subject's eye E (reference image), and analyzes the target cross section image to identify an image of the same portion (target image). Next, the z-shift amount calculation unit 232 obtains the z coordinate of the reference image and the z coordinate of the target image, and calculates the difference between these two z coordinates. In this example, the calculated difference is used as the z-direction shift amount between the reference sub-volume and the target sub-volume.

[0230] A specific example is shown in FIG. 8. Reference numeral 401 denotes the intersection state between a strip 402 corresponding to the reference subvolume and a strip 403 corresponding to the target subvolume. In this example, attention is focused on the intersection region surrounded by a white circle. The diameter of the white circle along the x direction (indicated by a dotted line) is the cross section of interest. The intersection region and the cross section of interest are also similarly indicated for each of the two strips 402 and 403. The z-shift amount calculation unit 232 constructs an image of this cross section of interest (reference cross section image) 412 from the reference subvolume corresponding to the strip 402, and constructs an image of this cross section of interest (target cross section image) 413 from the target subvolume corresponding to the strip 403. Furthermore, the z-shift amount calculation unit 232 analyzes the reference cross section image 412 to identify the z position (z coordinate) 422 of the image of the retinal surface, and analyzes the target cross section image 413 to identify the z position (z coordinate) 423 of the image of the retinal surface. In addition, the z-shift amount calculation unit 232 calculates the difference Δz between two z positions (two z coordinates) 422 and 423 identified from the two cross-sectional images 412 and 413, respectively, as the z-direction shift amount between the reference subvolume and the target subvolume.

[0231] The method for calculating the z-direction shift amount between the reference subvolume and the target subvolume is not limited to this, and any method that utilizes their intersection area may be used. For example, as described above, the z-shift amount calculation unit 232 may calculate a correlation coefficient between the reference subvolume and the target subvolume and determine the z-direction shift amount based on this correlation coefficient.

[0232] In this case, similar to step S9 in FIG. 6B, the correlation coefficients calculated for each subvolume pair are recorded by the information recording unit 234. For example, the information recording unit 234 records the correlation coefficients obtained in calculating the z-direction shift amount in association with the corresponding identifier. As described above, this identifier may be any of the coordinates in the definition coordinate system of the image data, the cycle number, the scan number, the strip number, the subvolume number, the merge process number, or an identifier different from these. By repeating steps S17 to S24, the correlation coefficients included in the correlation coefficient set of this Lissajous scan are accumulated. Here, the correlation coefficients in the x and y directions recorded in step S9 and the correlation coefficients in the z direction recorded in step S21 are associated with each other via the identifier. This makes it possible to generate scanning position information corresponding to three-dimensional correlation coefficients.

[0233] (S22: Save the z-direction shift amount) The z-direction shift amount calculated for the subvolume pair in step S21 is stored, for example, in the storage unit 212. Similar to storing the xy-direction shift amount (step S11), the z-direction shift amount may be recorded, for example, together with time information corresponding to the subvolume pair.

[0234] (S23: Perform registration between the reference subvolume and the target subvolume) The image data correcting unit 233 applies registration in the z direction to the reference sub-volume and the target sub-volume based on the z-direction shift amount calculated in step S21.

[0235] This registration is a process of adjusting the relative positions in the z direction between the reference sub-volume and the target sub-volume so as to cancel out the z-direction shift amount Δz in Figure 8, that is, so that the image of the retinal surface in the reference cross-sectional image 412 and the image of the retinal surface in the target cross-sectional image 413 are located at the same z position (equal z coordinate).

[0236] (S24: Construct a merged image of the reference subvolume and the target subvolume) The image data correcting unit 233 constructs a merged image of the reference subvolume and the target subvolume whose relative positions have been adjusted in the z direction by the registration in step S23, thereby obtaining a merged image of the reference subvolume and the target subvolume whose relative positions have been adjusted in the xy and z directions.

[0237] (S25: Have all subvolumes been processed?) A series of processes from steps S17 to S24 are executed for all sub-volumes in a predetermined order (for example, the order assigned to the first to Nth strips) (S25: No). In this example, the merged image constructed in step S24 is set as the reference strip in the next routine. By applying this series of processes to all sub-volumes in a sequential manner, merged images of all sub-volumes are obtained (S25: Yes).

[0238] (S26: Save the volume with all subvolumes merged) The merged image finally obtained by repeating steps S17 to S24 is an image in which motion artifacts in the x, y, and z directions have been corrected, and is an image that represents the entire application range of the Lissajous scan in step S1. The main control unit 211 can store this final merged image in the storage unit 212 (and / or other storage device).

[0239] The following processing is for more precise z-direction motion artifact correction. In steps S16 to S25, z-direction motion artifact correction is performed in subvolume units (strip units). In contrast, in steps S27 to S32 described below, z-direction motion artifact correction is performed in cycle units. This cycle-unit correction is performed by the data processing unit 230 (for example, the image data correction unit 233).

[0240] Note that precise z-direction motion artifact correction does not need to be performed on a cycle-by-cycle basis, but may be performed on a cycle group that constitutes a part of a subvolume basis.Furthermore, precise z-direction motion artifact correction does not need to be performed for all cycles (all cycle groups), but may be performed for only some cycles (some cycle groups).

[0241] (S27: Select cycles from merged volumes) The image data correcting unit 233 selects one cycle (partial data corresponding to one cycle) from the volume in which all the sub-volumes are merged.

[0242] The cycle selected first may be any cycle. For example, the cycles may be selected sequentially according to the scan order (chronological order, scan number, cycle number) of the multiple cycles in the Lissajous scan in step S1. Alternatively, the cycle selection order may be set based on the order (strip number, subvolume number) assigned to multiple strips or multiple subvolumes.

[0243] (S28: Remove the cycle from the volume) Next, the image data correcting unit 233 extracts the partial data corresponding to the cycle selected in step S27 from the volume.

[0244] (S29: Calculate the z-direction shift between the volume after cycle removal and the cycle) Next, the image data correcting unit 233 calculates the amount of shift in the z direction between the partial data corresponding to the cycle extracted in step S28 and the volume from which the cycle was extracted.

[0245] The method for calculating the z-direction shift amount in this step may be the same as the method in step S21. For example, the image data correction unit 233 analyzes the partial data (cross-sectional image of interest) of the cycle removed from the volume in step S28 to identify an image (first image) of a predetermined portion of the eye E, and analyzes the volume from which the cross-sectional image of interest has been extracted to identify an image (second image) of the same portion. The analysis for identifying the second image does not need to be applied to the entire volume from which the cross-sectional image of interest has been extracted; for example, it may be applied only to a portion adjacent to the cross-sectional image of interest (adjacent cycle). Furthermore, the image data correction unit 233 calculates the z coordinate of the first image and the z coordinate of the second image, and calculates the difference between these two z coordinates. In this example, the calculated difference is used as the z-direction shift amount between the cycle and the volume after the cycle has been removed.

[0246] (S30: Update the z-direction shift amount) Next, the main controller 211 replaces the z-direction shift amount calculated in step S21 with the new z-direction shift amount calculated in step S29. Here, the z-direction shift amount of the subvolume including the partial data corresponding to the cycle extracted in step S28 is updated.

[0247] When a correlation coefficient is used in both the calculation of the z-direction shift amount in step S21 and the calculation of the z-direction shift amount in step S29, the information recorded in step S21 (correlation coefficient, identifier, etc.) can be replaced with the information acquired in step S29 (correlation coefficient, identifier, etc.), or both the information recorded in step S21 and the information acquired in step S29 can be recorded in association with each other. Here, the information recorded in step S21 and the information acquired in step S29 can be associated with each other via the identifier. When a correlation coefficient is used in only one of the calculation of the z-direction shift amount in step S21 and the calculation of the z-direction shift amount in step S29, the information acquired in one of the processes can be recorded.

[0248] (S31: Perform registration and merging) Next, the image data correcting unit 233 performs registration between the partial data corresponding to the cycle removed in step S28 and the volume after the cycle removal so as to cancel the z-direction shift amount calculated in step S29. Furthermore, the image data correcting unit 233 constructs a merged image of the partial data whose relative position in the z direction has been adjusted by this registration and the volume after the cycle removal. This results in a volume whose relative position has been adjusted in the x and y directions and whose relative position has been precisely adjusted in the z direction.

[0249] (S32: Have all cycles been processed?) A series of processes from steps S27 to S31 are executed for all cycles in a predetermined order (S32: No). By applying this series of processes to all cycles in a sequential manner, a volume in which the relative positions in the z direction between all cycles have been adjusted is obtained (S32: Yes). The main controller 211 can store this final volume in the memory 212 (and / or another memory device).

[0250] (S33: Identify correlation coefficients that satisfy predetermined conditions from the set of correlation coefficients) In this operation example, the routine including step S9 (and at least one of the routine including step S21 and the routine including step S29) is repeatedly performed to accumulate information such as correlation coefficients and identifiers. The accumulated information corresponds to the above-mentioned set of correlation coefficients.

[0251] The information recording unit 234 identifies correlation coefficients that satisfy a predetermined condition from among the correlation coefficients included in the set of correlation coefficients. This predetermined condition corresponds to the aforementioned judgment condition. As a result, the aforementioned subset is obtained.

[0252] (S34: Generate scanning position information) The information recording unit 234 generates scanning position information corresponding to the subset identified in step S33. For example, the information recording unit 234 generates the scanning position information from an identifier associated with each correlation coefficient belonging to the subset identified in step S33. As described above, the identifier is information indicating the position (scanning position) where the Lissajous scan in step S1 was applied (i.e., an expression of the scanning position), and may include, for example, at least one of coordinates in a defined coordinate system of the image data, a cycle number, a scan number, a strip number, a subvolume number, a merge process number, and an identifier different from these.

[0253] The scanning position information thus generated includes information indicating the scanning position corresponding to the correlation coefficient that satisfies the determination condition.

[0254] When the judgment condition is the first example described above (the value of the correlation coefficient is small), the scanning position information includes information indicating the scanning position corresponding to the correlation coefficient whose value is smaller than the first threshold, that is, information indicating the position of the cycle pair with low correlation in the intersection region.

[0255] When the judgment condition is the second example described above (that the value of the correlation coefficient is large), the scanning position information includes information indicating the scanning position corresponding to the correlation coefficient whose value is larger than the second threshold, that is, information indicating the position of the cycle pair with high correlation in the intersection region.

[0256] When the judgment condition is the third example described above (the value of the correlation coefficient is included in the specific range), the scanning position information includes information indicating the scanning position corresponding to the correlation coefficient whose value falls within this specific range, that is, information indicating the position of the cycle pair whose correlation in the intersection region is included in the specific range.

[0257] Similarly, even when a different judgment condition is used than these examples, the scan position information includes information indicating the scan position corresponding to the correlation coefficient that satisfies the applied judgment condition, i.e., information indicating the position of the cycle pair whose correlation in the intersection region meets this judgment condition.

[0258] (S35: Record the scanning position information) The information recording unit 234 records the scanning position information generated in step S34 (end). The information recording unit 234 may record this scanning position information and the final volume (three-dimensional image data with motion artifacts corrected) generated in step S32 in association with each other. Furthermore, the information recording unit 234 may record any information generated in this operation example in association with the scanning position information. The type of information to be recorded together with the scanning position information may be set in advance or may be specified by the user or the ophthalmic examination apparatus 1.

[0259] The recorded scanning position information is used for any processing. This processing may be, for example, one or more of information presentation (displaying scanning position information, displaying information generated from scanning position information, etc.), information provision (transmission of scanning position information, transmission of information generated from scanning position information, etc.), statistical processing, image analysis, image processing, image evaluation, machine learning, and generation of training data for machine learning. Information recorded together with the scanning position information can also be used in these processing.

[0260] <Effects> Some effects of the ophthalmic examination apparatus 1 according to the first embodiment will be described.

[0261] The ophthalmic examination apparatus 1 is a scanning imaging apparatus that constructs image data using optical scanning, and includes a data set collection unit, a correlation coefficient calculation unit, and an information recording unit as its functional elements.

[0262] The dataset collection unit functions to collect a dataset by applying an optical scan to the sample according to a two-dimensional pattern including a series of cycles that intersect with each other. In this embodiment, a group of elements for collecting OCT data corresponds to the dataset collection unit, and for example, the fundus camera unit 2, the OCT unit 100, the image data construction unit 220, etc. correspond to the dataset collection unit.

[0263] The correlation coefficient calculation unit functions to calculate a correlation coefficient between cycles based on the data set collected by the data set collection unit. In this embodiment, the correlation coefficient calculation unit 2315, the z-shift amount calculation unit 232, etc. correspond to the correlation coefficient calculation unit.

[0264] The information recording unit functions to record scanning position information corresponding to a subset of the correlation coefficient set calculated from the data set by the correlation coefficient calculation unit, the subset having correlation coefficients that satisfy a predetermined condition as elements. In this embodiment, the information recording unit 234 and the like correspond to the information recording unit.

[0265] The ophthalmic examination apparatus 1 described above can record scanning position information that was not recorded in conventional techniques, and can also record scanning position information according to a correlation coefficient, thereby providing a new way of using data acquired by scanning imaging. Some non-limiting examples of such uses will be described in the following embodiments.

[0266] In some aspects of this embodiment, the elements of the subset may have correlation coefficients below a predetermined threshold. That is, the information recording unit of this aspect may be configured to record scanning position information corresponding to a subset of the correlation coefficient set calculated from the data set by the correlation coefficient calculation unit, the subset having elements with correlation coefficients below a predetermined threshold. According to this aspect, scanning position information indicating low-quality portions or portions not imaged in an image constructed using optical scanning can be generated and recorded. By using this scanning position information, for example, image interpolation, rescanning, etc. can be performed.

[0267] Some application examples of the first embodiment will be described in the following embodiments. In the following embodiments, unless otherwise specified, the same features as those of the first embodiment (similar configurations, similar elements, similar functions, similar actions, similar effects, etc.) can be employed. Note that the same applications as the following embodiments can also be made for embodiments equivalent to or similar to the first embodiment.

[0268] <Second embodiment> A second embodiment will now be described. The ophthalmic examination apparatus of this embodiment has the same configuration as that of the first embodiment shown in Figures 1 to 4A, and includes an image data processing unit 231A shown in Figure 9 instead of the image data processing unit 231 shown in Figure 4B. The image data processing unit 231A of this embodiment is obtained by adding a pixel value determination unit 235 to the image data processing unit 231 of the first embodiment.

[0269] The information recording unit 234 of this embodiment is configured to identify a subset having elements with correlation coefficients below a predetermined threshold from a set of correlation coefficients calculated from a data set collected by optical scanning such as a Lissajous scan, and record scanning position information corresponding to this subset. The ophthalmic examination apparatus of this embodiment has a functional element that performs a process of replacing low-quality parts in the image data constructed by the image data processing unit 231A (e.g., final three-dimensional image data generated by the image data correction unit 233 or a merged image generated by the merge processing unit 2318) with higher-quality pixels, and / or a process of complementing parts that were not imaged in this image data. An example of this functional element is the pixel value determination unit 235.

[0270] The pixel value determination unit 235 is configured to determine the value of a pixel corresponding to the scanning position information recorded by the information recording unit 234, based on the values ​​of one or more neighboring pixels. The pixel value determination unit 235 is realized by cooperation between hardware including a processor and pixel value determination processing software.

[0271] The pixel value determination unit 235 may perform any method of pixel value determination processing. For example, the pixel value determination unit 235 may be capable of performing any interpolation and / or any extrapolation. Examples of interpolation methods include nearest neighbor interpolation, linear interpolation, bilinear interpolation, trilinear interpolation, bicubic interpolation, curve fitting, and polynomial interpolation. Examples of extrapolation methods include linear extrapolation and Richardson extrapolation.

[0272] For example, after step S35 in FIG. 6F of the first embodiment, the pixel value determination unit 235 can determine the value of a pixel corresponding to the scanning position information recorded in step S35 based on the values ​​of one or more neighboring pixels. This scanning position information includes information indicating a scanning position corresponding to a correlation coefficient smaller than the first threshold value described in the first embodiment. In other words, this scanning position information includes information indicating the position of a cycle pair with low correlation in the intersection region. The pixel value determination unit 235 functions to determine the pixel value of each pixel present at such a scanning position (e.g., the cycle pair or the intersection region).

[0273] The operation of the ophthalmic examination apparatus according to this embodiment will be described. An example of the operation of the ophthalmic examination apparatus is shown in Fig. 10. Unless otherwise specified, the matters described in the first embodiment can be applied to this example of operation.

[0274] (S41: Specify the number of cycles) After inputting the patient ID and setting the scan mode (designating the Lissajous scan), the number of cycles in the Lissajous scan to be applied to the subject's eye E can be designated as an optional step. The number of cycles corresponds to the density of the scan. The number of cycles is designated, for example, by the user or the ophthalmic examination apparatus.

[0275] (S42: Perform preparation operations) The ophthalmic examination apparatus performs preparatory operations similar to those of the conventional method, such as presenting a fixation target, alignment, focus adjustment, and OCT optical path length adjustment. The ophthalmic examination apparatus may be configured to determine a judgment condition (e.g., the first threshold value described above) for identifying a subset of the correlation coefficient set based on image data acquired in the preparatory operations. Note that a default judgment condition may be used, or a judgment condition determined from the data set collected by the Lissajous scan in step S43 may be used.

[0276] (S43: Perform Lissajous scan) Upon receiving a predetermined scan start trigger signal, the scan control unit 2111 applies a Lissajous scan consisting of the number of cycles designated in step S41 to the subject's eye E (fundus oculi Ef).

[0277] (S44: Build a strip for each cycle) The image data constructing unit 220 constructs strips corresponding to the cycles specified in step S41 based on the data set collected by the Lissajous scan in step S43. This process may be performed in the same manner as in the first embodiment (step S2 in FIG. 6A), for example.

[0278] (S45: Calculate the correlation coefficient for each cycle pair) The image data processing unit 231A (correlation coefficient calculation unit 2315, etc.) calculates the correlation coefficient for each cycle pair formed based on the multiple cycles designated in step S41.

[0279] The processing of this step may be performed in the same manner as the motion artifact correction of the first embodiment (steps S3 to S8 in FIGS. 6A and 6B), for example.

[0280] Alternatively, the processing in this step may be such that, for each cycle pair formed based on the multiple cycles specified in step S41, a correlation coefficient between the two cycles constituting the cycle pair is calculated. In this case, the motion artifact correction of the first embodiment may be performed separately.

[0281] (S46: Set the pixel values ​​of strips whose correlation coefficient is below the threshold to zero) The image data processing unit 231A (for example, the information recording unit 234) compares each correlation coefficient calculated in step S45 with the first threshold value described above to identify correlation coefficients whose values ​​are smaller than the first threshold value. This identifies a subset of correlation coefficients whose values ​​are smaller than the first threshold value in the correlation coefficient set generated in step S45.

[0282] Furthermore, the image data processing unit 231A (e.g., the image data correction unit 233 or the information recording unit 234) sets the pixel value of the pixel at the scanning position corresponding to each element (low-value correlation coefficient) of this subset to zero. In some exemplary aspects, the pixel value of each pixel in a strip corresponding to at least one low-value correlation coefficient may be set to zero. In some exemplary aspects, the pixel value of each pixel in a strip where all corresponding correlation coefficients are low may be set to zero. In some exemplary aspects, the pixel value of each pixel in a strip where the number of corresponding low-value correlation coefficients is equal to or greater than a predetermined number may be set to zero. In these examples, the strips where pixel values ​​are set to zero are represented as solid black and correspond to the "unimaged portion" in the first embodiment. Note that the processing of this step is not limited to these examples.

[0283] (S47: Record the position of the pixel whose pixel value is set to zero (scanning position information)) The image data processing unit 231A (information recording unit 234) records the positions of the pixels whose pixel values ​​have been set to zero in step S46 as scanning position information.

[0284] (S48: Complement pixels with a pixel value of zero) The image data processing unit 231A (pixel value determination unit 235) interpolates the pixels recorded as scanning position information in step S47 (i.e., the pixels whose pixel values ​​were set to zero in step S46). The pixels to be subjected to this interpolation process may be one or more of all the pixels recorded as scanning position information in step S47. For example, all pixels that can be subjected to this interpolation process can be interpolated. In this step, the pixel value (zero) of at least one pixel in the strip that was represented as solid black is replaced based on the pixel values ​​of one or more pixels (neighboring pixels) located nearby.

[0285] (S49: Display image) The image data processing unit 231A constructs three-dimensional image data (final volume, final merged image) based on all strips including the strips complemented in step S48, and constructs a rendering image thereof. The main control unit 211 causes the display unit 241 to display this rendering image (END).

[0286] The ophthalmic examination apparatus of this embodiment includes, in addition to the data set collection unit, correlation coefficient calculation unit, and information recording unit similar to those of the first embodiment, a first pixel value determination unit that determines the value of a pixel corresponding to the scanning position information recorded by the information recording unit based on the values ​​of one or more neighboring pixels. In this embodiment, the pixel value determination unit 235 corresponds to the first pixel value determination unit.

[0287] According to the ophthalmic examination apparatus of this embodiment, as in the first embodiment, it is possible to record scanning position information that was not recorded in conventional techniques, and moreover, it is possible to record scanning position information according to a correlation coefficient. Furthermore, according to the ophthalmic examination apparatus of this embodiment, it is possible to perform image interpolation based on the recorded scanning position information. Therefore, it is possible to provide a new method of using data acquired by scanning imaging.

[0288] <Third embodiment> A third embodiment will now be described. The ophthalmic examination apparatus of this embodiment has a configuration similar to that of the first embodiment shown in Figures 1 to 4A, and includes an image data processing unit 231B shown in Figure 11 instead of the image data processing unit 231 shown in Figure 4B. The image data processing unit 231B of this embodiment is obtained by adding a correlation coefficient map creation unit 236 and a pixel value determination unit 237 to the image data processing unit 231 of the first embodiment.

[0289] As in the second embodiment, the information recording unit 234 of this embodiment is configured to identify a subset having elements with correlation coefficients equal to or less than a predetermined threshold from a set of correlation coefficients calculated from a data set collected by optical scanning such as a Lissajous scan, and record scanning position information corresponding to this subset. The ophthalmic examination apparatus of this embodiment has functional elements that perform a process of replacing low-quality parts in the image data constructed by the image data processing unit 231B (for example, the final three-dimensional image data generated by the image data correction unit 233 or the merged image generated by the merge processing unit 2318) with higher-quality pixels, and / or a process of complementing parts that were not imaged in this image data.

[0290] More specifically, the ophthalmic examination apparatus of this embodiment is configured to create a map (referred to as a correlation coefficient map) that represents the distribution of correlation coefficients and to determine pixel values ​​based on this correlation coefficient map. An example of a functional element that creates the correlation coefficient map is the correlation coefficient map creation unit 236, and an example of a functional element that determines pixel values ​​is the pixel value determination unit 237.

[0291] The correlation coefficient map creation unit 236 is configured to create a correlation coefficient map based on a set of correlation coefficients acquired by the image data processing unit 231B (e.g., the correlation coefficient calculation unit 2315). The correlation coefficient map is information that represents the distribution of correlation coefficients in at least a part of an area to which optical scanning such as Lissajous scanning has been applied. The correlation coefficient map creation unit 236 creates the correlation coefficient map, for example, by associating each correlation coefficient included in the set of correlation coefficients with the pixel position (scanning position) corresponding to that correlation coefficient. The correlation coefficient map creation unit 236 is realized by cooperation between hardware including a processor and correlation coefficient map creation processing software.

[0292] The pixel value determination unit 237 is configured to determine the value of a pixel corresponding to a scanning position belonging to the scanning position information recorded by the information recording unit 234, based on the correlation coefficient map created by the correlation coefficient map creation unit 236, and based on the values ​​of one or more neighboring pixels. The pixel value determination unit 237 is realized by cooperation between hardware including a processor and pixel value determination processing software.

[0293] The pixel value determination unit 237 may perform any method of pixel value determination processing. For example, the pixel value determination unit 237 may be capable of performing any interpolation (interpolation) and / or any extrapolation, as in the second embodiment. An example of processing performed by the pixel value determination unit 237 will be described in an operation example described later.

[0294] The operation of the ophthalmic examination apparatus according to this embodiment will now be described. An example of the operation of the ophthalmic examination apparatus is shown in Fig. 12. Unless otherwise specified, the matters described in the first embodiment and the second embodiment can be applied to this example of operation.

[0295] (S51: Specify the number of cycles) After inputting the patient ID and setting the scan mode (designating the Lissajous scan), the number of cycles in the Lissajous scan to be applied to the subject's eye E can be designated as an optional step. The number of cycles corresponds to the density of the scan. The number of cycles is designated, for example, by the user or the ophthalmic examination apparatus.

[0296] The number of cycles specified in this step is set to G (G is a positive integer). The number of cycles G may be any number, for example, 1024. As described in the first embodiment, the G cycles are ordered, and the gth cycle is represented by Cg (g=1, 2, . . . , G).

[0297] (S52: Perform preparation operations) The ophthalmic examination apparatus performs preparatory operations similar to those of the conventional method, such as presenting a fixation target, alignment, focus adjustment, and OCT optical path length adjustment. The ophthalmic examination apparatus may be configured to determine a judgment condition (e.g., the above-mentioned first threshold) for identifying a subset of the correlation coefficient set based on image data acquired in the preparatory operations. Note that a default judgment condition may be used, or a judgment condition determined from the data set collected by the Lissajous scan in step S53 may be used.

[0298] (S53: Perform Lissajous scan) Upon receiving a predetermined scan start trigger signal, the scan control unit 2111 applies a Lissajous scan consisting of G cycles C1 to CG specified in step S51 to the subject's eye E (fundus oculi Ef).

[0299] (S54: Build a strip for each cycle) The image data constructing unit 220 constructs strips corresponding to each cycle Cg specified in step S51 based on the data set collected by the Lissajous scan in step S53. This process may be performed, for example, in the same manner as in the first embodiment (step S2 in FIG. 6A).

[0300] For each cycle Cg specified in step S51, the strip corresponding to this cycle Cg is stored in a storage area Ag associated with this cycle Cg. G storage areas A1 to AG associated with the G cycles C1 to CG, respectively, are provided in the storage unit 212, for example.

[0301] (S55: Calculate the correlation coefficient for each cycle pair) The image data processing unit 231B (correlation coefficient calculation unit 2315, z shift amount calculation unit 232, etc.) calculates the correlation coefficient for each cycle pair formed based on the G cycles C1 to CG specified in step S51.

[0302] The processing in this step calculates, for example, a correlation coefficient between two cycles (cycle pair) Cg1 and Cg2 selected from G cycles C1 to CG (where g1=1 to G, g2=1 to G, and g1≠g2). In some exemplary aspects, the image data processing unit 231B sets one of the G cycles C1 to CG as a reference cycle and calculates correlation coefficients between this reference cycle and each of the other cycles.

[0303] As a specific example, the image data processing unit 231B sets the first cycle C1 as the reference cycle, and calculates the correlation coefficients between this reference cycle C1 and each of the other cycles C2 to CG.<C1、C2> ,<C1、C3> ,...,<C1、CG> G-1 correlation coefficients R2, R3, ..., RG corresponding to the above are obtained. Here,<C1、Cg> indicates a cycle pair consisting of the first cycle and the gth cycle (g=2 to G).

[0304] Cycle Pair<C1、Cg> The correlation coefficient Rg corresponding to this cycle Cg (or this cycle pair<C1、Cg> The G-1 storage areas B1 to BG associated with the G-1 correlation coefficients R2 to RG, respectively, are provided in the storage unit 212, for example.

[0305] In this way, the set of correlation coefficients generated in this example includes G-1 correlation coefficients R2 to RG, and each correlation coefficient Rg (g=2 to G) corresponds to the corresponding cycle Cg or the corresponding cycle pair Cg.<C1、Cg> is associated with.

[0306] (S56: Build image data) The image data processing unit 231B constructs image data based on the data set collected by the Lissajous scan in step S53. The processing in this step may be performed in the same manner as the combined processing of image data construction and motion artifact correction in the first embodiment (for example, steps S3 to S14 in FIGS. 6A and 6B, steps S3 to S25 in FIGS. 6A to 6D, or steps S3 to S32 in FIGS. 6A to 6E). The image data generated in this step is stored in the storage unit 212, for example.

[0307] (S57: Create a correlation coefficient map) The correlation coefficient map creating unit 236 creates a correlation coefficient map that represents the distribution of correlation coefficients in the area where the Lissajous scan is applied in step S53, based on the correlation coefficient set acquired in step S55. For example, the correlation coefficient map may include correlation coefficients Rg (g=2 to G) that belong to the correlation coefficient set, and cycles Cg or cycle pairs<C1、Cg> It is information that expresses the association between

[0308] An example of this step will be described below. Note that although this example describes the case of assigning correlation coefficients to pixels of two-dimensional image data, correlation coefficients may also be assigned to pixels of three-dimensional image data. All of the various image data handled in this embodiment are based on data sets collected by the same Lissajous scan, and therefore it is possible to naturally associate the assignment of correlation coefficients to pixels of certain image data with the assignment of correlation coefficients to pixels of other image data.

[0309] The correlation coefficient map in this example is created by assigning corresponding correlation coefficients to pixels of the image data (merged image). The correlation coefficients assigned to pixels are called pixel correlation coefficients. Here, one pixel correlation coefficient may be assigned to one pixel, or one pixel correlation coefficient may be assigned to two or more pixels (pixel groups). Examples of the latter include assigning the same pixel correlation coefficient to each pixel in the intersection area (common area) between the reference cycle C1 and the cycle Cg, or assigning the same pixel correlation coefficient to each pixel in the area of ​​the cycle Cg excluding this intersection area (non-intersection area).

[0310] The pixel correlation coefficient in this example is determined based on the correlation coefficient Rg calculated in step S55. For example, as shown in FIG. 13, a cycle pair consisting of two arbitrary cycles Cg(1) and Cg(2)<Cg(1)、Cg(2)> For (the corresponding strip pair), let D(g(1), g(2)) denote the intersection state of two cycles Cg(1) and Cg(2) (the corresponding strip pair), where g(1) = 2~G, g(2) = 2~G, and g(1) ≠ g(2).

[0311] 13, the intersection regions of the two cycles Cg(1) and Cg(2) are indicated by diagonal lines. In this example, the larger of the correlation coefficient of cycle Cg(1) and the correlation coefficient of cycle Cg(2) is assigned to each pixel belonging to each intersection region. Furthermore, the correlation coefficient of cycle Cg(1) is assigned to each pixel belonging to the non-intersection region of cycle Cg(1) (i.e., the part of cycle Cg(1) excluding these intersection regions), and the correlation coefficient of cycle Cg(2) is assigned to each pixel belonging to the non-intersection region of cycle Cg(2) (i.e., the part of cycle Cg(2) excluding these intersection regions).

[0312] For example, if the correlation coefficient of cycle Cg(1) is 0.5 and the correlation coefficient of cycle Cg(2) is 0.9, then max{0.5, 0.9} = 0.9 is assigned to each pixel belonging to the intersection region, 0.5 is assigned to each pixel belonging to the non-intersection region of cycle Cg(1), and 0.9 is assigned to each pixel belonging to the non-intersection region of cycle Cg(2).

[0313] In this example, the method of assigning correlation coefficients to each pixel in the reference cycle C1 may be arbitrary. For example, a cycle other than cycle C1 may be set as the reference cycle and similar processing may be performed to assign correlation coefficients to each pixel belonging to cycle C1. Alternatively, the value of the correlation coefficient assigned to the pixels belonging to the reference cycle C1 may be determined from the correlation coefficients assigned to the pixels belonging to cycles C2 to C-G.

[0314] By performing this process for each cycle pair, a correlation coefficient map can be created.

[0315] In some exemplary embodiments, in order to assign correlation coefficients to all pixels of image data (merged image) based on the Lissajous scan, the process as in the above example may be performed while changing the combination of cycle pairs in various ways. For example, the following series of processes may be performed.

[0316] First, a certain cycle Cg(1) is set as the reference cycle, and the process in the above example is executed for each pair of this reference cycle Cg(1) and other cycles Cg(2) to Cg(G).

[0317] Next, all cycles are paired (i.e., multiple pairs are set) so as to cover all pixels of the image data (merged image) based on the Lissajous scan. Furthermore, the process in the above example is performed for each set cycle pair.

[0318] Next, similar to the image data construction process of step S56 (the sequential process executed in steps S3 to S14 in FIGS. 6A and 6B of the first embodiment), the strips corresponding to these cycles are sequentially (recursively) merged, and the process of the above example is sequentially (recursively) executed. A correlation coefficient is assigned to every pixel of the image data constructed in this way.

[0319] (S58: Set the pixel values ​​of strips whose correlation coefficient is below the threshold to zero) The image data processing unit 231B (for example, the information recording unit 234) compares the value of each correlation coefficient recorded in the correlation coefficient map created in step S57 with the first threshold value described above, thereby identifying correlation coefficients whose values ​​are smaller than the first threshold value. As a result, of the correlation coefficients recorded in the correlation coefficient map created in step S57, correlation coefficients whose values ​​are smaller than the first threshold value are identified. In other words, a subset of the correlation coefficient set is identified.

[0320] Furthermore, the image data processing unit 231B (e.g., the image data correction unit 233 or the information recording unit 234) sets the pixel values ​​of the image data corresponding to the positions in the correlation coefficient map corresponding to each element (low correlation coefficient) of this subset to zero.

[0321] (S59: Record the position of the pixel whose pixel value is set to zero (scanning position information)) The image data processing unit 231B (information recording unit 234) records the positions of the pixels whose pixel values ​​have been set to zero in step S58 as scanning position information.

[0322] (S60: Complement pixels with a pixel value of zero) The image data processing unit 231B (pixel value determination unit 237) interpolates the pixels recorded as scanning position information in step S59 (i.e., the pixels whose pixel values ​​were set to zero in step S58). In this step, the pixel value (zero) of at least one pixel in the strip that was previously displayed as solid black is replaced based on the pixel values ​​of one or more pixels (neighboring pixels) located nearby.

[0323] An example of processing executed by the pixel value determination unit 237 will be described. The position (coordinates) of the pixel to be interpolated is (x i , y i ), and the pixel value (brightness value) of this target pixel is expressed as I(x i , y i ) and the corresponding correlation coefficient is R(x i , y i ) is expressed as

[0324] First, calculate a provisional luminance value of the target pixel before normalization (standardization). In this example, four pixels located above, below, left, and right of the target pixel are considered as neighboring pixels of the target pixel. As with the expression for the target pixel, the positions (coordinates) of the neighboring pixels are expressed as (x j , y j ), and the pixel value (brightness value) of this target pixel is expressed as I(x j , y j ) and the corresponding correlation coefficient is R(x j , y j ) is expressed as

[0325] In this example, the provisional luminance value T(x i , y i ) is calculated using the following formula: T(x i , y i )=I(x i , y i )×R(x i , y i )+I(x i+1 , y i+1 )×R(x i+1 , y i+1 )+I(x i+1 , y i-1 )×R(x i+1 , y i-1 )+I(x i-1 , y i+1 )×R(x i-1 , y i+1 )+I(x i-1 , y i-1 )×R(x i-1 , y i-1This formula represents the sum of the products of the luminance value I and the correlation coefficient R for the target pixel and its four neighboring pixels.

[0326] The correlation coefficient R(x i , y i ) is zero, so it is also possible to use the formula: T(x i , y i )=I(x i+1 , y i+1 )×R(x i+1 , y i+1 )+I(x i+1 , y i-1 )×R(x i+1 , y i-1 )+I(x i-1 , y i+1 )×R(x i-1 , y i+1 )+I(x i-1 , y i-1 )×R(x i-1 , y i-1 ).

[0327] Next, the calculated provisional brightness value T(x i , y i ) by the sum of the correlation coefficients of the five pixels consisting of the target pixel and its four neighboring pixels. Then, the luminance value I(x i , y i ) = 0. The luminance value of the target pixel interpolated in this way is expressed as follows: I(x i , y i )=T(x i , y i ) / {R(x i , y i )+R(x i+1 , y i+1 )+R(x i+1 , y i-1 )+R(x i-1 , y i+1 )+R(x i-1 , y i-1 )}=T(x i , y i) / {R(x i+1 , y i+1 )+R(x i+1 , y i-1 )+R(x i-1 , y i+1 )+R(x i-1 , y i-1 )}.

[0328] By applying this processing to each pixel recorded in the scanning position information (i.e., each pixel whose pixel value is set to zero), it is possible to complement areas where no image information was present.

[0329] (S61: Display image) The image data processing unit 231B constructs a rendering image of the image data (for example, a final volume or a final merged image) interpolated in step S60. The main control unit 211 causes the display unit 241 to display this rendering image (END).

[0330] The ophthalmic examination apparatus of this embodiment includes, in addition to the data set collection unit, correlation coefficient calculation unit, and information recording unit similar to those of the first embodiment, a map creation unit that creates a correlation coefficient map that represents the distribution of correlation coefficients in the area where optical scanning is applied, based on the set of correlation coefficients acquired by the correlation coefficient calculation unit. In this embodiment, the correlation coefficient map creation unit 236 corresponds to the correlation coefficient map.

[0331] As with the first embodiment, the ophthalmic examination apparatus of this embodiment can record scanning position information, which was not recorded in conventional techniques, and can also record scanning position information according to the correlation coefficient. Furthermore, the ophthalmic examination apparatus of this embodiment can create a correlation coefficient map based on the recorded scanning position information. This makes it possible to grasp the distribution state of the correlation coefficients, analyze the distribution state of the correlation coefficients, and provide the distribution state of the correlation coefficients to the user, thereby providing a new method for using data acquired by scanning imaging.

[0332] The ophthalmic examination apparatus of this embodiment further includes a second pixel value determination unit that determines the value of a pixel corresponding to the scanning position information recorded by the information recording unit based on the correlation coefficient map. In this embodiment, the pixel value determination unit 237 corresponds to the second pixel value determination unit.

[0333] According to the ophthalmic examination apparatus of the present embodiment, it is possible to perform image interpolation based on the correlation coefficient map, and it is possible to provide a new method of using data acquired by scanning imaging. Note that in some exemplary aspects, the second pixel value determination unit may not be provided.

[0334] <Fourth embodiment> A fourth embodiment will now be described. The ophthalmic examination apparatus of this embodiment has a configuration similar to that of the first embodiment shown in Figures 1 to 4A, and includes an image data processing unit 231C shown in Figure 14 instead of the image data processing unit 231 shown in Figure 4B. The image data processing unit 231C of this embodiment is obtained by adding a scanning area setting unit 238 and an image synthesis processing unit 239 to the image data processing unit 231 of the first embodiment.

[0335] As in the second and third embodiments, the information recording unit 234 of this embodiment is configured to identify a subset having elements with correlation coefficients equal to or less than a predetermined threshold from a set of correlation coefficients calculated from a data set collected by optical scanning such as a Lissajous scan, and record scanning position information corresponding to this subset. The ophthalmic examination apparatus of this embodiment is equipped with a functional element (i.e., a functional element for performing rescanning) that recollects data of low-quality parts or parts that were not imaged in the image data constructed by the image data processing unit 231C (e.g., the final three-dimensional image data generated by the image data correction unit 233 or the merged image generated by the merge processing unit 2318).

[0336] The rescanning mode in this embodiment may be any mode, for example, a Lissajous scan or another scan mode. When a Lissajous scan is performed in the rescanning, the rescanning may include, for example, a cycle with a low correlation coefficient identified based on the scanning position information. When a rescanning is performed in a scan mode different from the Lissajous scan, the rescanning may be, for example, a three-dimensional scan (such as a raster scan) that includes a portion with a low correlation coefficient identified based on the scanning position information (i.e., a low-quality portion or a portion not imaged in the image data constructed by the image data processing unit 231C).

[0337] The ophthalmic examination apparatus of this embodiment is configured to apply a new optical scan to the subject's eye E based on the scanning position information recorded by the information recording unit. The scanning area setting unit 238 is configured to set an area (referred to as a re-scanning area) to which this new optical scan is applied based on the scanning position information. Setting of the re-scanning area may be performed manually and / or automatically.

[0338] In some exemplary aspects when the rescanning area is set manually, the main control unit 211 causes the display unit 241 to display information based on the scanning position information (e.g., the correlation coefficient map of the third embodiment) and / or an image constructed based on the Lissajous scan (e.g., an image constructed by the image data corrector 233) together with a graphical user interface (GUI) for setting the rescanning area. The user determines a desired rescanning area by referring to the displayed information and / or image, and inputs this rescanning area using the operation unit 242 and the GUI. The scanning area setting unit 238 may be configured, for example, to compare the rescanning area input by the user with the scanning position information and / or the scanning protocol 2121, correct the rescanning area as necessary, and set the final rescanning area.

[0339] In some exemplary embodiments when the re-scanning area is set automatically, the scan area setting unit 238 may be configured to set the re-scanning area based on, for example, the scan position information and the scan protocol 2121.

[0340] Note that the scanning position information in this embodiment is information indicating portions of an image with low quality or portions that have not been imaged. The scanning area setting unit 238 sets the re-scanning area so as to include such portions. In the case of manual setting, the scanning area setting unit 238 determines, for example, whether portions of an image with low quality or portions that have not been imaged, indicated in the scanning position information, are included in the re-scanning area specified by the user, and if it is determined that they are not included, the re-scanning area is modified so as to include those portions. In the case of automatic setting, the scanning area setting unit 238 sets the re-scanning area so as to include, for example, portions of an image with low quality or portions that have not been imaged, indicated in the scanning position information.

[0341] The image synthesis processing unit 239 is configured to cooperate with other elements of the image data processing unit 231C to process a data set (referred to as the first data set) collected in the first optical scan (Lissajous scan) and a data set (referred to as the second data set) collected in the rescan to construct image data.

[0342] For example, the image synthesis processing unit 239 is configured to synthesize (merge) a first data set or image data based thereon (referred to as first image data) with a second data set or image data based thereon (referred to as second image data). The first image data includes areas of low quality or areas not imaged, as indicated in the scanning position information. On the other hand, the second image data is data constructed from a second data set collected in a rescan that is performed to include such areas. By synthesizing such first image data and second image data, it is possible to complement the first image data with the second image data.

[0343] In some exemplary aspects, the image data processing unit 231C may be configured to execute the following three processing steps. First, the image data processing unit 231C applies the same processing as in the first embodiment to the first data set to construct first image data corresponding to an area obtained by excluding at least a portion of the area corresponding to the scanning position information from the area to which the initial Lissajous scan is applied (first processing step). The image data processing unit 231C also applies the same processing as in the first embodiment to the second data set to construct second image data corresponding to the re-scanned area (second processing step). Furthermore, the image data processing unit 231C (image synthesis processing unit 239) synthesizes the first image data constructed in the first processing step with the second image data constructed in the second processing step (third processing step). By executing these processing steps, the first image data and the second image data can be suitably synthesized. For example, by setting the construction area of ​​the first image data so that there is an overlapping area between the first image data and the second image data, and aligning the area in the first image data corresponding to this overlapping area with the area in the second image data corresponding to this overlapping area, it is possible to improve the accuracy and precision of the registration between the first image data and the second image data.

[0344] The operation of the ophthalmic examination apparatus according to this embodiment will now be described. An example of the operation of the ophthalmic examination apparatus is shown in Fig. 15. Unless otherwise stated, the matters described in the first to third embodiments can be applied to this example of operation.

[0345] (S71: Specify the number of cycles) After inputting the patient ID and setting the scan mode (designating the Lissajous scan), the number of cycles in the Lissajous scan to be applied to the subject's eye E can be designated as an optional step. The number of cycles corresponds to the density of the scan. The number of cycles is designated, for example, by the user or the ophthalmic examination apparatus.

[0346] The number of cycles specified in this step is set to G (G is a positive integer). The number of cycles G may be any number, for example, 1024. As described in the first embodiment, the G cycles are ordered, and the gth cycle is represented by Cg (g=1, 2, . . . , G).

[0347] (S72: Perform preparation operations) The ophthalmic examination apparatus performs preparatory operations similar to those of the conventional method, such as presenting a fixation target, alignment, focus adjustment, and OCT optical path length adjustment. The ophthalmic examination apparatus may be configured to determine a judgment condition (e.g., the above-mentioned first threshold) for identifying a subset of the correlation coefficient set based on image data acquired in the preparatory operations. Note that a default judgment condition may be used, or a judgment condition determined from the data set collected by the Lissajous scan in step S73 may be used.

[0348] (S73: Perform Lissajous scan) Upon receiving a predetermined scan start trigger signal, the scan control unit 2111 applies a Lissajous scan consisting of G cycles C1 to CG specified in step S71 to the subject's eye E (fundus oculi Ef).

[0349] (S74: Build a strip for each cycle) The image data constructing unit 220 constructs strips corresponding to each cycle Cg specified in step S71 based on the data set collected by the Lissajous scan in step S73. This process may be performed, for example, in the same manner as in the first embodiment (step S2 in FIG. 6A).

[0350] For each cycle Cg specified in step S71, the strip corresponding to this cycle Cg is stored in a storage area Ag associated with this cycle Cg. G storage areas A1 to AG associated with the G cycles C1 to CG, respectively, are provided in the storage unit 212, for example.

[0351] (S75: Calculate the correlation coefficient for each cycle pair) The image data processing unit 231C (correlation coefficient calculation unit 2315, z shift amount calculation unit 232, etc.) calculates the correlation coefficient for each cycle pair formed based on the G cycles C1 to CG specified in step S71.

[0352] The processing in this step calculates, for example, a correlation coefficient between two cycles (cycle pair) Cg1 and Cg2 selected from G cycles C1 to CG (where g1=1 to G, g2=1 to G, and g1≠g2). In some exemplary aspects, the image data processing unit 231C sets one of the G cycles C1 to CG as a reference cycle and calculates correlation coefficients between this reference cycle and each of the other cycles.

[0353] As a specific example, the image data processing unit 231C sets the first cycle C1 as the reference cycle, and calculates the correlation coefficients between this reference cycle C1 and each of the other cycles C2 to CG.<C1、C2> ,<C1、C3> ,...,<C1、CG> G-1 correlation coefficients R2, R3, ..., RG corresponding to the above are obtained. Here,<C1、Cg> indicates a cycle pair consisting of the first cycle and the gth cycle (g=2 to G).

[0354] Cycle Pair<C1、Cg> The correlation coefficient Rg corresponding to this cycle Cg (or this cycle pair<C1、Cg> The G-1 storage areas B1 to BG associated with the G-1 correlation coefficients R2 to RG, respectively, are provided in the storage unit 212, for example.

[0355] In this way, the set of correlation coefficients generated in this example includes G-1 correlation coefficients R2 to RG, and each correlation coefficient Rg (g=2 to G) corresponds to the corresponding cycle Cg or the corresponding cycle pair Cg.<C1、Cg> is associated with.

[0356] (S76: Record the identifier of the cycle where the correlation coefficient is below the threshold) The image data processing unit 231C (information recording unit 234) compares the value of the correlation coefficient of each cycle pair calculated in step S75 with the first threshold value described above, thereby identifying correlation coefficients whose values ​​are smaller than the first threshold value. As a result, correlation coefficients whose values ​​are smaller than the first threshold value are identified from among the elements (correlation coefficients) of the correlation coefficient set acquired in step S75. In other words, a subset of the correlation coefficient set is identified.

[0357] The image data processing unit 231C (information recording unit 234) records the scanning position information corresponding to this subset. This scanning position information is represented by a predetermined identifier. This identifier may be, for example, any of a cycle number, a strip number, and coordinates in a defined coordinate system of the image data.

[0358] (S77: Rescan) The scan area setting unit 238 sets a re-scan area based on the scan position information (cycle identifier) ​​generated in step S76. The scan control unit 2111 applies a re-scan to the eye E based on this re-scan area.

[0359] (S78: Build a strip for each cycle of rescanning) The image data constructing unit 220 constructs strips corresponding to each cycle of the rescanning based on the data set collected in the rescanning in step S77. This process may be performed, for example, in the same manner as in the first embodiment (step S2 in FIG. 6A).

[0360] (S79: Each strip based on the rescan replaces the corresponding strip based on the Lissajous scan) The ophthalmic examination apparatus (for example, the main control unit 211 or the image data processing unit 231C) identifies a corresponding strip from among the strips constructed in step S74 for each strip constructed in step S78. Here, the correspondence between the strips based on the Lissajous scan and the strips based on the rescan can be determined by referring to the cycle identifier.

[0361] For each strip constructed in step S78, the ophthalmic examination device (e.g., the main control unit 211 or the image data processing unit 231C) replaces the identified corresponding strip (strip based on the Lissajous scan) with the strip (strip based on the rescan).

[0362] This allows low-quality portions of the image obtained by the Lissajous scan (strips corresponding to cycles with low correlation coefficients) to be replaced with images obtained by rescanning. Note that a similar correlation coefficient calculation may be performed based on the strips obtained by rescanning, and the quality may be reevaluated. If the quality of the strips obtained by rescanning is low, further scanning may be performed.

[0363] (S80: Constructing 3D image data) The image data processing unit 231C constructs three-dimensional image data based on the set of strips obtained in step S79 (i.e., a set consisting of high-quality strips based on the Lissajous scan and strips based on the rescan). The processing in this step may be performed, for example, in the same manner as the combined processing of image data construction and motion artifact correction in the first embodiment (e.g., steps S3 to S32 in FIGS. 6A to 6E). The image data generated in this step is stored in, for example, the storage unit 212.

[0364] (S81: Display image) The image data processing unit 231C constructs a rendering image of the three-dimensional image data interpolated in step S80. The main control unit 211 causes the display unit 241 to display this rendering image (END).

[0365] The ophthalmic examination apparatus of this embodiment includes a data set collection unit, a correlation coefficient calculation unit, and an information recording unit similar to those of Embodiment 1. The data set collection unit of this embodiment is configured to apply a first optical scan (such as a Lissajous scan) to the subject's eye according to a two-dimensional pattern including a series of cycles that intersect with each other, and to apply a second optical scan (rescan) to the subject's eye based on the scanning position information recorded by the information recording unit.

[0366] According to the ophthalmic examination apparatus of this embodiment, as in the first embodiment, it is possible to record scanning position information that was not recorded in conventional technology, and moreover, it is possible to record scanning position information according to the correlation coefficient. Furthermore, according to the ophthalmic examination apparatus of this embodiment, it is possible to perform a second optical scan (rescan) based on the recorded scanning position information. This makes it possible to perform the second optical scan based on the information obtained by the first optical scan, and it is possible to provide a new method of using data acquired by scanning imaging.

[0367] Furthermore, the data set collection unit of the ophthalmic examination apparatus of this embodiment may be configured to apply the second optical scan to an area set based on the scanning position information recorded by the information recording unit.

[0368] According to the ophthalmic examination apparatus of this embodiment, it is possible to automatically set a re-scanning area from information obtained by the first optical scan and to perform the second optical scan based on this re-scanning area. In this embodiment, the automatic setting of the re-scanning area is executed by the scan area setting unit 238.

[0369] In some exemplary aspects, the re-scan area may be set manually. For example, the ophthalmic examination apparatus of some exemplary aspects may be configured to display the correlation coefficient map of the third embodiment or information based thereon, and to perform a re-scan using a re-scan area specified by the user based on this display.

[0370] Furthermore, the ophthalmic examination apparatus of this embodiment may be configured to use a data set collected by a rescan to construct image data. To this end, the ophthalmic examination apparatus of this embodiment may include a processing unit that processes a first data set collected by a first optical scan (such as a Lissajous scan) and a second data set collected by a second optical scan (such as a rescan) to construct image data. This processing unit includes an image synthesis processing unit 239 and may further include other elements of the image data processing unit 231C.

[0371] According to the ophthalmic examination apparatus of this embodiment, it is possible to construct image data by combining a data set collected by the first optical scan and a data set collected by the second optical scan, thereby making it possible to obtain image data of higher quality.

[0372] Furthermore, the processing unit of the ophthalmic examination apparatus of this embodiment may be capable of executing the following series of steps. In a first step, the processing unit processes a first data set collected by the first optical scanning to construct first image data of an area (referred to as a target area) obtained by excluding at least a portion of an area corresponding to the scanning position information recorded by the information recording unit from the application area of ​​the first optical scanning. In a second step, the processing unit processes a second data set collected by the second optical scanning to construct second image data of an area set based on the scanning position information (i.e., a re-scanning area set by the scanning area setting unit 238). In a third step, the processing unit combines the first image data constructed in the first step with the second image data constructed in the second step.

[0373] According to the ophthalmic examination apparatus of this embodiment, it is possible to generate composite image data from first image data constructed from a data set collected by a first optical scan and second image data constructed from a data set collected by a second optical scan, thereby making it possible to obtain image data of higher quality.

[0374] <Fifth embodiment> A fifth embodiment will be described. In the first to fourth embodiments, a scanning imaging device configured to be able to perform optical scanning has been described, but in this embodiment, an information processing device configured to receive from the outside a data set collected by optical scanning and process the data set will be described.

[0375] The information processing device of this embodiment may be configured as a part of a scanning imaging device, but such a case will be excluded from the following description. However, it will be understood by those skilled in the art that any matter in the following description can also be applied to an information processing device configured as a part of a scanning imaging device.

[0376] The information processing device of this embodiment may be configured in the same manner as the scanning imaging device (ophthalmic examination device) of the first to fourth embodiments, except that it does not have a functional element that performs optical scanning (the data set collection unit in the first to fourth embodiments).

[0377] Any of the items described in the first to fourth embodiments can be combined with the information processing device of this embodiment, and the information processing device will achieve functions and effects according to the combined items.

[0378] An example of the configuration of the information processing device of this embodiment is shown in Fig. 16. This information processing device 300 includes a receiving unit 301, a correlation coefficient calculation unit 302, and an information recording unit 303.

[0379] The receiving unit 301 receives a data set collected by applying an optical scan that follows a two-dimensional pattern including a series of cycles that intersect with each other to a sample (e.g., an eye to be inspected). That is, the receiving unit 301 receives a data set collected from the sample by a data collecting unit of a scanning imaging device that can perform optical scanning such as a Lissajous scan.

[0380] The reception unit 301 has a function of acquiring data from an external device. This external device may be, for example, a computer, a storage device, a recording medium, or an information system (for example, a hospital information system, an electronic medical record system, or an image archiving system). The external device may be, for example, a device directly connected to the information processing device 300, a device connected to the information processing device 300 via a local area network (LAN), or a device connected to the information processing device 300 via a wide area network (WAN). The reception unit 301 may include, for example, a communication interface for performing data communication via a communication line and / or a drive device for reading data from a recording medium.

[0381] The correlation coefficient calculation unit 302 is configured to calculate a correlation coefficient between cycles based on a data set externally received by the reception unit 301. The correlation coefficient calculation unit 302 may be configured similarly to the image data processing unit 231 of the first embodiment.

[0382] In some exemplary aspects, the correlation coefficient calculation unit 302 may include the image data construction unit 220, mask image generation unit 2311, range adjustment unit 2312, composite image generation unit 2313, cross-correlation function calculation unit 2314, and correlation coefficient calculation unit 2315 of the first embodiment, and may further include a z-shift amount calculation unit 232. Note that when the data set accepted by the acceptance unit 301 is image data (e.g., a strip), the correlation coefficient calculation unit 302 may not have a functional element corresponding to the image data construction unit 220.

[0383] The information recording unit 303 records scanning position information corresponding to a subset having correlation coefficients satisfying predetermined conditions as elements from the set of correlation coefficients calculated by the correlation coefficient calculation unit 302 from the data set accepted by the acceptance unit 301. The information recording unit 303 may be configured similarly to the information recording unit 234 of the first embodiment.

[0384] Although not shown, the information processing device 300 of this embodiment may include at least one of the pixel value determination unit 235 of the second embodiment, the correlation coefficient map creation unit 236 (and pixel value determination unit 237) of the third embodiment, and the scanning area setting unit 238 (and image synthesis processing unit 239) of the fourth embodiment. Note that functional elements that can be combined with the information processing device 300 of this embodiment are not limited to these, and may be any of the items described in the first to fourth embodiments.

[0385] According to the information processing device 300, it is possible to record scanning position information that has not been recorded in conventional techniques, and moreover, it is possible to record scanning position information according to a correlation coefficient, so that it is possible to provide a new method of using data acquired by scanning imaging. Some non-limiting examples of this method of use are described in the first to fourth embodiments.

[0386] <Other embodiments> The embodiments of the present disclosure are not limited to the first to fifth embodiments and their modifications. For example, the first to fifth embodiments provide embodiments relating to various methods, embodiments relating to various programs, embodiments relating to various recording media, and the like.

[0387] For example, the first embodiment provides an embodiment of a method for controlling a scanning imaging apparatus. This embodiment is a method for controlling a scanning imaging apparatus including a scanner that performs optical scanning and a processor that constructs image data from data collected by the optical scanning. The scanner may be the data set collection unit of the first embodiment, and the processor may be the image data construction unit 220 and the image data processing unit 231 (at least a part of the image data processing unit 231) of the first embodiment.

[0388] A control method for a scanning imaging apparatus according to this embodiment may include the following three steps: a first step is executed by controlling a scanner to apply an optical scan to a sample according to a two-dimensional pattern including a series of cycles that intersect with each other to collect a data set; a second step is executed by controlling a processor to calculate a correlation coefficient between the cycles based on the data set collected in the first step; and a third step is executed by controlling a processor to record scanning position information corresponding to a subset of the set of correlation coefficients calculated in the second step from the data set collected in the first step, the subset having correlation coefficients that satisfy a predetermined condition as elements.

[0389] Any of the features described in the first to fourth embodiments can be combined with the control method for the scanning imaging device according to this embodiment, and the control method for the scanning imaging device will have the functions and effects according to the combined features.

[0390] According to this control method for a scanning imaging device, it is possible to record scanning position information that has not been recorded in conventional techniques, and moreover, it is possible to record scanning position information according to a correlation coefficient, so that it is possible to provide a new method of using data acquired by scanning imaging. Some non-limiting examples of this method of use are described in the first to fourth embodiments.

[0391] The fifth embodiment provides an embodiment of a method for controlling an information processing device. This embodiment is a method for controlling an information processing device including a processor. This processor may be the image data creation unit 220 and the image data processing unit 231 (at least a part of the image data processing unit 231) of the first embodiment.

[0392] The control method for an information processing device according to this embodiment may include the following three steps. A first step is executed by controlling a processor to receive a data set collected by applying an optical scan according to a two-dimensional pattern including a series of cycles that intersect with each other to a sample. A second step is executed by controlling a processor to calculate a correlation coefficient between the cycles based on the data set received in the first step. A third step is executed by controlling a processor to record scanning position information corresponding to a subset of the set of correlation coefficients calculated in the second step from the data set received in the first step, the subset having correlation coefficients that satisfy a predetermined condition as elements.

[0393] Any of the items described in the first to fourth embodiments can be combined with the control method of the information processing device according to this embodiment, and the control method of the information processing device will have the effects and advantages according to the combined items.

[0394] According to this control method for an information processing device, it is possible to record scanning position information that has not been recorded in conventional techniques, and moreover, it is possible to record scanning position information according to a correlation coefficient, so that it is possible to provide a new method of using data acquired by scanning imaging. Some non-limiting examples of this method of use are described in the first to fourth embodiments.

[0395] The first embodiment and / or the fifth embodiment provide an embodiment of a method for processing information.

[0396] An information processing method according to this embodiment may include the following three steps: a first step is performed to acquire a data set collected from a sample by optical scanning according to a two-dimensional pattern including a series of cycles that intersect with each other; in the information processing method according to the first embodiment, the first step is performed to collect a data set by applying optical scanning according to a two-dimensional pattern including a series of cycles that intersect with each other to a sample; in the information processing method according to the fifth embodiment, the first step is performed to externally receive a data set collected by applying optical scanning according to a two-dimensional pattern including a series of cycles that intersect with each other to a sample; a second step is performed to calculate a correlation coefficient between cycles based on the data set acquired in the first step; and a third step is performed to record scanning position information corresponding to a subset of the set of correlation coefficients calculated in the second step from the data set acquired in the first step, the subset having correlation coefficients that satisfy a predetermined condition as elements.

[0397] Any of the items described in the first to fourth embodiments can be combined with the information processing method according to this embodiment, and the information processing method will achieve functions and effects according to the combined items.

[0398] According to this information processing method, it is possible to record scanning position information that has not been recorded in conventional techniques, and moreover, it is possible to record scanning position information according to the correlation coefficient, so that it is possible to provide a new method of using data acquired by scanning imaging. Some non-limiting examples of this method of use are described in the first to fourth embodiments.

[0399] The present disclosure provides a program for causing a computer to execute a method according to an embodiment. Some exemplary embodiments provide a program for causing a computer to execute a control method for a scanning imaging apparatus. Some exemplary embodiments provide a program for causing a computer to execute a control method for an information processing apparatus. Some exemplary embodiments provide a program for causing a computer to execute an information processing method.

[0400] The present disclosure provides a computer-readable non-transitory recording medium having a program recorded thereon according to an embodiment. Some exemplary embodiments provide a computer-readable non-transitory recording medium having a program recorded thereon that causes a computer to execute a control method for a scanning imaging apparatus. Some exemplary embodiments provide a computer-readable non-transitory recording medium having a program recorded thereon that causes a computer to execute a control method for an information processing apparatus. Some exemplary embodiments provide a computer-readable non-transitory recording medium having a program recorded thereon that causes a computer to execute an information processing method. The non-transitory recording medium of the embodiments may be in any form, examples of which include a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

[0401] The present disclosure merely illustrates some exemplary embodiments and is not intended to limit the invention. Those who intend to implement the present invention can make any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention. [Explanation of symbols]

[0402] 1. Ophthalmic examination equipment 231, 231A, 231B, 231C Image data processing unit 2315 Correlation coefficient calculation unit 232 z-shift amount calculation unit 234 Information Recording Unit 235 Pixel value determination unit 236 Correlation coefficient map creation section 237 Pixel value determination unit 238 Scanning area setting unit 239 Image synthesis processing unit

Claims

1. 1. A scanning imaging device that constructs image data using optical scanning, comprising: a data set collector for collecting a first data set by applying a first optical scan to the sample according to a two-dimensional pattern including a series of intersecting cycles; a correlation coefficient calculation unit that calculates a correlation coefficient between cycles based on the first data set; an information recording unit that records scanning position information corresponding to a subset of the set of correlation coefficients calculated from the first data set by the correlation coefficient calculation unit, the subset having correlation coefficients equal to or less than a predetermined threshold as elements; A scanning imaging device comprising:

2. a first pixel value determination unit that determines a value of a pixel corresponding to the scanning position information recorded by the information recording unit based on values ​​of one or more neighboring pixels; 2. The scanning imaging device of claim 1.

3. a map creating unit that creates a correlation coefficient map in an application area of ​​the first optical scanning based on the set of correlation coefficients acquired by the correlation coefficient calculating unit, 2. The scanning imaging device of claim 1.

4. a second pixel value determining unit that determines a pixel value corresponding to the scanning position information recorded by the information recording unit based on the correlation coefficient map; 4. The scanning imaging device of claim 3.

5. the data set collection unit applies a second optical scan to the sample based on the scanning position information recorded by the information recording unit.

2. The scanning imaging device of claim 1.

6. the data set collection unit applies the second optical scan to an area set based on the scanning position information recorded by the information recording unit.

6. The scanning imaging device of claim 5.

7. a processor for processing the first data set collected by the first optical scan and the second data set collected by the second optical scan to construct image data.

7. The scanning imaging device of claim 6.

8. The processing unit processing the first data set to construct first image data of an area obtained by excluding at least a portion of an area corresponding to the scanning position information recorded by the information recording unit from an application area of ​​the first optical scanning; processing the second data set to construct second image data for the area defined based on the scan position information; combining the first image data and the second image data; 8. The scanning imaging device of claim 7.

9. The dataset collection unit: a deflector capable of deflecting light for optical scanning in two directions different from each other; applying the first optical scan to the sample by repeating a change in deflection direction along one of the two directions with a first period while repeating a change in deflection direction along the other direction with a second period different from the first period; The scanning imaging device according to any one of claims 1 to 8.

10. a receiving unit configured to receive a data set collected by applying an optical scan to a sample according to a two-dimensional pattern including a series of intersecting cycles; a correlation coefficient calculation unit that calculates a correlation coefficient between cycles based on the data set; an information recording unit that records scanning position information corresponding to a subset of the set of correlation coefficients calculated from the data set by the correlation coefficient calculation unit, the subset having correlation coefficients equal to or less than a predetermined threshold as elements; An information processing device comprising:

11. 1. A method for controlling a scanning imaging device including a scanner for performing an optical scan and a processor for constructing image data from data collected from the optical scan, comprising: controlling the scanner to apply an optical scan to the sample according to a two-dimensional pattern comprising a series of intersecting cycles to collect a data set; controlling the processor to calculate a correlation coefficient between cycles based on the data set; controlling the processor to record scanning position information corresponding to a subset of the set of correlation coefficients calculated from the data set, the subset having correlation coefficients equal to or less than a predetermined threshold as elements; method.

12. A method for controlling an information processing device including a processor, comprising: controlling the processor to accept a data set collected by applying an optical scan to the sample according to a two-dimensional pattern including a series of intersecting cycles; controlling the processor to calculate a correlation coefficient between cycles based on the data set; controlling the processor to record scanning position information corresponding to a subset of the set of correlation coefficients calculated from the data set, the subset having correlation coefficients equal to or less than a predetermined threshold as elements; method.

13. obtaining a data set collected from the sample by scanning the light according to a two-dimensional pattern comprising a series of cycles that intersect with each other; calculating a correlation coefficient between cycles based on the data set; recording scanning position information corresponding to a subset of the set of correlation coefficients calculated from the data set, the subset having correlation coefficients equal to or less than a predetermined threshold as elements; Information processing methods.

14. A program that causes a computer to execute the method according to any one of claims 11 to 13.

15. A computer-readable non-transitory recording medium on which the program of claim 14 is recorded.

Images