Scanning imaging device, information processing device, control method for scanning imaging device, control method for information processing device, information processing method, program, and recording medium

Abstract

To provide a new utilization method of data acquired in scan type imaging.SOLUTION: An ophthalmologic inspection device 1 is the scan type imaging device for constructing image data by using optical scan and collects a data set by applying the optical scan according to a two-dimensional pattern including a series of cycles crossing each other to a sample. The ophthalmologic inspection device 1 calculates a correlation coefficient between the cycles on the basis of the data set. The ophthalmologic inspection device 1 further generates correspondence information expressing a correspondence between an aggregation of the correlation coefficients calculated from the data set and an aggregation of position information in an application area of optical scan executed for collecting the data set. Moreover, the ophthalmologic inspection device 1 records the correspondence information in association with the image data that is constructed on the basis of the data set.SELECTED DRAWING: Figure 4B

Description

[Technical Field]

[0001] The present invention relates to a scanning imaging device, an information processing device, a control method for a scanning imaging device, a control method for an information processing device, an information processing method, a program, and a recording medium. [Background technology]

[0002] Scanning imaging is one type of imaging technology. Scanning imaging involves sequentially irradiating a beam at multiple locations on a sample, collecting data, and then constructing an image of the sample from the collected data. Scanning imaging techniques include, for example, spot-type scanning techniques that use a beam that projects a spot-shaped image, line-type scanning techniques that use a beam that projects a line-shaped image, and area-type scanning techniques that use a beam that projects an area-shaped image. Representative examples of these techniques include Fourier-domain optical coherence tomography (OCT), line-scan cameras, and full-field OCT, respectively.

[0003] Optical Coherence Tomography (OCT) is an example of a scanning imaging modality that uses light. OCT is a technology that can image light-scattering media with a resolution of micrometers or less, and is used in medical imaging and non-destructive testing.

[0004] OCT is a technology based on low-coherence interferometry. In medical imaging, a probe light (measurement light) in the near-infrared region is used, taking into account its depth and invasiveness. However, scanning imaging using light (electromagnetic waves) in wavelength bands other than near-infrared, ultrasound, and radiation are also known.

[0005] Ophthalmology is one of the most advanced fields in the application of OCT, with products equipped with Fourier-domain OCT functionality now widespread in clinics. Ophthalmic imaging diagnostics not only provides two-dimensional imaging, but also three-dimensional imaging, structural analysis, and functional analysis, making it a widely used, powerful diagnostic tool. In addition to Fourier-domain OCT, scanning laser ophthalmoscopes (SLOs), line scan cameras, and full-field OCT are also used in ophthalmology.

[0006] There are various scanning modes used in OCT and SLO, but the so-called "Lissajous scan" for the purpose of correcting motion artifacts has been attracting attention in recent years (see, for example, Patent Documents 1 to 4 and Non-Patent Documents 1 and 2).

[0007] A Lissajous scan is a scan performed according to a two-dimensional pattern generated as a Lissajous curve, obtained by synthesizing two mutually orthogonal simple harmonic motions. In a Lissajous scan, the measurement light is scanned at high speed to trace multiple loops (multiple cycles that intersect with each other) of a certain size, so the difference in data acquisition time from one cycle can be virtually ignored. Furthermore, because the intersection areas of different cycles can be referenced to align the cycles, it is possible to correct motion artifacts caused by sample movement. Focusing on these characteristics of the Lissajous scan, the field of ophthalmology is attempting to address motion artifacts caused by eye movement.

[0008] In the technique described in Non-Patent Document 1, a data set acquired by a Lissajous scan is divided into multiple sub-volumes that do not involve relatively large motion, and a front projection image (en face projection) of each sub-volume is constructed. Such front projection images are called strips. Images with motion artifacts corrected can be obtained by performing registration between these strips. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-17915 [Patent Document 2] Japanese Patent Application Publication No. 2018-68578 [Patent Document 3] Japanese Patent Application Laid-Open No. 2018-140004 [Patent Document 4] Japanese Patent Application Laid-Open No. 2018-140049 [Non-patent literature]

[0010] [Non-Patent Document 1] Yiwei Chen, Young-Joo Hong, Shuichi Makita, and Yoshiaki Yasuno, “Three-dimensional eye motion correction by Lissajous scan optical coherence tomography”, Biomedical Optics EXPRESS, Vol. 8, No. 3, 1 Mar 2017, PP. 1783-1802 [Non-patent document 2] Yiwei Chen, Young-Joo Hong, Shuichi Makita, and Yoshiaki Yasuno, “Eye-motion-corrected optical coherence tomography angiography using Lissajous scanning”, Biomedical Optics EXPRESS, Vol. 9, No. 3, 1 Mar 2018, PP. 1111-1129 Summary of the Invention [Problem to be solved by the invention]

[0011] One object of the present invention is to provide a novel method for utilizing data acquired by scanning imaging. [Means for solving the problem]

[0012] An exemplary aspect of the embodiment is a scanning imaging device that constructs image data using optical scanning, and includes a dataset collection unit, a correlation coefficient calculation unit, a correspondence information generation unit, and an information recording unit. The dataset collection unit is configured to collect a first dataset by applying a first optical scan to a sample according to a two-dimensional pattern including a series of cycles that intersect with each other. The correlation coefficient calculation unit is configured to calculate a correlation coefficient between the cycles based on this first dataset. The correspondence information generation unit is configured to generate correspondence information that represents a correspondence between a set of correlation coefficients calculated from the first dataset by the correlation coefficient calculation unit and a set of position information in an application area of ​​the first optical scan. The information recording unit is configured to record the correspondence information in association with first image data constructed based on the first dataset. [Effects of the Invention]

[0013] Exemplary aspects of the embodiment allow for novel uses of data acquired through scanning imaging. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 2] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 3] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 4A] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 4B] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 4C] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 5] 1 is a schematic diagram showing an example of a Lissajous scan pattern performed by a scanning imaging device (ophthalmic examination device) according to an exemplary aspect of the embodiment. [Figure 6] 1 is a schematic diagram showing an example of correspondence information generated by a scanning imaging device (ophthalmic examination device) according to an exemplary aspect of an embodiment. [Figure 7A] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of the embodiment. [Figure 7B] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 7C] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 7D] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of the embodiment. [Figure 7E] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of the embodiment. [Figure 7F] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 8A] 1A to 1C are diagrams for explaining an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 8B] 1A to 1C are diagrams for explaining an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 8C]1A to 1C are diagrams for explaining an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 9] 1A to 1C are diagrams for explaining an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 10] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 11] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 12] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 13] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 14] 1A to 1C are diagrams for explaining an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 15] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 16] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 17] 1 is a schematic diagram illustrating an example of the configuration of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 18] 1 is a flowchart illustrating an example of the operation of a scanning imaging apparatus (ophthalmic examination apparatus) according to an exemplary aspect of an embodiment. [Figure 19] 1 is a schematic diagram illustrating an example of a configuration of an information processing device according to an exemplary aspect of an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Some exemplary aspects of the embodiments will be described with reference to the drawings. The exemplary aspects described below relate to a scanning imaging device, an information processing device, a control method for a scanning imaging device, a control method for an information processing device, an information processing method, a program, and a recording medium, but the embodiments are not limited to these aspects.

[0016] Furthermore, while the exemplary aspects described below provide some applications in the field of ophthalmology, embodiments are not limited to these aspects and may be applied to any field, such as applications in medical departments other than ophthalmology, applications in medical methods such as diagnostic methods, and applications in fields other than the medical field (biology, non-destructive testing, etc.).

[0017] The matters disclosed in the documents cited in this specification and any other matters related to known techniques can be combined with the exemplary embodiments. Furthermore, unless otherwise specified, no distinction is made between "image data" and an "image" based thereon, and no distinction is made between a "site" of a subject (especially the subject's eye) and an "image" thereof.

[0018] A scanning imaging apparatus according to an exemplary embodiment described below is an apparatus used in ophthalmic examinations (referred to as an ophthalmic examination apparatus), and is configured to be capable of measuring and imaging the fundus of a living eye using Fourier-domain OCT (particularly, swept-source OCT). The type of OCT that can be employed in the embodiments is not limited to swept-source OCT, and may be, for example, spectral-domain OCT or time-domain OCT. Furthermore, although the scanning imaging apparatus according to the exemplary embodiment described below uses a spot-type scanning technique, it will be understood by those skilled in the art that any aspect of the present disclosure can be applied to a scanning imaging apparatus that uses other types of scanning techniques (e.g., a line-type scanning technique or an area-type scanning technique).

[0019] A scanning imaging device according to some exemplary embodiments may be capable of using a scanning imaging modality other than OCT. For example, some exemplary embodiments may employ any optical scanning imaging modality such as SLO. The scanning imaging modality applicable to some exemplary embodiments may not be an optical scanning imaging modality, but may be, for example, a scanning imaging modality using electromagnetic waves other than light, a scanning imaging modality using ultrasound, or the like.

[0020] In addition to processing data collected by OCT scans and / or SLO scans, the ophthalmic examination apparatus according to some exemplary embodiments may be capable of processing data acquired by other imaging modalities. The other imaging modalities may be any ophthalmic imaging modality, such as a fundus camera, a slit lamp microscope, or an ophthalmic surgical microscope. The ophthalmic examination apparatus according to some exemplary embodiments may have the functionality of imaging modalities other than OCT and SLO. Furthermore, the ophthalmic examination apparatus according to some exemplary embodiments may have a stereo alignment function using two or more anterior eye cameras, as disclosed in Japanese Patent Application Laid-Open No. 2013-248376 and Japanese Patent Application Laid-Open No. 2016-49243.

[0021] The object (sample) to which scanning imaging is applied may be any object. In the ophthalmology application described in the exemplary embodiments, scanning imaging is applied to the fundus, but it may also be applied to any part of the eye, such as the anterior segment or the vitreous body. The configurations and functions of some exemplary embodiments may be applied to measurement and imaging of biological parts (tissues) other than the eye, and may also be applied to measurement and imaging of non-biological objects (e.g., objects that move) or parts thereof. That is, the industrial application fields of some exemplary embodiments are not limited to ophthalmology-related fields, but may also include medical, veterinary, and biological fields, and may more generally include fields related to any object (sample) that moves locally and / or globally. Note that the industrial application fields of some exemplary embodiments may also include fields related to objects that do not move.

[0022] Any aspects of the embodiments described below are merely illustrative and are not intended to limit the invention.

[0023] A first aspect of the embodiment is a scanning imaging device that constructs image data using optical scanning, and includes a dataset collection unit, a correlation coefficient calculation unit, a correspondence information generation unit, and an information recording unit. The dataset collection unit is configured to collect a first dataset by applying a first optical scan to a sample according to a two-dimensional pattern including a series of cycles that intersect with each other. The correlation coefficient calculation unit is configured to calculate a correlation coefficient between the cycles based on this first dataset. The correspondence information generation unit is configured to generate correspondence information that represents a correspondence between a set of correlation coefficients calculated from the first dataset by the correlation coefficient calculation unit and a set of positional information in an application area of ​​the first optical scan. The information recording unit is configured to record the correspondence information in association with first image data constructed based on the first dataset.

[0024] A second aspect of the embodiment is a scanning imaging device of the first aspect, further including a processing unit configured to perform a predetermined process based on the corresponding information recorded by the information recording unit.

[0025] A third aspect of the embodiment is a scanning imaging device of the second aspect, in which the processing unit is configured to perform the following predetermined processes: a process of identifying a subset from the set of correlation coefficients acquired by the correlation coefficient calculation unit, the subset having correlation coefficients below a predetermined threshold as elements; a process of identifying a subset of positional information corresponding to this subset of correlation coefficients based on the correspondence information; and a process of determining the value of a pixel of the first image data corresponding to this subset of positional information based on the values ​​of one or more neighboring pixels.

[0026] In a fourth aspect of the embodiment, in the scanning imaging apparatus of the second aspect, the processing unit is configured to perform the predetermined processing of identifying a subset of correlation coefficients acquired by the correlation coefficient calculation unit, the subset having correlation coefficients equal to or less than a predetermined threshold, and identifying a subset of position information corresponding to the subset of correlation coefficients based on the correspondence information. Furthermore, the dataset collection unit is configured to apply a second optical scan to the sample based on the subset of position information identified by the processing unit.

[0027] A fifth aspect of the embodiment is a scanning imaging device of the fourth aspect, wherein the processing unit is further configured to perform the following predetermined processing: a process of setting an application area of ​​the second optical scanning based on this subset of positional information; and a process of synthesizing first image data constructed based on the first data set collected by the first optical scanning and second image data constructed based on the second data set collected by the second optical scanning.

[0028] A sixth aspect of the embodiment is a scanning imaging device of any of the second to fifth aspects, wherein the processing unit is configured to perform, as a predetermined process, a process of evaluating the quality of at least a portion of the first image data based on a set of correlation coefficients acquired by the correlation coefficient calculation unit, and a process of recording the result of this quality evaluation in association with the first image data.

[0029] A seventh aspect of the embodiment is a scanning imaging device according to any one of the first to sixth aspects, wherein the dataset collection unit includes a deflector capable of deflecting light for optical scanning in two directions different from each other, and is configured to apply a first optical scan to the sample by repeatedly changing the deflection direction along one of the two directions in a first period while repeatedly changing the deflection direction along the other direction in a second period different from the first period.

[0030] An eighth aspect of the embodiment is an information processing device including a receiving unit, a correlation coefficient calculation unit, a correspondence information generation unit, and an information recording unit. The receiving unit is configured to receive a data set collected by applying an optical scan according to a two-dimensional pattern including a series of cycles that intersect with each other to a sample. The correlation coefficient calculation unit is configured to calculate a correlation coefficient between the cycles based on the data set. The correspondence information generation unit is configured to generate correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set by the correlation coefficient calculation unit and a set of positional information in an application area of ​​the optical scan performed to collect the data set. The information recording unit is configured to record the correspondence information in association with image data constructed based on the data set.

[0031] A ninth aspect of the embodiment is a method for controlling a scanning imaging device including a scanner that performs optical scanning and a processor that constructs image data from data collected by the optical scanning, comprising the following steps: controlling the scanner to apply an optical scan to a sample according to a two-dimensional pattern including a series of cycles that intersect with each other to collect a data set; controlling the processor to calculate correlation coefficients between cycles based on the data set; controlling the processor to generate correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set and a set of position information in the application area of ​​the optical scanning performed to collect the data set; and controlling the processor to record the correspondence information in association with image data constructed based on the data set.

[0032] A tenth aspect of the embodiment is a method for controlling an information processing device including a processor, comprising the following steps: controlling the processor to accept a data set collected by applying an optical scan to a sample according to a two-dimensional pattern including a series of cycles that intersect with each other; controlling the processor to calculate correlation coefficients between cycles based on the data set; controlling the processor to generate correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set and a set of position information in the application area of ​​the optical scan performed to collect the data set; and controlling the processor to record the correspondence information in association with image data constructed based on the data set.

[0033] An eleventh aspect of the embodiment is a method for processing information, comprising the steps of: obtaining a dataset collected from a sample with an optical scan that follows a two-dimensional pattern including a series of cycles that intersect with each other; calculating correlation coefficients between the cycles based on the dataset; generating correspondence information representing a correspondence between a set of correlation coefficients calculated from the dataset and a set of position information in the application area of ​​the optical scan performed to collect the dataset; and recording the correspondence information in association with image data constructed based on the dataset.

[0034] A twelfth aspect of the embodiment is a program for causing a computer to execute the method according to any one of the ninth to eleventh aspects.

[0035] A thirteenth aspect of the embodiment is a computer-readable non-transitory recording medium having the program of the twelfth aspect recorded thereon.

[0036] The aspects of the embodiment are not limited to the above-described aspects 1 to 13. For example, features that can be combined with aspect 1 (for example, features related to aspects 2 to 7, any feature in the embodiments described below, any known technology, etc.) can be combined with aspects 8 to 13.

[0037] First Embodiment A first embodiment will be described. This embodiment functions to generate a set of correlation coefficients between cycles from a data set collected by applying an optical scan such as a Lissajous scan to a sample, generate correspondence information representing the correspondence between this set of correlation coefficients and a set of position information in the application area of ​​the optical scan, and record this correspondence information. Furthermore, this embodiment functions to execute a predetermined process based on the recorded correspondence information. Some non-limiting examples of this predetermined process will be described in the following embodiments.

[0038] <Configuration of ophthalmic examination device> The ophthalmic examination apparatus 1 shown in FIG. 1 is an exemplary embodiment of a scanning imaging apparatus according to the present embodiment. The ophthalmic examination apparatus 1 includes a fundus camera unit 2, an OCT unit 100, and an arithmetic and control unit 200. The fundus camera unit 2 includes a group of elements (optical elements, mechanisms, etc.) for capturing an image of the subject's eye E from the front. The OCT unit 100 includes some of the group of elements (optical elements, mechanisms, etc.) for performing an OCT scan on the subject's eye E. Another group of elements for the OCT scan is provided in the fundus camera unit 2. The arithmetic and control unit 200 includes one or more processors that perform various calculations and controls. In addition to these, the ophthalmic examination apparatus 1 may also include elements for supporting the subject's face and elements for switching the region to which the OCT scan is applied. Examples of the former elements include a chin rest and a forehead rest. An example of the latter element is a lens unit used to switch the region to which the OCT scan is applied from the fundus to the anterior segment.

[0039] The functionality of some elements disclosed herein is implemented using circuitry or processing circuitry. Circuitry or processing circuitry includes any of a general-purpose processor, a special-purpose processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA), conventional circuitry, and any combination thereof, configured and / or configured to perform the disclosed functions. A processor is considered to be processing circuitry or circuitry that includes transistors and / or other circuitry. In this disclosure, circuitry, unit, means, or similar terms is hardware that performs the disclosed functions or hardware that is programmed to perform the disclosed functions. The hardware may be hardware disclosed herein or may be known hardware that is programmed and / or configured to perform the described functions. In the case of a processor, where the hardware can be considered to be a type of circuitry, the circuitry, unit, means, or similar terms is a combination of hardware and software, and the software is used to configure the hardware and / or processor.

[0040] <Fundus camera unit 2> The fundus camera unit 2 is provided with an optical system for photographing the fundus Ef of the subject's eye E. The acquired image of the fundus Ef (called a fundus image, fundus photograph, etc.) is a front image such as an observed image or a photographed image. The observed image is obtained by, for example, video shooting using near-infrared light, and is used for alignment, focusing, tracking, etc. The photographed image is a still image obtained using, for example, flash light in the visible or infrared range.

[0041] The fundus camera unit 2 includes an illumination optical system 10 and an imaging optical system 30. The illumination optical system 10 irradiates illumination light onto the subject's eye E. The imaging optical system 30 detects return light of the illumination light from the subject's eye E. The measurement light from the OCT unit 100 is guided to the subject's eye E through an optical path within the fundus camera unit 2, and the return light is guided to the OCT unit 100 through the same optical path.

[0042] Light (observation illumination light) output from an observation light source 11 of an illumination optical system 10 is reflected by a concave mirror 12, passes through a condenser lens 13, and is transmitted through a visible light cut filter 14 to become near-infrared light. The observation illumination light is then focused near an imaging light source 15, reflected by a mirror 16, and passes through a relay lens system 17, a relay lens 18, an aperture 19, and a relay lens system 20. The observation illumination light is then reflected by the peripheral portion (the area surrounding the hole) of a perforated mirror 21, passes through a dichroic mirror 46, and is refracted by an objective lens 22 to illuminate the subject's eye E (fundus oculi Ef). Return light of the observation illumination light from the subject's eye E is refracted by the objective lens 22, passes through the dichroic mirror 46, passes through a hole formed in the central region of the perforated mirror 21, passes through a dichroic mirror 55, passes through an imaging focusing lens 31, and is reflected by a mirror 32. Furthermore, this returned light passes through the half mirror 33A, is reflected by the dichroic mirror 33, and is imaged on the light receiving surface of the image sensor 35 by the imaging lens 34. The image sensor 35 detects the returned light at a predetermined frame rate. The focus (focal position) of the photographing optical system 30 is adjusted to match the fundus Ef or the anterior segment of the eye.

[0043] Light (photography illumination light) output from the photography light source 15 is irradiated onto the fundus oculi Ef through the same path as the observation illumination light. Return light of the photography illumination light from the subject's eye E is guided to the dichroic mirror 33 through the same path as the return light of the observation illumination light, passes through the dichroic mirror 33, is reflected by a mirror 36, and is imaged by an imaging lens 37 on the light-receiving surface of an image sensor 38.

[0044] The liquid crystal display (LCD) 39 displays a fixation target (fixation target image). A portion of the light beam output from the LCD 39 is reflected by the half mirror 33A, reflected by the mirror 32, passes through the photographing focusing lens 31 and the dichroic mirror 55, and passes through the hole in the aperture mirror 21. The light beam that passes through the hole in the aperture mirror 21 passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef. By changing the display position of the fixation target image on the LCD 39, the direction in which the gaze of the subject's eye E is guided (fixation direction, fixation position) can be changed. Instead of a display device such as an LCD, for example, a light-emitting element array, or a combination of a light-emitting element and a mechanism for moving it, may be used.

[0045] The alignment optical system 50 generates an alignment index used to align the optical system with the subject's eye E. Alignment light output from a light-emitting diode (LED) 51 passes through an aperture 52, an aperture 53, and a relay lens 54, is reflected by a dichroic mirror 55, passes through the hole in the aperture mirror 21, transmits through the dichroic mirror 46, and is projected onto the subject's eye E via the objective lens 22. Return light of the alignment light from the subject's eye E (such as corneal reflection light) is guided to the image sensor 35 via the same path as the return light of the observation illumination light. Manual alignment or automatic alignment can be performed based on the received light image (alignment index image).

[0046] As in the conventional example, the alignment target image in this example consists of two bright spot images whose positions change depending on the alignment state. When the relative position between the subject's eye E and the optical system changes in the x and y directions (directions expressed by the x and y coordinates), the two bright spot images displace together in the x and y directions. When the relative position between the subject's eye E and the optical system changes in the z direction (direction expressed by the z coordinate), the relative position (distance) between the two bright spot images changes. When the distance between the subject's eye E and the optical system in the z direction matches the predetermined working distance, the two bright spot images overlap. When the position of the subject's eye E matches the position of the optical system in the x and y directions, the two bright spot images are presented within or near a specified alignment target. When the distance between the subject's eye E and the optical system in the z direction matches the working distance and the position of the subject's eye E matches the position of the optical system in the x and y directions, the two bright spot images overlap and are presented within the alignment target.

[0047] In auto-alignment, the data processing unit 230 detects the positions of the two bright spot images, and the main control unit 211 controls the moving mechanism 150 (described later) based on the positional relationship between the two bright spot images and the alignment target. In manual alignment, the main control unit 211 displays the two bright spot images together with the observed image of the subject's eye E on the display unit 241, and the user operates the moving mechanism 150 using the operation unit 242 while referring to the two displayed bright spot images.

[0048] The focusing optical system 60 generates a split index used for focus adjustment of the subject's eye E. The focusing optical system 60 moves along the optical path (illumination optical path) of the illumination optical system 10 in conjunction with movement of the photographing focusing lens 31 along the optical path (photographing optical path) of the photographing optical system 30. The reflecting rod 67 is inserted into and removed from the illumination optical path. When performing focus adjustment, the reflecting surface of the reflecting rod 67 is tilted and positioned in the illumination optical path. Focusing light output from the LED 61 passes through the relay lens 62, is split into two beams by the split index plate 63, passes through the two-hole diaphragm 64, is reflected by the mirror 65, and is first imaged and reflected on the reflecting surface of the reflecting rod 67 by the condenser lens 66. The focusing light then passes through the relay lens 20, is reflected by the aperture mirror 21, passes through the dichroic mirror 46, and is projected onto the subject's eye E via the objective lens 22. The return light (fundus reflected light, etc.) of the focusing light from the subject's eye E passes through the same path as the return light of the alignment light and is guided to the image sensor 35. Manual focusing or autofocusing can be performed based on the received light image (split target image).

[0049] Diopter correction lenses 70 and 71 can be selectively inserted into the photographing optical path between the aperture mirror 21 and the dichroic mirror 55. The diopter correction lens 70 is a plus lens (convex lens) for correcting severe hyperopia. The diopter correction lens 71 is a minus lens (concave lens) for correcting severe myopia.

[0050] The dichroic mirror 46 combines the optical path for fundus imaging and the optical path for OCT (measurement arm). The dichroic mirror 46 reflects light in the wavelength band used for OCT and transmits light for fundus imaging. The measurement arm is provided with, in order from the OCT unit 100 side, a collimator lens unit 40, a retroreflector 41, a dispersion compensation member 42, an OCT focusing lens 43, an optical scanner 44, and a relay lens 45.

[0051] The retroreflector 41 is movable in the direction of the arrow shown in Figure 1, thereby changing the length of the measurement arm. Changing the length of the measurement arm is used, for example, to correct the optical path length according to the axial length of the eye, adjust the interference state, and so on.

[0052] The dispersion compensation member 42, together with a dispersion compensation member 113 (described later) disposed in the reference arm, acts to match the dispersion characteristics of the measurement light LS with the dispersion characteristics of the reference light LR.

[0053] The OCT focusing lens 43 is moved along the measurement arm to adjust the focus of the measurement arm. The movement of the imaging focusing lens 31, the movement of the focus optical system 60, and the movement of the OCT focusing lens 43 can be controlled in a coordinated manner.

[0054] The optical scanner 44 is disposed at a position that is substantially optically conjugate with the pupil of the subject's eye E. The optical scanner 44 deflects the measurement light LS guided by the measurement arm. The optical scanner 44 is, for example, a galvanometer scanner capable of two-dimensional scanning, including a galvanometer mirror for scanning in the x direction and a galvanometer mirror for scanning in the y direction.

[0055] <OCTユニット100> As shown in FIG. 2, the OCT unit 100 is provided with an optical system for applying swept-source OCT. This optical system includes an interference optical system. This interference optical system splits light from a wavelength-tunable light source (swept-wavelength light source) into measurement light and reference light, and generates interference light by superimposing the return light of the measurement light guided to the subject's eye E by the measurement arm on the reference light guided by the reference arm, and detects this interference light. Data (detection signal) obtained by the interference optical system is a signal representing the spectrum of the interference light, and is sent to the arithmetic and control unit 200.

[0056] The light source unit 101 includes, for example, a near-infrared tunable laser that changes the output wavelength at high speed at least in the near-infrared wavelength band. Light L0 output from the light source unit 101 is guided by an optical fiber 102 to a polarization controller 103, where its polarization state is adjusted. The light L0 is further guided by an optical fiber 104 to a fiber coupler 105, where it is split into measurement light LS and reference light LR. The optical path of the measurement light LS is called a measurement arm, and the optical path of the reference light LR is called a reference arm.

[0057] The reference light LR is guided by an optical fiber 110 to a collimator 111, where it is converted into a parallel beam, and then guided to a retroreflector 114 via an optical path length correction element 112 and a dispersion compensation element 113. The optical path length correction element 112 acts to match the optical path length of the reference light LR with that of the measurement light LS. The dispersion compensation element 113, together with a dispersion compensation element 42 arranged in the measurement arm, acts to match the dispersion characteristics between the reference light LR and the measurement light LS. The retroreflector 114 is movable along the optical path of the reference light LR incident thereon, thereby changing the length of the reference arm. Changing the reference arm length is used, for example, to correct the optical path length according to the axial length of the eye, adjust the interference state, and so on.

[0058] The reference light LR that has passed through the retroreflector 114 passes through the dispersion compensation member 113 and the optical path length correction member 112, is converted from a parallel beam into a convergent beam by the collimator 116, and enters an optical fiber 117. The reference light LR that has entered the optical fiber 117 is guided to a polarization controller 118 where its polarization state is adjusted, is guided through an optical fiber 119 to an attenuator 120 where its light amount is adjusted, and is guided through an optical fiber 121 to a fiber coupler 122.

[0059] On the other hand, the measurement light LS generated by the fiber coupler 105 is guided by the optical fiber 127 and converted into a parallel beam by the collimator lens unit 40, passes through the retroreflector 41, the dispersion compensation member 42, the OCT focusing lens 43, the optical scanner 44, and the relay lens 45, is reflected by the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the subject's eye E. The measurement light LS is scattered and reflected at various depth positions in the subject's eye E. The returning light of the measurement light LS from the subject's eye E travels the same path as the outward path in the opposite direction and is guided to the fiber coupler 105, and reaches the fiber coupler 122 via the optical fiber 128.

[0060] The fiber coupler 122 generates interference light by superimposing the measurement light LS incident via the optical fiber 128 and the reference light LR incident via the optical fiber 121. The fiber coupler 122 splits the generated interference light at a predetermined splitting ratio (for example, 1:1) to generate a pair of interference lights LC. The pair of interference lights LC are guided to the detector 125 via optical fibers 123 and 124, respectively.

[0061] The detector 125 includes, for example, a balanced photodiode. The balanced photodiode has a pair of photodetectors that respectively detect a pair of interference light LC, and outputs the difference between the pair of detection results obtained by these. The detector 125 sends this output (detection signal) to a data acquisition system (DAS) 130.

[0062] A clock KC is supplied to the data collection system 130 from the light source unit 101. The clock KC is generated in the light source unit 101 in synchronization with the output timing of each wavelength swept within a predetermined wavelength range by the wavelength-tunable light source. The light source unit 101, for example, branches the light L0 of each output wavelength to generate two branched lights, optically delays one of the branched lights, combines the branched lights, detects the resulting combined light, and generates the clock KC based on the detection result. The data collection system 130 samples the detection signal input from the detector 125 based on the clock KC. The data collection system 130 sends the sampling result to the arithmetic and control unit 200.

[0063] In this example, both an element for changing the measurement arm length (e.g., retroreflector 41) and an element for changing the reference arm length (e.g., retroreflector 114 or reference mirror) are provided, but only one of these elements may be provided. Furthermore, the element for changing the difference between the measurement arm length and the reference arm length (optical path length difference) is not limited to these and may be any element (optical member, mechanism, etc.).

[0064] <Control system / processing system> 3, 4A, 4B, and 4C show configuration examples of the control system and processing system of the ophthalmic examination apparatus 1. The control unit 210, image data creation unit 220, and data processing unit 230 are provided in, for example, the arithmetic control unit 200. The ophthalmic examination apparatus 1 may include a communication device for performing data communication with an external device. The ophthalmic examination apparatus 1 may also include a drive device (reader / writer) for reading data from a recording medium and writing data to the recording medium.

[0065] <Control unit 210> The control unit 210 executes various types of control. The control unit 210 includes a main control unit 211 and a storage unit 212. In this embodiment, the main control unit 211 includes a scanning control unit 2111, and the storage unit 212 stores a scanning protocol 2121, as shown in FIG. 4A.

[0066] <Main control unit 211> The main control unit 211 includes a processor and controls each element (including the elements shown in FIGS. 1 to 4C) of the ophthalmic examination apparatus 1. The main control unit 211 is realized by cooperation between hardware including the processor and control software. The scan control unit 2111 controls OCT scanning of a scan area of ​​a predetermined shape and size.

[0067] The imaging focusing driver 31A moves the imaging focusing lens 31 arranged in the imaging optical path and the focus optical system 60 arranged in the illumination optical path under the control of the main controller 211. The retroreflector (RR) driver 41A moves the retroreflector 41 provided in the measurement arm under the control of the main controller 211. The OCT focusing driver 43A moves the OCT focusing lens 43 arranged in the measurement arm under the control of the main controller 211. The retroreflector (RR) driver 114A moves the retroreflector 114 arranged in the reference arm under the control of the main controller 211. Each driver includes an actuator such as a pulse motor that operates under the control of the main controller 211. The optical scanner 44 operates under the control of the main controller 211 (scanning controller 2111).

[0068] The movement mechanism 150 is typically configured to move the fundus camera unit 2 three-dimensionally. Such a movement mechanism 150 includes, for example, an x-stage configured to be movable in ±x directions (left and right directions), an x-movement mechanism configured to move the x-stage, a y-stage configured to be movable in ±y directions (up and down directions), a y-movement mechanism configured to move the y-stage, a z-stage configured to be movable in ±z directions (depth direction), and a z-movement mechanism configured to move the z-stage. Each of the x-movement mechanism, y-movement mechanism, and z-movement mechanism includes an actuator such as a pulse motor that operates under the control of the main controller 211.

[0069] <Storage section 212> The storage unit 212 stores various types of data. The data stored in the storage unit 212 includes OCT images, fundus images, subject eye information, and control information. The subject eye information includes subject information such as patient ID and name, left / right eye identification information, and electronic medical record information. The control information is information related to specific controls. The control information in this embodiment includes a scanning protocol 2121.

[0070] The scanning protocol 2121 is an agreement regarding the control content for an OCT scan of a scanning area of ​​a predetermined shape and a predetermined size, and includes a set of various control parameters (scanning control parameters). The scanning protocol 2121 includes a protocol for each scanning mode. The scanning protocol 2121 of this embodiment includes at least a protocol for Lissajous scanning, and may further include protocols for, for example, B-scan (line scanning), cross scanning, radial scanning, and raster scanning.

[0071] The scanning control parameters of this embodiment include at least a parameter indicating the content of control over the optical scanner 44. These parameters include, for example, a parameter indicating the scan pattern, a parameter indicating the scan speed, and a parameter indicating the scan interval. The scan pattern indicates the shape of the scan path, and examples thereof include a Lissajous pattern, a line pattern, a cross pattern, a radial pattern, and a raster pattern. The scan speed is defined, for example, as the repetition rate of A-scans. The scan interval is defined, for example, as the interval between adjacent A-scans, that is, the array interval of scan points.

[0072] As with the conventional technologies disclosed in Patent Documents 1 to 4 and Non-Patent Documents 1 and 2, the "Lissajous scan" of this embodiment may be not only a "narrowly defined" Lissajous scan whose path is a pattern (Lissajous pattern, Lissajous figure, Lissajous curve, Lissajous function, Bowditch curve) drawn by the locus of points obtained by an ordered pair of two mutually perpendicular simple harmonic motions, but also a "broadly defined" Lissajous scan that follows a predetermined two-dimensional pattern including a series of cycles.

[0073] The optical scanner 44 of this embodiment includes, for example, a first galvanometer mirror that deflects the measurement light LS in the x direction and a second galvanometer mirror that deflects the measurement light LS in the y direction. Lissajous scanning is achieved by controlling the first galvanometer mirror so that the deflection direction along the x direction repeatedly changes in a first period, while controlling the second galvanometer mirror so that the deflection direction along the y direction repeatedly changes in a second period. Here, the first period and the second period are different from each other.

[0074] For example, the Lissajous scan of this embodiment may not only be a scan of a narrowly defined Lissajous pattern obtained by combining two sine waves, but also a scan of a pattern obtained by adding a specific term (e.g., an odd-order polynomial) to a sine wave, or a scan of a pattern based on a triangular wave.

[0075] A "cycle" generally refers to an object consisting of a plurality of sampling points having a certain length. The cycle in this embodiment may be, for example, a closed curve or an almost closed curve (a substantially closed curve, an approximately closed curve). In other words, the start point and end point of the cycle in this embodiment may coincide or nearly coincide.

[0076] Typically, the scanning protocol 2121 is set based on the Lissajous function. As shown in equation (9) of Non-Patent Document 1, the Lissajous function is expressed by the following parametric equation system: x(t i )=A·cos(2π·(f A / n)·t i ), y(t i )=A·cos(2π·(f A (n-2) / n 2 )·t i ).

[0077] Here, "x" is the horizontal axis of the two-dimensional coordinate system in which the Lissajous curve is defined, "y" is the vertical axis, and "t i ” is the acquisition time of the i-th A-line in the Lissajous scan, “A” is the scan range (amplitude), and “f A" indicates the A-line acquisition rate (scan speed, A-scan repetition rate), and n indicates the number of A-lines in each cycle in the x-direction (horizontal axis).

[0078] An example of the distribution of scan lines (scan pattern) in such an exemplary Lissajous scan is shown in FIG.

[0079] Any pair of cycles included in a Lissajous scan (in the narrow or broad sense) intersects with each other at at least one point (particularly, two or more points). Utilizing such intersections makes it possible to perform registration between pairs of data collected from any pair of cycles in a Lissajous scan, and makes it possible to implement the image construction method and motion artifact correction method disclosed in Non-Patent Document 1 (or Non-Patent Document 2). Hereinafter, unless otherwise specified, a case where the method described in Non-Patent Document 1 is applied will be described. However, it is also possible to apply the method described in Non-Patent Document 2, or a method equivalent and / or similar to the method described in Non-Patent Document 1 or 2.

[0080] The control information stored in the storage unit 212 is not limited to the above example. For example, the control information may include information for performing focus control (focus control parameters).

[0081] The focus control parameters are parameters that indicate the content of control over the OCT focusing drive unit 43A. Examples of focus control parameters include a parameter that indicates the focal position of the measurement arm, a parameter that indicates the moving speed of the focal position, and a parameter that indicates the moving acceleration of the focal position. The parameter that indicates the focal position is, for example, a parameter that indicates the position of the OCT focusing lens 43. The parameter that indicates the moving speed of the focal position is, for example, a parameter that indicates the moving speed of the OCT focusing lens 43. The parameter that indicates the moving acceleration of the focal position is, for example, a parameter that indicates the moving acceleration of the OCT focusing lens 43. The moving speed may or may not be constant. The same applies to the moving acceleration.

[0082] Such focus control parameters enable focus adjustment according to the shape of the fundus oculi Ef (typically a concave shape with a deep center and a shallow periphery) and aberration distribution. Focus control is performed, for example, in conjunction with scan control (repeated control of Lissajous scan). This allows motion artifacts to be corrected and high-quality images that are in focus across the entire scan range to be obtained.

[0083] <Scanning control unit 2111> The scanning control unit 2111 controls at least the optical scanner 44 based on the scanning protocol 2121. The scanning control unit 2111 may further control the light source unit 101 in cooperation with the control of the optical scanner 44 based on the scanning protocol 2121. The scanning control unit 2111 is realized by cooperation between hardware including a processor and scanning control software including the scanning protocol 2121.

[0084] <Image data construction unit 220> The image data constructing unit 220 includes a processor and constructs OCT image data of the fundus Ef based on signals (sampling data) input from the data acquisition system 130. This OCT image data construction includes noise removal (noise reduction), filtering, fast Fourier transform (FFT), and the like, similar to conventional Fourier domain OCT (swept-source OCT). When another type of OCT method is adopted, the image data constructing unit 220 constructs OCT image data by performing known processing corresponding to that OCT type.

[0085] For example, the image data constructing unit 220 is configured to at least execute a process of constructing image data (A-scan image data) corresponding to each scan point (each A-line) from the sampling data. The image data constructed by the image data constructing unit 220 may be data before being imaged, and may be, for example, a signal profile (reflection profile, scattering profile) along the depth direction (z direction, optical axis direction, A-line direction).

[0086] As described above, in this embodiment, a Lissajous scan is applied to the fundus Ef. The image data constructing unit 220, together with the data processing unit 230, can construct three-dimensional image data of the fundus Ef by applying, for example, the image construction method and motion artifact correction method disclosed in Non-Patent Document 1 to the data set acquired through collection by the Lissajous scan and sampling by the data acquisition system 130.

[0087] The image data constructor 220 and / or the data processor 230 can apply rendering to the 3D image data to generate a display image. Examples of applicable rendering methods include volume rendering, surface rendering, maximum intensity projection (MIP), minimum intensity projection (MIP), and multiplanar reconstruction (MPR).

[0088] The image data constructing unit 220 and / or the data processing unit 230 can construct an en face OCT image based on the three-dimensional image data. For example, the image data constructing unit 220 and / or the data processing unit 230 can construct projection data by projecting the three-dimensional image data in the z direction (A-line direction, optical axis direction, depth direction). The image data constructing unit 220 and / or the data processing unit 230 can also construct a shadowgram by projecting partial three-dimensional image data, which is a part of the three-dimensional image data, in the z direction. This partial three-dimensional image data may be set using, for example, any segmentation method. Segmentation is a process of identifying a partial region in an image. In this example, segmentation can be performed to identify an image region corresponding to one or more tissues (regions) of the fundus oculi Ef.

[0089] The segmentation in the embodiment is not limited to the shadowgram construction process, and any known technique such as threshold processing, edge detection, filtering, machine learning (e.g., semantic segmentation) can be used.

[0090] The ophthalmic examination apparatus 1 may be capable of performing OCT angiography (OCT-angiography). OCT angiography is an imaging technique that constructs an image in which blood vessels are emphasized (see, for example, Non-Patent Document 2 and JP2015-515894A). Generally, fundus tissue (structure) does not change over a short period of time, but blood flow inside blood vessels also changes over a short period of time. OCT angiography generates an image by emphasizing areas where such temporal changes exist (blood flow signals). Note that OCT angiography is also called OCT motion contrast imaging. Furthermore, images obtained by OCT angiography are called angiographic images, angiograms, motion contrast images, etc.

[0091] When OCT angiography is possible, the ophthalmic examination apparatus 1 repeatedly scans the same region of the fundus Ef a predetermined number of times. For example, the ophthalmic examination apparatus 1 repeatedly executes the above-described scanning control (repeated control of Lissajous scanning) a predetermined number of times. As a result, multiple pieces of three-dimensional data (time-series three-dimensional data sets) are collected by the data acquisition system 130 from the application region of the Lissajous scan. The image data construction unit 220 and / or the data processing unit 230 can construct a motion contrast image from this three-dimensional data set. This motion contrast image is an angiographic image in which the temporal change in the interference signal caused by the blood flow in the fundus Ef is emphasized. This angiographic image is three-dimensional angiographic image data that represents the three-dimensional distribution of blood vessels in the fundus Ef.

[0092] The image data constructing unit 220 and / or the data processing unit 230 can construct any two-dimensional angiographic image data and / or any pseudo three-dimensional angiographic image data from this three-dimensional angiographic image data. For example, the image data constructing unit 220 and / or the data processing unit 230 can construct two-dimensional angiographic image data representing any cross section of the fundus Ef by applying multiplanar reconstruction to the three-dimensional angiographic image data. Furthermore, the image data constructing unit 220 and / or the data processing unit 230 can construct frontal angiographic image data of the fundus Ef by applying projection imaging or shadowgramming to the three-dimensional angiographic image data.

[0093] In this embodiment, the image data constructor 220 and / or the data processor 230 construct multiple strips from data collected by the data acquisition system 130. As described in Non-Patent Document 1, the image data constructor 220 and / or the data processor 230 divide the volume (three-dimensional data) collected by Lissajous scanning into multiple subvolumes without relatively large intervening motion, and construct a front projection image of each subvolume. These front projection images are strips. By applying registration and merging processes to the multiple strips obtained in this manner, an image in which motion artifacts have been corrected can be obtained. As will be described later, in this example, the registration and merging processes are performed by the data processor 230.

[0094] The image data constructing unit 220 is realized by cooperation between hardware including a processor and image constructing software. Note that, in some exemplary embodiments, the image data constructing unit 220 and the data processing unit 230 may be integrally configured.

[0095] <Data processing unit 230> The data processing unit 230 includes a processor and applies various types of data processing to the image of the subject's eye E. For example, the data processing unit 230 is realized by cooperation between hardware including a processor and data processing software. The same is true for each functional element included in the data processing unit 230.

[0096] The data processing unit 230 may be configured to perform registration between two images acquired of the fundus oculi Ef. For example, the data processing unit 230 may be configured to perform registration between three-dimensional image data acquired by OCT and a front image acquired by the fundus camera unit 2. The data processing unit 230 may also be configured to perform registration between two OCT images acquired by OCT. The data processing unit 230 may also be configured to perform registration between two front images acquired by the fundus camera unit 2. The data processing unit 230 may also be configured to apply registration to the analysis results of the OCT images and the analysis results of the front images. These registrations can be performed by known methods, and include, for example, feature point extraction and affine transformation.

[0097] Furthermore, the data processing unit 230 is configured to process a data set obtained using a Lissajous scan. As previously described, the image data constructor 220 is configured to construct strips from data collected by the Lissajous scan, for example. The data processing unit 230 is configured to construct a motion artifact corrected image by applying registration and merging operations to the strips.

[0098] A characteristic of Lissajous scanning is that any two strips have an overlapping area. Similar to the technique described in Non-Patent Document 1, the data processing unit 230 is configured to perform strip-to-strip registration using the overlapping area. The data processing unit 230 first orders the multiple strips constructed by the image data construction unit 220 according to size (e.g., area) (first to Nth strips) and designates the largest strip, the first strip, as the initial reference strip. Next, the data processing unit 230 registers the second strip using the first strip as a reference and merges (combines) the first and second strips. The data processing unit 230 registers the third strip using the resulting merged strip as a reference strip and merges the merged strip with the third strip. By sequentially performing these registration and merging processes in the above order, the first to Nth strips are aligned and stitched together, resulting in an image with motion artifacts corrected.

[0099] In such registration, a cross-correlation function is used to determine the relative position between a reference strip and other strips. However, since the strips can have arbitrary shapes, the correlation between the strips is calculated using a mask that treats each strip as an image of a specific shape (e.g., a square) (see Appendix A of Non-Patent Document 1).

[0100] As mentioned above, the strip is an intensity image of a predetermined gradation, and the mask image is a binary image, so the difference between the absolute values ​​of the pixel values ​​of the strip and the mask becomes large, and there is a risk that the effect of the mask in the correlation calculation using the strip and the mask will be ignored and an accurate correlation coefficient will not be obtained. In particular, when single-precision floating-point (float type) calculations are used to calculate the correlation coefficient between strips from the perspective of cost, etc., this problem becomes more pronounced due to the influence of rounding errors caused by the small number of significant digits.

[0101] <Image data processing unit 231> In addition to addressing the problem of such rounding errors, in order to realize the process of recording correspondence information, which will be described later, this embodiment employs a data processing unit 230 equipped with an image data processing unit 231 shown in FIG. 4B.

[0102] 4B , the image data processing unit 231 includes a mask image generation unit 2311, a range adjustment unit 2312, a composite image generation unit 2313, a cross-correlation function calculation unit 2314, a correlation coefficient calculation unit 2315, an xy shift amount calculation unit 2316, a registration unit 2317, a merge processing unit 2318, a z shift amount calculation unit 232, an image data correction unit 233, a correspondence information generation unit 234, and an information recording unit 235.

[0103] In the example described below, the image data processing unit 231 is configured to designate one of any two strips as a reference strip and to register the other strip (registering strip) with respect to this reference strip.

[0104] <Mask image generation unit 2311> The mask image generator 2311 generates a reference mask image corresponding to the reference strip and a target mask image corresponding to the target strip. Some examples of mask images are described below, but the present invention is not limited to these.

[0105] The contour shape of the mask image is, for example, rectangular, typically square. The shape of the reference mask image and the shape of the target mask image may be the same. Furthermore, the dimensions of the reference mask image and the dimensions of the target mask image may be the same.

[0106] The range of pixel values ​​of the mask image is set to be included in the closed interval [0,1], for example. Typically, the mask image may be a binary image with pixel values ​​of 0 or 1. As a specific example, the mask image is a binary image in which the pixel value in the area corresponding to the domain of the strip is 1 and the other pixels are 0. In other words, as shown in equation (20) of Non-Patent Document 1, the pixel value of the exemplary mask image is 1 in the image area of ​​the corresponding strip and 0 in the other areas.

[0107] <Range Adjustment Unit 2312> The range adjuster 2312 is configured to adjust the pixel value range of the reference strip and the pixel value range of the target strip based on the pixel value range of the reference mask image and the pixel value range of the target mask image. Typically, the range adjuster 2312 adjusts the pixel value range of the reference strip and the pixel value range of the target strip so as to reduce the difference between the pixel value range of the strip and the pixel value range of the mask image.

[0108] In general, the range adjuster 2312 adjusts the pixel value range of the reference strip, the pixel value range of the target strip, and the pixel value range of the mask image relatively. In a typical example, the reference mask image applied to the reference strip and the target mask image applied to the target strip are different, and the range adjuster 2312 adjusts the pixel value range of the reference strip, the pixel value range of the target strip, the pixel value range of the reference mask image, and the pixel value range of the target strip relatively.

[0109] The range adjustment unit 2312 is configured to adjust the range of pixel values ​​of the reference strip and the range of pixel values ​​of the reference mask image so as to reduce the range of pixel values ​​of the reference strip and the range of pixel values ​​of the reference mask image, and to relatively adjust the range of pixel values ​​of the target strip and the range of pixel values ​​of the target strip so as to reduce the range of pixel values ​​of the target strip and the range of pixel values ​​of the target mask image.

[0110] The range adjustment unit 2312 may be configured to adjust the pixel value range of the strip and the pixel value range of the mask image so that the pixel value range of one of the strip and the mask image matches the pixel value range of the other. For example, the range adjustment unit 2312 may be configured to adjust the pixel value range of the reference strip and the pixel value range of the reference mask image so that the pixel value range of the reference strip and the pixel value range of the reference mask image match, and to relatively adjust the pixel value range of the target strip and the pixel value range of the target strip so that the pixel value range of the target strip and the pixel value range of the target mask image match.

[0111] For example, the range adjuster 2312 is configured to normalize the range of pixel values ​​of the strip according to the range of pixel values ​​of the mask image, some examples of this normalization (standardization) are described below, but are not limited to these.

[0112] A first example of normalization will be described. When the range of pixel values ​​of the mask image is included in the closed interval [0, 1], the range adjustment unit 2312 divides the value of each pixel in the strip by the maximum pixel value in the strip. In this example, the range adjustment unit 2312 first compares the values ​​of all pixels in the strip to identify the maximum value (maximum pixel value), and then divides the value of each pixel in the strip by the maximum pixel value. This causes the range of pixel values ​​of the strip to match the closed interval [0, 1], which is the same as the range of pixel values ​​of the mask image.

[0113] A second example of normalization will be described. When the range of pixel values ​​of the mask image is included in the closed interval [0, 1], the range adjustment unit 2312 divides the value of each pixel of the strip by the maximum value of the range of pixel values ​​of this strip. The range of pixel values ​​of the strip is set in advance, and the range adjustment unit 2312 divides the value of each pixel of the strip by the upper limit (maximum value) of this range. In this example, the range of pixel values ​​of the strip is also matched to the same closed interval [0, 1] as the range of pixel values ​​of the mask image.

[0114] The processing performed by the range adjustment unit 2312 can reduce the difference between the absolute values ​​of the pixel values ​​of the strip and the absolute values ​​of the pixel values ​​of the mask, so that the effect of the mask is not ignored in the correlation calculation using the strip and the mask, and it becomes possible to accurately calculate the correlation coefficient. In particular, even when single-precision floating-point (float) calculation is used, it is possible to eliminate (reduce) the influence of rounding errors caused by a small number of significant digits.

[0115] Although this specification mainly describes an example of changing only the range of pixel values ​​of the strip, it is also possible to change only the range of pixel values ​​of the mask image, or to change both the range of pixel values ​​of the strip and the range of pixel values ​​of the mask image.

[0116] <Synthetic image generation unit 2313> For the strips and mask images to which the pixel value range adjustment by the range adjustment unit 2312 has been applied, the composite image generation unit 2313 generates two composite images by combining the mask image with each of the reference strip and the target strip. Typically, the composite image generation unit 2313 combines the reference mask image with the reference strip to generate a reference composite image, and combines the target mask image with the target strip to generate a target composite image.

[0117] The process of combining the strip and the mask image may be performed in the same manner as in Non-Patent Document 1. However, unlike the method in Non-Patent Document 1, in this embodiment, the difference between the pixel value range of the strip and the pixel value range of the mask image is adjusted to be small. Typically, the pixel value ranges of the reference strip and the target strip are normalized to match the pixel value ranges of the reference mask image and the target mask image.

[0118] For example, in the same manner as equation (19) in Non-Patent Document 1, the composite image generation unit 2313 may be configured to embed a strip with a normalized pixel value range into an image of the same size and shape as the mask image.

[0119] Furthermore, the composite image generation unit 2313 generates a composite image of the embedded image of the strip and the mask image. This composite image corresponds to "f´(r)m f (r)" (such as the second line on page 1800) in Non-Patent Document 1. However, as described above, the value of the embedded image of the strip is different from that in Equation (19).

[0120] <Cross-correlation function calculation unit 2314> The cross-correlation function calculation unit 2314 obtains a plurality of cross-correlation functions based on two composite images generated by the composite image generation unit 2313. The calculation of the cross-correlation function is performed in the same manner as in Non-Patent Document 1. For example, the cross-correlation function calculation unit 2314 calculates six cross-correlation functions (image cross-correlation) included in Equation (33) of Non-Patent Document 1 based on a reference composite image generated from a reference strip and a reference mask image and a target composite image generated from a target strip and a target mask image.

[0121] <Correlation coefficient calculation unit 2315> The correlation coefficient calculation unit 2315 calculates a correlation coefficient based on the plurality of cross-correlation functions calculated by the cross-correlation function calculation unit 2314. This calculation follows Equation (33) of Non-Patent Document 1.

[0122] <XY shift amount calculation unit 2316> The XY shift amount calculation unit 2316 calculates the shift amount in the XY direction (lateral direction, horizontal direction) between the reference strip and the target strip based on the correlation coefficient calculated by the correlation coefficient calculation unit 2315.

[0123] For example, the calculation for calculating the XY direction shift amount includes an operation corresponding to "rough lateral motion correction" (page 1787) in Non-Patent Document 1, and the XY shift amount calculation unit 2316 is configured to estimate the XY direction shift amount by obtaining the maximum value of the cross-correlation function.

[0124] Furthermore, the xy shift amount calculation unit 2316 may be configured to perform a calculation equivalent to "fine lateral motion correction" (page 1789) in Non-Patent Document 1 in order to calculate small lateral shift amounts caused by eye movements such as slow drift and tremor.

[0125] <Registration Department 2317> The registration unit 2317 performs lateral registration based on the lateral shift amount calculated by the xy shift amount calculation unit 2316. For example, this registration includes processing equivalent to "rough lateral motion correction" (page 1787) in Non-Patent Document 1. The registration unit 2317 can perform registration between the reference strip and the target strip so as to cancel out the lateral shift amount calculated by the xy shift amount calculation unit 2316.

[0126] When the xy shift amount calculation unit 2316 performs a calculation equivalent to "fine lateral motion correction" (page 1789) in Non-Patent Document 1, the registration unit 2317 can remove small lateral motion artifacts between the reference strip and the target strip by performing a registration equivalent to "fine lateral motion correction" (page 1789) in Non-Patent Document 1.

[0127] <Merge processing section 2318> The merge processing unit 2318 constructs a merged image of the reference strip and the target strip whose relative positions have been adjusted by the registration unit 2317. This processing is also performed in the same manner as the method described in Non-Patent Document 1.

[0128] As described above, the image data processing unit 231 sequentially executes the above-described series of processes on the plurality of strips constructed by the image data construction unit 220 in accordance with the order according to the size. As a result, a merged image in which lateral motion artifacts are corrected is obtained from the plurality of strips constructed by the image data construction unit 220. This merged image is typically an image representing the entire range to which the resampled scan is applied.

[0129] As described above, the plurality of strips are a plurality of front projection images based on a plurality of sub-volumes obtained by dividing the volume (3D data) collected by the resampled scan. Therefore, the merged image constructed from the plurality of strips by the image data processing unit 231 provides a plurality of sub-volumes (and their merged images) with lateral position adjustment. In this example, the registration and merging processes in the depth direction orthogonal to the lateral direction are executed by the z-shift amount calculation unit 232 and the image data correction unit 233 (described later).

[0130] <z-shift amount calculation unit 232> The z-shift amount calculation unit 232 calculates the shift amount in the z direction (A-line direction, optical axis direction, depth direction) between a plurality of sub-volumes obtained by dividing the volume collected by the resampled scan (that is, the plurality of sub-volumes that are the basis of the plurality of strips).

[0131] This example is configured such that after the image data processing unit 231 performs position adjustment in the xy direction (lateral direction, horizontal direction), the z-shift amount calculation unit 232 and the image data correction unit 233 perform position adjustment in the z direction. That is, the z-shift amount calculation unit 232 of this example is configured to calculate the shift amount in the z direction using the data (for example, xy direction shift amount data, merged image) obtained by the image data processing unit 231.

[0132] An example of processing executed by the z-shift amount calculation unit 232 will be described. Similar to the pair of strips (reference strip and target strip) considered in calculating the shift amounts in the x and y directions, a pair of sub-volumes is also considered in calculating the shift amounts in the z direction. The pair of sub-volumes may be two sub-volumes corresponding to the reference strip and target strip considered in calculating the shift amounts in the x and y directions, and these sub-volumes are referred to as the reference sub-volume and the target sub-volume, respectively.

[0133] Based on the result of the xy-direction registration between the corresponding reference strips and target strips, the reference sub-volumes and target sub-volumes are aligned in the xy-direction. Also, as mentioned above, any two strips in a Lissajous scan overlap (intersect) at four points, and therefore any two sub-volumes also overlap (intersect) at four points.

[0134] The z-shift amount calculation unit 232 first identifies an intersection area (common area) between the reference sub-volume and the target sub-volume whose positions have been adjusted in the x and y directions. Each intersection area is three-dimensional image data.

[0135] Next, the z-shift amount calculation unit 232 sets a cross section in the identified intersection area. This cross section is, for example, a plane formed by an arbitrary axis (for example, the x-axis, the y-axis, or an axis obliquely intersecting both the x-axis and the y-axis) in the xy plane and the z-axis. Note that the cross section is not limited to a plane, and may be a curved surface, etc.

[0136] Next, the z-shift amount calculation unit 232 constructs an image of the set cross section from the reference sub-volume, and also constructs an image of the same cross section from the target sub-volume. The cross-sectional image constructed from the reference sub-volume is called the reference cross-sectional image, and the cross-sectional image constructed from the target sub-volume is called the target cross-sectional image. The reference cross-sectional image and the target cross-sectional image are images that represent the same cross section in the intersection region between the reference sub-volume and the target sub-volume that have been registered in the x- and y-directions.

[0137] Next, the z-shift amount calculation unit 232 analyzes the reference cross-sectional image to identify an image of a predetermined portion of the subject's eye E, and analyzes the target cross-sectional image to identify an image of the same portion. This portion may be any portion, and may be, for example, the surface of the fundus Ef (the retinal surface, the internal limiting membrane, or the boundary between the retina and the vitreous body). Furthermore, if an artificial object has been implanted in the subject's eye E, an image of this artificial object may be identified. The analysis for identifying the image of the predetermined portion may include, for example, segmentation.

[0138] Next, the z-shift amount calculation unit 232 calculates the z-coordinate of an image (reference image) of a predetermined region identified from the reference cross-sectional image, and calculates the z-coordinate of an image (target image) of a predetermined region identified from the target cross-sectional image. Typically, the image of the predetermined region consists of multiple pixels, and the z-coordinates of these pixels are not constant. For example, the global shape of the retinal surface is generally a curved shape convex in the +z direction, and images of local areas (examples of the reference image and target image) are at least partially tilted with respect to the z-axis. The z-shift amount calculation unit 232 can calculate the z-coordinate of the image of the predetermined region based on at least one pixel of a group of pixels constituting the image of the predetermined region. For example, a statistic of the z-coordinates can be calculated from the group of pixels, and this statistic can be used as the z-coordinate of the image. This statistic can be any representative value (summary statistic), such as a maximum value, minimum value, mean value, median, mode, or quantile. On the other hand, if the image of the predetermined region consists of a single pixel, the z-coordinate of this pixel can be used as the z-coordinate of the predetermined region.

[0139] Next, the z-shift amount calculation unit 232 calculates the difference between the z-coordinate of the reference image and the z-coordinate of the target image (the shift amount in the z direction). In some exemplary aspects, processing control can be performed so that correction processing is performed by the image data correction unit 233 when the z-direction shift amount exceeds a threshold. This threshold is, for example, zero or a positive value, and may be a preset fixed value or a variable value.

[0140] In the above example, the z-direction shift amount is calculated by constructing two-dimensional image data (cross-sectional images) from three-dimensional image data (sub-volumes), but in some exemplary aspects, the z-direction shift amount may be calculated from three-dimensional image data without constructing two-dimensional image data. In this case, the same processing as that performed on the pair of reference cross-sectional images and target cross-sectional images can be performed on the pair of reference sub-volumes and target sub-volumes.

[0141] In the above example, two-dimensional image data (cross-sectional image) is constructed from three-dimensional image data (subvolume), and an image of a specified area is identified from this two-dimensional image data to determine the amount of shift in the z direction. However, in some exemplary embodiments, an image of a specified area may be identified from the three-dimensional image data, and two-dimensional image data of the image of this specified area may be constructed to determine the amount of shift in the z direction.

[0142] The z-shift amount calculation unit 232 may be configured to calculate the z-direction shift amount between the reference subvolume and the target subvolume based on a correlation coefficient calculated based on a pair of the reference subvolume and the target subvolume. The method for calculating the correlation coefficient between the subvolumes may be the same as the correlation coefficient calculation method executed by the correlation coefficient calculation unit 2315, for example.

[0143] The method for calculating the z-direction shift amount is not limited to the above-mentioned methods, and may be any method applicable to a pair of a reference subvolume and a target subvolume, or may be any method applicable to a pair of an object other than a subvolume. An example of an object other than a subvolume is a rendered image of the subvolume (an image having a dimension in the z direction, such as an image projected in the x direction or y direction).

[0144] <Image data correction unit 233> The image data correction unit 233 is configured to perform z-direction position adjustment on the multiple sub-volumes whose positions have been adjusted in the x- and y-directions by the image data processing unit 231, based on the z-direction shift amount obtained by the z-shift amount calculation unit 232.

[0145] Similar to the registration unit 2317 that performs registration based on the xy direction shift amount, the image data correction unit 233 performs registration between the reference subvolume and the target subvolume so as to cancel out the corresponding z direction shift amount obtained by the z shift amount calculation unit 232.

[0146] As mentioned above, in some exemplary embodiments, the image data corrector 233 may be controlled to perform registration between the reference subvolume and the target subvolume only if the corresponding z-direction shift exceeds a threshold.

[0147] The image data obtained as a result of the position adjustment by the image data corrector 233 is a group of subvolumes that have been registered in all of the x-, y-, and z-directions, i.e., a group of subvolumes that have been three-dimensionally registered. The group of subvolumes before application of such registration is a volume acquired by Lissajous scanning. The data processor 230 recombines the group of subvolumes obtained by three-dimensional registration to obtain a volume to which three-dimensional registration has been applied.

[0148] The 3D registered dataset can then be used for any processing (image processing, post-processing, analysis, evaluation, visualization, rendering, etc.) This allows the desired processing to be performed using a 3D dataset that has been corrected for motion artifacts with higher accuracy and precision than ever before.

[0149] In addition, the z-shift amount calculation unit 232 and the image data correction unit 233 may be configured to perform registration equivalent to "axial motion correction (rough axial motion correction and / or fine axial motion correction)" (pages 1789 to 1790) of Non-Patent Document 1 in order to remove motion artifacts in the z direction.

[0150] <Correspondence information generation unit 234> First, the premise of the processing executed by the correspondence information generation unit 234 will be described. As described above, the correlation coefficient calculation unit 2315 calculates the correlation coefficient between strips, and the z-shift amount calculation unit 232 calculates the correlation coefficient between subvolumes. Both the strips and the subvolumes are defined based on cycles. Therefore, the correlation coefficient between strips calculated by the correlation coefficient calculation unit 2315 can be regarded as the correlation coefficient between cycles, and the correlation coefficient between subvolumes calculated by the z-shift amount calculation unit 232 can be regarded as the correlation coefficient between cycles.

[0151] Thus, in the present disclosure, the correlation coefficient between cycles includes the correlation coefficient between strips and / or the correlation coefficient between sub-volumes. That is, some exemplary embodiments may handle only the correlation coefficient between strips, some exemplary embodiments may handle only the correlation coefficient between sub-volumes, and some exemplary embodiments may handle both the correlation coefficient between strips and the correlation coefficient between sub-volumes. Furthermore, some exemplary embodiments may handle the correlation coefficient between objects that are different from both strips and sub-volumes. In any case, it is possible to perform processing similar to that of this embodiment.

[0152] Furthermore, the correlation coefficient between strips may be the correlation coefficient between two strips, the correlation coefficient between one strip and two or more strips (strip group), or the correlation coefficient between two strip groups. The strip group may be, for example, a merged strip (described above) based on two or more strips. In this embodiment, in either case, the correlation coefficient calculation unit 2315 performs the calculation, and the content of the calculation may be the same as the processing described above.

[0153] Similarly, the correlation coefficient between subvolumes may be the correlation coefficient between two subvolumes, the correlation coefficient between one subvolume and two or more subvolumes (subvolume group), or the correlation coefficient between two subvolume groups. In any case, the z-shift amount calculation unit 232 performs the calculation, and the content of the calculation may be the same as the processing described above. The same applies when the correlation coefficient between objects other than strips and subvolumes is handled.

[0154] As described above, the ophthalmic examination apparatus 1 applies optical scanning (e.g., Lissajous scanning) according to a two-dimensional pattern including a series of cycles that intersect with each other to the subject's eye E to collect a data set (data set collection unit), and calculates a correlation coefficient between the cycles based on this data set (correlation coefficient calculation unit: correlation coefficient calculation unit 2315 and / or z-shift amount calculation unit 232).

[0155] In this embodiment, multiple correlation coefficients are calculated. For example, when a correlation coefficient between two strips is calculated, if the number of strips (number of cycles) is N, the maximum number of correlation coefficients calculated from the N strips is N×(N−1) (if overlapping combinations are combined, the maximum number is N×(N−1) / 2). The number of correlation coefficients calculated in this case is determined by the combination of the strips (i.e., the number of strip pairs considered). Similarly, when a correlation coefficient between one strip and a strip group is calculated or when a correlation coefficient between two strip groups is calculated, multiple correlation coefficients are calculated. In these cases, the number of correlation coefficients calculated is determined by the number of strip groups considered, the combination of the strips and strip groups, the combination of the strip groups, and the like. For example, when the aforementioned sequential calculation performed by the correlation coefficient calculation unit 2315 is applied, N−1 correlation coefficients are calculated. In this way, a set of correlation coefficients (referred to as a correlation coefficient set) is generated from the collected data set. The set of correlation coefficients may include as elements all of the correlation coefficients calculated from the data set collected by optical scanning, or may include as elements only a portion of all of these correlation coefficients.

[0156] The correspondence information generating unit 234 is configured to generate information representing the correspondence between the correlation coefficient set thus generated and a set of position information in the application area of ​​optical scanning (referred to as a position information set). The information representing the correspondence between the correlation coefficient set and the position information set is referred to as correspondence information.

[0157] The area of ​​application of the optical scanning is the area of ​​the fundus Ef to which the optical scanning performed to obtain this set of correlation coefficients was applied, in other words, the area of ​​the fundus Ef from which the data set used to generate this set of correlation coefficients was collected.

[0158] The position information in the application area of ​​optical scanning is information that represents a position within this application area. The position information set is a set whose elements are a plurality of pieces of position information that respectively correspond to a plurality of positions within this application area.

[0159] The method of expressing the location information may be any predetermined method of expression, and for example, the location information may be expressed using any of the following exemplary identifiers:

[0160] A first example of an identifier representing position information is the coordinates of the definition coordinate system of image data constructed by the image data processing unit 231. When this example identifier is applied, for example, position information may be defined for each pixel (in other words, using pixels as units), or for a predetermined image region (in other words, using pixel groups as units), or position information may be defined using both of these. A position information set when this example identifier is applied is a set whose elements are multiple coordinates corresponding to multiple positions in the application area of ​​optical scanning.

[0161] A second example of an identifier representing position information is a number (called a cycle number) based on the order of cycles. When this example identifier is applied, for example, position information may be defined for each cycle (in other words, in units of cycles (i.e., cycle numbers)), or for several cycles (in other words, in units of cycle groups (i.e., several cycle numbers)), or position information may be defined using both of these. When this example identifier is applied, a position information set is a set whose elements are multiple cycle numbers corresponding to multiple positions in the optical scanning application area. In some exemplary embodiments, one cycle may be divided into two or more parts (called subcycles), and each subcycle may be assigned a number. The number assigned to the subcycle may take any form. For example, if the first cycle is divided into two subcycles (a first subcycle and a second subcycle), the first subcycle may be assigned the cycle number "1" and the second subcycle may be assigned the cycle number "1.5." Alternatively, the first subcycle can be assigned the cycle number "1-1," and the second subcycle can be assigned the cycle number "1-2." Similar "sub-identifiers" can also be used for identifiers other than cycle numbers. For example, similar sub-identifiers can be defined for any of the following: scan number, strip number, subvolume number, and merge process number.

[0162] A third example of an identifier representing position information is a number based on the order of scans (called a scan number). When this example identifier is applied, for example, position information may be defined for each cycle of scans, or for several cycles of scans, or position information may be defined using both of these. In this embodiment, since a scan is performed for each cycle, the scan number in this example identifier is substantially the same as the cycle number in the second example. When this example identifier is applied, the position information set is a set whose elements are multiple scan numbers corresponding to multiple positions in the application area of ​​optical scanning.

[0163] A fourth example of an identifier representing position information is a number (called a strip number) based on the ordering of strips. When this example identifier is applied, for example, position information may be defined for each strip (in other words, in units of strips (i.e., strip numbers)), or for several strips (in other words, in units of strip groups (i.e., several strip numbers)), or position information may be defined using both of these. Since there is a one-to-one correspondence between cycles and strips in the processing performed by the image data processing unit 231 described above, the strip number in this example identifier is substantially the same value as each of the cycle number in the second example and the scan number in the third example. When this example identifier is applied, the position information set is a set whose elements are multiple strip numbers, each corresponding to multiple positions in the application area of ​​optical scanning.

[0164] A fifth example of an identifier representing position information is a number (called a subvolume number) based on the ordering of subvolumes. When this example identifier is applied, for example, position information may be defined for each subvolume (in other words, in units of subvolumes (i.e., subvolume numbers)), or for several subvolumes (in other words, in units of subvolume groups (i.e., several subvolume numbers)), or position information may be defined using both of these. Since strips and subvolumes correspond one-to-one in the processing performed by the image data processing unit 231 described above, the subvolume number in this example identifier is substantially the same value as the cycle number in the second example, the scan number in the third example, and the strip number in the fourth example. When this example identifier is applied, the position information set is a set whose elements are multiple subvolume numbers, each corresponding to multiple positions in the optical scanning application area.

[0165] A sixth example of an identifier representing position information is a number (called a merge process number) based on the order in which the merge processes are applied. In the processing executed by the image data processing unit 231 described above, the merge process is executed for each strip or each subvolume, so the merge process number in this example identifier is substantially the same value as the cycle number in the second example, the scan number in the third example, the strip number in the fourth example, and the subvolume number in the fifth example. When this example identifier is applied, the position information set is a set whose elements are multiple merge process numbers that respectively correspond to multiple positions in the application area of ​​optical scanning.

[0166] The seventh example of the identifier representing position information is a parameter indicating a scanning position in optical scanning. This example identifier may be a predetermined scanning control parameter, for example, any of the scanning control parameters used in the scanning protocol 2121 described above. As a specific example, the scanning control parameter of this example identifier may include any parameter indicating the content of control over the optical scanner 44 (e.g., the content of control regarding the orientation of each galvanometer mirror included in the optical scanner 44), such as a parameter indicating a scan pattern, a parameter indicating a scan speed, or a parameter indicating a scan interval. By taking into account the configuration (design) of the optical system for performing optical scanning (in this embodiment, the optical system of the fundus camera unit 2, the optical system of the OCT unit 100, etc.), the scan number of this example identifier is substantially the same as the coordinate of the first example. Similarly, by taking into account the configuration (design) of the optical system for performing optical scanning, the first to seventh example identifiers are all the same. A position information set when this example identifier is applied is a set whose elements are multiple scanning control parameter values ​​corresponding to multiple positions in the application area of ​​optical scanning.

[0167] The correspondence information generating unit 234 generates correspondence information that indicates the correspondence between the correlation coefficient set and the position information set. That is, the correspondence information generating unit 234 generates correspondence information that indicates the correspondence between the elements of the correlation coefficient set and the elements of the position information set.

[0168] The correspondence information is generated by establishing a correspondence between elements of the set of correlation coefficients and elements of the set of location information, which may be at least partially a one-to-one correspondence, at least partially a one-to-many correspondence, at least partially a many-to-one correspondence, or at least partially a many-to-many correspondence.

[0169] The elements of the correlation coefficient set and the elements of the position information set can be associated with each other via the above-mentioned identifiers, for example. For example, the correspondence information generating unit 234 can associate the elements of the correlation coefficient set with the elements of the position information set based on the fact that the correlation coefficient is calculated for each cycle, the position information is defined using an identifier, and the above-mentioned first to seventh example identifiers are all the same value.

[0170] For example, when the identifier of the positional information is a scan number, the correspondence information generating unit 234 may be configured to identify, for a correlation coefficient that is an arbitrary element of the correlation coefficient set, a cycle number corresponding to the correlation coefficient, identify positional information corresponding to this cycle number, and associate the positional information with the correlation coefficient.

[0171] As another example, when the identifier of the positional information is not a cycle number, for example, when the identifier of the positional information is any one of a coordinate, a scan number, a strip number, a subvolume number, a merge processing number, and a scan control parameter, the correspondence information generating unit 234 may be configured to identify, for a correlation coefficient that is an arbitrary element of the correlation coefficient set, a cycle number corresponding to the correlation coefficient, identify the value of the identifier corresponding to this cycle number, identify positional information corresponding to this identifier value, and associate this positional information with the correlation coefficient.

[0172] Several examples of the correspondence information generated in this manner will be described below. An example of the correspondence information is shown in Fig. 6. The correspondence information 270 in this example is a table showing the correspondence between N pieces of position information P1 to PN and N pieces of correlation coefficients R1 to RN. In the correspondence information 270, the position information Pn is associated with the correlation coefficient Rn (n = 1 to N).

[0173] The form of the correspondence information is not limited to the table format shown in Fig. 6 and may be any format. For example, when the identifier of the location information is a coordinate, or when the identifier of the location information is the same value as the coordinate, the correspondence information generating unit 234 can generate, as the correspondence information, a map that expresses the spatial distribution of the elements (plurality of correlation coefficients) of the correlation coefficient set according to the spatial distribution (for example, one-dimensional distribution, two-dimensional distribution, or three-dimensional distribution) of the elements (plurality of location information) of the location information set. Such a map will be described in other embodiments.

[0174] <Information Recording Unit 235> The information recording unit 235 records the correspondence information generated by the correspondence information generating unit 234. In this embodiment, the information recording unit 235 records the correspondence information in association with image data constructed based on a data set collected by optical scanning.

[0175] The processes executed by the information recording unit 235 include a process of associating the correspondence information with the image data, a process of saving the correspondence information in a predetermined storage device, and a process of saving the image data in a predetermined storage device. The correspondence information and the image data may be saved in the same storage device or in separate storage devices. Examples of such storage devices include a storage device (not shown) included in the information recording unit 235, the storage unit 212, a storage device accessible by the ophthalmic examination apparatus 1 (such as an in-facility database system or a cloud-based database system), and a recording medium to which data can be written by the ophthalmic examination apparatus 1.

[0176] The range of information included in the correspondence information may be arbitrary. In some exemplary aspects, the correspondence information may represent the correspondence between a subset of a set consisting of all correlation coefficients calculated by the image data processing unit 231 (this subset is the correlation coefficient set in these aspects) and a position information set whose elements are position information corresponding to each element of this subset.

[0177] The content of the information included in the correspondence information may be arbitrary. The correspondence information may include at least correlation coefficients and position information. For example, if the set of correlation coefficients included in the correspondence information is a subset of the set consisting of all correlation coefficients obtained by the image data processing unit 231, the correspondence information may include, in addition to the correlation coefficients and position information, information regarding the judgment conditions for determining this subset and the judgment results. For example, some exemplary aspects of the correspondence information may include information indicating the content of the judgment conditions and / or information indicating the content of the judgment results. Examples of information indicating the content of the judgment conditions include thresholds and ranges used to determine whether the correlation coefficient values ​​are large or small (e.g., a first threshold, a second threshold, a specific range, etc., which will be described later). Examples of information indicating the content of the judgment results include a judgment result indicating whether the judgment conditions are satisfied, the degree to which the judgment conditions are satisfied, the degree to which the judgment conditions are deviated from the judgment conditions, etc.

[0178] When the set of position information included in the correspondence information is a subset of the set of position information corresponding to the application area of ​​optical scanning, the correspondence information may include, in addition to the correlation coefficient and the position information, information regarding the judgment conditions and the judgment results for determining this subset. In some exemplary embodiments, the correspondence information may include information indicating the contents of the judgment conditions and / or information indicating the contents of the judgment results. For example, when considering a set of position information corresponding to a region corresponding to a predetermined region of the fundus Ef (e.g., the optic disc, the macula, etc.) or an area including this region, the correspondence information may include information indicating the position of this region in the application area of ​​optical scanning, information indicating the type of this region (e.g., the name of the region), segmentation conditions for identifying this region, etc. As another example, when considering a set of position information corresponding to a region corresponding to a lesion of the fundus Ef or an area including this region, the correspondence information may include information indicating the position of this region in the application area of ​​optical scanning, information indicating the type of this region (e.g., the name of the lesion), processing conditions for detecting this lesion, etc. Note that the information included in the correspondence information is not limited to these examples.

[0179] <Processing unit 260> 4C, the data processing unit 230 of the ophthalmic examination apparatus 1 of this embodiment includes a processing unit 260. In this embodiment, the processing unit 260 is provided in the scanning imaging device (ophthalmic examination apparatus 1), but in some exemplary embodiments, a similar processing unit may be provided in a device (e.g., an information processing device) separate from the scanning imaging device.

[0180] The processing unit 260 is configured to execute predetermined processing based on the correspondence information recorded by the information recording unit 235. The content of the processing executed by the processing unit 260 may be arbitrary, and some non-limiting examples thereof will be described in the embodiments described below.

[0181] <User Interface 240> The user interface 240 includes a display unit 241 and an operation unit 242. The display unit 241 includes the display device 3. The operation unit 242 includes various operation devices and input devices. The user interface 240 may include a device that combines a display function and an operation function, such as a touch panel. It is also possible to construct an embodiment that does not include at least a part of the user interface 240. For example, the display device may be an external device connected to the ophthalmic examination apparatus 1.

[0182] <Data Receiving Unit 250> The data receiving unit 250 acquires data from an external device. This external device may be, for example, a computer, a storage device, a recording medium, or an information system (for example, a hospital information system, an electronic medical record system, or an image archiving system). The external device may be, for example, a device directly connected to the ophthalmic examination apparatus 1, a device connected to the ophthalmic examination apparatus 1 via a local area network (LAN), or a device connected to the ophthalmic examination apparatus 1 via a wide area network (WAN).

[0183] The data receiving unit 250 may include, for example, a communication interface, a drive device, etc. The data receiving unit 250 may also include a scanner for reading data recorded on paper sheets, etc. The data receiving unit 250 may also include an input device for a user to input data. The input device may be a keyboard, a pen tablet, etc.

[0184] <Operation> The following describes the operation of the ophthalmic examination apparatus 1. An example of the operation of the ophthalmic examination apparatus 1 is shown in Figures 7A to 7F. In this operation example, the generation and recording of correspondence information, which are features of this embodiment, and processing based on the correspondence information, as well as image data construction in Lissajous scanning (which includes motion artifact correction) will be described in detail.

[0185] In some exemplary embodiments, only some of the steps in this example operation may be performed. For example, some exemplary embodiments may not include a step of performing processing based on the correspondence information (i.e., processing performed by the processing unit 260). In some exemplary embodiments, some of the steps in this example operation may be replaced with other steps (e.g., similar steps). In some exemplary embodiments, any steps may be combined in this example operation.

[0186] Prior to step S1, the same preparatory operations as in the conventional method are performed, such as inputting the patient ID, setting the scanning mode (specifying Lissajous scan), presenting the fixation target, alignment, focus adjustment, and OCT optical path length adjustment.

[0187] (S1: Lissajous Scan) Upon receiving a predetermined scan start trigger signal, the scan control unit 2111 starts applying an OCT scan (Lissajous scan) to the subject's eye E (fundus oculi Ef).

[0188] The scanning start trigger signal is generated, for example, in response to the completion of a predetermined preparatory operation (alignment, focus adjustment, OCT optical path length adjustment, etc.), or in response to a scanning start instruction operation being performed using the operation unit 242.

[0189] The scan control unit 2111 applies a Lissajous scan to the fundus Ef by controlling the optical scanner 44, the OCT unit 100, etc. based on a scan protocol 2121 (a protocol corresponding to the Lissajous scan). The data collected by the Lissajous scan is sent to the image data construction unit 220.

[0190] (S2: Building multiple strips) The image data constructing unit 220 constructs a plurality of strips based on the data collected in step S1 in the manner described above. The constructed plurality of strips are sent to the data processing unit 230.

[0191] (S3: Set the reference strip and target strip) The data processing unit 230 orders the multiple strips constructed in step S2 according to their dimensions (such as area). In this example, the number of multiple strips constructed in step S2 is assumed to be N (N is an integer equal to or greater than 2). Furthermore, these N strips are referred to as the first strip, second strip, ..., Nth strip according to the order specified by the ordering. Furthermore, any strip from the first to Nth strips may be referred to as the nth strip (n = 1, 2, ..., N). In this way, the first strip is the largest strip, the second strip is the second largest strip, the nth strip is the nth largest strip, and the Nth strip is the smallest strip.

[0192] In this example, the data processing unit 230 sets the first strip as the initial reference strip and the second strip as the initial target strip.

[0193] The reference strip and the target strip correspond to the arbitrary shaped images f(r) and g(r) in Appendix A of Non-Patent Document 1, respectively.

[0194] (S4: Set the reference mask image and the target mask image) The mask image generation unit 2311 generates a mask image (reference mask image) corresponding to the reference strip and a target mask image corresponding to the target strip.

[0195] At this stage, the mask image generation unit 2311 generates two mask images corresponding to the first strip and the second strip, respectively. The data processing unit 230 sets the first mask image corresponding to the first strip as an initial reference mask image, and sets the second mask image corresponding to the second strip as an initial target mask image.

[0196] The reference mask image and the target mask image are rectangular shaped binary image masks m in Appendix A of Non-Patent Document 1, respectively. f (r) and m g Equivalent to (r).

[0197] (S5: Normalize the reference strip and the target strip) The range adjustment unit 2312 normalizes the reference strip and the target strip set in step S3 in the manner described above. The strips are normalized according to the range of pixel values ​​of the mask image. At this stage, the range adjustment unit 2312 applies normalization to the first strip (reference strip) and the second strip (target strip). Note that the process performed by the range adjustment unit 2312 is not limited to normalization, and may be any of the range adjustment processes described above or a process similar thereto.

[0198] In this example, the pixel value range of each mask image is assumed to be in the closed interval [0, 1]. In particular, each mask image in this example is assumed to be a binary image in which the pixel values ​​within the region corresponding to the domain of the corresponding strip are set to 1 and the other pixel values ​​are set to 0.

[0199] The range adjustment unit 2312 normalizes the strip, for example, by dividing the value of each pixel in the strip by the maximum pixel value in the strip, or by dividing the value of each pixel in the strip by the maximum value of the range of pixel values ​​(tone range) of the strip.

[0200] Furthermore, the data processing unit 230 (synthetic image generation unit 2313) embeds the strip with the normalized pixel value range into an image with the same dimensions and shape as the corresponding mask image. The embedded image of the normalized first strip f(r) is similar to the rectangular shaped image f'(r) in Appendix A of Non-Patent Document 1, but the absolute value (|f(r)|) of the value range within the image area of ​​the first strip f(r) is normalized to 1 or less. This embedded image is also represented as f'(r).

[0201] Similarly, the embedded image of the normalized second strip g(r) is an image similar to the rectangular image g'(r), but the absolute value of the range within the image area of ​​the second strip g(r) (|g(r)|) is normalized to be less than or equal to 1. This embedded image is also denoted as g'(r).

[0202] (S6: Generate a reference composite image and a target composite image) The composite image generator 2313 generates a composite image of the embedded image of the normalized strip and the corresponding mask image.

[0203] At this stage, the composite image generation unit 2313 generates a composite image of the embedded image of the normalized first strip (reference strip) and the first mask image (reference mask image), and also generates a composite image of the embedded image of the normalized second strip (target strip) and the second mask image (target mask image). The former composite image is called the reference composite image, and the latter is called the target composite image.

[0204] Here, the reference composite image and the target composite image are the combination of two rectangle shaped images f'(r)m in Appendix A of Non-Patent Document 1, respectively. f (r) and g´(r)m g (r), where, as mentioned above, the absolute value of the range of the first strip f(r) in the image area (|f(r)|) is less than or equal to 1, and the absolute value of the range of the second strip g(r) in the image area (|g(r)|) is less than or equal to 1.

[0205] Image 311 in Figure 8A is an example of an embedded image f'(r) of the normalized first strip f(r), and image 321 is an example of a first mask image. Combining these two images 311 and 321 produces a reference composite image f'(r)m f (r) is obtained.

[0206] Similarly, image 312 in Figure 8B is an example of an embedded image g'(r) of the normalized second strip g(r), and image 322 is an example of a second mask image. Combining these two images 312 and 322 produces a reference composite image g'(r)m g (r) is obtained.

[0207] (S7: Calculate multiple cross-correlation functions) The cross-correlation function calculation unit 2314 calculates a plurality of cross-correlation functions based on the reference composite image and the target composite image generated in step S6. In this example, the cross-correlation function calculation unit 2314 calculates six cross-correlation functions included in equation (33) of Non-Patent Document 1 based on the reference composite image and the target composite image generated in step S6.

[0208] (S8: Calculate the correlation coefficient) The correlation coefficient calculation unit 2315 calculates a correlation coefficient based on the multiple cross-correlation functions calculated in step S7. In this example, the correlation coefficient calculation unit 2315 calculates the correlation coefficient (ρ(r')) from the multiple cross-correlation functions calculated in step S7 according to equation (33) in Non-Patent Document 1.

[0209] (S9: Record the correlation coefficient) The correspondence information generation unit 234 (or, for example, another element of the image data processing unit 231) records the correlation coefficient (ρ(r')) calculated in step S8. For example, the correspondence information generation unit 234 records this correlation coefficient and a corresponding identifier (position information) in association with each other. As described above, this identifier may be any of the coordinates in the definition coordinate system of the image data, the cycle number, the scan number, the strip number, or an identifier different from these. By repeating steps S3 to S14, correlation coefficients that are elements of the correlation coefficient set of this Lissajous scan (and position information that is an element of the position information set) are accumulated.

[0210] (S10: Calculate the shift amount in the x and y directions between the reference strip and the target strip) The xy shift amount calculation unit 2316 calculates the amount of shift in the xy directions between the first strip (reference strip) f(r) and the second strip (target strip) g(r) based on the correlation coefficient calculated in step S8. In this example, the relative amount of shift in the xy directions (Δx, Δy) between the reference strip f(r) and the target strip g(r) is found by detecting the peak of the correlation coefficient ρ(r') calculated in step S8.

[0211] (S11: Save the shift amount in the x and y directions) The x and y direction shift amounts calculated for the strip pair in step S10 are stored in, for example, the storage unit 212. The x and y direction shift amounts may be recorded together with time information corresponding to the strip pair. This time information may be time parameters corresponding to the Lissajous scan or information based thereon, or sequence information regarding the Lissajous scan or information based thereon. Examples of time information include a cycle number, a scan number, a strip number, etc. The x and y direction shift amounts sequentially acquired by repeating steps S3 to S13 are recorded together with the time information to generate x and y position history data.

[0212] (S12: Perform registration between the reference strip and the target strip) The registration unit 2317 applies x- and y-direction registration (rough lateral motion correction) to the reference strip and the target strip based on the x- and y-direction shift amounts calculated in step S10. At this stage, registration between the first strip f(r) and the second strip g(r) is performed.

[0213] Furthermore, the registration unit 2317 may apply any xy registration to the reference strip and the target strip, such as the "fine lateral motion correction" described above.

[0214] (S13: Construct a merged image of the reference strip and the target strip) The merge processing unit 2318 constructs a merged image of the reference strip and the target strip whose relative positions have been adjusted by the registration in step S12. Image 330 in Figure 8C shows an example of a merged image of the first strip f(r) and the second strip g(r).

[0215] Here, an example of implementing the processes in steps S7 to S13 will be described. First, f'(r)mf (r), g´(r)m g (r), m f (r), m g (r), (f´(r)m f (r)) 2 , and (g´(r)m g (r)) 2 By setting the imaginary part of each of the above to 0, we get only the real part.

[0216] Next, f´(r)m f (r), g´(r)m g (r), m f (r), m g (r), (f´(r)m f (r)) 2 , and (g´(r)m g (r)) 2 A fast Fourier transform (FFT) is applied to each real part of

[0217] Next, based on the group of functions obtained by these fast Fourier transforms, the six cross-correlation functions shown in step S7 of FIG. 7A are derived.

[0218] An inverse fast Fourier transform (IFFT) is then applied to each of the six derived cross-correlation functions.

[0219] Next, the correlation coefficient ρ(r′) of equation (33) in Non-Patent Document 1 is calculated.

[0220] Then, the relative shift amount (Δx, Δy) between the reference strip f(r) and the target strip g(r) is calculated by identifying the peak position of the correlation coefficient ρ(r'), and registration and merging processes between the reference strip f(r) and the target strip g(r) are performed based on this shift amount.

[0221] By sequentially applying this series of processes to the N strips, merged images of all N strips can be obtained. In addition, the xy direction shift amount (Δx, Δy) calculated for the strip pair can be saved together with the time information (i.e., associated with the time information).

[0222] (S14: Have you processed all the strips?) A series of processes from steps S3 to S13 are executed for all of the N strips obtained in step S2, in the order described above.

[0223] When N is 3 or more, if a merged image of the first strip and the second strip is created in step S13 (S14: No), the process returns to step S3. In step S3, the merged image of the first strip and the second strip is set as a new reference strip, and the third strip is set as a new target strip. By performing the processes of steps S4 to S13 based on the new reference strip and the new target strip, a merged image of the new reference strip and the new target strip is obtained. This new merged image is a merged image of the first to third strips. By sequentially applying this series of processes to the N strips, merged images of all N strips are obtained (S14: Yes).

[0224] (S15: Save the image with the xy motion artifacts corrected) The merged image finally obtained by the above-described repetitive processing is an image in which motion artifacts in the x and y directions have been corrected, and is an image that represents the entire application range of the Lissajous scan in step S1. The main control unit 211 can store this final merged image in the storage unit 212 (and / or other storage device).

[0225] The data processing unit 230 can use the registration results based on the N strips to register the data (three-dimensional data) collected in step S1 and / or the three-dimensional image data constructed by the image data construction unit 220 from this three-dimensional data. In other words, the data processing unit 230 can use the final merged image to register the three-dimensional data or the three-dimensional image data. This registration includes converting the coordinate system defining the Lissajous scan into a three-dimensional Cartesian coordinate system (x-y-z coordinate system). This coordinate conversion corresponds to "remapping" in the method described in Non-Patent Document 1. In this way, three-dimensional image data (volume) in which x- and y-direction motion artifacts have been corrected is obtained. The main control unit 211 can store this x- and y-direction motion artifact-corrected volume in the storage unit 212 (and / or another storage device) together with or instead of the final merged image.

[0226] (S16: Select the reference strip) The process then moves on to processing for correcting z-direction motion artifacts. In z-direction motion artifact correction, first, the z-shift amount calculation unit 232 selects one strip (reference strip) from multiple strips corresponding to the volume for which x- and y-direction motion artifact correction has been performed. The reference strip selected here may be the first strip selected in step S3 of the first iteration, or a strip different from this.

[0227] (S17: Select the target strip) Next, the z-shift amount calculation unit 232 selects one strip (target strip) other than the reference strip selected in step S16 from the multiple strips corresponding to the volume in which the x- and y-direction motion artifacts have been corrected. The reference strip selected here may be the second strip selected in step S3 of the first iteration, or may be a strip other than this.

[0228] (S18: Set a reference sub-volume corresponding to the reference strip) Next, the z-shift amount calculation unit 232 sets the sub-volume corresponding to the reference strip selected in step S16 as the reference sub-volume. Here, the front projection image of the reference sub-volume is the reference strip.

[0229] (S19: Set the target subvolume corresponding to the target strip) Similarly, the z-shift amount calculation unit 232 sets the sub-volume corresponding to the target strip selected in step S17 as the target sub-volume. Here, the front projection image of the target sub-volume is the target strip.

[0230] (S20: Identifying the intersection region between the reference subvolume and the target subvolume) Next, the z-shift amount calculation section 232 identifies an intersection area (common area) between the reference sub-volume set in step S18 and the target sub-volume set in step S19.

[0231] As described above, there are four intersection regions between the two sub-volumes. The z-shift amount calculation unit 232 identifies one or more intersection regions. When two or more intersection regions are identified, the following step S21 is executed for each of the identified intersection regions.

[0232] (S21: Calculate the z-direction shift amount between the reference subvolume and the target subvolume) Next, the z-shift amount calculation unit 232 calculates the amount of shift in the z direction between the reference subvolume and the target subvolume based on the intersecting region between the subvolumes identified in step S20.

[0233] For example, the z-shift amount calculation unit 232 sets a cross section (cross section of interest) in the intersection region identified in step S20. The cross section of interest may be any cross section in the intersection region. Next, the z-shift amount calculation unit 232 constructs an image of the cross section of interest (reference cross section image) from the reference sub-volume, and constructs an image of the cross section of interest (target cross section image) from the target sub-volume. Next, the z-shift amount calculation unit 232 analyzes the reference cross section image to identify an image of a predetermined portion of the subject's eye E (reference image), and analyzes the target cross section image to identify an image of the same portion (target image). Next, the z-shift amount calculation unit 232 obtains the z coordinate of the reference image and the z coordinate of the target image, and calculates the difference between these two z coordinates. In this example, the calculated difference is used as the z-direction shift amount between the reference sub-volume and the target sub-volume.

[0234] A specific example is shown in FIG. 9. Reference numeral 401 denotes the intersection state between a strip 402 corresponding to the reference subvolume and a strip 403 corresponding to the target subvolume. In this example, attention is focused on the intersection region surrounded by a white circle. The diameter of the white circle along the x direction (indicated by a dotted line) is the cross section of interest. The two strips 402 and 403 also respectively show intersection regions and cross sections of interest. The z-shift amount calculation unit 232 constructs an image of this cross section of interest (reference cross section image) 412 from the reference subvolume corresponding to the strip 402, and constructs an image of this cross section of interest (target cross section image) 413 from the target subvolume corresponding to the strip 403. Furthermore, the z-shift amount calculation unit 232 analyzes the reference cross section image 412 to identify the z position (z coordinate) 422 of the image of the retinal surface, and analyzes the target cross section image 413 to identify the z position (z coordinate) 423 of the image of the retinal surface. In addition, the z-shift amount calculation unit 232 calculates the difference Δz between two z positions (two z coordinates) 422 and 423 identified from the two cross-sectional images 412 and 413, respectively, as the z-direction shift amount between the reference subvolume and the target subvolume.

[0235] The method for calculating the z-direction shift amount between the reference subvolume and the target subvolume is not limited to this, and any method that utilizes their intersection area may be used. For example, as described above, the z-shift amount calculation unit 232 may calculate a correlation coefficient between the reference subvolume and the target subvolume and determine the z-direction shift amount based on this correlation coefficient.

[0236] In this case, similar to step S9 of FIG. 7B, the correlation coefficients calculated for each subvolume pair are recorded by the correspondence information generator 234. For example, the correspondence information generator 234 associates the correlation coefficients obtained in the calculation of the z-direction shift amount with the corresponding identifiers (position information) and records them. As described above, this identifier may be any of the coordinates in the definition coordinate system of the image data, the cycle number, the scan number, the strip number, the subvolume number, the merge process number, or an identifier different from these. By repeating steps S17 to S24, correlation coefficients that are elements of the correlation coefficient set for this Lissajous scan (and position information that is elements of the position information set) are accumulated. The correspondence information generator 234 can associate the correlation coefficients in the x and y directions recorded in step S9 with the correlation coefficients in the z direction recorded in step S21 via the identifiers (position information). This makes it possible to obtain three-dimensional correlation coefficients.

[0237] (S22: Save the z-direction shift amount) The z-direction shift amount calculated for the subvolume pair in step S21 is stored, for example, in the storage unit 212. Similar to storing the xy-direction shift amount (step S11), the z-direction shift amount may be recorded, for example, together with time information corresponding to the subvolume pair.

[0238] (S23: Perform registration between the reference subvolume and the target subvolume) The image data correcting unit 233 applies registration in the z direction to the reference sub-volume and the target sub-volume based on the z-direction shift amount calculated in step S21.

[0239] This registration is a process of adjusting the relative positions in the z direction between the reference sub-volume and the target sub-volume so as to cancel out the z-direction shift amount Δz in Figure 9, that is, so that the image of the retinal surface in the reference cross-sectional image 412 and the image of the retinal surface in the target cross-sectional image 413 are located at the same z position (equal z coordinate).

[0240] (S24: Construct a merged image of the reference subvolume and the target subvolume) The image data correcting unit 233 constructs a merged image of the reference subvolume and the target subvolume whose relative positions have been adjusted in the z direction by the registration in step S23, thereby obtaining a merged image of the reference subvolume and the target subvolume whose relative positions have been adjusted in the xy and z directions.

[0241] (S25: Have all subvolumes been processed?) A series of processes from steps S17 to S24 are executed for all sub-volumes in a predetermined order (for example, the order assigned to the first to Nth strips) (S25: No). In this example, the merged image constructed in step S24 is set as the reference strip in the next routine. By applying this series of processes to all sub-volumes in a sequential manner, merged images of all sub-volumes are obtained (S25: Yes).

[0242] (S26: Save the volume with all subvolumes merged) The merged image finally obtained by repeating steps S17 to S24 is an image in which motion artifacts in the x, y, and z directions have been corrected, and is an image that represents the entire application range of the Lissajous scan in step S1. The main control unit 211 can store this final merged image in the storage unit 212 (and / or other storage device).

[0243] The following processing is for more precise z-direction motion artifact correction. In steps S16 to S25, z-direction motion artifact correction is performed in subvolume units (strip units). In contrast, in steps S27 to S32 described below, z-direction motion artifact correction is performed in cycle units. This cycle-unit correction is performed by the data processing unit 230 (for example, the image data correction unit 233).

[0244] Note that precise z-direction motion artifact correction does not need to be performed on a cycle-by-cycle basis, but may be performed on a cycle group that constitutes a part of a subvolume basis.Furthermore, precise z-direction motion artifact correction does not need to be performed for all cycles (all cycle groups), but may be performed for only some cycles (some cycle groups).

[0245] (S27: Select cycles from merged volumes) The image data correcting unit 233 selects one cycle (partial data corresponding to one cycle) from the volume in which all the sub-volumes are merged.

[0246] The cycle selected first may be any cycle. For example, the cycles may be selected sequentially according to the scan order (chronological order, scan number, cycle number) of the multiple cycles in the Lissajous scan in step S1. Alternatively, the cycle selection order may be set based on the order (strip number, subvolume number) assigned to multiple strips or multiple subvolumes.

[0247] (S28: Remove the cycle from the volume) Next, the image data correcting unit 233 extracts the partial data corresponding to the cycle selected in step S27 from the volume.

[0248] (S29: Calculate the z-direction shift between the volume after cycle removal and the cycle) Next, the image data correcting unit 233 calculates the amount of shift in the z direction between the partial data corresponding to the cycle extracted in step S28 and the volume from which the cycle was extracted.

[0249] The method for calculating the z-direction shift amount in this step may be the same as the method in step S21. For example, the image data correction unit 233 analyzes the partial data (cross-sectional image of interest) of the cycle removed from the volume in step S28 to identify an image (first image) of a predetermined portion of the eye E, and analyzes the volume from which the cross-sectional image of interest has been extracted to identify an image (second image) of the same portion. The analysis for identifying the second image does not need to be applied to the entire volume from which the cross-sectional image of interest has been extracted; for example, it may be applied only to a portion adjacent to the cross-sectional image of interest (adjacent cycle). Furthermore, the image data correction unit 233 calculates the z coordinate of the first image and the z coordinate of the second image, and calculates the difference between these two z coordinates. In this example, the calculated difference is used as the z-direction shift amount between the cycle and the volume after the cycle has been removed.

[0250] (S30: Update the z-direction shift amount) Next, the main controller 211 replaces the z-direction shift amount calculated in step S21 with the new z-direction shift amount calculated in step S29. Here, the z-direction shift amount of the subvolume including the partial data corresponding to the cycle extracted in step S28 is updated.

[0251] When a correlation coefficient is used in both the calculation of the z-direction shift amount in step S21 and the calculation of the z-direction shift amount in step S29, the information recorded in step S21 (correlation coefficient, identifier (position information), etc.) can be replaced with the information acquired in step S29 (correlation coefficient, identifier (position information), etc.), or both the information recorded in step S21 and the information acquired in step S29 can be recorded in association with each other. Here, the information recorded in step S21 and the information acquired in step S29 can be associated with each other via the identifier (position information). When a correlation coefficient is used in only one of the calculation of the z-direction shift amount in step S21 and the calculation of the z-direction shift amount in step S29, the information acquired in one of the processes can be recorded. The various processes described herein are executed by, for example, the correspondence information generating unit 234.

[0252] (S31: Perform registration and merging) Next, the image data correcting unit 233 performs registration between the partial data corresponding to the cycle removed in step S28 and the volume after the cycle removal so as to cancel the z-direction shift amount calculated in step S29. Furthermore, the image data correcting unit 233 constructs a merged image of the partial data whose relative position in the z direction has been adjusted by this registration and the volume after the cycle removal. This results in a volume whose relative position has been adjusted in the x and y directions and whose relative position has been precisely adjusted in the z direction.

[0253] (S32: Have all cycles been processed?) A series of processes from steps S27 to S31 are executed for all cycles in a predetermined order (S32: No). By applying this series of processes to all cycles in a sequential manner, a volume in which the relative positions in the z direction between all cycles have been adjusted is obtained (S32: Yes). The main controller 211 can store this final volume in the memory 212 (and / or another memory device).

[0254] (S33: Generate a correlation coefficient set and a location information set) In this operation example, information such as correlation coefficients, identifiers (location information), etc. is accumulated by repeatedly performing a routine including step S9 (and at least one of a routine including step S21 and a routine including step S29). Correspondence information generation unit 234 generates a correlation coefficient set by collecting multiple correlation coefficients accumulated by this iterative processing. Similarly, correspondence information generation unit 234 generates a location information set by collecting multiple identifiers accumulated by this iterative processing.

[0255] As described above, a correlation coefficient set may be generated from some of the correlation coefficients accumulated by the iterative process, or a location information set may be generated from some of the identifiers accumulated by the iterative process. More generally, the correspondence information generation unit 234 according to this operation example is configured to generate a correlation coefficient set from at least some of the correlation coefficients accumulated by the iterative process, and is configured to generate a location information set from some of the identifiers accumulated by the iterative process.

[0256] When generating a correlation coefficient set from some (but not all) of the correlation coefficients accumulated by the above-described iterative process, the correspondence information generating unit 234 executes a process of selecting elements of the correlation coefficient set from the accumulated correlation coefficients. This selection process includes, for example, a process of identifying correlation coefficients that satisfy a predetermined condition from the accumulated correlation coefficients. This predetermined condition may be any of the aforementioned determination conditions.

[0257] Similarly, when generating a location information set from some (but not all) of the plurality of identifiers accumulated by the above-mentioned iterative process, the correspondence information generating unit 234 includes, for example, a process of identifying identifiers that satisfy a predetermined condition from among the plurality of accumulated identifiers in order to select elements of the location information set from the plurality of accumulated identifiers. This predetermined condition may be any of the judgment conditions described above.

[0258] (S34: Generate correspondence information) The correspondence information generation unit 234 generates correspondence information indicating the correspondence between the correlation coefficient set and the position information set generated in step S33. The correspondence between the elements of the correlation coefficient set and the elements of the position information set for generating the correspondence information is established, for example, via an identifier, as described above.

[0259] (S35: Record the correspondence information in association with the image data) The information recording unit 235 records the correspondence information generated in step S34. In this operation example, the information recording unit 235 may record this correspondence information and the image data (final volume) generated in step S32 in association with each other.

[0260] The correspondence information and image data may be stored in the same storage device or in separate storage devices. In either case, the correspondence information and image data are associated with each other, and it is possible to identify one from the other.

[0261] The information recording unit 235 may record any information generated in this operation example in association with the correspondence information. For example, the information recorded in association with the correspondence information may be at least one of the following: the data set collected in step S1; the strip constructed in step S2; the x- and y-direction shift amounts saved in step S11; the merged image constructed in step S13; the image saved in step S15; the reference strip selected in step S16; the reference subvolume set in step S18; the intersection region identified in step S20; the z-direction shift amount saved in step S22; the merged image constructed in step S24; the volume saved in step S26; the z-direction shift amount updated in step S30; and the final volume created in step S32.

[0262] The information recorded in association with the corresponding information may include other information (e.g., analysis data, OCT angiography images) generated from the data set collected in step S1, or may include other data acquired by the ophthalmic examination device 1 (e.g., observed images, photographed images, OCT images, etc.).

[0263] The information recorded in association with the correspondence information may include any information acquired by a means other than the ophthalmic examination apparatus 1. Examples of such information include data acquired by any medical device (e.g., examination data, image data, analysis data), data generated by any information processing device (e.g., analysis data, statistical data, training data for machine learning), and data input by a doctor or the like (e.g., electronic medical record data, interpretation data).

[0264] The type of information to be recorded in association with the correspondence information may be set in advance, or may be specified by the user or the ophthalmic examination apparatus 1.

[0265] For example, the ophthalmic examination apparatus 1 stores in advance a list (referred to as an information list) indicating the types of information to be recorded in association with the corresponding information. This information list is stored, for example, in the storage unit 212. Some exemplary embodiments may store two or more information lists selectable by the user or the ophthalmic examination apparatus 1. The ophthalmic examination apparatus 1 of this example (for example, the information recording unit 235) is configured to collect the types of information indicated in the information list and record the collected information in association with the corresponding information.

[0266] In another example of the ophthalmic examination apparatus 1, the main control unit 211 causes the display unit 241 to display the information types shown in the information list. The user selects a desired information type from the displayed information types. The information recording unit 235 collects one or more types of information selected by the user and records the collected information in association with corresponding information.

[0267] In yet another example of the ophthalmic examination apparatus 1, the information recording unit 235 selects an information type from an information list and / or selects an information list from two or more information lists based on predetermined information. This predetermined information may include, for example, at least one of information input by a user (e.g., a selection result of an examination mode), information input to the ophthalmic examination apparatus 1 (e.g., electronic medical record data, an interpretation report, examination data acquired by another ophthalmic examination apparatus, data generated by an information processing apparatus), and information acquired by the ophthalmic examination apparatus 1 (e.g., data acquired or generated in this operation example or data generated from this data, data acquired by an OCT scan other than the Lissajous scan in step S1 or data generated from this data, an observed image, or a captured image).

[0268] (S36: Execute the specified process) The correspondence information (and image data) recorded in step S35 may be used in any processing. In this operation example, the processing unit 260 executes a predetermined processing (end).

[0269] The processing performed by the processing unit 260 using the correspondence information (and image data) may include, for example, any one or more of the following group: presentation of information (e.g., displaying the correspondence information, displaying information generated from the correspondence information); provision of information (e.g., transmitting the correspondence information, transmitting information generated from the correspondence information); statistical processing; image analysis; image processing; image evaluation; machine learning; generation of training data for machine learning. In these processes, information recorded together with the correspondence information may also be used.

[0270] In this operation example, the ophthalmic examination apparatus 1 performs the predetermined process using the correspondence information (and image data), but this predetermined process may be performed by another device. In some exemplary aspects, the ophthalmic examination apparatus 1 performs up to step S35, and the other device performs the predetermined process using the stored correspondence information (and image data).

[0271] <Effects> Some effects of the ophthalmic examination apparatus 1 according to the first embodiment will be described.

[0272] The ophthalmic examination apparatus 1 is a scanning imaging apparatus that constructs image data using optical scanning, and includes as its functional elements a data set collection unit, a correlation coefficient calculation unit, a correspondence information generation unit, and an information recording unit.

[0273] The dataset collection unit functions to collect a dataset by applying an optical scan to the sample according to a two-dimensional pattern including a series of cycles that intersect with each other. In this embodiment, a group of elements for collecting OCT data corresponds to the dataset collection unit, and for example, the fundus camera unit 2, the OCT unit 100, the image data construction unit 220, etc. correspond to the dataset collection unit.

[0274] The correlation coefficient calculation unit functions to calculate a correlation coefficient between cycles based on the data set collected by the data set collection unit. In this embodiment, the correlation coefficient calculation unit 2315, the z-shift amount calculation unit 232, etc. correspond to the correlation coefficient calculation unit.

[0275] The correspondence information generation unit functions to generate correspondence information that represents the correspondence between a set of correlation coefficients (correlation coefficient set) calculated by the correlation coefficient calculation unit from the dataset collected by the dataset collection unit, and a set of position information (position information set) in the application area of ​​the optical scanning performed by the dataset collection unit to collect this dataset. In this embodiment, the correspondence information generation unit 234 and the like correspond to the correspondence information generation unit.

[0276] The information recording unit functions to record the correspondence information generated by the correspondence information generating unit in association with the image data constructed based on the dataset collected by the dataset collecting unit. In this embodiment, the information recording unit 235 and the like correspond to the information recording unit.

[0277] The ophthalmic examination apparatus 1 described above can store correlation coefficient sets and position information sets, which have not been recorded in conventional techniques, and can also record correspondence information indicating the correspondence between the correlation coefficient sets and the position information sets, thereby providing new ways of using data acquired by scanning imaging. Some non-limiting examples of such uses will be described in the following embodiments.

[0278] The ophthalmic examination apparatus 1 of this embodiment may further include a processing unit. The processing unit functions to execute predetermined processing based on the correspondence information recorded by the information recording unit. In this embodiment, the processing unit 260 and the like correspond to the processing unit. Such an ophthalmic examination apparatus 1 makes it possible to provide a new method of using data (correspondence information, image data) acquired by scanning imaging. Note that a similar processing unit may be provided in other devices.

[0279] Some application examples of the first embodiment will be described in the following embodiments. In the following embodiments, unless otherwise specified, the same features as those of the first embodiment (similar configurations, similar elements, similar functions, similar actions, similar effects, etc.) can be employed. Note that the same applications as the following embodiments can also be made for embodiments equivalent to or similar to the first embodiment.

[0280] <Second embodiment> A second embodiment will be described. The ophthalmic examination apparatus of this embodiment has the same configuration as that of the first embodiment shown in Figures 1 to 4C, and also has a processing unit 260A shown in Figure 10 as the processing unit 260 of Figure 4C.

[0281] The processing unit 260A is configured to execute a process of complementing data in areas where the correlation coefficient is low, based on correspondence information that indicates the correspondence between the correlation coefficient set and the position information set. The processing unit 260A is realized by cooperation between hardware including a processor and processing software. The processing unit 260A includes a correlation coefficient identification unit 261, a position information identification unit 262, and a pixel value determination unit 263.

[0282] The correlation coefficient identification unit 261 is configured to identify a subset having correlation coefficients that satisfy a predetermined condition as elements from among the correlation coefficients that belong to the correlation coefficient set in the correspondence information generated by the correspondence information generation unit 234. Some exemplary aspects of this process will be described later. The correlation coefficient identification unit 261 is realized by cooperation between hardware including a processor and correlation coefficient identification processing software.

[0283] The location information identification unit 262 is configured to identify, based on the correspondence information, a subset of the location information set corresponding to the subset identified from the correlation coefficient set by the correlation coefficient identification unit 261. By referring to the correspondence between the correlation coefficient set and the location information set indicated in the correspondence information, the location information identification unit 262 identifies location information corresponding to each correlation coefficient belonging to the subset identified by the correlation coefficient identification unit 261, and obtains a subset (which is a subset of the location information set) having the identified location information as an element. The location information identification unit 262 is realized by cooperation between hardware including a processor and location information identification processing software.

[0284] The pixel value determination unit 263 is configured to determine the value of a pixel of image data corresponding to the subset identified from the position information set by the position information identification unit 262, based on the values ​​of one or more neighboring pixels. This image data may be any image data created based on a data set collected by optical scanning for generating correspondence information, and may be image data created by the image data correction unit 233, for example. Some exemplary aspects of the processing performed by the pixel value determination unit 263 will be described later. The pixel value determination unit 263 is realized by cooperation between hardware including a processor and pixel value determination processing software.

[0285] Several exemplary aspects of the correlation coefficient specifying unit 261 will be described. In this embodiment, the correlation coefficient specifying unit 261 is configured to specify correlation coefficients that are equal to or less than a predetermined threshold from among the correlation coefficients belonging to the correlation coefficient set, and to obtain a subset having the specified correlation coefficients as elements. This subset is a subset of the correlation coefficient set. This makes it possible to specify correlation coefficients (subsets) that correspond to low-quality parts or parts that were not imaged in an image constructed using optical scanning, among the correlation coefficients belonging to the correlation coefficient set.

[0286] More generally, in order to identify a subset of the correlation coefficient set, the correlation coefficient identification unit 261 determines whether each correlation coefficient belonging to the correlation coefficient set satisfies a predetermined condition. This determination condition may be any predetermined condition, such as any of the following exemplary conditions. Note that although the first example is used in this embodiment, other examples may be used in other embodiments, and in those cases, it is possible to perform the same processing as in this embodiment.

[0287] A first example of the determination condition is that the value of the correlation coefficient is small. When this example condition is applied, the correlation coefficient identification unit 261, for example, compares each correlation coefficient included in the correlation coefficient set with a preset threshold (referred to as a first threshold) to identify a subset (a set having low-value correlation coefficients as elements) made up of correlation coefficients with values ​​equal to or less than this first threshold. The correlation coefficients belonging to this subset are information indicating the state of low-quality parts or parts that have not been visualized in the image data.

[0288] The position information identifying unit 262 identifies a subset of the position information set corresponding to such a subset of the correlation coefficient set. This subset of the position information set is information indicating the positions of low-quality parts or parts that have not been imaged in the image data.

[0289] The pixel value determination unit 263 determines the pixel value of the image data corresponding to the subset identified from the position information set by the position information identification unit 262 based on the values ​​of one or more neighboring pixels. This makes it possible to complement pixel values ​​of low-quality parts of the image data or pixel values ​​of parts that have not been visualized based on the values ​​of surrounding (neighboring) pixels.

[0290] It should be noted that the method of using the information acquired by the correlation coefficient identification unit 261 (subset of the correlation coefficient set) and / or the information acquired by the position information identification unit 262 (subset of the position information set) is not limited to image completion, and can also be used for rescanning, which will be described later. In addition to these, the method of use may be presentation of information, provision of information, statistical processing, image analysis, image processing, image evaluation, machine learning, generation of training data for machine learning, etc., and elements according to the method of use may be provided in the processing unit.

[0291] A second example of the determination condition is that the value of the correlation coefficient is large. When this example condition is applied, the correlation coefficient identification unit 261, for example, compares each correlation coefficient included in the correlation coefficient set with a preset threshold (referred to as the second threshold), and identifies a subset (a set having high-value correlation coefficients as elements) made up of correlation coefficients with values ​​equal to or greater than the second threshold. The correlation coefficients belonging to this subset are information indicating the state of high-quality parts of the image data.

[0292] The position information specifying unit 262 specifies a subset of the position information set corresponding to such a subset of the correlation coefficient set. This subset of the position information set is information indicating the positions of high-quality portions in the image data.

[0293] In this example, the information (subset of the correlation coefficient set) acquired by the correlation coefficient identification unit 261 and / or the information (subset of the position information set) acquired by the position information identification unit 262 may be used, for example, to present information, provide information, statistical processing, image analysis, image processing, image evaluation, machine learning, generate training data for machine learning, etc. Note that in this example, the pixel value determination unit 263 does not need to be provided, and an element according to the method of use may be provided in the processing unit.

[0294] A third example of the determination condition is that the value of the correlation coefficient falls within a predetermined range. When this example condition is applied, the correlation coefficient specification unit 261, for example, compares each correlation coefficient included in the correlation coefficient set with a predetermined range (referred to as a specific range) to obtain a subset of correlation coefficients whose values ​​fall within this specific range. The correlation coefficients belonging to this subset are information indicating the state of a portion of the image data whose quality falls within a certain range.

[0295] The position information specifying unit 262 specifies a subset of the position information set corresponding to such a subset of the correlation coefficient set. This subset of the position information set is information indicating the positions of parts of the image data whose quality falls within a certain range.

[0296] In this example, the information acquired by the correlation coefficient identification unit 261 (subset of the correlation coefficient set) and / or the information acquired by the position information identification unit 262 (subset of the position information set) may be used, for example, image completion, rescanning, information presentation, information provision, statistical processing, image analysis, image processing, image evaluation, machine learning, generation of training data for machine learning, etc. In this example, the pixel value determination unit 263 does not need to be provided, and an element according to the method of use may be provided in the processing unit.

[0297] The first threshold value, the second threshold value, and the specific range (upper and lower limits) may all be set by any method. These values ​​may be, for example, default values ​​determined based on actual measurements and / or simulation values, individual values ​​determined based on image data to which the correlation coefficient calculation has been applied, or individual values ​​determined based on image data acquired in preparation for acquiring the image data (e.g., image data acquired during a preparatory operation such as alignment). The default values ​​and individual values ​​may typically be determined by applying statistical processing to the image data. For example, one or more of the first threshold value, the second threshold value, and the specific range may be determined based on the results of statistical calculations (e.g., histograms, maps, statistics, etc.) of predetermined parameters (e.g., correlation coefficients, pixel values, etc.) generated from the image data.

[0298] The judgment conditions may be determined depending on the use of the information generated by the data processing unit 230. Any number of judgment conditions may be prepared. When two or more judgment conditions are provided, these judgment conditions may be selectively used. Furthermore, two or more judgment conditions may be usable in the same optical scanning process. In this case, two or more pieces of information (for example, two or more subsets of a correlation coefficient set, two or more subsets of a position information set) corresponding to the two or more judgment conditions may be generated.

[0299] An exemplary embodiment of the pixel value determination unit 263 will be described. The pixel value determination unit 263 of this embodiment determines the value of a pixel of image data corresponding to position information belonging to a subset of the position information set identified by the position information identification unit 262, based on the values ​​of one or more neighboring pixels. In other words, the pixel value determination unit 263 performs a process of replacing low-quality areas in the image data constructed by the image data processing unit 231 (for example, the final three-dimensional image data generated by the image data correction unit 233 or the merged image generated by the merge processing unit 2318) with higher-quality areas, and / or a process of complementing portions of this image data that were not visualized.

[0300] The pixel value determination unit 263 may perform any method of pixel value determination processing. For example, the pixel value determination unit 263 may be capable of performing any interpolation (interpolation) and / or any extrapolation. Examples of interpolation methods include nearest neighbor interpolation, linear interpolation, bilinear interpolation, trilinear interpolation, bicubic interpolation, curve fitting, and polynomial interpolation. Examples of extrapolation methods include linear extrapolation and Richardson extrapolation.

[0301] In some exemplary aspects, the pixel value determination unit 263 can determine the value of a pixel of image data corresponding to each piece of position information belonging to a subset identified from the set of position information by the position information identification unit 262, based on the values ​​of one or more neighboring pixels. Here, since the correspondence information and the image data are generated from a data set collected in the same Lissajous scan, correspondence between the position information (e.g., the above-mentioned identifier) ​​and the pixel (coordinate) is possible.

[0302] Furthermore, each piece of position information belonging to the subset identified from the position information set by the position information identification unit 262 is position information corresponding to a correlation coefficient equal to or less than the first threshold value described above, and is position information for a cycle pair with low correlation in the intersection region. The pixel value determination unit 263 functions to determine the pixel value of each pixel present at the position indicated by such position information (for example, the cycle pair or the intersection region).

[0303] The operation of the ophthalmic examination apparatus according to this embodiment will now be described. An example of the operation of the ophthalmic examination apparatus is shown in Fig. 11. Unless otherwise specified, the matters described in the first embodiment can be combined with this example of operation.

[0304] (S41: Specify the number of cycles) After inputting the patient ID and setting the scan mode (designating the Lissajous scan), the number of cycles in the Lissajous scan to be applied to the subject's eye E can be designated as an optional step. The number of cycles corresponds to the density of the scan. The number of cycles is designated, for example, by the user or the ophthalmic examination apparatus.

[0305] (S42: Perform preparation operations) The ophthalmic examination apparatus performs preparatory operations similar to those of the conventional method, such as presenting a fixation target, alignment, focus adjustment, and OCT optical path length adjustment. The ophthalmic examination apparatus may be configured to determine a judgment condition (e.g., the first threshold value described above) for identifying a subset of the correlation coefficient set based on image data acquired in the preparatory operations. Note that a default judgment condition may be used, or a judgment condition determined from the data set collected by the Lissajous scan in step S43 may be used.

[0306] (S43: Perform Lissajous scan) Upon receiving a predetermined scan start trigger signal, the scan control unit 2111 applies a Lissajous scan consisting of the number of cycles designated in step S41 to the subject's eye E (fundus oculi Ef).

[0307] (S44: Build a strip for each cycle) The image data constructing unit 220 constructs strips corresponding to the cycles specified in step S41 based on the data set collected by the Lissajous scan in step S43. This process may be performed in the same manner as in the first embodiment (step S2 in FIG. 7A), for example.

[0308] (S45: Calculate the correlation coefficient for each cycle pair) The image data processing unit 231 (correlation coefficient calculation unit 2315, etc.) calculates the correlation coefficient for each cycle pair formed based on the multiple cycles designated in step S41.

[0309] The processing of this step may be performed in the same manner as the motion artifact correction of the first embodiment (steps S3 to S8 in FIGS. 7A and 7B), for example.

[0310] Alternatively, the processing in this step may be such that, for each cycle pair formed based on the multiple cycles specified in step S41, a correlation coefficient between the two cycles constituting the cycle pair is calculated. In this case, the motion artifact correction of the first embodiment may be performed separately.

[0311] (S46: Set the pixel values ​​of strips whose correlation coefficient is below the threshold to zero) The image data processing unit 231 (for example, the image data correction unit 233) compares each correlation coefficient calculated in step S45 with the first threshold value described above to identify correlation coefficients with values ​​equal to or less than the first threshold value. Furthermore, the image data processing unit 231 (for example, the image data correction unit 233) sets the pixel value of the pixel at the position corresponding to the identified correlation coefficient (low correlation coefficient) to zero.

[0312] In some exemplary aspects, the pixel value of each pixel in a strip corresponding to at least one low correlation coefficient may be set to zero. In some exemplary aspects, the pixel value of each pixel in a strip where all of the corresponding correlation coefficients are low may be set to zero. In some exemplary aspects, the pixel value of each pixel in a strip where the number of corresponding low correlation coefficients is equal to or greater than a predetermined number may be set to zero. In these examples, strips with pixel values ​​set to zero are represented as solid black and correspond to the "unimaged portion" in the first embodiment. Note that the processing of this step is not limited to these examples.

[0313] (S47: Build image data) The image data processing unit 231 constructs image data based on a plurality of strips including the strips whose pixel values ​​were set to zero in step S46. This image data is, for example, three-dimensional image data constructed by the image data correction unit 233. Note that part of this image data is not visualized (i.e., its pixel values ​​are zero).

[0314] (S48: Generate correspondence information) The image data processing unit 231 (correspondence information generating unit 234) generates correspondence information that indicates the correspondence between the multiple correlation coefficients (correlation coefficient set) calculated in step S45 and the multiple pieces of position information (position information set) in the application area of ​​the Lissajous scan in step S43. This processing may be executed, for example, in the same manner as in the first embodiment (steps S33 and S34 in FIG. 7F).

[0315] (S49: Record the correspondence information and image data) The image data processing unit 231 (information recording unit 235) records the image data constructed in step S47 and the correspondence information generated in step S48 in association with each other. This process may be performed, for example, in the same manner as in the first embodiment (step S35 in FIG. 7F).

[0316] The following steps S50 to S54 are an example of the predetermined processing executed in step S36 of Fig. 7F in the first embodiment. In this operation example, steps S50 to S54 are executed by the ophthalmic examination apparatus 1, but at least one of steps S50 to S54 may be executed by another apparatus.

[0317] The time difference between the execution of step S49 and the execution of step S50 may be arbitrary. For example, steps S50 to S54 may be executed immediately after step S49, or the correspondence information and image data may be read and steps S50 to S54 may be executed at any time after step S49.

[0318] (S50: Identify correlation coefficients below a threshold) Data processing unit 230 (correlation coefficient identification unit 261 of processing unit 260A) identifies correlation coefficients whose values ​​are equal to or less than a predetermined threshold (the above-mentioned first threshold) from among the correlation coefficients belonging to the correlation coefficient set related to the correspondence information recorded in step S49. As a result, a subset having low-value correlation coefficients as elements is identified from the correlation coefficient set.

[0319] (S51: Identify the corresponding location information) The data processing unit 230 (the location information identification unit 262 of the processing unit 260A) identifies the location information corresponding to the correlation coefficient identified in step S50 from among the location information belonging to the location information set related to the correspondence information recorded in step S49. As a result, a subset having location information corresponding to a low correlation coefficient as an element is identified from the location information set.

[0320] (S52: Identify corresponding pixels in the image data) The data processing unit 230 (pixel value determination unit 263 of processing unit 260A) identifies pixels corresponding to the position information identified in step S51 from among the pixels of the image data recorded together with the correspondence information in step S49. As a result, pixels corresponding to low correlation coefficients are identified from among the pixels constituting this image data.

[0321] In this operation example, since the values ​​of pixels corresponding to low correlation coefficients are set to zero in step S46, instead of executing steps S50 to S52, pixels having a pixel value of zero may be identified from among the pixels of the recorded image data in step S49. Note that the processing executed in steps S50 to S52 is not limited to this operation example or an aspect similar thereto, but can be applied to any aspect.

[0322] (S53: Determine pixel value) The data processing unit 230 (pixel value determination unit 263 of processing unit 260A) determines the value of the pixel identified in step S52 based on the values ​​of one or more pixels (one or more neighboring pixels) located near this pixel. This step is to complement the pixel whose pixel value was set to zero in step S46.

[0323] The pixel to be interpolated may be one or more of all pixels identified from the correspondence information recorded in step S49. For example, all pixels that can be the target of the interpolation process may be interpolated. As a result, the pixel value (zero) of at least one pixel in the strip that is represented as solid black by the process of step S46 is replaced based on the pixel values ​​of one or more neighboring pixels.

[0324] (S54: Display image) The ophthalmic examination apparatus 1 (for example, the main control unit 211) displays an image based on the image data interpolated in step S53. For example, the ophthalmic examination apparatus 1 constructs a rendering image of the three-dimensional image data interpolated in step S53, and displays this rendering image on the display unit 241 (END).

[0325] The ophthalmic examination apparatus of this embodiment includes a data set collection unit, a correlation coefficient calculation unit, a correspondence information generation unit, an information recording unit, and a processing unit, similar to the first embodiment.

[0326] The processing unit of this embodiment is configured to perform the following processes: a process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the correlation coefficient set acquired by the correlation coefficient calculation unit (first process); a process of identifying a subset of the position information set corresponding to the subset identified from the correlation coefficient set based on correspondence information (second process); and a process of determining a pixel value of image data corresponding to the subset identified from the position information set based on values ​​of one or more neighboring pixels (third process). In the exemplary aspect (processing unit 260A) shown in Fig. 10, the correlation coefficient identification unit 261 performs the first process, the position information identification unit 262 performs the second process, and the pixel value determination unit 263 performs the third process; however, the aspects of this embodiment are not limited to this.

[0327] According to the ophthalmic examination apparatus of this embodiment, as in the first embodiment, it is possible to store correlation coefficient sets and position information sets that were not recorded in conventional technology, and further, it is possible to record correspondence information that represents the correspondence between the correlation coefficient sets and the position information sets, thereby making it possible to provide a new method of using data acquired by scanning imaging.

[0328] Furthermore, the ophthalmic examination apparatus of this embodiment can execute predetermined processing based on data (correspondence information, image data) acquired by scanning imaging. More specifically, the ophthalmic examination apparatus of this embodiment can complement (image complement) image data constructed based on a data set collected by optical scanning based on the recorded correspondence information. Therefore, it is possible to provide a novel method for utilizing data acquired by scanning imaging. Note that a similar processing unit may be provided in other devices.

[0329] <Third embodiment> A third embodiment will now be described. The ophthalmic examination apparatus of this embodiment has the same configuration as that of the first embodiment shown in FIGS. 1 to 4C, and also includes a correspondence information generation unit 234A of FIG. 12 as the correspondence information generation unit 234 of FIG. 4B. The correspondence information generation unit 234A includes a correlation coefficient map creation unit 236. The correlation coefficient map created by the correlation coefficient map creation unit 236 is an example of correspondence information that represents the correspondence between a correlation coefficient set and a position information set. As with the second embodiment, the ophthalmic examination apparatus of this embodiment may include a processing unit 260A of FIG. 10 as the processing unit 260 of FIG. 4C.

[0330] The ophthalmic examination apparatus of this embodiment is configured to obtain image data and a set of correlation coefficients from a data set collected by optical scanning such as a Lissajous scan, create a correlation coefficient map (a map showing the distribution of correlation coefficients) as correspondence information showing the correspondence between the set of correlation coefficients and a set of position information in the area where the optical scanning is applied, and record the correlation coefficient map and image data in association with each other. Furthermore, the ophthalmic examination apparatus of this embodiment is configured to identify a subset of the set of correlation coefficients whose elements have correlation coefficients equal to or less than a predetermined threshold, identify a subset of the set of position information corresponding to the subset of the set of correlation coefficients based on the correlation coefficient map, and determine pixel values ​​of image data corresponding to the subset of the set of position information based on the values ​​of one or more neighboring pixels. Here, the image data may be final 3D image data generated by the image data correction unit 233 or a merged image generated by the merge processing unit 2318. The ophthalmic examination apparatus of this embodiment performs a process of replacing low-quality portions of the image data with higher-quality pixels and / or a process of complementing portions of the image data that were not imaged.

[0331] The correlation coefficient map creation unit 236 is configured to create a correlation coefficient map based on a set of correlation coefficients acquired by the image data processing unit 231 (e.g., the correlation coefficient calculation unit 2315). The correlation coefficient map is information representing the distribution of correlation coefficients in at least a part of an area to which optical scanning such as Lissajous scanning has been applied. The correlation coefficient map creation unit 236 creates the correlation coefficient map, for example, by associating each correlation coefficient included in the set of correlation coefficients with a pixel position (scanning position in optical scanning) corresponding to that correlation coefficient. The correlation coefficient map creation unit 236 is realized by cooperation between hardware including a processor and correlation coefficient map creation processing software.

[0332] The operation of the ophthalmic examination apparatus according to this embodiment will now be described. An example of the operation of the ophthalmic examination apparatus is shown in Fig. 13. Unless otherwise specified, the matters described in the first embodiment and the matters described in the second embodiment can be combined with this example of operation.

[0333] (S61: Specify the number of cycles) After inputting the patient ID and setting the scan mode (designating the Lissajous scan), the number of cycles in the Lissajous scan to be applied to the subject's eye E can be designated as an optional step. The number of cycles corresponds to the density of the scan. The number of cycles is designated, for example, by the user or the ophthalmic examination apparatus.

[0334] The number of cycles specified in this step is set to G (G is a positive integer). The number of cycles G may be any number, for example, 1024. As described in the first embodiment, the G cycles are ordered, and the gth cycle is represented by Cg (g=1, 2, . . . , G).

[0335] (S62: Perform preparation operations) The ophthalmic examination apparatus performs preparatory operations similar to those of the conventional method, such as presenting a fixation target, alignment, focus adjustment, and OCT optical path length adjustment. The ophthalmic examination apparatus may be configured to determine a judgment condition (e.g., the first threshold value described above) for identifying a subset of the correlation coefficient set based on image data acquired in the preparatory operations. Note that a default judgment condition may be used, or a judgment condition determined from the data set collected by the Lissajous scan in step S63 may be used.

[0336] (S63: Perform Lissajous scan) Upon receiving a predetermined scan start trigger signal, the scan control unit 2111 applies a Lissajous scan consisting of G cycles C1 to CG specified in step S61 to the subject's eye E (fundus oculi Ef).

[0337] (S64: Build a strip for each cycle) The image data constructing unit 220 constructs strips corresponding to each cycle Cg specified in step S61 based on the data set collected by the Lissajous scan in step S63. This process may be performed, for example, in the same manner as in the first embodiment (step S2 in FIG. 7A).

[0338] For each cycle Cg specified in step S61, the strip corresponding to this cycle Cg is stored in a storage area Ag associated with this cycle Cg. G storage areas A1 to AG associated with the G cycles C1 to CG, respectively, are provided in the storage unit 212, for example.

[0339] (S65: Calculate the correlation coefficient for each cycle pair) The image data processing unit 231 (correlation coefficient calculation unit 2315, etc.) calculates the correlation coefficient for each cycle pair formed based on the G cycles C1 to CG specified in step S61.

[0340] The processing in this step calculates, for example, a correlation coefficient between two cycles (cycle pair) Cg1 and Cg2 selected from G cycles C1 to CG (where g1=1 to G, g2=1 to G, and g1≠g2). In some exemplary aspects, the image data processing unit 231 sets one of the G cycles C1 to CG as a reference cycle and calculates correlation coefficients between this reference cycle and each of the other cycles.

[0341] As a specific example, the image data processing unit 231 sets the first cycle C1 as the reference cycle, and calculates the correlation coefficients between this reference cycle C1 and each of the other cycles C2 to CG.<C1、C2> ,<C1、C3> ,...,<C1、CG> G-1 correlation coefficients R2, R3, ..., RG corresponding to the above are obtained. Here,<C1、Cg> indicates a cycle pair consisting of the first cycle and the gth cycle (g=2 to G).

[0342] Cycle Pair<C1、Cg> The correlation coefficient Rg corresponding to this cycle Cg (or this cycle pair<C1、Cg> The G-1 storage areas B1 to BG associated with the G-1 correlation coefficients R2 to RG, respectively, are provided in the storage unit 212, for example.

[0343] In this way, the set of correlation coefficients generated in this example includes G-1 correlation coefficients R2 to RG, and each correlation coefficient Rg (g=2 to G) corresponds to the corresponding cycle Cg or the corresponding cycle pair Cg.<C1、Cg> is associated with.

[0344] (S66: Build image data) The image data processing unit 231 constructs image data based on the data set collected by the Lissajous scan in step S63. The processing in this step may be performed in the same manner as the combined processing of image data construction and motion artifact correction in the first embodiment (e.g., steps S3 to S14 in FIGS. 7A and 7B, steps S3 to S25 in FIGS. 7A to 7D, or steps S3 to S32 in FIGS. 7A to 7E). The image data generated in this step is stored in the storage unit 212, for example.

[0345] (S67: Set the pixel values ​​of strips whose correlation coefficient is below the threshold to zero) The image data processing unit 231 (for example, the image data correction unit 233) compares each correlation coefficient calculated in step S65 with the first threshold value described above to identify correlation coefficients that are equal to or less than the first threshold value. Furthermore, the image data processing unit 231 (for example, the image data correction unit 233) identifies pixels corresponding to the identified correlation coefficients (low correlation coefficients) from among the pixels of the image data constructed in step S66, and sets the pixel values ​​of the identified pixels to zero.

[0346] In some exemplary aspects, the pixel value of each pixel in an image region corresponding to a strip corresponding to at least one low correlation coefficient may be set to zero. In some exemplary aspects, the pixel value of each pixel in an image region corresponding to a strip where all of the corresponding correlation coefficients are low may be set to zero. In some exemplary aspects, the pixel value of each pixel in an image region corresponding to a strip where the number of corresponding low correlation coefficients is equal to or greater than a predetermined number may be set to zero. In these examples, the image region where the pixel value is set to zero is represented as solid black, and corresponds to the "unimaged portion" in the first embodiment. Note that the processing of this step is not limited to these examples.

[0347] (S68: Create a correlation coefficient map) The correlation coefficient map creating unit 236 creates a correlation coefficient map that represents the distribution of correlation coefficients in the area where the Lissajous scan is applied in step S63, based on the correlation coefficient set acquired in step S65. For example, the correlation coefficient map may include correlation coefficients Rg (g=2 to G) belonging to the correlation coefficient set and cycles Cg or cycle pairs<C1、Cg> The correlation coefficient map obtained in this step is an example of correspondence information that represents the correspondence between a set of correlation coefficients and a set of location information.

[0348] An example of this step will be described below. Note that although this example describes the case of assigning correlation coefficients to pixels of two-dimensional image data, correlation coefficients may also be assigned to pixels of three-dimensional image data. All of the various image data handled in this embodiment are based on data sets collected by the same Lissajous scan, and therefore it is possible to naturally associate the assignment of correlation coefficients to pixels of certain image data with the assignment of correlation coefficients to pixels of other image data.

[0349] The correlation coefficient map in this example is created by assigning corresponding correlation coefficients to pixels of the image data (merged image). The correlation coefficients assigned to pixels are called pixel correlation coefficients. Here, one pixel correlation coefficient may be assigned to one pixel, or one pixel correlation coefficient may be assigned to two or more pixels (pixel groups). An example of the latter is when the same pixel correlation coefficient is assigned to each pixel in the intersection region (common region) between the reference cycle C1 and the cycle Cg, or when the same pixel correlation coefficient is assigned to each pixel in the region of the cycle Cg excluding this intersection region (non-intersection region). In some exemplary embodiments, two or more pixel correlation coefficients may be assigned to one pixel.

[0350] The pixel correlation coefficient in this example is determined based on the correlation coefficient Rg calculated in step S65. For example, as shown in FIG. 14, a cycle pair consisting of two arbitrary cycles Cg(1) and Cg(2)<Cg(1)、Cg(2)> For (the corresponding strip pair), let D(g(1), g(2)) denote the intersection state of two cycles Cg(1) and Cg(2) (the corresponding strip pair), where g(1) = 2~G, g(2) = 2~G, and g(1) ≠ g(2).

[0351] In Figure 14, the intersection regions of the two cycles Cg(1) and Cg(2) are indicated by diagonal lines. In this example, the larger of the correlation coefficient of cycle Cg(1) and the correlation coefficient of cycle Cg(2) is assigned to each pixel belonging to each intersection region. Furthermore, the correlation coefficient of cycle Cg(1) is assigned to each pixel belonging to the non-intersection region of cycle Cg(1) (i.e., the part of cycle Cg(1) excluding these intersection regions), and the correlation coefficient of cycle Cg(2) is assigned to each pixel belonging to the non-intersection region of cycle Cg(2) (i.e., the part of cycle Cg(2) excluding these intersection regions).

[0352] For example, if the correlation coefficient of cycle Cg(1) is 0.5 and the correlation coefficient of cycle Cg(2) is 0.9, then max{0.5, 0.9} = 0.9 is assigned to each pixel belonging to the intersection region, 0.5 is assigned to each pixel belonging to the non-intersection region of cycle Cg(1), and 0.9 is assigned to each pixel belonging to the non-intersection region of cycle Cg(2).

[0353] In this example, the method of assigning correlation coefficients to each pixel in the reference cycle C1 may be arbitrary. For example, a cycle other than cycle C1 may be set as the reference cycle and similar processing may be performed to assign correlation coefficients to each pixel belonging to cycle C1. Alternatively, the value of the correlation coefficient assigned to the pixels belonging to the reference cycle C1 may be determined from the correlation coefficients assigned to the pixels belonging to cycles C2 to C-G.

[0354] By performing this process for each cycle pair, a correlation coefficient map can be created.

[0355] In some exemplary embodiments, in order to assign correlation coefficients to all pixels of image data (merged image) based on the Lissajous scan, the process as in the above example may be performed while changing the combination of cycle pairs in various ways. For example, the following series of processes may be performed.

[0356] First, a certain cycle Cg(1) is set as the reference cycle, and the process in the above example is executed for each pair of this reference cycle Cg(1) and other cycles Cg(2) to Cg(G).

[0357] Next, all cycles are paired (i.e., multiple pairs are set) so as to cover all pixels of the image data (merged image) based on the Lissajous scan. Furthermore, the process in the above example is performed for each set cycle pair.

[0358] Next, similar to the image data construction process of step S66 (for example, the sequential processes executed in steps S3 to S14 of FIGS. 7A and 7B in the first embodiment), the strips corresponding to these cycles are sequentially (recursively) merged, and the process of the above example is sequentially (recursively) executed. A correlation coefficient is assigned to every pixel of the image data constructed in this way.

[0359] (S69: Record the correlation coefficient map and image data) The image data processing unit 231 (information recording unit 235) records the image data constructed in step S66 and the correlation coefficient map generated in step S68 in association with each other. This process may be performed, for example, in the same manner as in the first embodiment (step S35 in FIG. 7F).

[0360] The following steps S70 to S74 are an example of the predetermined processing executed in step S36 of Fig. 7F in the first embodiment. In this operation example, steps S70 to S74 are executed by the ophthalmic examination apparatus 1, but at least one of steps S70 to S74 may be executed by another apparatus.

[0361] The time interval between the execution of step S69 and the execution of step S70 may be arbitrary. For example, steps S70 to S74 may be executed immediately after step S69, or the correspondence information and image data may be read and steps S70 to S74 may be executed at any time after step S69.

[0362] (S70: Identify correlation coefficients below a threshold) Data processing unit 230 (correlation coefficient identification unit 261 of processing unit 260A) identifies correlation coefficients whose values ​​are equal to or less than a predetermined threshold (the above-mentioned first threshold) from among the correlation coefficients belonging to the correlation coefficient set related to the correlation coefficient map recorded in step S69. As a result, a subset having low-value correlation coefficients as elements is identified from the correlation coefficient set.

[0363] (S71: Identify the corresponding location information) The data processing unit 230 (the position information identifying unit 262 of the processing unit 260A) identifies the position information corresponding to the correlation coefficient identified in step S70 from among the position information belonging to the position information set related to the correlation coefficient map recorded in step S69. As a result, a subset having as its elements position information corresponding to a low correlation coefficient is identified from the position information set.

[0364] (S72: Identify corresponding pixels in the image data) The data processing unit 230 (pixel value determination unit 263 of processing unit 260A) identifies pixels corresponding to the position information identified in step S71 from among the pixels of the image data recorded together with the correlation coefficient map in step S69. As a result, pixels corresponding to low correlation coefficients are identified from among the pixels constituting this image data.

[0365] In this operation example, since the values ​​of pixels corresponding to low correlation coefficients are set to zero in step S67, instead of executing steps S70 to S72, pixels having a pixel value of zero may be identified from among the pixels of the recorded image data in step S69. Note that the processing executed in steps S70 to S72 is not limited to this operation example or an aspect similar thereto, but can be applied to any aspect.

[0366] (S73: Determine pixel value) The data processing unit 230 (pixel value determination unit 263 of processing unit 260A) determines the value of the pixel identified in step S72 based on the values ​​of one or more pixels (one or more neighboring pixels) located near this pixel. This step is to complement the pixel whose pixel value was set to zero in step S67.

[0367] The pixel to be interpolated may be one or more of all pixels identified from the correlation coefficient map recorded in step S69. For example, all pixels that can be the target of the interpolation process can be interpolated. As a result, the pixel value (zero) of at least one pixel in the strip that is represented as solid black by the process of step S67 is replaced based on the pixel values ​​of one or more neighboring pixels.

[0368] An example of the processing executed in this step will be explained. The position (coordinates) of the pixel to be interpolated is (x i , y i ), and the pixel value (brightness value) of this target pixel is expressed as I(x i , y i ) and the corresponding correlation coefficient is R(x i , y i ) is expressed as

[0369] First, calculate a provisional luminance value of the target pixel before normalization (standardization). In this example, four pixels located above, below, left, and right of the target pixel are considered as neighboring pixels of the target pixel. As with the expression for the target pixel, the positions (coordinates) of the neighboring pixels are expressed as (x j , y j ), and the pixel value (brightness value) of this target pixel is expressed as I(x j , y j ) and the corresponding correlation coefficient is R(x j , y j ) is expressed as

[0370] In this example, the provisional luminance value T(x i , y i ) is calculated using the following formula: T(x i , y i )=I(x i , y i )×R(x i , y i )+I(x i+1 , y i+1 )×R(x i+1 , y i+1 )+I(x i+1 , y i-1 )×R(x i+1 , yi-1 )+I(x i-1 , y i+1 )×R(x i-1 , y i+1 )+I(x i-1 , y i-1 )×R(x i-1 , y i-1 This formula represents the sum of the products of the luminance value I and the correlation coefficient R for the target pixel and its four neighboring pixels.

[0371] The correlation coefficient R(x i , y i ) is zero, so it is also possible to use the formula: T(x i , y i )=I(x i+1 , y i+1 )×R(x i+1 , y i+1 )+I(x i+1 , y i-1 )×R(x i+1 , y i-1 )+I(x i-1 , y i+1 )×R(x i-1 , y i+1 )+I(x i-1 , y i-1 )×R(x i-1 , y i-1 ).

[0372] Next, the calculated provisional brightness value T(x i , y i ) by the sum of the correlation coefficients of the target pixel and its four neighboring pixels. Then, the brightness value I(x i , y i ) = 0. The luminance value of the target pixel interpolated in this way is expressed as follows: I(x i , y i )=T(x i , y i ) / {R(x i , y i )+R(x i+1 , yi+1 )+R(x i+1 , y i-1 )+R(x i-1 , y i+1 )+R(x i-1 , y i-1 )}=T(x i , y i ) / {R(x i+1 , y i+1 )+R(x i+1 , y i-1 )+R(x i-1 , y i+1 )+R(x i-1 , y i-1 )}.

[0373] By applying this process to each pixel identified in step S72 (for example, each pixel whose pixel value was set to zero in step S67), it is possible to complement the area where there was no image information. Note that by performing a similar process on pixels in an area to which low-quality image information is assigned, it is possible to complement this area (improve the quality of the image information).

[0374] (S74: Display image) The ophthalmic examination apparatus 1 (for example, the main control unit 211) displays an image based on the image data interpolated in step S73. For example, the ophthalmic examination apparatus 1 constructs a rendering image of the three-dimensional image data interpolated in step S73, and displays this rendering image on the display unit 241 (END).

[0375] The ophthalmic examination apparatus of this embodiment includes a data set collection unit, a correlation coefficient calculation unit, a correspondence information generation unit, an information recording unit, and a processing unit, similar to the first embodiment. The correspondence information generation unit of this embodiment includes a correlation coefficient map creation unit. The correlation coefficient map creation unit creates a correlation coefficient map based on the correlation coefficient set acquired by the correlation coefficient calculation unit. The correlation coefficient map is a map that represents the spatial distribution of correlation coefficients, and is an example of correspondence information that represents the correspondence between the correlation coefficient set and the position information set.

[0376] As in the second embodiment, the processing unit of this embodiment is configured to perform the following operations: identifying a subset from the correlation coefficient set, the subset having elements with correlation coefficients equal to or less than a predetermined threshold (first operation); identifying a subset of the position information set corresponding to the identified subset from the correlation coefficient set based on a correlation coefficient map (second operation); and determining a pixel value of image data corresponding to the identified subset from the position information set based on values ​​of one or more neighboring pixels (third operation). Some exemplary aspects of this embodiment may include the processing unit 260A of the second embodiment, and may be configured such that the correlation coefficient identification unit 261 performs the first operation, the position information identification unit 262 performs the second operation, and the pixel value determination unit 263 performs the third operation. However, aspects of this embodiment are not limited to this.

[0377] According to the ophthalmic examination apparatus of this embodiment, as in the first embodiment, it is possible to store correlation coefficient sets and position information sets that were not recorded in conventional technology, and further, it is possible to record a correlation coefficient map as correspondence information that represents the correspondence between the correlation coefficient sets and the position information sets, thereby making it possible to provide a new method of using data acquired by scanning imaging.

[0378] Furthermore, the ophthalmic examination apparatus of this embodiment can perform predetermined processing based on data (correlation coefficient map, image data) acquired by scanning imaging. More specifically, the ophthalmic examination apparatus of this embodiment can complement (image complement) image data constructed based on a data set collected by optical scanning based on the recorded correlation coefficient map. Therefore, it is possible to provide a novel method for utilizing data acquired by scanning imaging. Note that a similar processing unit may be provided in other devices.

[0379] <Fourth embodiment> A fourth embodiment will now be described. The ophthalmic examination apparatus of this embodiment has the same configuration as that of the first embodiment shown in FIGS. 1 to 4C, and also includes a processing unit 260B shown in FIG. 15 as the processing unit 260 of FIG. 4C. The ophthalmic examination apparatus of this embodiment may also include a correlation coefficient map creation unit 236 shown in FIG.

[0380] The processing unit 260B is configured to perform optical scanning to re-acquire data for areas with low correlation coefficients based on correspondence information representing the correspondence between the correlation coefficient set and the position information set. The processing unit 260B is realized by cooperation between hardware including a processor and processing software. The processing unit 260B includes a correlation coefficient specifying unit 264, a position information specifying unit 265, a scanning area setting unit 266, and an image synthesis processing unit 267. Note that some exemplary aspects of this embodiment may not include the image synthesis processing unit 267.

[0381] As in the first to third embodiments, the ophthalmic examination apparatus of this embodiment is configured to generate image data and corresponding information from a data set collected by optical scanning such as Lissajous scanning, and record them in association with each other. The ophthalmic examination apparatus of this embodiment is provided with a functional element (i.e., a functional element for performing rescanning) that recollects data of low-quality parts or parts that have not been imaged in the image data constructed by the image data processing unit 231 (for example, the final three-dimensional image data generated by the image data correction unit 233 or the merged image generated by the merge processing unit 2318).

[0382] The rescanning mode in this embodiment may be any mode, for example, a Lissajous scan or another scan mode. When a Lissajous scan is performed in the rescanning, the rescanning may include, for example, a cycle with a low correlation coefficient identified based on the correspondence information. When a rescanning is performed in a scan mode different from the Lissajous scan, the rescan may be, for example, a three-dimensional scan (such as a raster scan) that includes a portion with a low correlation coefficient identified based on the correspondence information (i.e., a low-quality portion or a portion not imaged in the image data constructed by the image data processing unit 231).

[0383] Similar to the correlation coefficient identification unit 261 of the second embodiment, the correlation coefficient identification unit 264 is configured to identify a subset having correlation coefficients that satisfy predetermined conditions as elements from among the correlation coefficients belonging to the correlation coefficient set related to the correspondence information.

[0384] Similar to the position information identification unit 262 of the second embodiment, the position information identification unit 265 is configured to identify, based on the correspondence information, a subset of the position information set that corresponds to the subset identified from the correlation coefficient set by the correlation coefficient identification unit 264. By referring to the correspondence between the correlation coefficient set and the position information set indicated in the correspondence information, the position information identification unit 262 identifies position information that corresponds to each correlation coefficient belonging to the subset identified by the correlation coefficient identification unit 264, and obtains a subset (which is a subset of the position information set) having the identified position information as an element.

[0385] In this embodiment, the correlation coefficient identification unit 264 is configured to identify correlation coefficients that are equal to or less than a predetermined threshold (the first threshold described above) from among the correlation coefficients belonging to the correlation coefficient set related to the correspondence information, and to obtain a subset having the identified correlation coefficients as elements. This subset is a subset of the correlation coefficient set. As a result, correlation coefficients (subset) corresponding to low-quality parts or parts not imaged in the image constructed using optical scanning are identified from the correlation coefficient set. The position information identification unit 265 identifies a subset of the position information set that corresponds to the subset identified from the correlation coefficient set by the correlation coefficient identification unit 264. The subset identified from the position information set by the position information identification unit 265 is information indicating the positions of low-quality parts or parts not imaged in the image constructed using optical scanning.

[0386] The ophthalmic examination apparatus of this embodiment applies a new optical scan to the subject's eye E in order to reacquire data corresponding to the subset identified from the position information set by the position information identifying unit 265. The scan area setting unit 266 is configured to set an area (referred to as a re-scan area) to which this new optical scan is applied. Setting of the re-scan area may be performed manually and / or automatically.

[0387] In some exemplary aspects when the rescanning area is set manually, the main control unit 211 causes the display unit 241 to display information based on the correspondence information (e.g., the correlation coefficient map of the third embodiment) and / or an image constructed based on the Lissajous scan (e.g., an image constructed by the image data corrector 233) together with a graphical user interface (GUI) for setting the rescanning area. The user determines a desired rescanning area by referring to the displayed information and / or image, and inputs this rescanning area using the operation unit 242 and the GUI. The scanning area setting unit 266 may be configured, for example, to compare the rescanning area input by the user with the correspondence information and / or the scanning protocol 2121, correct the rescanning area as necessary, and set the final rescanning area.

[0388] In some exemplary embodiments when the re-scan area is set automatically, the scan area setting unit 266 may be configured to set the re-scan area based on, for example, the correspondence information and the scan protocol 2121.

[0389] In this embodiment, portions of an image with low quality or portions that have not been imaged are identified based on the correspondence information. The scanning area setting unit 266 sets the re-scanning area to include such portions. In the case of manual setting, the scanning area setting unit 266 determines, for example, whether portions of an image with low quality or portions that have not been imaged are included in the re-scanning area specified by the user, and if it is determined that these portions are not included, the re-scanning area is modified to include these portions. In the case of automatic setting, the scanning area setting unit 266 sets the re-scanning area to include, for example, portions of an image with low quality or portions that have not been imaged.

[0390] The image synthesis processing unit 267, an optional element of this embodiment, is configured to cooperate with other elements of the ophthalmic examination apparatus to process a data set (referred to as a first data set) collected in the first optical scan (Lissajous scan) and a data set (referred to as a second data set) collected in the rescan to construct image data.

[0391] For example, the image synthesis processing unit 267 is configured to synthesize (merge) a first data set or image data based thereon (referred to as first image data) with a second data set or image data based thereon (referred to as second image data). The first image data includes areas of low quality or areas that were not imaged. On the other hand, the second image data is data constructed from a second data set collected in a rescan that is performed to include such areas. By synthesizing such first image data and second image data, it is possible to complement the first image data with the second image data.

[0392] In some exemplary embodiments, the processing unit 260B may be configured to perform the following three processing steps. First, the processing unit 260B applies the same processing as in the first embodiment to the first data set to construct first image data corresponding to an area obtained by excluding at least a portion of an area corresponding to a low-quality area or an area not imaged from the area where the initial Lissajous scan was performed (first processing step). The processing unit 260B also applies the same processing as in the first embodiment to the second data set to construct second image data corresponding to a re-scanned area (second processing step). Furthermore, the processing unit 260B (image synthesis processing unit 267) synthesizes the first image data constructed in the first processing step with the second image data constructed in the second processing step (third processing step). By performing these processing steps, the first image data and the second image data can be suitably synthesized. For example, by setting the construction area of ​​the first image data so that there is an overlapping area between the first image data and the second image data, and aligning the area in the first image data corresponding to this overlapping area with the area in the second image data corresponding to this overlapping area, it is possible to improve the accuracy and precision of the registration between the first image data and the second image data.

[0393] The operation of the ophthalmic examination apparatus according to this embodiment will now be described. An example of the operation of the ophthalmic examination apparatus is shown in Fig. 16. Unless otherwise specified, the matters described in the first to third embodiments can be combined with this example of operation.

[0394] (S81: Steps S41 to S49 in FIG. 11) In this operation example, first, steps S41 to S49 of FIG. 11 of the second embodiment are executed, whereby the correspondence information and image data generated from the data set collected by the Lissajous scan are recorded in association with each other.

[0395] The operation of the ophthalmic examination apparatus according to this embodiment is not limited to this. In an operation example different from this operation example, other steps may be performed instead of at least part of steps S41 to S49 in Fig. 11 of the second embodiment, part of steps S41 to S49 in Fig. 11 of the second embodiment may be omitted, or other steps may be combined with steps S41 to S49 in Fig. 11 of the second embodiment. In some exemplary aspects, steps S61 to S69 in Fig. 13 of the third embodiment may be performed.

[0396] (S82: Identify correlation coefficients below a threshold) Data processing unit 230 (correlation coefficient identification unit 264 of processing unit 260B) identifies correlation coefficients whose values ​​are equal to or less than a predetermined threshold (the above-mentioned first threshold) from among the correlation coefficients belonging to the correlation coefficient set related to the correspondence information recorded in step S81. As a result, a subset having low-value correlation coefficients as elements is identified from the correlation coefficient set.

[0397] (S83: Identify the corresponding location information) The data processing unit 230 (the location information identification unit 265 of the processing unit 260B) identifies the location information corresponding to the correlation coefficient identified in step S82 from among the location information belonging to the location information set related to the correspondence information recorded in step S81. As a result, a subset having location information corresponding to a low correlation coefficient as an element is identified from the location information set.

[0398] (S84: Identifying corresponding pixels in image data) The data processing unit 230 (scanning area setting unit 266 of processing unit 260B) sets a rescanning area based on the subset identified from the position information set in step S83. This sets a rescanning area corresponding to a low correlation coefficient. In other words, this step sets a rescanning area that includes parts of low-quality images or parts that were not imaged in the image data recorded in step S81.

[0399] The rescan area setting in this step may be any of the following techniques: the manual setting described above; other manual settings; the automatic setting described above; other automatic settings; semi-automatic settings; or a technique including at least a partial combination of at least two of these exemplary setting techniques.

[0400] (S85: Rescan) The scan control unit 2111 applies a rescan to the eye E based on the rescan area set in step S84.

[0401] In this example, a case where a Lissajous scan is performed as a rescan will be described, but the present embodiment is not limited to this.

[0402] In an aspect where Lissajous scanning is applied to rescanning, as in this example, the process of constructing new image data from two pieces of image data based on two Lissajous scans (the Lissajous scan in step S81 (step S43) and the Lissajous scan in the rescanning in step S85) can be performed by the strip replacement process in step S87, which will be described later. Therefore, when Lissajous scanning is applied to rescanning, in some exemplary aspects, the image synthesis processing unit 267 may not be provided. Note that even when Lissajous scanning is applied to rescanning, in some exemplary aspects, the process of constructing new image data from two pieces of image data may include image data synthesis by the image synthesis processing unit 267. Furthermore, when Lissajous scanning is applied to rescanning, in some exemplary aspects, the image synthesis processing unit 267 may be configured to perform strip replacement processing.

[0403] In contrast, in an aspect equipped with an image synthesis processing unit 267, the rescanning of step S85 may be performed in any scan mode, and the image data based on the Lissajous scan of step S81 (step S43) and the image data based on the rescanning of step S85 in any scan mode can be synthesized using the method described above (or a similar method).

[0404] (S86: Build a strip for each cycle of rescanning) The image data constructing unit 220 constructs strips corresponding to each cycle of the rescanning based on the data set collected in the rescanning in step S85. This process may be performed, for example, in the same manner as in the first embodiment (step S2 in FIG. 7A).

[0405] (S87: Replace the corresponding strip of image data with each strip based on the rescan) The ophthalmic examination apparatus (for example, the main control unit 211 or the processing unit 260B (image synthesis processing unit 267)) identifies a corresponding strip from among the strips constructed in step S81 (step S44) for each strip constructed in step S86. Here, the correspondence between the strip based on the Lissajous scan in step S81 (step S43) and the strip based on the rescan in step S85 can be established by referring to position information (such as a cycle identifier).

[0406] The ophthalmic examination apparatus (for example, the main control unit 211 or the processing unit 260B (image synthesis processing unit 267)) replaces the identified corresponding strip (which is any of the strips obtained in step S81 (step S44)) with the strip (which is the strip obtained in step S86) for each strip constructed in step S86.

[0407] This allows low-quality portions (strips corresponding to cycles with low correlation coefficients) in the image obtained by the Lissajous scan in step S81 (step S44) to be replaced with the image obtained by the rescan in step S85. Note that a similar correlation coefficient calculation may be performed based on the strips obtained by the rescan, and the quality may be reevaluated. If the quality of the strips obtained by the rescan is determined to be low, for example, a further scan may be performed.

[0408] (S88: Constructing 3D image data) The processing unit 260B (image synthesis processing unit 267) constructs three-dimensional image data based on the strip set obtained in step S87 (i.e., a set including, as elements, the high-quality strip set based on the Lissajous scan in step S81 (step S44) and the strip set based on the rescan in step S85). The processing in this step may be performed in the same manner as the combined processing of image data construction and motion artifact correction in the first embodiment (e.g., steps S3 to S32 in FIGS. 7A to 7E).

[0409] (S89: Save 3D image data) The ophthalmic examination apparatus (for example, the main control unit 211 or the processing unit 260B) stores the three-dimensional image data acquired in step S88 in, for example, the storage unit 212. Here, this three-dimensional image data is stored in association with, for example, the corresponding information (and image data) recorded in step S81 (step S49).

[0410] (S90: Display image) The ophthalmic examination apparatus (for example, the image data processing unit 231 or the processing unit 260B) constructs a rendering image of the three-dimensional image data acquired in step S88. The main control unit 211 causes the display unit 241 to display this rendering image (END).

[0411] The ophthalmic examination apparatus of this embodiment includes a data set collection unit, a correlation coefficient calculation unit, a correspondence information generation unit, an information recording unit, and a processing unit, similar to the first embodiment.

[0412] The processing unit of this embodiment is configured to execute a process (first process) of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the correlation coefficient set acquired by the correlation coefficient calculation unit, and a process (second process) of identifying a subset of the position information set corresponding to the subset identified from the correlation coefficient set, based on the correspondence information. The dataset collection unit of this embodiment is configured to apply a rescan to the sample (fundus of the subject's eye) based on the subset identified from the position information set by the processing unit.

[0413] According to the ophthalmic examination apparatus of this embodiment, as in the first embodiment, it is possible to store correlation coefficient sets and position information sets that were not recorded in conventional technology, and further, it is possible to record correspondence information that represents the correspondence between the correlation coefficient sets and the position information sets, thereby making it possible to provide a new method of using data acquired by scanning imaging.

[0414] Furthermore, the ophthalmic examination apparatus of this embodiment can perform rescanning based on data (correspondence information, image data) acquired by scanning imaging. More specifically, the ophthalmic examination apparatus of this embodiment can perform rescanning to acquire data that can be used to complement (image complement) image data constructed based on a data set collected by optical scanning, based on the recorded correspondence information. Therefore, it is possible to provide a novel method for utilizing data acquired by scanning imaging. Note that a similar processing unit may be provided in other devices.

[0415] The ophthalmic examination apparatus of this embodiment may also be configured to execute a process (third process) of setting an application area for rescanning (rescan area) based on a subset identified from the position information set by the processing unit. This makes it possible to provide a new way of using data acquired by scanning imaging, and, for example, to improve the convenience of settings for rescanning (for example, simplifying, facilitating, or automating the settings).

[0416] Furthermore, the ophthalmic examination apparatus of this embodiment may be configured to perform a process (fourth process) of combining image data (first image data) constructed based on a data set (first data set) collected in a first optical scan (first optical scan) with image data (second image data) constructed based on a data set (second data set) collected in a second optical scan (second optical scan: more generally, any one or more optical scans from the second scan onward). This makes it possible to provide a new way of using data acquired by scanning imaging, and, for example, improve the convenience of combining two or more image data acquired by two (or more) optical scans (e.g., simplifying, facilitating, and automating the combination). Furthermore, according to the ophthalmic examination apparatus of this embodiment, it is possible to construct image data by combining a data set collected in the first optical scan with a data set collected in the second optical scan, thereby making it possible to acquire higher quality image data.

[0417] The processes that can be executed by the processing unit of this embodiment are not limited to those described above. In some exemplary embodiments, the processing unit can use the information (correspondence information, image data) recorded by the information recording unit to control preparatory operations for rescanning. In some exemplary embodiments, the recorded correspondence information and / or image data can be used for any preparatory operations such as fixation, alignment, focusing, auto-Z, Z-lock, etc.

[0418] In some exemplary aspects, any parameter for rescanning a portion where the correlation coefficient value is small may be controlled based on the recorded correspondence information and / or image data. Examples of such control include control to increase the intensity of the measurement light when rescanning a portion where the correlation coefficient value is small, control to reduce the scanning speed when rescanning a portion where the correlation coefficient value is small (to improve the signal-to-noise ratio), control to increase the scanning speed when rescanning a portion where the correlation coefficient value is small (to reduce motion artifacts caused by eye movement), etc.

[0419] <Fifth embodiment> A fifth embodiment will be described. The ophthalmic examination apparatus of this embodiment has the same configuration as that of the first embodiment shown in FIGS. 1 to 4C, and also includes a processing unit 260C shown in FIG. 17 as the processing unit 260 of FIG. 4C. The ophthalmic examination apparatus of this embodiment may also include a correlation coefficient map creation unit 236 shown in FIG.

[0420] As in the first to fourth embodiments, the ophthalmic examination apparatus of this embodiment is configured to generate image data and corresponding information from a data set collected by optical scanning such as a Lissajous scan, and record them in association with each other. The ophthalmic examination apparatus of this embodiment has a functional element that evaluates at least partial quality of the image data constructed by the image data processing unit 231 (for example, the final three-dimensional image data generated by the image data correction unit 233 or the merged image generated by the merge processing unit 2318).

[0421] The processing unit 260C, which provides an example of this functional element, is configured to evaluate the quality of at least a portion of the image data recorded in association with the recorded correspondence information based on a set of correlation coefficients related to the recorded correspondence information. The processing unit 260C includes an image quality evaluation unit 268 configured to perform this image data quality evaluation process. The image quality evaluation unit 268 is realized by cooperation between hardware including a processor and image quality evaluation software.

[0422] In some exemplary aspects, the image quality evaluation unit 268 may be configured to identify a subset having correlation coefficients satisfying a predetermined condition from among the correlation coefficients belonging to the correlation coefficient set related to the correspondence information, similarly to the correlation coefficient identification unit 261 of the second embodiment. For example, the image quality evaluation unit 268 may be configured to identify correlation coefficients that are equal to or less than a predetermined threshold (the first threshold described above) from among the correlation coefficients belonging to the correlation coefficient set, and to obtain a subset having the identified correlation coefficients as its elements. This makes it possible to identify correlation coefficients corresponding to low-quality portions of an image constructed using optical scanning from among the correlation coefficients belonging to the correlation coefficient set (a subset of the correlation coefficient set having such correlation coefficients as its elements).

[0423] Furthermore, similar to the position information identification unit 262 of the second embodiment, the image quality evaluation unit 268 may be configured to identify a subset of the position information set corresponding to the subset identified from the correlation coefficient set, based on the correspondence information. This makes it possible to identify position information corresponding to a correlation coefficient that satisfies a predetermined condition. For example, it is possible to identify position information corresponding to a low-quality portion in an image constructed using optical scanning (a subset of the position information set having such position information as elements).

[0424] Additionally, the image quality evaluation unit 268 may be configured to evaluate the quality of partial data of image data corresponding to location information belonging to at least a portion of the subset identified from the set of location information. This makes it possible to evaluate the quality of partial data evaluated as having low quality based on the correlation coefficient from a perspective other than the correlation coefficient (from the perspective of image quality). Furthermore, it is possible to obtain information (such as correlation) that represents the relationship between the quality evaluation based on the correlation coefficient and the quality evaluation based on other criteria. Such information can be collected and the resulting information group can be applied to machine learning or to creating training data for machine learning. Note that the use of this information group is not limited to these examples.

[0425] Furthermore, based on information representing the relationship between the quality evaluation based on the correlation coefficient and the quality evaluation based on other criteria, a decision can be made regarding processing of the image data recorded together with this correspondence information. Some exemplary aspects of the ophthalmic examination apparatus (e.g., the processing unit 260C) may be configured to determine whether to perform a predetermined process (e.g., the image interpolation of the second or third embodiment, or the rescanning of the fourth embodiment) based on this information. Some exemplary aspects of the ophthalmic examination apparatus (e.g., the processing unit 260C) may be configured to determine the extent to which the predetermined process (e.g., the image interpolation of the second or third embodiment, or the rescanning of the fourth embodiment) should be applied based on this information. Note that the use of this information is not limited to these examples.

[0426] In some exemplary aspects, the image quality evaluation unit 268 may be configured to make a decision regarding the execution of the image completion of the second or third embodiment based on position information corresponding to a low-quality portion in an image constructed using optical scanning (a subset of the position information set having such position information as elements). For example, for each piece of position information belonging to the subset of the position information set, the image quality evaluation unit 268 identifies one or more neighboring pixels of a pixel corresponding to the position information (which is a pixel of the image data recorded together with this corresponding information) and evaluates the image quality of each identified neighboring pixel. If the image quality of a neighboring pixel is evaluated to be high (e.g., equal to or greater than a predetermined threshold), the neighboring pixel is used for image completion. If the image quality of a neighboring pixel is evaluated to be low (e.g., lower than a predetermined threshold), the neighboring pixel can be excluded from image completion. Alternatively, if the image quality of a neighboring pixel is evaluated to be low (e.g., lower than a predetermined threshold), new neighboring pixels can be searched for (e.g., the range of neighboring pixels used for image completion can be expanded), and image completion can be performed using the newly specified neighboring pixels.

[0427] The image quality assessment unit 268 may assess the quality of the entire image data recorded together with the corresponding information. In some exemplary embodiments, the image quality assessment unit 268 may generate one assessment result by applying one quality assessment process to the entire image data. In some exemplary embodiments, the image quality assessment unit 268 may generate two or more assessment results by applying two or more quality assessment processes to the entire image data, respectively.

[0428] In some exemplary embodiments, the image quality assessment unit 268 may divide the image data into multiple partial images and apply one quality assessment process to each partial image, thereby generating multiple assessment results corresponding to the multiple partial images. In some exemplary embodiments, the image quality assessment unit 268 may divide the image data into multiple partial images and apply two or more quality assessment processes to each partial image, thereby generating multiple assessment results corresponding to the multiple partial images. The process of dividing the image data into multiple partial images may be performed based on correspondence information. For example, the image quality assessment unit 268 may be configured to perform this division process based on the magnitude of the correlation coefficient value.

[0429] In some exemplary embodiments, the image quality assessment unit 268 may be configured to determine one or more assessment methods to be used in the quality assessment process of the image data based on the correspondence information. For example, the image quality assessment unit 268 may be configured to be able to execute two or more assessment methods and to determine the assessment method based on a value (statistic) calculated by applying a statistical operation to the set of correlation coefficients. This statistic may be any representative value (summary statistic), examples of which include a maximum value, minimum value, average value, median value, mode, and quantile. Furthermore, the image quality assessment unit 268 may be configured to create a histogram of correlation coefficients from the set of correlation coefficients and determine the assessment method based on this histogram.

[0430] In some exemplary embodiments, the image quality assessment unit 268 can divide the image data into multiple partial images and determine the assessment method to be applied to each partial image based on the correspondence information. For example, the image quality assessment unit 268 can determine the assessment method to be applied to each partial image based on the magnitude of the correlation coefficient value. This makes it possible to apply different assessment methods to partial images with relatively high quality and partial images with relatively low quality, for example.

[0431] The quality assessment method performed by the image quality assessment unit 268 may be any method that can be used to assess the quality of an image, and examples thereof include sharpness assessment (e.g., point spread function (PSF) or modulation transfer function (MTF)), noise assessment (e.g., root mean square (RMS) granularity, noise spectral density (NSDI, Wiener spectrum), tone assessment (e.g., quantization bit rate, gamma characteristics, contrast), and spatial characteristic assessment (e.g., luminance unevenness, distortion). The image quality assessment unit 268 may also include a model (typically including a convolutional neural network) trained to perform image quality assessment using machine learning.

[0432] The operation of the ophthalmic examination apparatus according to this embodiment will be described. An example of the operation of the ophthalmic examination apparatus is shown in Fig. 18. Unless otherwise specified, the matters described in the first to fourth embodiments can be combined with this example of operation.

[0433] (S101: Steps S41 to S49 in FIG. 11) In this operation example, first, steps S41 to S49 of FIG. 11 of the second embodiment are executed, whereby the correspondence information and image data generated from the data set collected by the Lissajous scan are recorded in association with each other.

[0434] The operation of the ophthalmic examination apparatus according to this embodiment is not limited to this. In an operation example different from this operation example, other steps may be performed instead of at least part of steps S41 to S49 in Fig. 11 of the second embodiment, part of steps S41 to S49 in Fig. 11 of the second embodiment may be omitted, or other steps may be combined with steps S41 to S49 in Fig. 11 of the second embodiment. In some exemplary aspects, steps S61 to S69 in Fig. 13 of the third embodiment may be performed.

[0435] (S102: Evaluate the quality of the image data) The data processing unit 230 (image quality evaluation unit 268 of the processing unit 260C) evaluates the quality of the image data (all or part) recorded together with the correspondence information based on the correlation coefficient set related to the correspondence information recorded in step S101. This quality evaluation process may be performed by, for example, any of the methods described above, but is not limited to these.

[0436] (S103: Save the quality evaluation results) The ophthalmic examination apparatus (for example, the main control unit 211 or the processing unit 260C) stores the result of the image quality evaluation performed in step S102, for example, in the storage unit 212. Here, this evaluation result is stored, for example, in association with the image data (and corresponding information) recorded in step S101 (step S49).

[0437] (S104: Display the image and the quality evaluation results) The ophthalmic examination apparatus (e.g., image data processing unit 231 or processing unit 260C) constructs a rendering image of this image data (if necessary). The main control unit 211 causes this rendering image and the result of the image quality evaluation performed in step S102 to be displayed on the display unit 241 (END).

[0438] The ophthalmic examination apparatus of this embodiment includes a data set collection unit, a correlation coefficient calculation unit, a correspondence information generation unit, an information recording unit, and a processing unit, similar to the first embodiment.

[0439] The processing unit of this embodiment is configured to perform a process of evaluating the quality of at least a portion of the image data based on the set of correlation coefficients obtained by the correlation coefficient calculation unit, and a process of recording the results of this quality evaluation process in association with the image data.

[0440] According to the ophthalmic examination apparatus of this embodiment, as in the first embodiment, it is possible to store correlation coefficient sets and position information sets that were not recorded in conventional technology, and further, it is possible to record correspondence information that represents the correspondence between the correlation coefficient sets and the position information sets, thereby making it possible to provide a new method of using data acquired by scanning imaging.

[0441] Furthermore, the ophthalmic examination apparatus of this embodiment is configured to evaluate the quality of image data recorded in association with the correspondence information (the set of correlation coefficients associated therewith) recorded by the information recording unit. Therefore, it is possible to provide a novel method for utilizing data acquired by scanning imaging. Note that a similar processing unit may be provided in other devices.

[0442] Sixth Embodiment Although various examples of processing performed using the correspondence information recorded by the information recording unit have been described in the first to fifth embodiments, the processing that can be performed using the correspondence information is not limited to these examples. In this embodiment, further non-limiting examples of processing that can be performed using the correspondence information will be described.

[0443] In some exemplary embodiments, an ophthalmic examination device (scanning imaging device) can complement images generated by the ophthalmic examination device using images acquired by other modalities. For example, low-quality or missing portions of an OCT fundus image generated by the ophthalmic examination device can be complemented using images generated by a fundus camera, an SLO, or another OCT. Such image complementation provides a novel application of image fusion.

[0444] In some exemplary embodiments, the correspondence information recorded by the information recording unit can be used for diagnosis. The ophthalmic examination apparatus (scanning imaging apparatus) may be configured to calculate statistics by applying statistical operations to a set of correlation coefficients related to the correspondence information, and to generate diagnostic data based on the statistics.

[0445] For example, the ophthalmic examination apparatus can calculate the sum of correlation coefficients belonging to a set of correlation coefficients related to the correspondence information and evaluate the state of eye movement of the subject's eye based on this sum. The state of eye movement to be evaluated may be, for example, the amount of eye movement.

[0446] As another example, the ophthalmic examination apparatus can generate distribution information of correlation coefficients belonging to a set of correlation coefficients related to the correspondence information and evaluate the state of nystagmus of the subject's eye based on this distribution information. This distribution information may be, for example, a frequency distribution of correlation coefficients (e.g., a histogram) or a position distribution of correlation coefficients based on the correspondence between a set of correlation coefficients and a set of position information (e.g., a map). Furthermore, the state of nystagmus that is the subject of this evaluation may be, for example, a classification of nystagmus (identification of the type of nystagmus). Types of nystagmus include physiological nystagmus and pathological nystagmus, and pathological nystagmus includes optic nystagmus and pathological otic nystagmus.

[0447] Seventh Embodiment A seventh embodiment will be described. In the first to sixth embodiments, a scanning imaging device configured to be able to perform optical scanning has been described, but in this embodiment, an information processing device configured to receive from the outside a data set collected by optical scanning and process it will be described.

[0448] The information processing device of this embodiment may be configured as a part of a scanning imaging device, but such a case will be excluded from the following description. However, it will be understood by those skilled in the art that any matter in the following description can also be applied to an information processing device configured as a part of a scanning imaging device.

[0449] The information processing device of this embodiment may have a configuration similar to any of the scanning imaging devices (ophthalmic examination devices) of the first to sixth embodiments, except that it does not have a functional element that performs optical scanning (the data set collection unit in the first to sixth embodiments).

[0450] Any of the items described in the first to sixth embodiments can be combined with the information processing device of this embodiment, and the information processing device will achieve functions and effects according to the combined items.

[0451] An example of the configuration of the information processing device of this embodiment is shown in Fig. 19. This information processing device 300 includes a receiving unit 301, a correlation coefficient calculation unit 302, a correspondence information generation unit 303, and an information recording unit 304.

[0452] The receiving unit 301 receives a data set collected by applying an optical scan that follows a two-dimensional pattern including a series of cycles that intersect with each other to a sample (e.g., an eye to be inspected). That is, the receiving unit 301 receives a data set collected from the sample by a data collecting unit of a scanning imaging device that can perform optical scanning such as a Lissajous scan.

[0453] The reception unit 301 has a function of acquiring data from an external device. This external device may be, for example, a computer, a storage device, a recording medium, or an information system (for example, a hospital information system, an electronic medical record system, or an image archiving system). The external device may be, for example, a device directly connected to the information processing device 300, a device connected to the information processing device 300 via a local area network (LAN), or a device connected to the information processing device 300 via a wide area network (WAN). The reception unit 301 may include, for example, a communication interface for performing data communication via a communication line and / or a drive device for reading data from a recording medium.

[0454] The correlation coefficient calculation unit 302 is configured to calculate a correlation coefficient between cycles based on a data set externally received by the reception unit 301. The correlation coefficient calculation unit 302 may include a configuration similar to that of the image data processing unit 231 of the first embodiment.

[0455] In some exemplary aspects, the correlation coefficient calculation unit 302 may include the image data construction unit 220, mask image generation unit 2311, range adjustment unit 2312, composite image generation unit 2313, cross-correlation function calculation unit 2314, and correlation coefficient calculation unit 2315 of the first embodiment, and may further include a z-shift amount calculation unit 232. Note that when the data set accepted by the acceptance unit 301 is image data (e.g., a strip), the correlation coefficient calculation unit 302 may not have a functional element corresponding to the image data construction unit 220.

[0456] The correspondence information generation unit 303 is configured to generate information (correspondence information) that indicates the correspondence between a set of correlation coefficients calculated by the correlation coefficient calculation unit 302 from the data set received by the reception unit 301 and a set of position information in the application area of ​​optical scanning from which the data set received by the reception unit 301 is collected. The correspondence information generation unit 303 may be configured similarly to the correspondence information generation unit 234 of the first embodiment.

[0457] The information recording unit 304 is configured to record the correspondence information generated by the correspondence information generating unit 303 in association with the image data constructed based on the data set accepted by the accepting unit 301. The information recording unit 304 may be configured similarly to the information recording unit 235 of the first embodiment.

[0458] Although not shown, the information processing device 300 of this embodiment may include one or more of at least a part of the data processing unit 230 of Fig. 4C, at least a part of the processing unit 260A of Fig. 10, at least a part of the correspondence information generating unit 234A of Fig. 12, at least a part of the processing unit 260B of Fig. 15, and at least a part of the processing unit 260C of Fig. 17. Note that functional elements that can be combined with the information processing device 300 of this embodiment are not limited to these, and may be any of the elements described in the first to sixth embodiments.

[0459] According to the information processing device 300, it is possible to store a set of correlation coefficients and a set of position information, which have not been recorded in conventional techniques, and further to record correspondence information indicating the correspondence between the set of correlation coefficients and the set of position information, thereby providing a new method of using data acquired by scanning imaging, some non-limiting examples of which have been described in the above-mentioned embodiments.

[0460] The information processing device 300 of this embodiment may further include a processing unit (not shown). The processing unit functions to execute predetermined processing based on the correspondence information recorded by the information recording unit. Such an information processing device 300 makes it possible to provide a new method of using data (correspondence information, image data) acquired by scanning imaging. Note that a similar processing unit may also be provided in other devices.

[0461] <Other embodiments> The embodiments of the present disclosure are not limited to the first to seventh embodiments and their modifications. For example, the first to seventh embodiments provide embodiments relating to various methods, embodiments relating to various programs, embodiments relating to various recording media, and the like.

[0462] For example, the first embodiment provides an embodiment of a method for controlling a scanning imaging apparatus. This embodiment is a method for controlling a scanning imaging apparatus including a scanner that performs optical scanning and a processor that constructs image data from data collected by the optical scanning. The scanner may be the data set collection unit of the first embodiment, and the processor may be the image data construction unit 220 and the image data processing unit 231 (at least a part of the image data processing unit 231) of the first embodiment.

[0463] The control method for a scanning imaging apparatus according to this embodiment may include the following four steps: a first step is executed by controlling a scanner to apply an optical scan according to a two-dimensional pattern including a series of cycles intersecting each other to a sample to collect a data set; a second step is executed by controlling a processor to calculate a correlation coefficient between the cycles based on the data set collected in the first step; a third step is executed by controlling the processor to generate correspondence information representing a correspondence between a set of correlation coefficients calculated in the second step from the data set collected in the first step and a set of positional information in the area where the optical scan was applied in the first step; and a fourth step is executed by controlling the processor to record the correspondence information generated in the third step in association with image data constructed based on the data set collected in the first step.

[0464] Any of the features described in the first to sixth embodiments can be combined with the control method for the scanning imaging device according to this embodiment, and the control method for the scanning imaging device will have the functions and effects according to the combined features.

[0465] According to this control method for a scanning imaging device, it is possible to store correlation coefficient sets and position information sets, which were not recorded in conventional techniques, and it is also possible to record correspondence information that represents the correspondence between the correlation coefficient sets and the position information sets, thereby providing a new way of using data acquired by scanning imaging.

[0466] The control method for a scanning imaging device of this embodiment may further include a fifth step of controlling a processor to execute a predetermined process based on the correspondence information recorded in the fourth step. Such a control method for a scanning imaging device makes it possible to provide a new way of using data (correspondence information, image data) acquired by scanning imaging. Note that the fifth step may be executed by another device.

[0467] The seventh embodiment provides an embodiment of a method for controlling an information processing device. This embodiment is a method for controlling an information processing device including a processor. This processor may be the image data creation unit 220 and the image data processing unit 231 (at least a part of the image data processing unit 231) of the first embodiment.

[0468] The control method for an information processing device according to this embodiment may include the following four steps. The first step is executed by controlling a processor to receive a data set collected by applying an optical scan according to a two-dimensional pattern including a series of intersecting cycles to a sample. The second step is executed by controlling a processor to calculate a correlation coefficient between the cycles based on the data set received in the first step. The third step is executed by controlling a processor to generate correspondence information representing a correspondence between a set of correlation coefficients calculated in the second step from the data set received in the first step and a set of positional information in an application area of ​​the optical scan performed to collect the data set. The fourth step is executed by controlling a processor to record the correspondence information generated in the third step in association with image data constructed based on the data set received in the first step.

[0469] Any of the features described in the first to seventh embodiments can be combined with the control method of the information processing device according to this embodiment, and the control method of the information processing device will have the effects and advantages according to the combined features.

[0470] According to this control method for an information processing device, it is possible to store correlation coefficient sets and position information sets, which were not recorded in conventional techniques, and further to record correspondence information that indicates the correspondence between the correlation coefficient sets and the position information sets, thereby providing a new method of using data acquired by scanning imaging.

[0471] The control method for the information processing device of this embodiment may further include a fifth step of controlling the processor to execute a predetermined process based on the correspondence information recorded in the fourth step. Such a control method for the information processing device makes it possible to provide a new way of using data (correspondence information, image data) acquired by scanning imaging. Note that the fifth step may be executed by another device.

[0472] The first embodiment and / or the seventh embodiment provide an embodiment of a method for processing information.

[0473] The information processing method according to this embodiment may include the following four steps: a first step is performed to acquire a dataset collected from a sample by optical scanning according to a two-dimensional pattern including a series of cycles that intersect with each other; a first step of the information processing method according to the first embodiment is performed to collect a dataset by applying optical scanning according to a two-dimensional pattern including a series of cycles that intersect with each other to a sample; a first step of the information processing method according to the seventh embodiment is performed to externally receive a dataset collected by applying optical scanning according to a two-dimensional pattern including a series of cycles that intersect with each other to a sample; a second step is performed to calculate a correlation coefficient between the cycles based on the dataset acquired in the first step; a third step is performed to generate correspondence information representing a correspondence between a set of correlation coefficients calculated in the second step from the dataset acquired in the first step and a set of positional information in an application area of ​​the optical scanning performed to collect the dataset; and a fourth step is performed to record the correspondence information generated in the third step in association with image data constructed based on the dataset acquired in the first step.

[0474] Any of the items described in the first to seventh embodiments can be combined with the information processing method according to this embodiment, and the information processing method will achieve functions and effects according to the items combined.

[0475] According to this information processing method, it is possible to store correlation coefficient sets and position information sets, which were not recorded in conventional techniques, and further to record correspondence information that indicates the correspondence between the correlation coefficient sets and the position information sets, thereby providing a new way of using data acquired by scanning imaging.

[0476] The information processing method of this embodiment may further include a fifth step of executing a predetermined process based on the correspondence information recorded in the fourth step. Such an information processing method makes it possible to provide a novel way of using data (correspondence information, image data) acquired by scanning imaging.

[0477] Any of the items described in the first to seventh embodiments can be combined with the information processing method according to this embodiment, and the information processing method will achieve functions and effects according to the items combined.

[0478] According to this information processing method, it is possible to store correlation coefficient sets and position information sets, which were not recorded in conventional techniques, and further to record correspondence information that indicates the correspondence between the correlation coefficient sets and the position information sets, thereby providing a new way of using data acquired by scanning imaging.

[0479] The present disclosure provides a program for causing a computer to execute a method according to an embodiment. Some exemplary embodiments provide a program for causing a computer to execute a control method for a scanning imaging apparatus. Some exemplary embodiments provide a program for causing a computer to execute a control method for an information processing apparatus. Some exemplary embodiments provide a program for causing a computer to execute an information processing method.

[0480] The present disclosure provides a computer-readable non-transitory recording medium having a program recorded thereon according to an embodiment. Some exemplary embodiments provide a computer-readable non-transitory recording medium having a program recorded thereon that causes a computer to execute a control method for a scanning imaging apparatus. Some exemplary embodiments provide a computer-readable non-transitory recording medium having a program recorded thereon that causes a computer to execute a control method for an information processing apparatus. Some exemplary embodiments provide a computer-readable non-transitory recording medium having a program recorded thereon that causes a computer to execute an information processing method. The non-transitory recording medium of the embodiments may be in any form, examples of which include a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

[0481] The present disclosure merely illustrates some exemplary embodiments and is not intended to limit the invention. Those who intend to implement the present invention can make any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention.

[0482] For example, novel uses of data acquired by scanning imaging achieved by the techniques of the present disclosure are not limited to the uses described in the various embodiments above. In some exemplary aspects, the techniques of the present disclosure can be used to detect temporal changes (e.g., changes in shape, size, position, etc.) of a sample (e.g., a living organ, tissue, etc.). In some exemplary aspects, the detected temporal changes can be used for any processing (e.g., device control, data processing, etc.). [Explanation of symbols]

[0483] 1. Ophthalmic examination equipment 220 Image Data Construction Department 230 Data Processing Unit 231 Image data processing unit 2315 Correlation coefficient calculation unit 232 z-shift amount calculation unit 234 Correspondence information generation unit 235 Information Recording Unit 260, 260A, 260B Processing section 261, 264 Correlation coefficient specification part 262, 265 Location information identification section 263 Pixel Value Determination Unit 266 Scanning area setting unit 267 Image synthesis processing unit

Claims

1. 1. A scanning imaging device that constructs image data using optical scanning, comprising: a data set collection unit that collects a data set by applying a Lissajous scan to the sample, the Lissajous scan being an optical scan that follows a Lissajous pattern including a series of cycles that intersect with each other; a correlation coefficient calculation unit that calculates a correlation coefficient in an intersection region between cycles based on the data set; a correspondence information generating unit that generates correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set by the correlation coefficient calculating unit and a set of position information in an application area of ​​the Lissajous scan; an information recording unit that records the correspondence information in association with image data constructed based on the data set; a processing unit that executes a predetermined process based on the correspondence information recorded by the information recording unit; Including, The processing unit performs the predetermined processing as follows: a process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the set of correlation coefficients acquired by the correlation coefficient calculation unit; identifying a subset of the position information corresponding to the subset of the correlation coefficients based on the correspondence information; determining a value of a pixel of the image data corresponding to the subset of location information based on values ​​of one or more neighboring pixels; To execute Scanning imaging device.

2. 1. A scanning imaging device that constructs image data using optical scanning, comprising: a data set collection unit that collects a first data set by applying a Lissajous scan, which is an optical scan that follows a Lissajous pattern including a series of cycles that intersect with each other, as a first optical scan to the sample; a correlation coefficient calculation unit that calculates a correlation coefficient in an intersection region between cycles based on the first data set; a correspondence information generating unit that generates correspondence information representing a correspondence between a set of correlation coefficients calculated from the first data set by the correlation coefficient calculating unit and a set of position information in an application area of ​​the first optical scanning; an information recording unit that records the correspondence information in association with first image data constructed based on the first data set; a processing unit that executes a predetermined process based on the correspondence information recorded by the information recording unit; Including, The processing unit performs the predetermined processing as follows: a process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the set of correlation coefficients acquired by the correlation coefficient calculation unit; identifying a subset of the position information corresponding to the subset of the correlation coefficients based on the correspondence information; Run the data set collection unit applies a second optical scan to the sample based on the subset of the position information determined by the processing unit. Scanning imaging device.

3. The processing unit performs the predetermined processing as follows: a process of setting an application area of ​​the second optical scanning based on the subset of the position information; combining the first image data constructed based on the first data set collected by the first optical scan and the second image data constructed based on the second data set collected by the second optical scan; Further execute 3. The scanning imaging device of claim 2.

4. 1. A scanning imaging device that constructs image data using optical scanning, comprising: a data set collection unit that collects a data set by applying a Lissajous scan to the sample, the Lissajous scan being an optical scan that follows a Lissajous pattern including a series of cycles that intersect with each other; a correlation coefficient calculation unit that calculates a correlation coefficient in an intersection region between cycles based on the data set; a correspondence information generating unit that generates correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set by the correlation coefficient calculating unit and a set of position information in an application area of ​​the Lissajous scan; an information recording unit that records the correspondence information in association with image data constructed based on the data set; a processing unit that executes a predetermined process based on the correspondence information recorded by the information recording unit; Including, The processing unit performs the predetermined processing as follows: evaluating the quality of at least a portion of the image data based on the set of correlation coefficients obtained by the correlation coefficient calculation unit; a process of recording the quality evaluation result in association with the image data; To execute Scanning imaging device.

5. a receiving unit that receives a data set collected by applying a Lissajous scan, which is an optical scan that follows a Lissajous pattern including a series of cycles that intersect with each other, to a sample; a correlation coefficient calculation unit that calculates a correlation coefficient in an intersection region between cycles based on the data set; a correspondence information generating unit that generates correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set by the correlation coefficient calculating unit and a set of position information in an application area of ​​the Lissajous scan; an information recording unit that records the correspondence information in association with image data constructed based on the data set; a processing unit that executes a predetermined process based on the correspondence information recorded by the information recording unit; Including, The processing unit performs the predetermined processing as follows: a process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the set of correlation coefficients acquired by the correlation coefficient calculation unit; identifying a subset of the position information corresponding to the subset of the correlation coefficients based on the correspondence information; determining a value of a pixel of the image data corresponding to the subset of location information based on values ​​of one or more neighboring pixels; To execute Information processing device.

6. a receiving unit that receives a data set collected by applying a Lissajous scan, which is an optical scan that follows a Lissajous pattern including a series of cycles that intersect with each other, to a sample; a correlation coefficient calculation unit that calculates a correlation coefficient in an intersection region between cycles based on the data set; a correspondence information generating unit that generates correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set by the correlation coefficient calculating unit and a set of position information in an application area of ​​the Lissajous scan; an information recording unit that records the correspondence information in association with image data constructed based on the data set; a processing unit that executes a predetermined process based on the correspondence information recorded by the information recording unit; Including, The processing unit performs the predetermined processing as follows: a process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the set of correlation coefficients acquired by the correlation coefficient calculation unit; identifying a subset of the position information corresponding to the subset of the correlation coefficients based on the correspondence information; Run the subset of positional information identified by the processing unit is used for further optical scanning of the sample. Information processing device.

7. a receiving unit that receives a data set collected by applying a Lissajous scan, which is an optical scan that follows a Lissajous pattern including a series of cycles that intersect with each other, to a sample; a correlation coefficient calculation unit that calculates a correlation coefficient in an intersection region between cycles based on the data set; a correspondence information generating unit that generates correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set by the correlation coefficient calculating unit and a set of position information in an application area of ​​the Lissajous scan; an information recording unit that records the correspondence information in association with image data constructed based on the data set; a processing unit that executes a predetermined process based on the correspondence information recorded by the information recording unit; Including, The processing unit performs the predetermined processing as follows: evaluating the quality of at least a portion of the image data based on the set of correlation coefficients obtained by the correlation coefficient calculation unit; a process of recording the quality evaluation result in association with the image data; To execute Information processing device.

8. 1. A method for controlling a scanning imaging device including a scanner for performing an optical scan and a processor for constructing image data from data collected from the optical scan, comprising: controlling the scanner to apply a Lissajous scan, an optical scan that follows a Lissajous pattern including a series of intersecting cycles, to the sample to collect a data set; controlling the processor to calculate a correlation coefficient at an intersection region between cycles based on the data set; controlling the processor to generate correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set and a set of position information in an application area of ​​the Lissajous scan; controlling the processor to record the correspondence information in association with image data constructed based on the data set; controlling the processor to execute a predetermined process based on the correspondence information; The predetermined processing is A process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the set of correlation coefficients; identifying a subset of the position information corresponding to the subset of the correlation coefficients based on the correspondence information; determining a value of a pixel of the image data corresponding to the subset of location information based on values ​​of one or more neighboring pixels; Including, method.

9. 1. A method for controlling a scanning imaging device including a scanner for performing an optical scan and a processor for constructing image data from data collected from the optical scan, comprising: controlling the scanner to apply a Lissajous scan, an optical scan that follows a Lissajous pattern including a series of intersecting cycles, to the sample to collect a data set; controlling the processor to calculate a correlation coefficient at an intersection region between cycles based on the data set; controlling the processor to generate correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set and a set of position information in an application area of ​​the Lissajous scan; controlling the processor to record the correspondence information in association with image data constructed based on the data set; controlling the processor to execute a predetermined process based on the correspondence information; The predetermined processing is A process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the set of correlation coefficients; identifying a subset of the position information corresponding to the subset of the correlation coefficients based on the correspondence information; Including, controlling the scanner to apply a further optical scan to the sample based on the subset of position information; method.

10. 1. A method for controlling a scanning imaging device including a scanner for performing an optical scan and a processor for constructing image data from data collected from the optical scan, comprising: controlling the scanner to apply a Lissajous scan, an optical scan that follows a Lissajous pattern including a series of intersecting cycles, to the sample to collect a data set; controlling the processor to calculate a correlation coefficient at an intersection region between cycles based on the data set; controlling the processor to generate correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set and a set of position information in an application area of ​​the Lissajous scan; controlling the processor to record the correspondence information in association with image data constructed based on the data set; controlling the processor to execute a predetermined process based on the correspondence information; The predetermined processing is assessing a quality of at least a portion of the image data based on the set of correlation coefficients; a process of recording the quality evaluation result in association with the image data; Including, method.

11. A method for controlling an information processing device including a processor, comprising: controlling the processor to accept a data set collected by applying a Lissajous scan to the sample, the Lissajous scan being an optical scan that follows a Lissajous pattern including a series of intersecting cycles; controlling the processor to calculate a correlation coefficient at an intersection region between cycles based on the data set; controlling the processor to generate correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set and a set of position information in an application area of ​​the Lissajous scan; controlling the processor to record the correspondence information in association with image data constructed based on the data set; controlling the processor to execute a predetermined process based on the correspondence information; The predetermined processing is A process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the set of correlation coefficients; identifying a subset of the position information corresponding to the subset of the correlation coefficients based on the correspondence information; determining a value of a pixel of the image data corresponding to the subset of location information based on values ​​of one or more neighboring pixels; Including, method.

12. A method for controlling an information processing device including a processor, comprising: controlling the processor to accept a data set collected by applying a Lissajous scan to the sample, the Lissajous scan being an optical scan that follows a Lissajous pattern including a series of intersecting cycles; controlling the processor to calculate a correlation coefficient at an intersection region between cycles based on the data set; controlling the processor to generate correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set and a set of position information in an application area of ​​the Lissajous scan; controlling the processor to record the correspondence information in association with image data constructed based on the data set; controlling the processor to execute a predetermined process based on the correspondence information; The predetermined processing is A process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the set of correlation coefficients; identifying a subset of the position information corresponding to the subset of the correlation coefficients based on the correspondence information; Including, the subset of position information is used for further optical scanning of the sample. method.

13. A method for controlling an information processing device including a processor, comprising: controlling the processor to accept a data set collected by applying a Lissajous scan to the sample, the Lissajous scan being an optical scan that follows a Lissajous pattern including a series of intersecting cycles; controlling the processor to calculate a correlation coefficient at an intersection region between cycles based on the data set; controlling the processor to generate correspondence information representing a correspondence between a set of correlation coefficients calculated from the data set and a set of position information in an application area of ​​the Lissajous scan; controlling the processor to record the correspondence information in association with image data constructed based on the data set; controlling the processor to execute a predetermined process based on the correspondence information; The predetermined processing is assessing a quality of at least a portion of the image data based on the set of correlation coefficients; a process of recording the quality evaluation result in association with the image data; Including, method.

14. applying a Lissajous scan, which is an optical scan that follows a Lissajous pattern that includes a series of intersecting cycles, to obtain a data set collected from the sample; calculating a correlation coefficient in the crossover region between cycles based on the data set; generating correspondence information representing correspondence between a set of correlation coefficients calculated from the data set and a set of position information in the application area of ​​the Lissajous scan; Recording the correspondence information in association with image data constructed based on the data set; Execute a predetermined process based on the correspondence information; The predetermined processing is A process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the set of correlation coefficients; identifying a subset of the position information corresponding to the subset of the correlation coefficients based on the correspondence information; determining a value of a pixel of the image data corresponding to the subset of location information based on values ​​of one or more neighboring pixels; Including, Information processing methods.

15. applying a Lissajous scan, which is an optical scan that follows a Lissajous pattern that includes a series of intersecting cycles, to obtain a data set collected from the sample; calculating a correlation coefficient in the crossover region between cycles based on the data set; generating correspondence information representing correspondence between a set of correlation coefficients calculated from the data set and a set of position information in the application area of ​​the Lissajous scan; Recording the correspondence information in association with image data constructed based on the data set; Execute a predetermined process based on the correspondence information; The predetermined processing is A process of identifying a subset having elements with correlation coefficients equal to or less than a predetermined threshold from the set of correlation coefficients; identifying a subset of the position information corresponding to the subset of the correlation coefficients based on the correspondence information; Including, the subset of position information is used for further optical scanning of the sample. Information processing methods.

16. applying a Lissajous scan, which is an optical scan that follows a Lissajous pattern that includes a series of intersecting cycles, to obtain a data set collected from the sample; calculating a correlation coefficient in the crossover region between cycles based on the data set; generating correspondence information representing correspondence between a set of correlation coefficients calculated from the data set and a set of position information in the application area of ​​the Lissajous scan; Recording the correspondence information in association with image data constructed based on the data set; Execute a predetermined process based on the correspondence information; The predetermined processing is assessing a quality of at least a portion of the image data based on the set of correlation coefficients; a process of recording the quality evaluation result in association with the image data; Including, Information processing methods.

17. A program that causes a computer to execute the method according to any one of claims 8 to 16.

18. A computer-readable non-transitory recording medium on which the program of claim 17 is recorded.

Images