Ophthalmic apparatus, method for controlling ophthalmic apparatus, and program

Abstract

To reduce positional deviation that occurs when a color fundus image and an En-Face image are superimposed and displayed, thereby facilitating observation of the fundus structure. [Solution] The ophthalmic device disclosed herein comprises an image acquisition means for acquiring a first image obtained by photographing a subject's eye using a first photographing method and a second image obtained by photographing the subject's eye using a second photographing method different from the first photographing method, an alignment means for deforming the second image so that the position of a first region included in the first image and the position of a second region included in the second image that corresponds to the first region are closer together, and a display control means for controlling the display of a superimposed image in which the first image and the deformed second image are superimposed on each other on a display unit.

Description

[Technical Field]

[0001] The present disclosure relates to an ophthalmic apparatus, a control method for an ophthalmic apparatus, and a program. [Background technology]

[0002] In the field of ophthalmology, ophthalmoscopes are commonly known as instruments used to observe the eye of a patient. In recent years, scanning laser ophthalmoscopes (SLOs) (hereinafter referred to as color SLOs) have been put to practical use. These ophthalmoscopes are equipped with light sources in multiple wavelength ranges and scan the eye of a patient to obtain color fundus images.

[0003] Color SLO uses a light source that emits light in the wavelength ranges corresponding to red (R), green (G), and blue (B). By using these light sources, color SLO can acquire color fundus images.

[0004] Optical Coherence Tomography (OCT) Angiography (hereafter referred to as OCTA) is also known, which uses OCT to non-invasively visualize fundus blood vessels. OCTA scans the same location multiple times (cluster scans) using measurement light to obtain multiple OCT tomographic images. OCTA then obtains an OCTA image, which is motion contrast data obtained from the interaction between the displacement of red blood cells and the measurement light, based on these multiple OCT tomographic images (clusters).

[0005] OCTA images are three-dimensional images, but it is possible to generate En-Face images, which are two-dimensional images obtained by selecting a portion of the three-dimensional image in the depth direction and projecting it onto a plane.

[0006] In addition, in order to observe the structure of the fundus using a color fundus image and an En-Face image, a method of displaying the color fundus image and the En-Face image in a superimposed manner is used.

[0007] Here, Patent Document 1 discloses a method of acquiring three-dimensional motion contrast data (OCTA image) of the fundus and generating an En-Face image.

[0008] Furthermore, Patent Document 2 discloses a method for generating a blended image in which an SLO image and an En-Face image are blended by blending the SLO image and the En-Face image. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Patent Publication No. 2019-209136 [Patent Document 2] Japanese Patent Publication No. 2023-10308 Summary of the Invention [Problem to be solved by the invention]

[0010] Here, for example, due to differences in aberration between the OCT optical system and the SLO optical system, the position of the blood vessel in the color fundus image may differ from the position of the blood vessel in the En-Face image, even for the same blood vessel. Therefore, when the color fundus image and the En-Face image are superimposed and displayed, the positions of the blood vessels may be displayed misaligned, making it difficult to observe the structure of the fundus.

[0011] Therefore, an object of the present disclosure is to reduce the positional deviation that occurs when a color fundus image and an En-Face image are displayed in a superimposed manner, thereby making it easier to observe the structure of the fundus. [Means for solving the problem]

[0012] The ophthalmologic apparatus of the present disclosure includes an image acquisition means for acquiring a first image obtained by photographing a subject's eye using a first photographing method and a second image obtained by photographing the subject's eye using a second photographing method different from the first photographing method, an alignment means for deforming the second image so that the position of a first region included in the first image and the position of a second region included in the second image that corresponds to the first region are closer together, and a display control means for controlling the display of a superimposed image in which the first image and the deformed second image are superimposed on a display unit. [Effects of the Invention]

[0013] According to the present disclosure, it is possible to reduce the positional deviation that occurs when a color fundus image and an En-Face image are displayed in a superimposed manner, making it easier to observe the structure of the fundus. [Brief explanation of the drawings]

[0014] [Figure 1] 1 shows an example of the optical configuration of an SLO device according to a first embodiment. [Figure 2] 3 is a block diagram showing an example of a functional configuration of a control unit according to the first embodiment. FIG. [Figure 3] 4 is a flowchart showing an example of a series of processes according to the first embodiment. [Figure 4] 10 is an example of a screen displaying an image according to an imaging result according to the first embodiment. [Figure 5] 10 is a flowchart showing an example of color SLO shooting processing according to the first embodiment. [Figure 6] 4A to 4C are examples of a color SLO image, a selected SLO image, and an En-Face image according to the first embodiment. [Figure 7] 10 is a flowchart showing an example of alignment of color SLO images according to the first embodiment. [Figure 8] 10 is an example of a superimposed display of a color SLO image and an En-Face image according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. However, the dimensions, materials, shapes, and relative positions of components described in the following embodiments are arbitrary and can be changed depending on the configuration of the device to which the present disclosure is applied or various conditions. Furthermore, in this specification and drawings, components having substantially the same or functionally similar configurations are designated by the same reference numerals between the drawings, and redundant description will be omitted.

[0016] First Embodiment As a first embodiment of the present disclosure, an example of a case in which the technology according to the present disclosure is applied to fundus imaging will be described below with reference to FIGS.

[0017] Below, we will explain an SLO / OCT combined device, which is an example of a fundus imaging device according to this embodiment. Note that in this embodiment, SLO and OCT are combined into one device, but they may also be separate devices. For this reason, in the following explanation, the device will be referred to as an SLO device when describing SLO imaging, and as an OCT device when describing OCT imaging or OCTA imaging. Note that the SLO device and OCT device are examples of image acquisition means.

[0018] The SLO device according to this embodiment switches the wavelength of the light (photography light) to be imaged for each frame (image capture frame) of the fundus of the subject's eye, irradiates the fundus with the light, scans the fundus with the light, and acquires image data of a color corresponding to the photography light. In particular, the SLO device according to this embodiment acquires image data of the image capture frame using light in the red wavelength region, light in the green wavelength region, or light in the blue wavelength region, performs a synthesis process, and generates a color SLO image. The red, green, and blue images that make up the color SLO image are referred to as the red SLO image, the green SLO image, and the blue SLO image, respectively. The SLO device and information processing method according to this embodiment will be described in more detail below.

[0019] (optical configuration) An example of the optical configuration of an SLO device according to this embodiment will be described with reference to Fig. 1. Fig. 1 schematically illustrates an example of the optical configuration of an SLO device according to this embodiment. The SLO device according to this embodiment includes a light output unit 100, a reflecting mirror 2, a perforated mirror 3, a focus lens 4, a light scanning unit 5, a lens 6, a dichroic mirror 7, a wavelength branching mirror 8, an objective lens 9, a light receiving unit 200, and an anterior eye imaging unit 300. The dichroic mirror 7 is a component where the optical paths of the OCT device described below converge, and the dichroic mirror 7, the wavelength branching mirror 8, and the objective lens 9 form an optical path common to the SLO device and the OCT device.

[0020] 1, the light output unit 100 is configured to be able to sequentially switch the wavelength of the imaging light among three or more wavelengths and output the light. Specifically, the light output unit 100 is provided with laser light sources 101IR and 101RGB, collimator lenses 103IR and 103RGB, and a wavelength branching mirror 105.

[0021] The laser light source 101IR is configured to emit infrared light. The laser light source 101RGB is a semiconductor laser in a CAN package mounted on a single substrate 104. The laser emitting units 101R, 101G, and 101B are light-emitting diodes with a common anode within the laser light source 101RGB. Based on instructions from the control unit 50, a drive current is applied from the substrate 104 to the laser emitting units 101R, 101G, and 101B, allowing them to switch between emitting light in the red wavelength region, light in the green wavelength region, and light in the blue wavelength region, respectively. Hereinafter, controlling the laser light source 101RGB to emit light from the laser emitting unit 101R may be referred to as "turning on the laser emitting unit 101R." The same applies to the laser emitting units 101G and 101B. By changing the pattern of the applied current, it is possible to select a single or multiple combinations of emitted light. Furthermore, the substrate 104 can control a Peltier element (not shown) to dissipate heat from a heat sink (not shown), thereby adjusting the temperature of the laser light source 101RGB.

[0022] The collimator lenses 103IR and 103RGB convert the light (laser light) emitted from the laser light sources 101IR and 101RGB into parallel light. The wavelength branching mirror 105 is configured to branch infrared light, light in the red wavelength region, light in the green wavelength region, and light in the blue wavelength region.

[0023] In this configuration, the infrared light emitted from the laser light source 101IR is converted into parallel light by the collimator lens 103IR, and then transmitted through the wavelength branching mirror 105.

[0024] The light in the red wavelength region emitted from the laser emission unit 101R is converted into parallel light by the collimator lens 103RGB, and then part of it is reflected and part of it is transmitted by the wavelength branching mirror 105. Similarly, the light in the green wavelength region emitted from the laser emission unit 101G and the light in the blue wavelength region emitted from the laser emission unit 101B are partly reflected and partly transmitted by the wavelength branching mirror 105. The transmitted light enters the light intensity monitor unit 110, and the light intensity is controlled based on the monitor output signal.

[0025] Arranged in this order on the optical path of the light emitted from the optical output unit 100 are a reflecting mirror 2, a perforated mirror 3, a focus lens 4, an optical scanning unit 5, a lens 6, a dichroic mirror 7, a wavelength branching mirror 8, and an objective lens 9. These optical members constitute the fundus observation optical system.

[0026] Light emitted from the light output unit 100 is reflected by a reflecting mirror 2, passes through a through-hole formed on the optical axis of a perforated mirror 3, and is guided to the light scanning unit 5 via a focus lens 4. The light scanning unit 5 may include, for example, a galvanometer mirror, a resonant mirror, or a polygon mirror arranged adjacent to each other in the optical axis direction (tandem arrangement) to scan light in the X and Y directions that are orthogonal to each other. In this embodiment, a polygon mirror 11 is used in the main scanning direction (high-speed scanning), and a galvanometer mirror 12 is used in the sub-scanning direction (slow-speed scanning). The light guided from the light output unit 100 to the light scanning unit 5 and emitted from the light scanning unit 5 passes through a lens 6, is reflected by a dichroic mirror 7 and a wavelength branching mirror 8, and is guided to the fundus Ef of the subject's eye E via an objective lens 9.

[0027] Here, the X and Y directions are directions that define a two-dimensional plane perpendicular to the Z direction, which is the depth direction of the eye E. For simplicity of explanation, the X direction will be referred to as the main scanning direction, and the Y direction perpendicular to the X direction will be referred to as the sub-scanning direction. However, the main scanning direction and the sub-scanning direction are not limited to the X direction and the Y direction, and may be any directions that intersect with each other.

[0028] Furthermore, the light reflected by the fundus Ef is guided to the perforated mirror 3 via the objective lens 9, wavelength branching mirror 8, dichroic mirror 7, lens 6, optical scanning unit 5, and focus lens 4, and is reflected by the peripheral portion of the perforated mirror 3. A light receiving unit 200 is disposed on the optical path of the light reflected by the perforated mirror 3.

[0029] The light receiving unit 200 is provided with a lens 201, an aperture 203, wavelength cut filters 204 and 205, and a light receiving element 202. In the example shown in FIG. 1 , the lens 201, the aperture 203, the wavelength cut filter 204, and the light receiving element 202 are arranged in this order on the optical path of light incident on the light receiving unit 200. For example, an APD (Avalanche Photo Diode), a PMT (Photomultiplier Tube), or an MPPC (Multi-Pixel Photon Counter) may be used as the light receiving element 202. The aperture 203 is realized by, for example, a confocal pinhole, and is arranged before the light receiving element 202 on the optical path of the light receiving unit 200.

[0030] 1, wavelength cut filter 204 is arranged on the optical path of light incident on light receiving unit 200, and wavelength cut filter 205 is arranged outside the optical path. With this configuration, light receiving unit 200 is configured so that the wavelength cut filter arranged on the optical path of the incident light can be selectively switched to either wavelength cut filter 204 or 205 by a drive unit (not shown). Wavelength cut filters 204 and 205 each have the property of blocking light of different wavelengths. Note that wavelength cut filters 204 and 205 can be realized, for example, by a dichroic filter, a notch filter, or the like.

[0031] Light incident on the light receiving unit 200 passes through the lens 201, the diaphragm 203, and the wavelength cut filter 204 or the wavelength cut filter 205, and is incident on the light receiving element 202. The light receiving element 202 converts the received (detected) light into a light intensity signal, and sends it to the control unit 50, which will be described later. Of the light incident on the light receiving unit 200, light having a wavelength corresponding to the cutoff characteristics of the wavelength cut filters 204 and 205 arranged on the optical path is blocked by the wavelength cut filters 204 and 205 before it is incident on the light receiving element 202.

[0032] Anterior-segment illumination light sources 303a and 303b are disposed around the objective lens 9. An image of the anterior segment of the subject's eye E illuminated by the anterior-segment illumination light sources 303a and 303b is guided to the wavelength branching mirror 8 via the objective lens 9 and passes through the wavelength branching mirror 8. An anterior-eye imaging unit 300 is disposed on the optical path of the light that has passed through the wavelength branching mirror 8.

[0033] The anterior eye imaging section 300 is provided with a lens 301 and an imaging element 302. The lens 301 and the imaging element 302 are arranged in this order on the optical path of light incident on the anterior eye imaging section 300. The light incident on the anterior eye imaging section 300 (an image of the anterior segment of the subject's eye E transmitted through the wavelength branching mirror 8) is imaged on the imaging surface (for example, a two-dimensional imaging surface) of the imaging element 302 via the lens 301. The imaging element 302 sends a video signal according to the imaging result to the control section 50, which will be described later.

[0034] (Functional configuration) An example of the functional configuration of a control unit that controls the operation of the SLO device shown in Fig. 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the functional configuration of a control unit according to this embodiment.

[0035] The control unit 50, which controls the operation of the SLO device according to this embodiment, is provided with a wavelength switching unit 51, an optical scanning control unit 52, an image generation unit 53, a synthesis processing unit 54, a display control unit 55, a storage unit 56, and an alignment unit 60. The control unit 50 controls these components to control the operation of the SLO device. The control unit 50 is also connected to the optical output unit 100, the focus lens 4, the optical scanning unit 5, the light receiving unit 200, the anterior eye imaging unit 300, the anterior eye illumination light sources 303a and 303b, and the display unit 400. The control unit 50 also controls the operation of the OCT device, as will be described in detail below.

[0036] An external fixation lamp 500, various driving units, a storage device, and the like may also be connected to the control unit 50. In this embodiment, an example will be described in which light of any wavelength emitted from the laser light source 101 is used as an internal fixation lamp. The control unit 50 may be connected to the display unit 400 and the storage device via any network such as the Internet or an intranet.

[0037] The wavelength switching unit 51 controls the operation of the light output unit 100, thereby selectively switching the wavelength of the light (imaging light) emitted from the light output unit 100. Therefore, the wavelength switching unit 51 can function as an example of a selection unit that selects the wavelength of light to be irradiated onto the subject's eye E from the multiple wavelengths output from the light output unit 100.

[0038] The optical scanning control unit 52 controls the operation of the optical scanning unit 5 to realize scanning of the fundus Ef of the subject's eye E with the light emitted from the light output unit 100. The optical scanning control unit 52 can control the optical scanning unit 5, for example, to scan the subject's eye E in a line with the light of each wavelength output from the light output unit 100.

[0039] The image generating unit 53 generates image data of an imaging frame of the fundus oculi Ef based on the result of receiving, by the light receiving element 202 of the light receiving unit 200, return light (for example, light in each of the red wavelength region, the green wavelength region, and the blue wavelength region) from the fundus oculi Ef of the subject's eye E. Furthermore, the image generating unit 53 can generate an image of the anterior segment (anterior segment image) based on the result of capturing an image of the anterior segment of the subject's eye E by the imaging element 302 of the anterior segment imaging unit 300.

[0040] The synthesis processing unit 54 generates a synthetic image by synthesizing the image data generated by the image generation unit 53. As a specific example, the synthesis processing unit 54 synthesizes image data of a red SLO image, a green SLO image, and a blue SLO image generated by the image generation unit 53 based on the detection results of light in the red wavelength region, light in the green wavelength region, and light in the blue wavelength region, respectively. In this way, the synthesis processing unit 54 can generate a three-color pseudo-color SLO image.

[0041] The display control unit 55 presents the images generated by the image generation unit 53 and the synthesis processing unit 54 to the user by displaying the images in a display area of ​​the display unit 400. The display unit 400 is an output device that presents display information such as images to the user by displaying the display information in a display area such as a screen, and may be configured using, for example, any display. The display control unit 55 is an example of display control means.

[0042] The storage unit 56 can store various generated images, various types of information, etc. It can also store an operating system (OS), device drivers for peripheral devices, and programs for implementing various application software including programs for performing the processes described below.

[0043] The alignment unit 60 aligns the red SLO image, green SLO image, and blue SLO image obtained by the SLO device with the En-Face image obtained by the OCT device. The alignment unit 60 is an example of alignment means.

[0044] The control unit 50 may be configured using, for example, a general-purpose computer. Alternatively, the control unit 50 may be configured using a computer dedicated to the SLO device. The control unit 50 includes a central processing unit (CPU) and a micro processing unit (MPU), not shown, and storage media including memories such as optical disks and read-only memories (ROMs). Each component of the control unit 50, other than the storage unit 56, may be configured using software modules executed by a processor such as a CPU or MPU. Furthermore, the functions of the control unit 50 that control each component other than the storage unit 56 may be configured using software modules executed by a processor such as a processor U or MPU. Furthermore, each of these components may be configured using a circuit that performs a specific function, such as an ASIC, or an independent device. The storage unit 56 may be configured using any storage medium, such as an optical disk or memory.

[0045] The control unit 50 may include one or more processors such as a CPU and storage media such as a ROM. Therefore, each component of the control unit 50 may be configured to function when at least one processor and at least one storage medium are connected and the at least one processor executes a program stored in at least one storage medium. The processor is not limited to a CPU or an MPU, but may also be a GPU (Graphics Processing Unit) or the like.

[0046] The processing of the wavelength switching unit 51, the optical scanning control unit 52, the image generating unit 53, the synthesis processing unit 54, and the display control unit 55 will be described in detail below in conjunction with the processing shown in FIGS.

[0047] (process) Next, an example of a series of processes related to imaging of the fundus oculi Ef of the subject's eye E using the SLO device shown in Fig. 1 will be described with reference to Fig. 3 and Fig. 4. Here, Fig. 3(a) is a flowchart showing an example of a series of processes according to this embodiment. First, the overall flow of the process will be described with reference to the flowchart shown in Fig. 3(a).

[0048] In step S0, the control unit 50 transitions to the alignment mode. Here, the alignment mode refers to a mode in which the user adjusts the device to the subject's eye while checking the anterior eye image and the fundus observation image. For example, the transition to the alignment mode is triggered by inputting new patient information into the input area 421 described below.

[0049] The control unit 50 (not shown) controls a drive unit (not shown) to insert a wavelength cut filter 204 into the optical path of the light receiving unit 200. The wavelength cut filter 204 has the property of blocking light of a wavelength similar to the light in the green wavelength region emitted from the laser emission unit 101G among the series of laser light sources 101 provided in the light output unit 100, and transmitting light of a wavelength similar to the light emitted from the other laser light sources 101.

[0050] In step S1, the control unit 50 acquires the imaging result by the anterior eye imaging unit 300. The image generation unit 53 generates an anterior eye image of the subject's eye E based on the acquired imaging result. The display control unit 55 causes the display unit 400 to display the generated anterior eye image.

[0051] 4 shows an example of a screen for displaying an image corresponding to an imaging result of the subject's eye E using the SLO device according to this embodiment. The screen shown in FIG. 4 presents display areas 401 and 402, a focus adjustment button 404, a mode selection button 405, an imaging button 406, a gain adjustment button 410, a display area 420, and an input area 421.

[0052] The display area 401 is an area where an anterior eye image is displayed according to the imaging result by the anterior eye imaging unit 300. The display area 402 is an area where a fundus image is displayed according to the light receiving unit 200 receiving the return light from the fundus Ef of the subject's eye E.

[0053] The focus adjustment button 404 is a button that receives instructions related to focus adjustment from the user. The mode selection button 405 is a button that receives instructions related to the imaging mode from the user. The imaging button 406 is a button that receives instructions related to imaging of the fundus oculi Ef from the user. The gain adjustment button 410 is a button that receives instructions related to the brightness of the image displayed in the display area 402 from the user.

[0054] The display area 420 is an area where information such as patient information and acquired imaging data is selectably displayed. The display control unit 55 can display a fundus image in the display area 402 based on imaging data selected in response to an instruction from a user. The input area 421 is an area where patient information linked to the imaging data is input.

[0055] 4 may be configured to be able to accept a user's designation of the display position of the fixation light. For example, in the example shown in Fig. 4, a marker 403 indicating the display position of the fixation light is displayed, and the control unit 50 can adjust the position of the marker 403 in response to an instruction from the user and control the position on the fundus Ef where the fixation light is displayed.

[0056] 3(a) again, in step S1, the display control unit 55 can display, for example, an anterior eye image corresponding to the imaging result by the anterior eye imaging unit 300 in a display area 401 on the screen displayed on the display unit 400.

[0057] Specifically, when the subject's eye E is placed within the imaging range of the fundus imaging device, the anterior segment of the subject's eye E is illuminated with light emitted from the anterior-segment illumination light sources 303a and 303b. An image of the anterior segment illuminated in this manner is guided to the wavelength branching mirror 8 via the objective lens 9, and after passing through the wavelength branching mirror 8, is formed on the imaging surface of the imaging element 302 via the lens 301. The imaging element 302 outputs a video signal according to the imaging result.

[0058] The video signal output from the imaging element 302 is converted into a digital signal by an A / D converter (not shown) and then input to the control unit 50. The image generation unit 53 generates an anterior eye image based on the digital data. The display control unit 55 displays the generated anterior eye image in a display area 401 on a screen displayed on the display unit 400. The A / D converter (not shown) may convert the video signal into digital data in real time. The A / D converter may also be provided in the control unit 50. In this case, the control unit 50 can acquire the video signal from the imaging element 302 and convert the acquired video signal into a digital signal.

[0059] The optical system of the SLO device according to this embodiment can be placed on, for example, a stage (not shown). In this case, the control unit 50 can move the optical system placed on the stage up, down, left, right, forward, and backward by controlling a stage drive unit (not shown). In addition, the optical axis of the optical system can be adjusted to approximately coincide with the center of the imaging surface of the image sensor 302. By performing such adjustment, the amount of eccentricity between the pupil center of the anterior eye image corresponding to the imaging result by the anterior eye imaging unit 300 and the imaging center of the anterior eye imaging unit 300 corresponds to the amount of eccentricity between the subject's eye E and the optical system.

[0060] In step S2, the control unit 50 adjusts the position so that the center of the pupil of the subject's eye E and the optical axis of the optical system are substantially aligned. Specifically, the control unit 50 determines the state of decentering and focus between the subject's eye E and the optical system by using a portion of the anterior segment image corresponding to the imaging result of the subject's eye E, particularly a portion corresponding to the iris pattern.

[0061] Based on the result of the determination, the control unit 50 controls the stage driver to adjust the position between the subject's eye E and the optical system so that the center of the pupil and the optical axis of the optical system are approximately aligned. Based on the result of the determination, the control unit 50 also controls the stage driver to adjust the position between the subject's eye E and the optical system in the optical axis direction so that the contrast of the iris pattern is increased. Through this control, the control unit 50 can maintain a constant distance (working distance) between the pupil of the subject's eye E, which is flush with the iris, and the objective lens 9 of the optical system. Displaying an anterior segment image in the display area 401 allows the user to check the decentering of the optical axis based on the anterior segment image. The user may also operate the stage to adjust the position between the subject's eye E and the optical system.

[0062] In step S3, the control unit 50 generates and displays a fundus image (fundus observation image) for alignment of the subject's eye E. Specifically, when the amount of eccentricity between the pupil center and the imaging center becomes equal to or less than a predetermined value as a result of the position adjustment in step S2, the wavelength switching unit 51 turns on the laser light source 101IR.

[0063] The infrared laser light (observation light) emitted from the laser light source 101IR is converted into parallel light by the collimator lens 103IR, reflected by the reflecting mirror 2, passes through the through-hole of the perforated mirror 3, and then transmits through the focus lens 4. The observation light is then guided to the wavelength branching mirror 8 via the optical scanning unit 5, lens 6, and dichroic mirror 7, and is reflected by the wavelength branching mirror 8 before being guided to the fundus Ef of the subject's eye E via the objective lens 9. At this time, the observation light is two-dimensionally scanned at the fundus Ef by the operation of the X-scan mirror and Y-scan mirror of the optical scanning unit 5.

[0064] The observation light is reflected and scattered by the layers constituting the retina of the fundus oculi Ef, and is guided as return light to the perforated mirror 3 via the objective lens 9, wavelength branching mirror 8, dichroic mirror 7, lens 6, optical scanning unit 5, and focus lens 4. The observation light is reflected by the peripheral portion of the perforated mirror 3, and is guided to the wavelength cut filter 204 via the lens 201 and diaphragm 203, and is transmitted through the wavelength cut filter 204 to be guided to the light receiving element 202.

[0065] The light intensity signal output from the light receiving element 202 in response to the result of receiving the observation light by the light receiving element 202 is converted into a digital signal by an A / D converter (not shown) and then input to the control unit 50. The A / D converter (not shown) may perform a process of converting the light intensity signal output from the light receiving element 202 into a digital signal in real time. The A / D converter may also be provided in the control unit 50. In this case, the control unit 50 can acquire the light intensity signal from the light receiving element 202 and convert the acquired light intensity signal into a digital signal.

[0066] The image generation unit 53 generates a fundus observation image based on the digital signal acquired by the control unit 50. The display control unit 55 displays the generated fundus observation image in a display area 402 on the screen displayed on the display unit 400. This allows the user to check the fundus observation image displayed in the display area 402 and adjust the focus by operating the focus adjustment button 404 so that the fundus observation image is displayed brighter.

[0067] In step S4, the control unit 50 adjusts the position where the fixation lamp is displayed. Specifically, after executing the process related to the generation and display of the fundus observation image in step S3, the control unit 50 turns on the laser emitter 101G and starts displaying the fixation lamp. The light in the green wavelength region emitted from the laser emitter 101G is converted into parallel light by the collimator lens 103RGB and guided to the fundus Ef of the subject's eye E via the same optical path as the observation light described above.

[0068] The display control unit 55 displays a marker 403 indicating the display position of the fixation light on the fundus oculi Ef in the display area 402. The user can specify the display position of the fixation light by adjusting the position of the marker 403 through a predetermined operation. The control unit 50 controls the emission timing of the laser emitter 101G in accordance with the user's specification of the display position of the fixation light, and displays the fixation light at the position specified by the user.

[0069] At this time, light having a wavelength similar to that of light in the green wavelength region emitted from the laser emission unit 101G is blocked by a wavelength cut filter 204 arranged in front of the light receiving element 202. This makes it possible to prevent the light from the fixation lamp (fixation lamp light) from appearing as noise in the fundus observation image. In addition, the wavelength cut filter 204 is arranged so as to be interposed between the diaphragm 203 and the light receiving element 202. This makes it possible to prevent the amount of light passing through the diaphragm 203 from being affected by a position error in the wavelength cut filter 204.

[0070] In step S6, the control unit 50 executes processing related to color SLO photography of a fundus image in response to the user's operation of selecting color SLO photography mode using the mode selection button 405. The processing flow related to color SLO photography of a fundus image will be described in detail later. The color SLO image generated in response to the imaging result is stored in the storage unit 56, such as a memory.

[0071] In step S7, the display control unit 55 executes processing related to the color SLO image generated based on the processing in step S5. As a specific example, the display control unit 55 may display a color SLO image of the fundus oculi Ef of the subject's eye E in the display area 402 on the screen of the display unit 400. The user may also use the gain adjustment button 410 to adjust the brightness of the image displayed in the display area 402. Furthermore, the system may be configured so that the user can adjust the color tone, tone curve, etc. of the color SLO image using an image adjustment button (not shown). This configuration allows the user to generate a color SLO image of desired image quality.

[0072] Note that, with regard to the processes of steps S1 to S4, two or more processes may be executed in parallel. For example, a process related to generating an anterior eye image and adjusting its position and a process related to generating a fundus observation image and adjusting the position of a fixation lamp may be executed in parallel. In this case, the laser light source 101IR that emits observation light (infrared light) and the laser emitter 101G that is used to display the fixation lamp may be turned on, and the laser emitters 101R and 101B may be turned off.

[0073] Furthermore, the light source used to display the fixation lamp is not limited to the laser emitter 101G. For example, the laser emitter 101R or the laser emitter 101B may be used to display the fixation lamp. In this case, among the laser emitters 101R, 101G, and 101B, the laser light source 101 used to display the fixation lamp may be turned on, and the other laser light sources 101 may be turned off. In this case, the wavelength cut filter 204 may be one that has the property of blocking light of the same wavelength as the light emitted by the laser light source 101 used to display the fixation lamp.

[0074] (Fundus imaging sequence) Next, details of the processing related to color SLO photography of the fundus oculi Ef shown as the processing in step S5 in FIG. 3(a) will be described with reference to FIG.

[0075] In step S500, the wavelength switching unit 51 turns off the laser light source 101IR to stop emission of the observation light from the laser light source 101IR. In addition, the optical scanning control unit 52 controls the optical scanning unit 5 to move the position (scanning position) where the light emitted from the light output unit 100 is irradiated when scanning the fundus Ef with the light to the start position.

[0076] The timing for stopping the emission of observation light and the timing for moving the scanning position to the start position may each be arbitrary. As a specific example, the process for stopping the emission of observation light and the process for moving the scanning position to the start position may be executed immediately after it is detected that the image capture button 406 has been pressed. As another example, the process for stopping the emission of observation light and the process for moving the scanning position to the start position may be executed after scanning for one frame using the observation light after the image capture button 406 has been pressed.

[0077] In step S501, when the control unit 50 detects that the optical scanning unit 5 has moved the scanning position to the start position, the wavelength switching unit 51 turns on the laser emission unit 101R. Also, if the laser emission unit 101B is on, the wavelength switching unit 51 turns off the laser emission unit 101B.

[0078] In step S502, the optical scanning control unit 52 controls the optical scanning unit 5 to perform scanning for one frame with light in the red wavelength region (red laser light) emitted from the laser emission unit 101R. As a result, the red laser light is irradiated onto the fundus Ef of the subject's eye E for one frame, and the returned light of the red laser light reflected by the fundus Ef is guided to the light receiving unit 200. At this time, the wavelength switching unit 51 controls the turn-on timing of the laser emission unit 101G so that a fixation lamp is displayed according to a position designated by the user. Specifically, while the red laser light is continuously irradiated for a period equivalent to one frame, the irradiation period of light in the green wavelength region (green laser light) emitted from the laser emission unit 101G is controlled to be shorter than the period equivalent to one frame. In other words, the green laser light is controlled to be turned on and off in a cycle shorter than that of the red laser light.

[0079] The return light of the green laser light reflected by the fundus Ef of the subject's eye E is blocked by the wavelength cut filter 204. Therefore, of the return light of the red laser light and the green laser light reflected by the fundus Ef of the subject's eye E, only the return light corresponding to the red laser light is received by the light receiving element 202. This makes it possible to prevent the fixation lamp light from appearing as noise in an image captured by the red laser light. Furthermore, since light with a different wavelength from the light to be captured is used to display the fixation lamp, it is expected that the visibility of the fixation lamp will be improved.

[0080] The light intensity signal output from the light receiving element 202 in response to the reception of the red laser light by the light receiving element 202 is converted into a digital signal and then input to the control unit 50. As a result, the control unit 50 acquires one frame's worth of digital signals in response to the results of scanning using the red laser light. The image generation unit 53 generates one frame's worth of image data of the red laser light based on the acquired digital signals, and the control unit 50 stores the image data in the storage unit 56. Note that the storage unit 56 can also store the acquired digital signals. When scanning for one frame using the red laser light is completed, the wavelength switching unit 51 turns off the laser emission unit 101R.

[0081] In step S503, the optical scanning control unit 52 controls the optical scanning unit 5 to move the scanning position to the start position for scanning on the fundus Ef with the light emitted from the light output unit 100. When the control unit 50 detects that the optical scanning unit 5 has moved the scanning position to the start position, the wavelength switching unit 51 turns on the laser emission unit 101B.

[0082] In step S504, the optical scanning control unit 52 controls the optical scanning unit 5 to perform scanning for one frame with light in the blue wavelength region (blue laser light) emitted from the laser emission unit 101B. As a result, the blue laser light is irradiated onto the fundus Ef of the subject's eye E for one frame, and the returned light of the blue laser light reflected by the fundus Ef is guided to the light receiving unit 200. At this time, the wavelength switching unit 51 controls the turn-on timing of the laser emission unit 101G so that a fixation lamp is displayed according to a position designated by the user. Specifically, while the blue laser light is continuously irradiated for a period equivalent to one frame, the irradiation period of the green laser light is controlled to be shorter than the period equivalent to one frame. In other words, the green laser light is controlled to be turned on and off in a cycle shorter than that of the blue laser light.

[0083] Return light of the green laser light reflected by the fundus Ef of the subject's eye E is blocked by the wavelength cut filter 204. Therefore, of the return light of the blue laser light and the green laser light reflected by the fundus Ef of the subject's eye E, only the return light corresponding to the blue laser light is received by the light receiving element 202. This makes it possible to prevent the fixation lamp light from appearing as noise in an image captured by the blue laser light. Furthermore, since light with a different wavelength from the light to be captured is used to display the fixation lamp, it is expected that the visibility of the fixation lamp will be improved.

[0084] The light intensity signal output from light receiving element 202 in response to the reception of the blue laser light by light receiving element 202 is converted into a digital signal and then input to control unit 50. As a result, control unit 50 acquires one frame's worth of digital signals in response to the results of scanning using the blue laser light. Image generation unit 53 generates one frame's worth of image data of the blue laser light based on the acquired digital signals, and control unit 50 stores the image data in storage unit 56. Note that storage unit 56 can also store the acquired digital signals. When scanning of one frame's worth of blue laser light is completed, wavelength switching unit 51 turns off laser emission unit 101B.

[0085] In step S505, the control unit 50 removes the wavelength cut filter 204 from the optical path of the light receiving unit 200, and inserts the wavelength cut filter 205, which has the property of blocking light of the same wavelength as the red laser light, into the optical path. At this time, the wavelength cut filter 205 is placed on the optical path between the diaphragm 203 and the light receiving element 202. This makes it possible to prevent a situation in which a positioning error in the wavelength cut filter 205 affects the amount of light passing through the diaphragm 203.

[0086] The wavelength cut filter switching mechanism may be, for example, a turret in which multiple filters are fitted, and the filters can be switched by rotating the turret. Alternatively, the wavelength cut filter may be switched by, for example, switching the optical path.

[0087] In step S506, the optical scanning control unit 52 controls the optical scanning unit 5 to move the scanning position to the start position for scanning on the fundus Ef with the light emitted from the light output unit 100. When the control unit 50 detects that the optical scanning unit 5 has moved the scanning position to the start position, the wavelength switching unit 51 turns on the laser emission unit 101G.

[0088] In step S507, the optical scanning control unit 52 controls the optical scanning unit 5 to perform scanning for one frame with the green laser light emitted from the laser emission unit 101G. As a result, the green laser light is irradiated onto the fundus Ef of the subject's eye E for one frame, and the returned light of the green laser light reflected by the fundus Ef is guided to the light receiving unit 200. At this time, the wavelength switching unit 51 controls the turn-on timing of the laser emission unit 101R so that the fixation lamp is displayed according to the position specified by the user. Specifically, while the green laser light is continuously irradiated for a period equivalent to one frame, the irradiation period of the red laser light is controlled to be shorter than the period equivalent to one frame. In other words, the red laser light is controlled to be turned on and off in a shorter cycle than the green laser light.

[0089] The return light of the red laser light reflected by the fundus Ef of the subject's eye E is blocked by the wavelength cut filter 205. Therefore, of the return light of the green laser light and the red laser light reflected by the fundus Ef of the subject's eye E, only the return light corresponding to the green laser light is received by the light receiving element 202. This makes it possible to prevent the fixation lamp light from appearing as noise in an image captured by the green laser light. Furthermore, since light with a different wavelength from the light to be captured is used to display the fixation lamp, it is expected that the visibility of the fixation lamp will be improved.

[0090] The light intensity signal output from light receiving element 202 in response to the reception of the green laser light by light receiving element 202 is converted into a digital signal and then input to control unit 50. As a result, control unit 50 acquires one frame's worth of digital signals in response to the results of scanning using the green laser light. Image generation unit 53 generates one frame's worth of image data of the green laser light based on the acquired digital signals, and control unit 50 stores the image data in storage unit 56. Note that storage unit 56 can also store the acquired digital signals. When scanning for one frame using the green laser light is completed, wavelength switching unit 51 turns off laser emission unit 101G.

[0091] Furthermore, in step S508, the control unit 50 removes the wavelength cut filter 205 from the optical path of the light receiving unit 200, and inserts the wavelength cut filter 204, which has the property of blocking light of the same wavelength as the green laser light, into the optical path. By executing the processes of steps S501 to S508 described above, the control unit 50 can acquire image data generated using the digital signals of the light intensity signals based on the red laser light, blue laser light, and green laser light for one frame at a common scanning position.

[0092] In step S509, the composition processor 54 combines the red SLO image, the green SLO image, and the blue SLO image to generate a three-color pseudo-color SLO image. The composition processor 54 corrects the gradation of the SLO images for each of the three color frames based on a pre-created look-up table (LUT), and then adds the SLO images for each of the three color frames to generate a color SLO image. The composition processor 54 may also use the signal-to-noise ratio or the like as a judgment criterion for each image, and perform noise removal processing such as applying a Gaussian filter if the signal-to-noise ratio is low.

[0093] Furthermore, when combining the images of each of the three color frames, the composition processor 54 may apply aberration correction to each frame image based on a table. This method allows for the generation of a good color SLO image. Furthermore, the composition processor 54 may be configured to automatically determine whether or not to apply aberration correction depending on factors such as the shooting speed, angle of view, and signal-to-noise ratio, or the user may be able to select whether or not to apply aberration correction.

[0094] Furthermore, the synthesis processing unit 54 may apply various corrections such as positional deviation correction when synthesizing the SLO images of the three color frames.

[0095] In step S510, the control unit 50 determines whether a predetermined number of frames have been acquired. Here, the predetermined number of frames may be the number of color SLO images specified by the user, a number of frames set in advance depending on a disease, or the like.

[0096] If the control unit 50 determines in step S510 that the predetermined number of frames of captured images have been acquired, it ends the shooting. On the other hand, if the control unit 50 determines in step S510 that the acquisition of captured images has not been completed, it returns the process to step S501.

[0097] As described above, the SLO device according to this embodiment includes the light output unit 100, the wavelength switching unit 51, the image generation unit 53, the control unit 50, and the synthesis processing unit 54. The light output unit 100 functions as an example of a light source that emits light of multiple wavelengths. The wavelength switching unit 51 functions as an example of a selection unit that selects the wavelength of light to be irradiated onto the subject's eye E from multiple wavelengths. The image generation unit 53 functions as an example of a first generation unit that generates image data based on the detection result of returned light obtained by irradiating the subject's eye E with light of the selected wavelength. The control unit 50 functions as an example of a control unit that controls the selection unit and the first generation unit to sequentially generate image data using light of each of the multiple wavelengths. The synthesis processing unit 54 functions as an example of a second generation unit that synthesizes image data generated using light of each wavelength to generate color SLO image data. Each time image data is generated by the image generation unit 53, the synthesis processing unit 54 updates the image data of the light of the wavelength corresponding to the generated image data and performs a synthesis process. The image generation unit 53 generates image data frame by frame.

[0098] The light output unit 100 according to this embodiment can emit light of at least red wavelengths, green wavelengths, and blue wavelengths, and therefore the SLO device can generate a color SLO image based on light of at least the three primary colors of RGB.

[0099] The SLO device according to this embodiment further includes a plurality of filters that block light of different wavelengths. The control unit 50 controls the light source to emit light of a wavelength different from the wavelength of light used to generate image data, as light used to prompt the subject's eye E to fixate. The control unit 50 also controls the driving of the plurality of filters to insert the filter that blocks the light used to prompt the subject's eye E to fixate into the optical path of the returned light. This allows the light source emitting light of a plurality of wavelengths to be used as an internal fixation light, which prompts the subject's eye E to fixate and allows a better image to be acquired.

[0100] Next, a method for acquiring an En-Face image using an OCT device will be described with reference to FIG.

[0101] The OCT device is composed of an optical coherence tomography acquisition unit 1000 that acquires optical coherence tomography signals, a common optical path with the SLO device, and a control unit 50. The control unit 50 is further composed of a signal processing unit 57, a signal acquisition control unit 58, a display control unit 55, an image generation unit 53, a map generation unit 59, and a display unit 400.

[0102] (Optical coherence tomography acquisition unit 1000) The light source 1101 is a swept source (hereinafter referred to as SS) light source, and emits light with a swept wavelength ranging from 1000 nm to 1100 nm, centered around 1050 nm, for example.

[0103] The light passing through the fiber 1102 is guided to the light branching element 1110 and branched into measurement light and reference light. The branching ratio at this time was set to 30 (measurement light):70 (reference light). The branched measurement light passes through the fiber 1111, exits from the fiber end 1111a, and is collimated by the collimator 1112. The collimated measurement light passes through a galvanometer scanner 1114 that scans the measurement light at the fundus Er of the subject's eye E, a scan lens 1115, and a focus lens 1116, and then passes through a dichroic mirror 7, a wavelength branching mirror 8, and an objective lens 9, which are common optical paths with the SLO device, before entering the subject's eye E. Here, although the galvanometer scanner 1114 is illustrated as a single mirror, it is actually composed of two galvanometer scanners (X-axis scanner 1114X and Y-axis scanner 1114Y) (not shown) that raster scan the fundus Er of the subject's eye E. The focus lens 1116 is fixed on a stage 1117, and can be moved in the optical axis direction to perform focus adjustment. The galvanometer scanner 1114 and the stage 1117 are controlled by a signal acquisition control unit 58, and can scan the measurement light over a desired range of the fundus Er of the subject's eye E (also referred to as the tomographic image acquisition range, the tomographic image acquisition position, and the measurement light irradiation position).

[0104] Although not described in detail in this embodiment, it is desirable to provide a tracking function that detects the movement of the fundus Er and scans the mirror of the galvano scanner 1114 while tracking the movement of the fundus Er. The tracking method can be performed using a general technique.

[0105] The measurement light is incident on the subject's eye E and focused on the fundus Er by a focus lens 1116 mounted on a stage 1117. The measurement light that has irradiated the fundus Er is reflected and scattered by each retinal layer, passes through the optical path described above, and returns to the optical branching element 11110. The return light of the measurement light that has entered the optical branching element 1110 passes through a fiber 1126 and enters the optical branching element 1128.

[0106] Meanwhile, the reference light branched by the optical branching element 11110 passes through a fiber 1119a, a polarization controller 150, and a fiber 1119b, exits at an end 1119c of the fiber, and is collimated by a collimator 1120. The polarization controller 150 can change the polarization of the reference light to a desired polarization state. The reference light passes through a dispersion compensation glass 1122, an ND filter 1123, and a collimator 1124, and is coupled to a fiber end 1127A of a fiber 1127. The collimator lens 1124 and one end of the fiber 1127 are fixed on a coherence gate stage 1125, and are controlled by a signal acquisition control unit 58 to be driven in the optical axis direction in response to differences in the axial length of the subject's eye, etc. Note that, although the optical path length of the reference light is changed in this embodiment, it is sufficient if the difference in optical path length between the optical path of the measurement light and the optical path of the reference light can be changed. Furthermore, although an example has been described in which the fiber 1119 and the fiber 1127 face each other in a straight line, the optical path may be bent by disposing a mirror in the optical path.

[0107] The reference light that passes through fiber 1127 enters optical branching element 1128. In optical branching element 1128, the return light of the measurement light and the reference light are multiplexed and branched. At this time, the branched lights have phases that are inverted relative to each other. The split return light of the measurement light and the reference light pass through fiber 1129 and enter one input port of detector 141. The other split return light of the measurement light and the reference light pass through fiber 1130 and enter the other input port of detector 1141. At this time, the return light of the measurement light that enters different input ports of detector 1141 has phases that are inverted relative to each other. Similarly to the return light of the measurement light, the reference light that enters different input ports of detector 1141 also has phases that are inverted relative to each other. As a result, the interference light observed by detector 141 has phases of interference fringes (signals in which signal intensity varies along the wavelength direction) that are inverted due to the different input ports. Detector 1141 is a differential detector, and when two interference signals with inverted phases are input, it can remove the DC component and output only the interference component.

[0108] The interference signal, which has been detected by the detector 1141 and has been converted into an electrical signal from which the DC component has been removed, is input to a signal processing unit 57, which is an example of a tomographic image generating unit.

[0109] The control unit 50 is composed of a signal processing unit 57, a signal acquisition control unit 58, a display unit 400, and a display control unit 55. The signal processing unit 57 further includes an image generation unit 53 and a map generation unit 59. The image generation unit 53 has a function of generating a luminance image from the electrical signal sent, and the map generation unit 59 has a function of generating layer information (retinal segmentation) from the luminance image.

[0110] The signal acquisition control unit 58 controls each unit as described above. The signal processing unit 57 generates an image based on the signal output from the detector 141, analyzes the generated image, and generates visible information of the analysis results.

[0111] The image and analysis results generated by the signal processing unit 57 are sent to the display control unit 55, which then displays them on the display screen of the display unit 400. Here, the display unit 400 is, for example, a liquid crystal display. Note that the image data generated by the signal processing unit 57 may be sent to the display control unit 55 and then transmitted to the display unit 400 via a wired or wireless connection. Furthermore, while the display unit 400 and the like are included in the control unit 50 in this embodiment, the present disclosure is not limited to this. The display unit 400 may be provided separately from the control unit 50, and for example, a tablet, which is an example of a device that can be carried by a user, may be used. In this case, the display unit may be equipped with a touch panel function, and may be configured to enable scanning of the touch panel to move the display position of the image, zoom in and out, change the displayed image, and the like.

[0112] The above is a description of the process for acquiring information about a tomography at a certain point on the subject's eye E. Acquiring information about a tomography in the depth direction of the subject's eye E in this way is called an A-scan, and a graph of the tomography in the depth direction is called an A-scan profile. In addition, the scanning direction for acquiring information about the subject's tomography, i.e., an image, in a direction perpendicular to the A-scan is called a B-scan, and the two-dimensional image obtained by the B-scan is called a two-dimensional tomography image. The B-scan is performed by the galvano scan 1114 described above.

[0113] The signal processing unit 57 generates two-dimensional motion contrast images from the tomographic images of each part, and can generate three-dimensional motion contrast images by arranging the generated motion contrast images of each part in a single coordinate system. The En-Face image generating means generates En-Face images, which are two-dimensional images projected onto a plane based on two arbitrary reference planes in the depth direction (Z direction), from three-dimensional images such as three-dimensional OCT images and three-dimensional motion contrast images. Note that En-Face images can be generated using various known methods, and detailed explanations thereof will be omitted.

[0114] Next, an example of a series of processes relating to imaging of the fundus oculi Ef of the subject's eye E using the SLO-OCT composite apparatus shown in Fig. 1 will be described with reference to Fig. 3. Fig. 3(b) is a flowchart showing an example of a series of processes according to this embodiment.

[0115] The following describes the operation performed when the user selects the color SLO-OCTA shooting mode using the mode selection button 405. In FIG. 3(b), steps S0 to S4 are the same as those described above, and therefore their explanation will be omitted.

[0116] In step S5, the control unit 50 operates the OCT device to perform OCTA imaging and generate an En-Face image. FIG. 6(c) shows the En-Face image generated at this time. Next, in step S6, color SLO imaging is performed. FIG. 6(a) shows the color SLO image generated at this time. Note that OCTA imaging is an example of a first imaging method. Also, color SLO imaging is an example of a second imaging method. Also, the En-Face image is an example of a first image. Also, the color SLO image is an example of a second image.

[0117] In step S8, these images are displayed on the display unit 400. In this embodiment, an input unit (not shown) is configured to allow selection of whether to display the En-Face image and the color SLO image side by side or in a superimposed manner. The following describes the superimposed display, which is suitable for observing detailed structures.

[0118] To superimpose different types of fundus images, the control unit 50 extracts blood vessels, which are characteristic regions of the fundus, from each image and aligns the extracted blood vessels. However, because color SLO images and En-Face images are acquired at different times using different devices, alignment is difficult. For example, alignment is difficult between images acquired at different times due to misalignment caused by eye movement during scanning. Alignment is also difficult between images acquired using different devices due to misalignment caused by differences in optical system aberrations. Furthermore, the brightness of blood vessels in color SLO images is generally lower than that of tissues surrounding the blood vessels because visible light is largely absorbed by hemoglobin. On the other hand, the brightness of blood vessels in En-Face images is generally higher than that of tissues surrounding the blood vessels because processing is performed to enhance the brightness of areas with motion contrast. Therefore, the brightness of blood vessels in color SLO images and En-Face images differ significantly, making alignment difficult. Furthermore, blood vessels vary in size, from large defects to small vessels (such as retinal arterioles). In conventional technology, it is difficult to perform high-precision alignment so that small-diameter blood vessels in a color SLO image and small-diameter blood vessels in an En-Face image match, and then superimpose the images. Therefore, in this embodiment, the following method is used to achieve a configuration in which alignment is performed with higher precision than in conventional technology, and superimposition display is performed. This will be explained with reference to FIG. 7.

[0119] In step S701, the control unit 50 selects the green SLO image from among the images (red SLO image, green SLO image, and blue SLO image) that constitute the color SLO image. The selected SLO image may hereinafter be referred to as the selected SLO image. Here, the absorption spectrum of hemoglobin (absorption rate relative to the wavelength of light) has a peak in the visible light wavelength range (around 550 nm or 430 nm) for both oxyhemoglobin and deoxyhemoglobin (high absorption rate in the visible light wavelength range). Therefore, the brightness of blood vessels in the green SLO image is lower than that of blood vessels in the red SLO image or the blue SLO image. In other words, the brightness difference between the brightness of blood vessels and the brightness of tissues surrounding the blood vessels is greater in the green SLO image than in the red SLO image or the blue SLO image. The greater the brightness difference, the more accurate the alignment. Therefore, the control unit 50 selects the green SLO image as the selected SLO image. In this embodiment, a green SLO image is selected, but if a 450 nm laser beam is used as the blue laser beam, a blue SLO image may be selected. In other words, an image with a large difference in brightness between the tissue surrounding the blood vessel and the blood vessel itself may be selected as the selected SLO image.

[0120] In step S702, the control unit 50 performs preprocessing on the selected SLO image. In this embodiment, the luminance of the selected SLO image is inverted and adjusted to emphasize blood vessels. In this adjustment, it is desirable to adjust the luminance value of the blood vessels to be approximately the same as the luminance value in the En-Face image. In addition, in the SLO image, differences in average luminance occur between the center and periphery of the image due to the effects of aberration, vignetting, focus, etc. In this adjustment, it is desirable to adjust the difference in average luminance between the center and periphery of the SLO image to be approximately the same as the difference in average luminance between the center and periphery of the En-Face image. Figure 6(b) shows the selected SLO image after these adjustments (preprocessing) have been performed.

[0121] In step S703, the control unit 50 extracts blood vessels from each of the selected SLO image and the En-Face image. As an example of the extraction process, threshold processing may be used to extract pixels that satisfy a predetermined threshold as blood vessels. The predetermined feature extraction process for extracting blood vessels is not limited to the above, and may also use, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2019-209136. Specifically, in addition to threshold processing, at least one of a process for emphasizing linear structures by applying a Hessian filter, which is one type of blood vessel enhancement process, and an edge selection sharpening process by weighting edge portions, which is one type of edge enhancement process, may also be used.

[0122] In step S704, the control unit 50 aligns the images based on the blood vessels extracted from each of the selected SLO image and the En-Face image. The control unit 50 aligns the images by using the En-Face image as a reference and deforming the selected SLO image using an affine transformation based on the correlation value. Note that the alignment in step S704 only requires a rough alignment. Therefore, alignment using a general linear transformation (alignment performed by translation, rotation, enlargement, and reduction) can be used. The En-Face image is acquired by tracking using near-infrared light of 1000 nm, which has low glare, and has less distortion, so it can be used as a reference image.

[0123] In step S705, the control unit 50 extracts the positions (positions of the feature points) of the characteristic regions (blood vessels). Note that a plurality of characteristic regions (blood vessels) are extracted.

[0124] In step S706, the control unit 50 performs registration (non-rigid registration) using non-linear transformation using the extracted multiple feature regions. Non-rigid registration is performed by deforming the selected SLO image so that the positions of feature regions (blood vessels) included in the En-Face image approach the positions of feature regions (blood vessels) included in the selected SLO image. Here, the feature regions (blood vessels) may be, for example, regions where blood vessels intersect or branch. Examples of regions where blood vessels branch in the SLO image after preprocessing are shown as regions 601 and 602 in FIG. 6(b). Examples of regions where blood vessels branch in the En-Face image are shown as regions 603 and 604 in FIG. 6(c). In step S706, the control unit 50 performs registration (non-rigid registration) using non-linear transformation so that region 601 approaches region 603 and region 602 approaches region 604. Note that a region where blood vessels intersect or branch off is an example of a first region or a second region.

[0125] In this embodiment, the En-Face image is used as the reference image and the selected SLO image is deformed, but this is not limited to this. An image with relatively little distortion may also be used as the reference image.

[0126] As described above, by performing alignment by deforming the captured image through nonlinear transformation, the accuracy of alignment is improved compared to when alignment is performed without deforming the image (when alignment is performed using linear transformation only). As a result, for example, the misalignment between the positions of blood vessels in a color fundus image and the positions of blood vessels in an En-Face image due to the difference in aberration between the OCT optical system and the SLO optical system is reduced, making it easier to observe the structure of the fundus.

[0127] In step S707, the control unit 50 also applies the transformation applied to the selected SLO image to the color SLO image. That is, the transformation applied to the green SLO image is applied to the red SLO image and the blue SLO image. As a result, each of the red SLO image, green SLO image, and blue SLO image that make up the color SLO image is aligned with the En-Face image, and the En-Face image and the color SLO image are aligned with high accuracy.

[0128] A method for superimposing and displaying the aligned color SLO image and En-Face image on the display screen of the display unit 400 will be described with reference to FIG.

[0129] FIG. 8 shows a state in which a color SLO image and an En-Face image are blended at a specified transmittance and displayed as a blended image superimposed on the display unit 400. The user specifies the transmittance to the control unit 50 using a slider 801 that adjusts the transmittance. The control unit 50 changes the blending process according to the specified transmittance. The blended image 810 shown in FIG. 8(a) has a transmittance of 0%, and is in a state in which the En-Face image is displayed. On the other hand, the blended image 811 shown in FIG. 8(b) has a transmittance of 80%, and is in a state in which the color SLO image and the En-Face image are superimposed and displayed. Note that the blended image 811 is an example of a superimposed image.

[0130] Note that the superimposed display is not limited to a color SLO image and an En-Face image. For example, a red SLO image and an En-Face image may be superimposed. Furthermore, the images to be superimposed may be selectable. Compared to blue or green laser light, red laser light is reflected by a deeper part of the fundus (the choroid). Therefore, by superimposing a red SLO image and an En-Face image, it is possible to visualize, for example, information about the choroid. This allows for more detailed observation of the fundus structure.

[0131] Furthermore, the depth of the retina when generating an En-Face image from an OCT image may be configured to be changeable according to the light source of the color SLO to be superimposed. For example, it is desirable to be able to superimpose an En-Face image of the superficial retina on a blue SLO image or a green SLO image, or to superimpose an En-Face image of a deep retina or including the choroid on a red SLO image.

[0132] Note that the angle of view of both the color SLO image and the En-Face image may be corrected based on the axial length. Specifically, for example, the angle of view of the En-Face image may be adjusted to fall within a predetermined range using the value of the axial length based on the Littmann formula.

[0133] Furthermore, when blue laser light is selected, the weighting of feature points around the optic disc, where an increase in luminance occurs due to the optic nerve fiber layer, may be reduced, thereby improving the accuracy of the superimposed display.

[0134] In this embodiment, alignment is first performed using linear transformation, and then nonlinear transformation is performed, but the procedure is not limited to this. For example, if the SLO device and OCT device have a common optical system, the imaging position is highly reproducible by guidance using a fixation light, and the aberrations and distortions of the SLO image and the OCT image are approximately the same, it is sufficient to apply only linear transformation.

[0135] <Other embodiments> The disclosed technology can also be realized by executing the following process. That is, the disclosed technology can also be realized by supplying software (programs) that realize one or more functions of the various embodiments described above to a system or device via a network or a storage medium, and having a computer (or a CPU, MPU, or the like) of the system or device read and execute the programs. The computer has one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions. In this case, the processor or circuit may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or a field-programmable gateway (FPGA). The processor or circuit may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0136] (Configuration 1) an image acquisition means for acquiring a first image obtained by photographing the subject's eye using a first photographing method and a second image obtained by photographing the subject's eye using a second photographing method different from the first photographing method; a positioning means for deforming the second image so that a position of a first region included in the first image and a position of a second region included in the second image, the second region corresponding to the first region, become closer to each other; a display control means for controlling a display of a superimposed image in which the first image and the transformed second image are superimposed on a display unit; An ophthalmic device comprising:

[0137] (Configuration 2) 2. The ophthalmologic apparatus according to configuration 1, wherein one of the first imaging method and the second imaging method is OCTA, and the other is SLO.

[0138] (Configuration 3) 3. The ophthalmologic apparatus according to configuration 2, wherein the first imaging method is OCTA and the second imaging method is SLO.

[0139] (Configuration 4) the first image is an En-Face image; The ophthalmologic apparatus of configuration 3, wherein the second image is a color SLO image obtained using a red SLO image obtained by scanning the test eye with a red laser light, a green SLO image obtained by scanning the test eye with a green laser light, and a blue SLO image obtained by scanning the test eye with a blue laser light.

[0140] (Configuration 5) The ophthalmologic apparatus of configuration 4, wherein the alignment means deforms the color SLO image so that the position of the first area included in the En-Face image and the position of the second area included in the green SLO image are closer to each other.

[0141] (Configuration 6) The ophthalmologic apparatus of configuration 5, wherein the alignment means inverts the brightness of the green SLO image and deforms the color SLO image so that the position of the first area included in the En-Face image and the position of the second area included in the green SLO image with the inverted brightness are closer to each other.

[0142] (Configuration 7) An ophthalmologic apparatus as described in configuration 5 or 6, wherein the alignment means inverts the brightness of the green SLO image, adjusts the brightness of the peripheral and central parts of the green SLO image, and deforms the color SLO image so that the position of the first area included in the En-Face image and the position of the second area included in the green SLO image with the adjusted brightness are closer to each other.

[0143] (Configuration 8) 8. The ophthalmologic apparatus according to any one of configurations 1 to 7, wherein the first area and the second area are areas including blood vessels.

[0144] (Configuration 9) 9. The ophthalmologic apparatus according to any one of configurations 1 to 8, wherein the first region and the second region are regions where blood vessels intersect or branch off.

[0145] (Configuration 10) The ophthalmologic apparatus according to any one of configurations 1 to 9, wherein the display control means controls displaying, on a display unit, a blended image obtained by blending the first image and the transformed second image.

[0146] (Configuration 11) The ophthalmologic apparatus according to configuration 10, wherein the display control means performs blending processing corresponding to the transmittance specified by the user, and controls the display of a blended image obtained by blending the first image and the deformed second image on the display unit.

[0147] (Method 1) an image acquiring step of acquiring a first image obtained by photographing the subject's eye using a first photographing method and a second image obtained by photographing the subject's eye using a second photographing method different from the first photographing method; a registration process of deforming the second image so that a position of a first region included in the first image and a position of a second region included in the second image, the second region corresponding to the first region, are closer to each other; a display control step of controlling a display unit to display a superimposed image in which the first image and the deformed second image are superimposed; A control method for an ophthalmic apparatus comprising:

[0148] (Program 1) A program that causes a computer to execute the method for controlling an ophthalmologic apparatus according to Method 1. [Explanation of symbols]

[0149] 1000 Optical coherence tomography acquisition unit 100 Optical output section of SLO device 200 SLO device light receiving unit 300 Anterior eye imaging unit

Claims

1. an image acquisition means for acquiring a first image obtained by photographing the subject's eye using a first photographing method and a second image obtained by photographing the subject's eye using a second photographing method different from the first photographing method; a positioning means for deforming the second image so that a position of a first region included in the first image and a position of a second region included in the second image, the second region corresponding to the first region, become closer to each other; a display control means for controlling a superimposed image in which the first image and the transformed second image are superimposed on a display unit; An ophthalmic device comprising:

2. 2. The ophthalmologic apparatus according to claim 1, wherein one of the first imaging method and the second imaging method is OCTA, and the other is SLO.

3. The ophthalmologic apparatus according to claim 2 , wherein the first imaging method is OCTA and the second imaging method is SLO.

4. the first image is an En-Face image; 4. The ophthalmologic apparatus according to claim 3, wherein the second image is a color SLO image obtained using a red SLO image obtained by scanning the test eye with red laser light, a green SLO image obtained by scanning the test eye with green laser light, and a blue SLO image obtained by scanning the test eye with blue laser light.

5. The ophthalmologic apparatus according to claim 4, wherein the alignment means deforms the color SLO image so that the position of the first region included in the En-Face image and the position of the second region included in the green SLO image are closer to each other.

6. The ophthalmologic apparatus of claim 5, wherein the alignment means inverts the brightness of the green SLO image and deforms the color SLO image so that the position of the first region included in the En-Face image and the position of the second region included in the green SLO image with the inverted brightness are closer to each other.

7. The ophthalmologic apparatus of claim 5, wherein the alignment means inverts the brightness of the green SLO image, adjusts the brightness of the peripheral portion and the brightness of the central portion of the green SLO image, and deforms the color SLO image so that the position of the first region included in the En-Face image and the position of the second region included in the green SLO image whose brightness has been adjusted are closer to each other.

8. The ophthalmologic apparatus according to claim 1 , wherein the first region and the second region are regions containing blood vessels.

9. The ophthalmologic apparatus according to claim 1 , wherein the first region and the second region are regions where blood vessels intersect or branch off.

10. The ophthalmologic apparatus according to claim 1 , wherein the display control means controls displaying, on a display unit, a blended image obtained by blending the first image and the transformed second image.

11. The ophthalmologic apparatus according to claim 10, wherein the display control means performs blending processing corresponding to a transmittance specified by a user, and controls displaying a blended image obtained by blending the first image and the transformed second image on a display unit.

12. an image acquiring step of acquiring a first image obtained by photographing the subject's eye using a first photographing method and a second image obtained by photographing the subject's eye using a second photographing method different from the first photographing method; a registration process of deforming the second image so that a position of a first region included in the first image and a position of a second region included in the second image, the second region corresponding to the first region, are brought closer together; a display control step of controlling a display unit to display a superimposed image in which the first image and the deformed second image are superimposed; A control method for an ophthalmic apparatus comprising:

13. A program that causes a computer to execute the method for controlling an ophthalmologic apparatus according to claim 12.

Images