Image processing method, image processing device, ophthalmic device, and program

Abstract

The image processing device irradiates the eye under examination with light from a light source using an optical system with a first numerical aperture, and detects interference light between the signal light propagated by the optical system with a second numerical aperture from the reflected light from the eye under examination due to the irradiated light and the reference light obtained by splitting the light from the light source, thereby acquiring information indicating the interference light (S200). It then performs a first process (S202) in which it projects the information indicating the interference light onto a four-dimensional frequency aperture formed by the optical system with the first numerical aperture and the optical system with the second numerical aperture in a four-dimensional space of the frequency of the light source and the frequency of the light from the eye under examination due to the signal light, and a second process (S204) in which it projects the projected information onto a three-dimensional space. [Effect] The generated OCT images have reduced information loss while maintaining resolution, resulting in high-precision OCT images that faithfully reflect the condition of the fundus in the examined eye.

Description

Technical Field

[0001] The technology of the present disclosure relates to an image processing method, an ophthalmic apparatus, and a program.

Background Art

[0002] An optical coherence tomography by optical coherence tomography can measure the structure of an eye to be examined using reflected light from different layers in the fundus of the eye to be examined, and can obtain a one-dimensional depth direction, a two-dimensional tomographic image, and a three-dimensional image. For example, a technique for generating data related to the structure of an eye to be examined using an optical coherence tomograph is known (U.S. Patent No. 10238281).

Summary of the Invention

[0003] A first aspect of the technology of the present disclosure is an image processing method in an image processing apparatus performed by a processor, irradiating light from a light source onto an irradiated object by an optical system with a first numerical aperture, and obtaining information indicating the interference light obtained by detecting the interference light between the signal light propagated by the optical system with a second numerical aperture from the irradiated object by the irradiated light and the reference light obtained by splitting the light from the light source, performing a first process of projecting the information indicating the interference light onto a four-dimensional frequency aperture formed by the optical system with the first numerical aperture and the optical system with the second numerical aperture in a four-dimensional space of the frequency of the light source and the frequency of the three-dimensional light indicating the irradiated object, and a second process of projecting the projected information onto a three-dimensional space An image processing method including this.

[0004] A second aspect of the technology of the present disclosure is in an image processing apparatus including a memory and a processor, the processor irradiating light from a light source onto an irradiated object by an optical system with a first numerical aperture, and obtaining information indicating the interference light obtained by detecting the interference light between the signal light propagated by the optical system with a second numerical aperture from the irradiated object by the irradiated light and the reference light obtained by splitting the light from the light source, The first process involves projecting information representing the interference light onto a four-dimensional frequency aperture formed by the optical system of the first numerical aperture and the optical system of the second numerical aperture, in a four-dimensional space between the frequency of the light source and the frequency of the three-dimensional light representing the irradiated object; and the second process involves projecting the projected information onto a three-dimensional space. This is an image processing device that performs the following action.

[0005] A third aspect of the technology of this disclosure is: On the computer, Light from a light source is irradiated onto the eye under examination using an optical system with a first numerical aperture, and information indicating the interference light obtained by detecting the interference light between the signal light propagated by the reflected light from the eye under examination using an optical system with a second numerical aperture and the reference light obtained by splitting the light from the light source, A first process involves projecting the information indicating the interference light onto a four-dimensional frequency aperture formed by the optical system of the first numerical aperture and the optical system of the second numerical aperture, in a four-dimensional space between the frequency of the light source and the frequency of the light from the eye being examined due to the signal light; and projecting the projected information into a three-dimensional space. The second process is performed This is a program that executes a process. [Brief explanation of the drawing]

[0006] [Figure 1] This is a schematic diagram of the ophthalmic system according to the embodiment. [Figure 2] This is a schematic diagram of the ophthalmic apparatus according to the present invention. [Figure 3] This is a conceptual diagram of an OCT image. [Figure 4] This is a conceptual diagram illustrating an example of how information about OCT images in an OCT system is represented in four-dimensional frequency space. [Figure 5] This is a conceptual diagram showing an example of a 4D aperture A4. [Figure 6] This is a conceptual diagram showing a part of the 4D aperture A4. [Figure 7] This is an explanatory diagram illustrating the concept of OCT image correction processing using the double projection method. [Figure 8] This is a conceptual diagram of the optical system in an OCT system. [Figure 9] This is a diagram illustrating the functions implemented by the image processing program. [Figure 10] This is a flowchart illustrating an example of an image processing workflow. [Figure 11A] This is a conceptual diagram showing an example of an OCT image according to this embodiment. [Figure 11B] This is a conceptual diagram showing a comparative example of OCT images. [Modes for carrying out the invention]

[0007] Hereinafter, an ophthalmic system 100 according to an embodiment of the present invention will be described with reference to the drawings. Figure 1 shows a schematic configuration of the ophthalmology system 100. As shown in Figure 1, the ophthalmology system 100 comprises an ophthalmology device 110, a server device (hereinafter referred to as "server") 140, and a display device (hereinafter referred to as "viewer") 150. The ophthalmology device 110 acquires fundus images. The server 140 stores multiple fundus images obtained by the ophthalmology device 110 capturing the funduses of multiple patients, and the axial length measured by an axial length measuring device (not shown), corresponding to the patient ID. The viewer 150 displays the fundus images and analysis results acquired by the server 140. In this embodiment, the case in which the eye under examination is applied as an example of the "object to be irradiated" in the present disclosure will be described. The ophthalmic device 110 is an example of the "ophthalmic device" in the present disclosure. Furthermore, the ophthalmic device is also an example of the "image processing device" in the present disclosure.

[0008] The ophthalmic device 110, server 140, and viewer 150 are interconnected via network 130. Network 130 can be any network, such as a LAN, WAN, the Internet, or a wide-area Ethernet network. For example, if the ophthalmic system 100 is installed in a single medical facility such as a hospital, a LAN can be used for network 130.

[0009] The viewer 150 is a client in a client-server system, and multiple units are connected via a network. Also, multiple servers 140 may be connected via a network to ensure the redundancy of the system. Further, if the ophthalmic device 110 has an image processing function and an image viewing function of the viewer 150, the ophthalmic device 110 can be in a stand-alone state to acquire, process, and view fundus images. Also, if the server 140 has an image viewing function of the viewer 150, the acquisition, processing, and viewing of fundus images are possible with the configuration of the ophthalmic device 110 and the server 140.

[0010] In addition, other ophthalmic devices (examination devices such as perimetry and tonometry) and a diagnostic support device that performs image analysis using AI (Artificial Intelligence) may be connected to the ophthalmic device 110, the server 140, and the viewer 150 via the network 130.

[0011] Next, the configuration of the ophthalmic device 110 will be described with reference to FIG. 2.

[0012] For convenience of explanation, a scanning laser ophthalmoscope is referred to as "SLO". Also, an optical coherence tomography is referred to as "OCT".

[0013] When the ophthalmic device 110 is installed in a horizontal plane, the horizontal direction is defined as the "X direction", the vertical direction with respect to the horizontal plane is defined as the "Y direction", and the direction connecting the center of the pupil in the anterior segment of the eye to be examined 12 and the center of the eyeball is defined as the "Z direction". Therefore, the X direction, the Y direction, and the Z direction are perpendicular to each other.

[0014] The ophthalmic device 110 includes an imaging device 14 and a control device 16. The imaging device 14 includes an SLO unit 18 and an OCT unit 20, and acquires fundus images of the test eye 12. Hereinafter, the two-dimensional fundus image acquired by the SLO unit 18 is referred to as an SLO image. Also, the cross-sectional image or en-face image of the retina created based on the OCT data acquired by the OCT unit 20 may be referred to as an OCT image.

[0015] The control device 16 includes a computer having a CPU (Central Processing Unit), a RAM (Random Access Memory) 16B, a ROM (Read-Only Memory) 16C, and an input / output port (I / O) 16D.

[0016] The control device 16 includes an input / output display device 16E connected to the CPU 16A via the I / O port 16D. The input / output display device 16E has a graphic user interface for displaying an image of the test eye 12 and receiving various instructions from the user. Examples of the graphic user interface include a touch panel display.

[0017] Also, the control device 16 includes an image processor 17 connected to the I / O port 16D. The image processor 17 generates an image of the test eye 12 based on the data obtained by the imaging device 14. The control device 16 is connected to a network 130 via a communication interface 16F. Note that the image processor 17 includes a memory 17M, which is a non-volatile storage device capable of storing an image processing program described later.

[0018] As described above, in Figure 2, the control device 16 of the ophthalmic device 110 is equipped with an input / display device 16E, but the technology of this disclosure is not limited thereto. For example, the control device 16 of the ophthalmic device 110 may not be equipped with an input / display device 16E, but rather with a separate input / display device that is physically independent of the ophthalmic device 110. In this case, the display device may be equipped with an image processing processor unit, and the image processing processor unit may display an SLO image or the like based on the image signal output from the ophthalmic device 110.

[0019] The imaging device 14 operates under the control of the CPU 16A of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The imaging optical system 19 includes an optical scanner 22 and a wide-angle optical system 30.

[0020] The optical scanner 22 scans the light emitted from the SLO unit 18 in two dimensions, in the X and Y directions. The optical scanner 22 can be any optical element capable of deflecting the light beam, such as a polygon mirror or a galvanometer mirror. A combination of these may also be used.

[0021] The wide-angle optical system 30 combines light from the SLO unit 18 and light from the OCT unit 20.

[0022] The wide-angle optical system 30 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a reflective-refractive optical system combining a concave mirror and a lens. By using a wide-angle optical system using an elliptical mirror or a wide-angle lens, it becomes possible to photograph not only the central part of the fundus but also the peripheral part of the fundus.

[0023] When using a system that includes an elliptical mirror, a configuration using an elliptical mirror as described in International Publication WO2016 / 103484 or International Publication WO2016 / 103489 is also acceptable. Each of the disclosures in International Publication WO2016 / 103484 and International Publication WO2016 / 103489 is incorporated herein by reference in its entirety.

[0024] The wide-angle optical system 30 enables observation of the fundus in a wide field of view (FOV) region 12A. The FOV region 12A indicates the range that can be captured by the imaging device 14. The FOV region 12A can be expressed as the field of view angle. In this embodiment, the field of view angle can be defined by the internal illumination angle and the external illumination angle. The external illumination angle is the illumination angle of the light beam irradiated from the ophthalmic device 110 onto the eye under examination 12, defined with respect to the pupil 27. The internal illumination angle is the illumination angle of the light beam irradiated onto the fundus F, defined with respect to the center O of the eyeball. The external illumination angle and the internal illumination angle are in a corresponding relationship. For example, if the external illumination angle is 120 degrees, the internal illumination angle corresponds to approximately 160 degrees. In this embodiment, the internal illumination angle is set to 200 degrees.

[0025] Here, SLO fundus images obtained by capturing with an internal illumination angle of 160 degrees or more are referred to as UWF-SLO fundus images. UWF stands for UltraWide Field. The wide-angle optical system 30, which sets the field of view (FOV) of the fundus to an ultra-wide angle, can capture the region from the posterior pole to beyond the equator of the fundus of the eye under examination 12.

[0026] The ophthalmic device 110 can image a region 12A with an internal illumination angle of 200°, using the center O of the eyeball of the eye being examined 12 as the reference position. Note that an internal illumination angle of 200° corresponds to an external illumination angle of 110°, with the pupil of the eyeball of the eye being examined 12 as the reference point. In other words, the wide-angle optical system 30 emits laser light from the pupil with an external illumination angle of 110° and images the fundus region with an internal illumination angle of 200°.

[0027] The SLO system is implemented by the control device 16, SLO unit 18, and imaging optical system 19 shown in Figure 2. Because the SLO system includes a wide-angle optical system 30, it enables fundus imaging in a wide field of view (FOV) area 12A.

[0028] The SLO unit 18 includes a light source 40 which contains B light (blue light), G light (green light), R light (red light), and IR light (infrared light (e.g., near-infrared light)). The light of each color from the light source 40 is guided along the same optical path.

[0029] The SLO unit 18 is configured to allow switching between combinations of light sources that emit or emit laser light of different wavelengths, such as a mode that emits R-light and G-light, and a mode that emits infrared light. However, the technology disclosed herein is not limited to having four light sources: a B-light source, a G-light source, an R-light source, and an IR-light source. For example, the SLO unit 18 may further include a white light source and emit light in various modes, such as a mode that emits G-light, R-light, and B-light, or a mode that emits only white light.

[0030] Light incident from the SLO unit 18 into the imaging optical system 19 is scanned in the X and Y directions by the optical scanner 22. The scanned light passes through the wide-angle optical system 30 and the pupil 27 and illuminates the fundus. The reflected light reflected by the fundus passes through the wide-angle optical system 30 and the optical scanner It is injected into the SLO unit 18 via 22.

[0031] The SLO unit 18 includes a beam splitter 60 that guides light from the posterior segment (fundus) of the eye under examination 12 to a detection element 70. The detection element 70 detects light from the posterior segment (fundus) of the eye under examination 12. Note that the beam splitter 60 and detection element 70 may be provided for each color. For example, beam splitters for B light, G light, R light, and IR light may be arranged on the optical axis, and detection elements of the corresponding colors may be placed downstream of each beam splitter. The beam splitter for B light should reflect B light and transmit other light. Similarly, the beam splitter for G light should reflect G light and transmit other light, the beam splitter for R light should reflect R light and transmit other light, and the beam splitter for IR light should reflect IR light.

[0032] The image processor 17, operating under the control of the CPU 16A, generates a UWF-SLO image using the signals from the detection element 70 based on light incident on the SLO unit 18 via the wide-angle optical system 30 and the optical scanner 22 (reflected light reflected by the fundus of the eye), i.e., light of each color.

[0033] Furthermore, the control device 16 controls the light sources 40 for each color so that they emit light simultaneously. By simultaneously photographing the fundus of the eye under examination 12 with B light, G light, and R light, G-color fundus images, R-color fundus images, and B-color fundus images are obtained where each position corresponds to the others. An RGB color fundus image is obtained from the G-color fundus images, R-color fundus images, and B-color fundus images. The control device 16 controls the light sources 40 for each color so that they emit light simultaneously, and by simultaneously photographing the fundus of the eye under examination 12 with G light and R light, G-color fundus images and R-color fundus images are obtained where each position corresponds to the others. An RG color fundus image is obtained from the G-color fundus images and R-color fundus images.

[0034] The wide-angle optical system 30 provides an ultra-wide field of view (FOV) of the fundus, allowing imaging of the region from the posterior pole to beyond the equator of the fundus of the eye under examination 12.

[0035] The OCT system is implemented by the control device 16, OCT unit 20, and imaging optical system 19 shown in Figure 2. Because the OCT system is equipped with a wide-angle optical system 30, it enables OCT imaging of the peripheral portion of the fundus, similar to the SLO fundus image acquisition described above. In other words, the wide-angle optical system 30, which provides an ultra-wide field of view (FOV) of the fundus, allows for OCT imaging of the region of the fundus of the eye 12, extending from the posterior pole to beyond the equator. OCT data of the peripheral portion of the fundus can be acquired, and tomographic images and the 3D structure of the fundus can be obtained by image processing the OCT data.

[0036] The OCT unit 20 includes an illumination / detection optical system 20E which includes a light source 20A, a sensor (detection element) 20B, a first optical coupler 20C, a reference optical system 20D, and a second optical coupler 200 (Figure 3).

[0037] Light emitted from the light source 20A is split by the first optical coupler 20C. One of the split beams of light is incident on the imaging optical system 19 via the illumination / detection optical system 20E as measurement light. The measurement light is irradiated onto the fundus of the eye via the wide-angle optical system 30 and the pupil 27. The measurement light reflected by the fundus of the eye is incident on the OCT unit 20 via the wide-angle optical system 30 and the optical scanner 22, and then incident on the sensor 20B via the illumination / detection optical system 20E and the first optical coupler 20C. The other beam of light split by the first optical coupler 20C is incident on the reference optical system 20D, and its reflected light is incident on the sensor 20B via the first optical coupler 20C as reference light.

[0038] The light incident on sensor 20B, that is, the measurement light reflected from the fundus, and the reference light are different. The light is interfered with to generate interference light. This interference light is received by the sensor 20B. The image processor 17, operating under the control of the CPU 16, generates OCT data detected by the sensor 20B. Based on this OCT data, it is possible to generate a tomographic image and an OCT image.

[0039] The OCT unit 20 described above can scan a predetermined range (for example, a rectangular range of 6 mm x 6 mm) in a single OCT scan. This predetermined range is not limited to 6 mm x 6 mm; it may be a square range of 12 mm x 12 mm or 23 mm x 23 mm, or a rectangular range such as 14 mm x 9 mm or 6 mm x 3.5 mm, or any other rectangular range. It may also be a range with a circular diameter of 6 mm, 12 mm, 23 mm, etc.

[0040] By using the wide-angle optical system 30, the ophthalmic device 110 can scan an area 12A with an internal illumination angle of 200°. That is, by controlling the optical scanner 22, OCT imaging is performed within a predetermined range. The ophthalmic device 110 can generate OCT data from this OCT imaging. Therefore, the ophthalmic device 110 can generate OCT images, including a tomographic image of the fundus (B-scan image), OCT volume data, and an en-face image (a frontal image generated from the OCT volume data), which is a cross-section of the OCT volume data.

[0041] OCT data (or image data of OCT images) is sent from the ophthalmic device 110 to the server 140 via the communication interface 16F and stored in the storage device.

[0042] Incidentally, OCT systems can simultaneously acquire depth information in the fundus. Figure 3 shows an example of an image of dots aligned on the optical axis (OCT image) simultaneously acquired by an OCT system using optical systems (e.g., objective lenses) with different numerical apertures (NA). Figure 3 shows an example of an image of dots aligned on the optical axis for optical systems with NA=0.1, NA=0.3, and NA=0.5. In the figure, the depth axis at the same position on the x-axis is denoted as cτ. Hereafter, the unit system c=1 will be used. The depth of focus of the optical system is denoted as DOF. As shown in the example in Figure 3, the image of dots acquired by an OCT system becomes an incomplete image of dots as the depth position in the fundus moves away from the focal position of the optical system (e.g., depth of focus). In other words, as the position from which information about the fundus is acquired moves away from the focal point of the imaging optical system 19 (e.g., depth of field), the resolution gradually decreases. As a result, the image gradually becomes an incomplete dot with blurring and distortion as it moves away from the depth of field. Some information about the fundus outside the depth of field (e.g., about 10 times the depth of field) is lost, and it is difficult to fully recover the lost information with commonly used image processing. Although there are techniques to correct blurring and distortion in images outside the depth of field by performing processes such as deconvolution and refocusing, the correction is incomplete because some information is lost at the time the information is acquired.

[0043] The reason why the reconstruction of OCT images by the OCT system is incomplete is thought to be due to the fact that the information acquired by the OCT system does not fully reflect all the information regarding the fundus of the examined eye 12.

[0044] Figure 4 shows an example of information regarding an OCT image. Figure 4 is an example of information regarding an OCT image formed by the optical system of an OCT system with an optical system having an NA=0.9 (e.g., an objective lens), shown in four-dimensional frequency space. In the figure, ν is the frequency of light from the light source 20A (in this embodiment, the frequency of light irradiating the fundus), and fx, fy, and fz are object frequencies (in this embodiment, the frequencies of reflected light in the fundus). The frequency ν may be replaced with the angular frequency ω (ω=2πν). For simplicity of explanation, the axis for the object frequency fy is omitted. Also, Figure 4 shows the same optical path, i.e., the optical system (e.g., objective lens) in the OCT system. Here, we show an example where the optical path due to the irradiation of the measurement light and the optical path due to the reflected light from the eye 12 under examination are the same, and both have an NA of 0.9.

[0045] Here, the object frequencies (fx, fy, fz) are uniform in the ν direction. Furthermore, in the 4-dimensional frequency space, since the frequencies in real space are within a predetermined range, a window function representing the 4-dimensional aperture A4 (4-D aperture) is defined as the device function of the OCT system. The 4-dimensional aperture A4 is the aperture space in the 4-dimensional frequency space. Therefore, the OCT system acquires only the frequencies within the 4-dimensional aperture A4 from the object frequencies. However, not all of the frequencies within the 4-dimensional aperture A4 (4-dimensional information) are acquired; instead, they are integrated in the fz direction during detection and acquired as 3-dimensional information. Thus, when the frequencies within the 4-dimensional aperture A4 are integrated in the fz direction, some of the 4-dimensional information is lost.

[0046] Therefore, this embodiment provides an OCT system that can reduce information loss, where a portion of the four-dimensional information is lost.

[0047] For example, in systems such as interference microscopes that enable 3D scanning, the image frequency is the ν integral of the object frequency within the 4D aperture A4. Therefore, by applying the ν integral of the object frequency within the 4D aperture A4, it is possible to reduce information loss, which is the loss of some of the 4D information.

[0048] In this embodiment, the OCT system, which reduces information loss, is equipped with an optical system that makes the 4D aperture A4 sufficiently thin. This makes it possible to reduce information loss, where some of the 4D information is lost even after fz integration. Specifically, the optical system is formed so that the NA of the irradiating optical system that irradiates light onto the eye 12 and the NA of the detection optical system that detects light from the eye 12 (e.g., reflected light from the fundus) are different. For example, if the NA of the irradiating optical system that irradiates light onto the eye 12 is made sufficiently small, for example, approximately zero, it becomes possible to form a thin 4D aperture A4 even if the NA of the detection optical system that detects light from the eye 12 is larger than the NA of the irradiating optical system. In this embodiment, a sufficiently small NA of the optical system is referred to as "NA is zero (NA=0)". That is, an optical system with zero NA includes an optical system with an NA greater than zero and an aperture angle smaller than the size of the image height formed by the light beam. Furthermore, an optical system with zero NA includes an optical system in which the optical system is formed so that the incident or outgoing light becomes a parallel beam, and the NA can be considered to be zero.

[0049] Furthermore, an optical system with zero numerical aperture (NA) can be applied to the optical system of an OCT system using full-field irradiation, which uniformly illuminates an object on the irradiated side, such as the eye 12 being examined, or illuminates a predetermined area of ​​the object on the irradiated side all at once.

[0050] Even when the 4D aperture A4 is formed thinly, the object frequencies within the 4D aperture A4 contain sufficient information (including fz information), and it is possible to reduce the information loss that occurs when fz integration causes some of the information (4D information) related to the eye 12 being examined to be lost.

[0051] In other words, in this embodiment, the optical system is formed such that the NA of the irradiating optical system and the NA of the detection optical system are different for the eye under examination 12. Specifically, an optical system is formed in which the irradiating optical system has a sufficiently small NA, or an NA of zero, and an optical system is formed in which the NA of the detection optical system is larger than the NA of the irradiating optical system. This makes it possible to form a thin 4D aperture A4, as shown in Figure 5. More specifically, as shown in Figure 6 by the 4D aperture A4 at a predetermined frequency (frequency ν of light from the light source 20A), the shape changes from a shape with information in the fz direction (a crescent shape in the figure) to a linear shape. A series of these linear apertures constitutes the 4D aperture A4 in this embodiment (Figure 5).

[0052] The optical system for forming a thin 4D aperture A4 consists of an optical system where one optical system has zero or near-zero NA, and the other optical system has a larger NA than the other optical system. For example, in optical systems such as microscopes, an example of a sufficiently small NA is NA=0.2. Therefore, in this embodiment, from the viewpoint of forming a thin 4D aperture A4, it is preferable that the NA of the irradiation-side optical system, which is the excitation side, has the following relationship as a condition for a small NA. 0 ≤ NA ≤ 0.2

[0053] A thinly formed four-dimensional aperture A4 is an example of the "four-dimensional frequency aperture" of the technology disclosed herein.

[0054] In the following description, as an example, we will explain the case where the optical system on the light irradiation side of the eye 12 is an optical system with an NA of zero, and the optical system on the light detection side of the eye 12 is an optical system with a finite NA. However, the technology of this disclosure is not limited to this. For example, the optical system on the detection side may be an optical system with an NA of zero, and the optical system on the irradiation side may be an optical system with a finite NA. Also, the optical system with an NA of zero may have a finite NA. In other words, the optical system on the irradiation side and the optical system on the detection side can be formed to have different NAs, and by making their NAs different, it is possible to make the 4D aperture A4 thinner than when both have the same NA.

[0055] In this embodiment, the OCT image is corrected using a 4D aperture A4 capable of reducing information loss. Specifically, the OCT image is corrected using a two-stage processing method (double projection method) that utilizes the frequency information of the OCT image. As shown conceptually in Figure 7, in the first stage of processing, the frequency information (I) of the OCT image G1 is projected onto a thinly formed 4D aperture A4. In the second stage of processing, the information (~I) of the image G2 on the projected 4D aperture A4 is further projected in the ν (=2π / ω) direction, i.e., into the (fx, fy, fz) plane. The information (~I') of the image G3 projected in the second stage of processing has reduced information loss of information (4D information) related to the eye 12 being examined. Therefore, the double projection method described above makes it possible to correct, for example, a distorted OCT image to an image equivalent to a normal microscope image. Furthermore, because information loss is reduced, resolution is maintained, making it possible to generate OCT images with higher accuracy compared to cases where a thinly formed 4D aperture A4 is not used.

[0056] In other words, the 4-dimensional aperture A4 (4-dimensional opening) is a quantity that does not depend on the object frequency. It is defined by the optical system of the first numerical aperture and the optical system of the second numerical aperture. In this embodiment, the 4D aperture can be represented as a window function capable of representing physical quantities determined by the configuration of the optical system in the OCT system. In this embodiment, the object frequency is a physical quantity that represents the eye under examination 12 by the components of light. Since the object frequency is not dependent on ν, it is a uniform 4D physical quantity in the ν direction. The frequency ν is the optical frequency of the light source and is also the optical frequency of the signal light. The value of the 3D function obtained by fz integration of the 4D function (acquirable image frequency) obtained by multiplying this object frequency by the 4D aperture A4 is acquired as the detection value by the optical system. That is, the value of the 3D function representing the fz integral of the 4D function obtained by multiplying the object frequency by the 4D aperture A4 is acquired by the sensor of the optical system of the OCT system.

[0057] Furthermore, if the value of NA is finite, the 4D aperture A4 has a finite thickness. Therefore, when using a 4D aperture A4 with a finite thickness, it is sufficient to define and use an aperture surface within the 4D aperture A4. It is preferable to apply a predetermined surface within the 4D aperture A4 that is far from the origin (for example, the outermost surface of the 4D aperture A4, which is the surface furthest from the origin within the 4D aperture A4).

[0058] The frequency of the OCT image used in the correction of OCT images using the 4D aperture A4 described above The information can be expressed by the following equation (1).

[0059]

number

[0060] Here, τ represents the depth of the eye under examination 12 (in the example of a time-type domain OCT, this is the adjustment amount used to adjust the optical path length in the reference optical system 20D, for example, the amount of mirror movement).

[0061] The information (~I) of the image G2 in the projected 4-dimensional aperture A4 can be expressed by (2) below.

[0062]

number

[0063] The information (~I') of the image G3 projected from the 4-dimensional aperture A4 can be expressed as follows (3).

[0064]

number

[0065] The control device 16 generates an OCT image using an image processor 17 that operates under the control of the CPU 16A, using equations (1) to (3) described above. The process for generating the OCT image will be described later.

[0066] Next, the configuration of the optical system in the OCT system according to this embodiment will be further described with reference to Figure 8. Note that in Figure 8, the wide-angle optical system 30 is omitted for simplicity of explanation.

[0067] The sensor 20B of the OCT unit 20 comprises a pair of lenses 20B-1 and a detection element 20B-2. The sensor 20B collimates the light branched by the first optical coupler 20C, i.e., the interference light resulting from the interference of the measurement light reflected from the fundus and the reference light, into parallel light using the pair of lenses 20B-1, and then focuses it onto the detection element 20B-2.

[0068] The reference optical system 20D comprises a pair of lenses 20D-1 and a mirror 20D-2. In the reference optical system 20D, the light branched by the first optical coupler 20C, i.e., the measurement light, is collimated into parallel light by the pair of lenses 20D-1 and then focused to the mirror 20D-2. The mirror 20D-2 is configured to be movable in the optical axis direction (indicated by arrow τ in Figure 8). The reflected light reflected by the mirror 20D-2 is then focused through the pair of lenses 20D-1 to the first The light is returned to the optical coupler 20C as a reference light.

[0069] In Figure 8, a configuration is shown in which the mirror 20D-2 is movable as the reference optical system 20D. However, the OCT system according to this embodiment is not limited to a configuration in which the mirror 20D-2 is movable, and may be fixed. That is, the OCT system according to this embodiment is a TD-OCT (Time-Domain OCT), also known as a time-type domain OCT. It can be applied to various types of OCT systems, including OCT, SS-OCT (Swept-Source OCT), a wavelength-swept type known as Fourier-domain OCT, and SD-OCT (Spectral-Domain OCT) using a spectrometer.

[0070] Therefore, the OCT system should be configured according to the OCT system being applied. For example, when applying time-domain OCT (TD-OCT), the mirror 20D-2 should be moved to perform the sweep. When applying SD-OCT, a type of Fourier-domain OCT, the mirror 20D-2 should be fixed and spectral detection should be performed. When performing spectral detection, a light source that emits multiple beams of light at multiple wavelengths should be used. When applying SS-OCT, for example, a single-pixel detector should be used for the detection element 20B-2, and the mirror 20D-2 should be fixed to sweep the wavelength of a broadband light source. When performing wavelength sweeping, for example, a broadband light source from a laser device that emits broadband laser light for wavelength sweeping should be used as the light source 20A. Furthermore, as an optical system with zero numerical aperture (NA), an optical system of an OCT system using full-field irradiation can be applied to uniformly illuminate an object on the irradiated side, such as the eye 12 under examination, or to illuminate a predetermined area of ​​the object on the irradiated side all at once. In other words, the technology of this disclosure is not limited to the laser scanning optical system that scans the laser light described above, but can also be applied to optical systems using a CCD (Charge Coupled Device) camera or the like. In this case, a CCD camera containing an element such as a CCD can be used as the detection element 20B-2.

[0071] The illumination / detection optical system 20E comprises a second optical coupler 200, an illumination optical system 210, and a detection optical system 220. The second optical coupler 200 has the function of guiding the light emitted from the light source 20A (measurement light branched by the first optical coupler 20C) to the illumination optical system 210 as illumination light, and the function of guiding the light from the detection optical system 220 (i.e., the reflected light from the eye under examination 12) to the first optical coupler 20C (i.e., toward the sensor 20B) as detection light. Therefore, the eye under examination 12 is irradiated with light propagating through the illumination optical system 210 as light from the light source 20A, and the reflected light from the light that has propagated through the illumination optical system 210 and irradiated the eye under examination 12 (reflected light from the eye under examination 12) is incident on the sensor 20B as measurement light propagating through the detection optical system 220. In Figure 8, the optical path through the illumination optical system 210 in the illumination / detection optical system 20E is shown by a solid line, and the optical path through the detection optical system 220 is shown by a dotted line.

[0072] The illumination optical system 210 includes a pair of lenses, lens 212 and lens 214. Lens 212 is positioned such that one end face of the illumination fiber 211 is located at the focal point on the incident side of lens 212 on the optical axis of the illumination optical system 210. The illumination fiber 211 is made of a single-mode fiber, and its other end face is connected to the second optical coupler 200. Lens 214 is positioned such that the reflective surface of the optical scanner 22 is located at the focal point on the exit side of lens 214 on the optical axis of the illumination optical system 210. Thus, lens 212 collimates the light from the light source 20A (measurement light branched by the first optical coupler 20C) into parallel light, and lens 214 focuses the parallel light collimated by lens 212 onto the reflective surface of the optical scanner 22. The light reflected by the optical scanner 22 is then collimated into parallel light by lens 12B of the eye under examination 12 and illuminates the fundus 12C. Therefore, the light from the light source 20A via the illumination optical system 210 is irradiated as parallel light onto the fundus 12C of the eye under examination 12.

[0073] The illumination optical system 210 is configured to emit parallel light toward the eye 12 under examination, thereby forming an optical system with zero numerical aperture (NA). The NA of the illumination optical system 210 is an example of the "first numerical aperture" of the technology disclosed herein. Furthermore, the illumination optical system 210 is an example of the "optical system with the first numerical aperture" of the technology disclosed herein.

[0074] The detection optical system 220 includes a beam splitter 222 and a pair of lenses, lens 214 and lens 216. The beam splitter 222 is positioned between lens 212 and lens 214 in the illumination optical system 210 and extracts the reflected light from the eye under examination 12 by its reflection function. The beam splitter 222 is positioned such that its reflective surface is located at the focal point on the exit side of the reflected light from the eye under examination 12 of lens 214 on the optical axis of the illumination optical system 210. Lens 224 is positioned such that its reflective surface is located at the focal point on the incident side of lens 224 on the optical axis of the detection optical system 220. Lens 226 is positioned such that one end face of the detection fiber 228 is located at the focal point on the exit side of lens 226 on the optical axis of the detection optical system 220. The detection fiber 228 is formed from a single-mode fiber, and its other end is connected to the second optical coupler 200. Thus, lens 224 collimates the light from beam splitter 222 into parallel light, and lens 226 focuses the parallel light collimated by lens 224 onto the end face of the detection fiber 228. Consequently, in the detection optical system 220, the light reflected at the fundus 12C of the eye under examination 12 is emitted toward the sensor 20B.

[0075] The detection optical system 220 has a focal point on the eye 12 side, thereby having a different numerical aperture (NA) than the illumination optical system 210, and forming an optical system with a finite NA. The NA of the detection optical system 220 is an example of the "second numerical aperture" of the technology of this disclosure. Furthermore, the detection optical system 220 is an example of the "optical system with a second numerical aperture" of the technology of this disclosure.

[0076] In this embodiment, the ophthalmic device 110 generates an OCT image using information obtained by reducing the information loss of 4D information using the OCT system which includes the irradiation optical system 210 with zero numerical aperture described above. The OCT image is generated by executing an image processing program using an image processor 17 that operates under the control of the CPU 16A. In this embodiment, the case in which the ophthalmic device 110 generates the OCT image is described, but it is of course possible to generate it using an external device such as a server 140.

[0077] The ROM 16C of the ophthalmic device 110 or the memory 17M of the image processor 17 stores the image processing program shown in Figure 10.

[0078] ROM16C and memory17M are examples of “memory” in the technology of this disclosure. CPU16A is an example of “processor” in the technology of this disclosure. Image processing program is an example of “program” in the technology of this disclosure.

[0079] In the ophthalmic apparatus 110 according to this embodiment, various functions are realized by the CPU 16A reading and executing an image processing program. The image processing program includes a display control function, an image processing function, and a processing function. That is, by the CPU 16A executing an image processing program having these functions, the CPU 16A operates as a display control unit 204, an image processing unit 206, and a processing unit 208, as shown in Figure 9. The image processing function includes an image processing function using the two-stage processing method (double projection method) described above.

[0080] Next, the image processing according to this embodiment will be described in detail with reference to Figure 10. The CPU 16A of the ophthalmic device 110 reads and executes the image processing program from the ROM 16C or memory 17M, thereby realizing the image processing shown in the flowchart of Figure 10. The image processing process shown in Figure 10 is an example of the image processing method disclosed herein.

[0081] In step S200, the CPU 16A, along with the image processing unit 206, acquires OCT data from the image processor 17. The OCT data is obtained by performing OCT imaging on the subject eye 12 using the ophthalmic device 110. The OCT data includes data at positions with different depths in the optical axis direction (Z axis direction). The OCT data is obtained by performing OCT imaging using the ophthalmic device 110 as described above.

[0082] In step S202, the image processing unit 206 performs a first-stage process in which it projects the information obtained by the OCT system (OCT data) onto a four-dimensional aperture. Specifically, the image processing unit 206 expresses the acquired OCT data using equation (1) above, and derives the frequency information of the OCT image using equation (2) above as a process to project the frequency information (I) of the OCT image G1 onto the thinly formed four-dimensional aperture A4. The derived information is temporarily stored in RAM 16B.

[0083] In the next step S204, the image processing unit 206 performs a second stage of processing to project the information projected onto the 4D aperture A4 into 3D space. That is, it performs a process to project the information of the image G2 (~I) on the projected 4D aperture A4 in the ν (=2π / ω) direction ((fx, fy, fz) plane). In the second stage of processing, the image processing unit 206 uses equation (3) above to derive the information of the OCT image after double projection projected from the 4D aperture A4 into 3D space, as a process to project the OCT data information projected onto the 4D aperture A4 into 3D space. The derived information is temporarily stored in RAM 16B.

[0084] Next, in step S206, the image processing unit 206 generates an OCT image using the information projected by the double projection method. The generated OCT image is stored in RAM 16B by the processing unit 208.

[0085] The OCT image corrected using the double projection method described above may generate an OCT image at a predetermined depth in the optical axis direction, i.e., in the depth direction of the eye 12 being examined, or it may generate OCT images at multiple different depths. For example, an OCT image may be generated at a predetermined depth, such as 10 times the depth of focus, based on the focal point of the detection optical system 220 on the eye 12 side. Alternatively, multiple OCT images at different depths may be generated based on the focal point, or each OCT image may be generated at a predetermined number of positions at predetermined intervals. If multiple OCT images are generated, each of the multiple OCT images can be stored in RAM 16B or memory 17M. The predetermined depth, different depths, and predetermined number can be predetermined and can be arbitrarily set by the user, for example.

[0086] Furthermore, when generating the OCT images described above, image processing such as noise reduction may be performed to improve the sharpness of the images.

[0087] As described above, the image processing unit 206 executes the image processing shown in the flowchart of Figure 10, thereby generating an OCT image at a predetermined depth, for example, based on the focal position of the detection optical system 220 on the side of the eye being examined 12. The generated OCT image has reduced information loss and maintains resolution, resulting in a high-precision OCT image that faithfully reflects the state of the fundus in the eye being examined 12.

[0088] Furthermore, the image processing described above may include a process to generate information for a display screen that shows the generated OCT images. The process to generate information for a display screen that shows the OCT images generates information for a display screen that shows a predetermined sample of OCT images side by side. It is possible. Furthermore, when multiple OCT images are generated, it is possible to generate information for a display screen that shows the multiple OCT images side by side. When controlling the display of multiple OCT images side by side, for example, by generating information for a display screen that shows the OCT image group arranged in the direction of increasing depth, it is possible to visualize and provide to the user changes in size and shape in the depth direction of the examined eye 12.

[0089] The OCT image obtained from the information of a single point detected by the ophthalmic device 110, corrected using the double projection method described above, is corrected from a distorted OCT image as shown in Figure 11B to an OCT image equivalent to a normal microscope image as shown in Figure 11A. Figure 11A shows the illumination / detection optical system 20E in the OCT unit 20, where the NA of the illumination optical system 210 and the NA of the detection optical system are different, and the OCT image is corrected using the double projection method. Here, the NA of the illumination optical system 210 is formed to have a zero NA by irradiating the eye under examination with a parallel light beam, and the optical system of the detection optical system 220 is formed to have an NA of 0.9. Also, Figure 11A generates an OCT image at a depth position 30 times the depth of focus. Figure 11B is a comparative example where the NA of the illumination optical system 210 and the NA of the detection optical system 220 are both 0.9 and are formed to match.

[0090] As shown in Figure 11A, by applying the OCT system according to this embodiment, it is possible to obtain an OCT image as a single-point image (point image) from the information of a single point detected in the eye 12 under examination. On the other hand, as shown in Figure 11B, if the NA of the illumination optical system 210 and the detection optical system 220 are matched, the OCT image will have a distorted shape, and the state of the fundus of the eye 12 under examination will be obtained as a distorted image. Therefore, as can be understood from Figures 11A and 11B, by forming a thin 4D aperture A4 and generating an OCT image from information obtained using the double projection method projected through the 4D aperture A4, it is possible to generate an OCT image with high accuracy.

[0091] As explained above, in this embodiment, the irradiation optical system 210 and the detection optical system 220 have different numerical apertures (NAs), and one of the optical systems (the irradiation optical system 210 in the above example) has a zero NA. This makes it possible to treat the aperture for the OCT image in the 4D space considering frequency as a thin aperture (4D aperture A4) that can suppress the influence even when fz integration is performed. By using this thinly formed 4D aperture A4, the information loss of information related to the eye 12 (4D information) in the OCT image projected by the double projection method is reduced. Therefore, by correcting the OCT image with the image processing described above, it is possible to correct a distorted OCT image to an image equivalent to a normal microscope image. Furthermore, since information loss is reduced, the resolution is maintained, and it is possible to generate OCT images with higher accuracy compared to cases where a thinly formed 4D aperture A4 is not used.

[0092] In the above embodiment, the case in which image processing (Figure 10) is performed on the ophthalmic device 110 is described, but the technology of this disclosure is not limited thereto, and may be performed on the ophthalmic device 110, server 140, viewer 150, or additional image processing devices further provided on the network 130, or a combination of any of these.

[0093] As described above, the technology of this disclosure preferably includes the following technologies, since it is preferable that image processing is realized using information obtained by utilizing a thinly formed 4D aperture A4.

[0094] (First technology) An acquisition unit that detects interference light between a signal light, obtained by detecting interference light between a signal light, which is the reflected light from the irradiated object propagated by a second numerical aperture optical system, and a reference light obtained by splitting the light from the light source, and acquires information indicating the interference light, which is transmitted by an optical system with a first numerical aperture, and the reflected light from the irradiated object. A processing unit that performs the following: a first process of projecting information indicating the interference light onto a four-dimensional frequency aperture formed by the optical system of the first numerical aperture and the optical system of the second numerical aperture, in a four-dimensional space of the frequency of the light source and the frequency of the three-dimensional light representing the irradiated object; and a second process of projecting the projected information onto a three-dimensional space. An image processing device equipped with the following features.

[0095] The image processing unit 206 is an example of the "acquisition unit" and "processing unit" of the technology of this disclosure.

[0096] (Second technology) A detection unit that detects interference light between signal light obtained by irradiating the eye under examination with light from a light source and reference light obtained by splitting the light from the light source, An illumination optical system formed with a first numerical aperture to irradiate the eye under examination with light from the aforementioned light source, A detection optical system formed with a second numerical aperture different from the first numerical aperture, such that the reflected light from the eye under examination due to the light irradiated by the illumination optical system is propagated to the detection unit as the signal light, A processing unit performs the following based on information indicating interference light detected by the detection unit: a first process of projecting the information indicating interference light onto a four-dimensional frequency aperture formed by the optical system of the first numerical aperture and the optical system of the second numerical aperture, in a four-dimensional space of the frequency of the light source and the frequency of the three-dimensional light representing the eye under examination; and a second process of projecting the projected information onto a three-dimensional space. An image generation unit generates multiple images corresponding to multiple planes with different depths in the optical axis direction based on the information processed by the processing unit, An ophthalmic device equipped with [unspecified features].

[0097] Sensor 20B is an example of the "acquisition unit" of the present disclosure. Irradiation optical system 210 is an example of the "irradiation optical system" of the present disclosure, and detection optical system 220 is an example of the "detection optical system" of the present disclosure. Image processing unit 206 is an example of the "processing unit" and "image generation unit" of the present disclosure.

[0098] Although the technology of this disclosure has been described above using embodiments, the technical scope of the technology of this disclosure is not limited to the scope described in the embodiments above. Various modifications or improvements can be made to the embodiments above without departing from the gist of the work, and such modified or improved forms are also included in the technical scope of the technology of this disclosure.

[0099] Furthermore, although the above embodiments describe processing performed by executing a program stored in a memory or other storage device, at least some of the program processing may be implemented in hardware. Also, the program processing flow described in the above embodiments is just one example, and unnecessary steps may be deleted, new steps added, or the processing order changed, without departing from the main point.

[0100] Furthermore, in order to have a computer execute the processing described in the above-described embodiment, a program in which the above-described processing is written in code that can be processed by a computer may be stored on a storage medium such as an optical disc and distributed.

[0101] In the embodiments described above, a CPU was used as an example of a general-purpose processor, but in the embodiments above, the term "processor" refers to a processor in a broad sense, including general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, etc.). This includes programmable logic devices, etc.

[0102] Furthermore, the operation of the processor in the above-described embodiment may not be performed by a single processor, but may be performed by multiple processors working together, or by multiple processors located in physically separate locations working together.

[0103] All documents, patent applications, and technical standards described herein are incorporated by reference in the same way as if each individual document, patent application, and technical standard were specifically and individually described as being incorporated by reference. In addition, the disclosure of Japanese application number 2022-048770, filed on 24 March 2022, is incorporated in its entirety by reference herein.

Claims

[Claim 1] An image processing method in an image processing device performed by a processor, Light from a light source is shone onto the object to be irradiated by an optical system with a first numerical aperture, and information indicating the interference light obtained by detecting the interference light between the signal light propagated by the reflected light from the object to be irradiated by the optical system with a second numerical aperture and the reference light obtained by splitting the light from the light source, The first process involves projecting the information representing the interference light onto a four-dimensional frequency aperture formed by the optical system of the first numerical aperture and the optical system of the second numerical aperture, in a four-dimensional space consisting of the frequency of the light emitted from the light source and the frequency of the three-dimensional light representing the irradiated object; and the second process involves projecting the projected information onto three-dimensional space. An image processing method that includes the following.

Images