Image processing method, image processing apparatus, and program

Abstract

In the present invention, a three-dimensional image of a vortex vein is generated on the basis of optical coherence tomography (OCT) volume data.　This image processing method is executed by a processor, and includes: a step for acquiring OCT volume data containing the choroid; and a step for extracting, on the basis of the OCT volume data, choroid blood vessels, and generating a three-dimensional image of the choroid blood vessels.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to an image processing method, an image processing device, and a program. [Background technology]

[0002] U.S. Patent No. 10,238,281 discloses a technique for generating volume data of an eye to be inspected using an optical coherence tomography. Conventionally, it has been desired to visualize blood vessels based on the volume data of the eye to be inspected. Summary of the Invention

[0003] An image processing method according to a first aspect of the technology of the present disclosure is an image processing method performed by a processor, and includes the steps of acquiring OCT volume data including the choroid, extracting choroidal blood vessels based on the OCT volume data, and generating a three-dimensional image of the choroidal blood vessels.

[0004] An image processing device according to a second aspect of the disclosed technique includes a memory and a processor connected to the memory, and the processor executes the steps of acquiring OCT volume data including the choroid, extracting choroidal blood vessels based on the OCT volume data, and generating a stereoscopic image of the choroidal blood vessels.

[0005] A program according to a third aspect of the technique of the present disclosure causes a computer to execute the steps of acquiring OCT volume data including the choroid, extracting choroidal blood vessels based on the OCT volume data, and generating a three-dimensional image of the choroidal blood vessels. [Brief explanation of the drawings]

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0007] Hereinafter, the ophthalmic system 100 according to an embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, the schematic configuration of the ophthalmic system 100 is illustrated. As shown in FIG. 1, the ophthalmic system 100 includes an ophthalmic device 110, a server device (hereinafter referred to as "server") 140, and a display device (hereinafter referred to as "viewer") 150. The ophthalmic device 110 acquires fundus images. The server 140 stores a plurality of fundus images obtained by photographing the fundus of a plurality of patients with the ophthalmic device 110 and the axial length measured by an axial length measuring device (not shown) in correspondence with the patient ID. The viewer 150 displays the fundus images and analysis results acquired by the server 140.

[0008] The server 140 is an example of the "image processing device" of the technology of the present disclosure.

[0009] The ophthalmic device 110, the server 140, and the viewer 150 are interconnected via the network 130. The network 130 is an arbitrary network such as a LAN, a WAN, the Internet, or a wide area Ethernet network. For example, when the ophthalmic system 100 is constructed in one hospital, a LAN can be adopted for the network 130.

[0010] The viewer 150 is a client in a client-server system, and multiple units are connected via the network. Also, in order to ensure the redundancy of the system, multiple servers 140 may be connected via the network. Alternatively, if the ophthalmic device 110 has an image processing function and the image viewing function of the viewer 150, the ophthalmic device 110 can be in a stand-alone state and can acquire, process, and view fundus images. Further, if the server 140 has the image viewing function of the viewer 150, the acquisition, processing, and viewing of fundus images can be achieved with the configuration of the ophthalmic device 110 and the server 140.

[0011] Note that a diagnostic support device that performs image analysis using other ophthalmic devices (examination devices such as visual field measurement and intraocular pressure measurement) and AI (Artificial Intelligence) may be connected to the ophthalmic device 110, the server 140, and the viewer 150 via the network 130.

[0012] Next, with reference to FIG. 2, the configuration of the ophthalmic device 110 will be described.

[0013] For convenience of explanation, a scanning laser ophthalmoscope is referred to as "SLO". Also, an optical coherence tomography is referred to as "OCT".

[0014] Note that when the ophthalmic device 110 is installed on a horizontal plane, the horizontal direction is the "X direction", the vertical direction with respect to the horizontal plane is the "Y direction", and the direction connecting the center of the pupil of the anterior segment of the eye to be examined 12 and the center of the eyeball is the "Z direction". Therefore, the X direction, the Y direction, and the Z direction are perpendicular to each other.

[0015] The ophthalmic device 110 includes a photographing device 14 and a control device 16. The photographing device 14 includes an SLO unit 18 and an OCT unit 20, and acquires a fundus image of the fundus of the eye to be examined 12. Hereinafter, the two-dimensional fundus image acquired by the SLO unit 18 is referred to as an SLO image. Also, a tomographic image or an en-face image of the retina created based on the OCT data acquired by the OCT unit 20 is referred to as an OCT image.

[0016] The control device 16 includes a computer having a CPU (Central Processing Unit), a RAM (Random Access Memory) 16B, a ROM (Read-Only Memory) 16C, and an input / output (I / O) port 16D.

[0017] The control device 16 includes an input / display device 16E connected to the CPU 16A via the I / O port 16D. The input / display device 16E has a graphic user interface for displaying an image of the eye under examination 12 and receiving various instructions from the user. Examples of the graphic user interface include a touch panel display.

[0018] Further, the control device 16 includes an image processor 17 connected to the I / O port 16D. The image processor 17 generates an image of the eye under examination 12 based on the data obtained by the imaging device 14. Note that the control device 16 is connected to the network 130 via the communication interface 16F.

[0019] As described above, in FIG. 2, the control device 16 of the ophthalmic device 110 includes the input / display device 16E, but the technology of the present disclosure is not limited thereto. For example, the control device 16 of the ophthalmic device 110 may not include the input / display device 16E and may include a separate input / display device physically independent of the ophthalmic device 110. In this case, the display device includes an image processing processor unit that operates under the control of the display control unit 204 of the CPU 16A of the control device 16. The image processing processor unit may display an SLO image or the like based on the image signal output-instructed by the display control unit 204.

[0020] The imaging device 14 operates under the control of the CPU 16A of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The imaging optical system 19 includes an optical scanner 22 and a wide-angle optical system 30.

[0021] The optical scanner 22 two-dimensionally scans the light emitted from the SLO unit 18 in the X direction and the Y direction. The optical scanner 22 may be an optical element capable of deflecting a light beam. For example, a polygon mirror, a galvanometer mirror, or the like can be used. Further, a combination thereof may also be used.

[0022] The wide-angle optical system 30 combines the light from the SLO unit 18 and the light from the OCT unit 20.

[0023] Note that the wide-angle optical system 30 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a catadioptric optical system combining a concave mirror and a lens. By using a wide-angle optical system using an elliptical mirror, a wide-angle lens, etc., it becomes possible to photograph the retina not only at the center of the fundus but also at the peripheral part of the fundus.

[0024] When using a system including an elliptical mirror, it may be configured to use the system using the elliptical mirror described in International Publication WO2016 / 103484 or International Publication WO2016 / 103489. The disclosures of International Publication WO2016 / 103484 and International Publication WO2016 / 103489 are each incorporated herein by reference in their entirety.

[0025] The wide-angle optical system 30 realizes observation with a wide field of view (FOV) 12A in the fundus. The FOV 12A indicates the range that can be photographed by the imaging device 14. The FOV 12A can be expressed as a viewing angle. The viewing angle can be defined by an internal irradiation angle and an external irradiation angle in the present embodiment. The external irradiation angle is the irradiation angle of the light beam irradiated from the ophthalmic device 110 to the eye to be examined 12, defined with respect to the pupil 27. The internal irradiation angle is the irradiation angle of the light beam irradiated to the fundus F, defined with respect to the center O of the eyeball. The external irradiation angle and the internal irradiation angle are in a corresponding relationship. For example, when the external irradiation angle is 120 degrees, the internal irradiation angle corresponds to about 160 degrees. In the present embodiment, the internal irradiation angle is 200 degrees.

[0026] Here, an SLO fundus image obtained by photographing with a photographing angle of 160 degrees or more at the internal irradiation angle is referred to as a UWF-SLO fundus image. Note that UWF is an abbreviation for UltraWide Field (ultra-wide angle). With a wide-angle optical system 30 having an ultra-wide angle as the visual field angle (FOV) of the fundus, it is possible to photograph a region extending from the posterior pole of the fundus of the eye to be examined 12 beyond the equator, and to photograph structures existing in the peripheral part of the fundus such as the vortex vein.

[0027] The ophthalmic device 110 can photograph a region 12A with an internal irradiation angle of 200° with the center O of the eyeball of the eye to be examined 12 as the reference position. Note that the internal irradiation angle of 200° is 110° at the external irradiation angle based on the pupil of the eyeball of the eye to be examined 12. That is, the wide-angle optical system 30 irradiates laser light from the pupil at a photographing angle of the external irradiation angle of 110°, and photographs a fundus region with an internal irradiation angle of 200°.

[0028] The SLO system is realized by the control device 16, the SLO unit 18, and the photographing optical system 19 shown in FIG. 2. Since the SLO system includes the wide-angle optical system 30, it enables fundus photography with a wide FOV 12A.

[0029] The SLO unit 18 includes a light source 40 for B light (blue light), a light source 42 for G light (green light), a light source 44 for R light (red light), and a light source 46 for IR light (infrared light (for example, near-infrared light)), and optical systems 48, 50, 52, 54, 56 that reflect or transmit the light from the light sources 40, 42, 44, 46 and guide it to one optical path. The optical systems 48, 56 are mirrors, and the optical systems 50, 52, 54 are beam splitters. The B light is reflected by the optical system 48, transmitted through the optical system 50, and reflected by the optical system 54. The G light is reflected by the optical systems 50, 54. The R light is transmitted through the optical systems 52, 54. The IR light is reflected by the optical systems 52, 56 and guided to one optical path, respectively.

[0030] The SLO unit 18 is configured to be able to switch between light sources that emit laser light with different wavelengths, such as a mode that emits R light and G light, or a combination of light sources that emit light, such as a mode that emits infrared light. In the example shown in FIG. 2, it includes four light sources: a light source 40 for B light, a light source 42 for G light, a light source 44 for R light, and a light source 46 for IR light. However, the technology of the present disclosure is not limited to this. For example, the SLO unit 18 may further include a light source for white light and emit light in various modes such as a mode that emits G light, R light, and B light, or a mode that emits only white light.

[0031] The light incident from the SLO unit 18 into the imaging optical system 19 is scanned in the X direction and the Y direction by the optical scanner 22. The scanned light is irradiated onto the fundus via the wide-angle optical system 30 and the pupil 27. The reflected light reflected by the fundus is incident on the SLO unit 18 via the wide-angle optical system 30 and the optical scanner 22.

[0032] The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits light other than B light among the light from the posterior part (fundus) of the eye to be examined 12, and a beam splitter 58 that reflects G light and transmits light other than G light among the light transmitted through the beam splitter 64. The SLO unit 18 includes a beam splitter 60 that reflects R light and transmits light other than R light among the light transmitted through the beam splitter 58. The SLO unit 18 includes a beam splitter 62 that reflects IR light among the light transmitted through the beam splitter 60. The SLO unit 18 includes a B light detection element 70 that detects the B light reflected by the beam splitter 64, a G light detection element 72 that detects the G light reflected by the beam splitter 58, an R light detection element 74 that detects the R light reflected by the beam splitter 60, and an IR light detection element 76 that detects the IR light reflected by the beam splitter 62.

[0033] The light (reflected light reflected by the fundus) incident on the SLO unit 18 via the wide-angle optical system 30 and the optical scanner 22 is reflected by the beam splitter 64 and received by the B light detection element 70 in the case of B light, and is reflected by the beam splitter 58 and received by the G light detection element 72 in the case of G light. The incident light is transmitted through the beam splitter 58 and reflected by the beam splitter 60 and received by the R light detection element 74 in the case of R light. The incident light is transmitted through the beam splitters 58 and 60 and reflected by the beam splitter 62 and received by the IR light detection element 76 in the case of IR light. The image processor 17 operating under the control of the CPU 16A generates a UWF-SLO image using the signals detected by the B light detection element 70, the G light detection element 72, the R light detection element 74, and the IR light detection element 76.

[0034] The UWF-SLO image generated using the signal detected by the B light detection element 70 is called a B-UWF-SLO image (B-color fundus image). The UWF-SLO image generated using the signal detected by the G light detection element 72 is called a G-UWF-SLO image (G-color fundus image). The UWF-SLO image generated using the signal detected by the R light detection element 74 is called an R-UWF-SLO image (R-color fundus image). The UWF-SLO image generated using the signal detected by the IR light detection element 76 is called an IR-UWF-SLO image (IR fundus image). The UWF-SLO image includes these R-color fundus images, G-color fundus images, B-color fundus images to IR fundus images. Also included is the UWF-SLO image of fluorescence taken with fluorescence.

[0035] Further, the control device 16 controls the light sources 40, 42, and 44 to emit light simultaneously. By photographing the fundus of the eye 12 simultaneously with B light, G light, and R light, G-color fundus images, R-color fundus images, and B-color fundus images in which each position corresponds to each other are obtained. An RGB color fundus image is obtained from the G-color fundus image, the R-color fundus image, and the B-color fundus image. The control device 16 controls the light sources 42 and 44 to emit light simultaneously, and by photographing the fundus of the eye 12 simultaneously with G light and R light, G-color fundus images and R-color fundus images in which each position corresponds to each other are obtained. An RG color fundus image is obtained from the G-color fundus image and the R-color fundus image. Further, a full-color fundus image may be generated using the G-color fundus image, the R-color fundus image, and the B-color fundus image.

[0036] The wide-angle optical system 30 can set the field of view (FOV) of the fundus to an ultra-wide angle and photograph a region exceeding the equator from the posterior pole of the fundus of the eye 12.

[0037] The OCT system is realized by the control device 16, the OCT unit 20, and the imaging optical system 19 shown in FIG. 2. Since the OCT system includes the wide-angle optical system 30, similar to the photographing of the SLO fundus image described above, OCT photographing of the peripheral part of the fundus is possible. That is, the OCT photographing of a region exceeding the equator 178 from the posterior pole of the fundus of the eye 12 can be performed by the wide-angle optical system 30 having an ultra-wide angle as the field of view (FOV) of the fundus. OCT data of structures existing in the peripheral part of the fundus such as vortex veins can be acquired, and a tomographic image of the vortex vein and a 3D structure of the vortex vein can be obtained by image processing of the OCT data.

[0038] The OCT unit 20 includes a light source 20A, a sensor (detection element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

[0039] The light emitted from the light source 20A is branched by the first optical coupler 20C. One of the branched lights is made into parallel light by the collimating lens 20E as measurement light and then enters the imaging optical system 19. The measurement light is irradiated onto the fundus through the wide-angle optical system 30 and the pupil 27. The measurement light reflected by the fundus enters the OCT unit 20 through the wide-angle optical system 30 and enters the second optical coupler 20F through the collimating lens 20E and the first optical coupler 20C.

[0040] The other light emitted from the light source 20A and branched by the first optical coupler 20C enters the reference optical system 20D as reference light and enters the second optical coupler 20F through the reference optical system 20D.

[0041] These lights incident on the second optical coupler 20F, that is, the measurement light reflected by the fundus and the reference light, are interfered by the second optical coupler 20F to generate interference light. The interference light is received by the sensor 20B. The image processor 17 operating under the control of the image processing unit 206 generates OCT data detected by the sensor 20B. It is also possible to generate OCT images such as tomographic images and en-face images by the image processor 17 based on the OCT data.

[0042] Here, the OCT unit 20 can scan a predetermined range (for example, a rectangular range of 6 mm × 6 mm) in one OCT imaging. The predetermined range is not limited to 6 mm × 6 mm, and may be a square range of 12 mm × 12 mm or 23 mm × 23 mm, or a rectangular range such as 14 mm × 9 mm or 6 mm × 3.5 mm, and can be any rectangular range. Also, it may be a range of a circular diameter such as 6 mm, 12 mm, or 23 mm.

[0043] By using the wide-angle optical system 30, the ophthalmic device 110 can scan the region 12A with an internal irradiation angle of 200°. That is, by controlling the optical scanner 22, OCT imaging of a predetermined range including the vortex vein is performed. The ophthalmic device 110 can generate OCT data by the OCT imaging.

[0044] Therefore, the ophthalmic device 110 can generate a tomographic image (B-scan image) of the fundus including the vortex vein, which is an OCT image, OCT volume data including the vortex vein, and an en-face image (frontal image generated based on the OCT volume data), which is a cross-section of the OCT volume data. Needless to say, the OCT image includes an OCT image of the central part of the fundus (posterior pole of the eyeball where the macula and optic disc are present).

[0045] The OCT data (or the image data of the OCT image) is sent from the ophthalmic device 110 to the server 140 via the communication interface 16F and stored in the storage device 254.

[0046] In this embodiment, the light source 20A is an example of a wavelength-sweeping type SS-OCT (Swept-Source OCT), but various types of OCT systems such as SD-OCT (Spectral-Domain OCT) and TD-OCT (Time-Domain OCT) may also be used.

[0047] Next, with reference to FIG. 3, the electrical configuration of the server 140 will be described. As shown in FIG. 3, the server 140 includes a computer main body 252. The computer main body 252 has a CPU 262, a RAM 266, a ROM 264, and an input / output (I / O) port 268. Connected to the input / output (I / O) port 268 are a storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I / F) 258. The storage device 254 is composed of, for example, a non-volatile memory. The input / output (I / O) port 268 is connected to the network 130 via the communication interface (I / F) 258. Therefore, the server 140 can communicate with the ophthalmic device 110 and the viewer 150.

[0048] The image processing program shown in FIG. 6 is stored in the ROM 264 or the storage device 254.

[0049] The ROM 264 or the storage device 254 is an example of the "memory" of the technology of the present disclosure. The CPU 262 is an example of the "processor" of the technology of the present disclosure. The image processing program is an example of the "program" of the technology of the present disclosure.

[0050] The server 140 stores each data received from the ophthalmic device 110 in the storage device 254.

[0051] Various functions realized by the CPU 262 of the server 140 executing the image processing program will be described. As shown in FIG. 4, the image processing program includes a display control function, an image processing function, and a processing function. By the CPU 262 executing the image processing program having these functions, the CPU 262 functions as a display control unit 204, an image processing unit 206, and a processing unit 208.

[0052] Next, with reference to FIG. 5, the main flowchart of the image processing by the server 140 will be described. By the CPU 262 of the server 140 executing the image processing program, the image processing (image processing method) shown in the flowchart of FIG. 5 is realized.

[0053] First, in step 6000, the image processing unit 206 acquires OCT volume data including the choroid from the storage device 254.

[0054] In the next step 6100, the image processing unit 206 extracts choroidal blood vessels based on the OCT volume data and executes a stereoscopic image generation process (described later) for generating a stereoscopic image (3D image) of the vortex vein vessels.

[0055] Then, in step 6200, the processing unit 208 outputs the generated stereoscopic image (3D image) of the vortex vein vessels, specifically, stores it in the RAM 266 or the storage device 254, and ends the image processing.

[0056] Based on the user's instructions, a display screen including a three-dimensional image of the vortex vein (examples of the display screen are shown in FIGS. 10 to 15 described later) is generated by the display control unit 204. The generated display screen is output as an image signal by the processing unit 208 to the viewer 150. The display screen is displayed on the display of the viewer 150.

[0057] Next, the three-dimensional image generation process of the vortex vein in step 6100 will be described in detail with reference to FIG. 6.

[0058] In step 620 of FIG. 6, the image processing unit 206 extracts a region corresponding to the choroid from the OCT volume data 400 (see FIG. 8) acquired in step 6000, and based on the extracted region, extracts (acquires) the OCT volume data of the choroid portion. In the present embodiment, the OCT volume data 400D including the vortex vein and the choroidal blood vessels around the vortex vein will be described as an example. In this case, the choroidal blood vessels refer to the vortex vein and the choroidal blood vessels around the vortex vein.

[0059] Specifically, the image processing unit 206 extracts the OCT volume data 400D of the region below the retinal pigment epithelial cell layer 400R (Retinal Pigment Epithelium, hereinafter referred to as the RPE layer) from the OCT volume data scanned so as to include the vortex vein and the choroidal blood vessels around the vortex vein.

[0060] Specifically, first, the image processing unit 206 identifies the RPE layer 400R by performing image processing for identifying the boundary surface of each layer on the OCT volume data 400. Also, the highest intensity layer in the OCT volume data may be identified as the RPE layer 400R.

[0061] Then, the image processing unit 206 extracts pixel data of the choroid region in a predetermined range deeper than the RPE layer 400R (a region in a predetermined range farther from the RPE layer as viewed from the center of the eyeball) as OCT volume data 400D. Since the OCT volume data of the deep region may not be uniform, the image processing unit 206 may extract, as the OCT volume data 400D, the region between the RPE layer 400R and the bottom surface 400E obtained by the above-described image processing for specifying the boundary surface, as shown in FIG. 8. The choroid region in a predetermined range deeper than the RPE layer 400R is an example of the "choroid portion" of the technology of the present disclosure.

[0062] Through the above processing, the OCT volume data 400D for generating a three-dimensional image of the choroidal blood vessels is extracted.

[0063] In step 630, the image processing unit 206 performs a noise removal process, particularly a speckle noise process, as a first preprocessing for performing a first blood vessel extraction process (linear blood vessel extraction) on the OCT volume data 400D. This is a process for eliminating the influence of speckle noise and performing linear blood vessel extraction that correctly reflects the blood vessel shape. Examples of the speckle noise process include Gaussian blur processing.

[0064] In the next step 640, the image processing unit 206 performs a first blood vessel extraction process (linear blood vessel extraction) on the OCT volume data 400D subjected to the first preprocessing, thereby extracting the first choroidal blood vessels that are linear portions from the OCT volume data 400D. Thereby, a first three-dimensional image is generated. The first blood vessel extraction process will be described.

[0065] The image processing unit 206 performs image processing using, for example, an eigenvalue filter, a Gabor filter, etc., and extracts the region of linear blood vessels from the OCT volume data 400D. In the OCT volume data 400D, the blood vessel region is a low-luminance pixel (a pixel that looks black), and the region where the low-luminance pixels are continuous remains as the blood vessel portion.

[0066] In addition, the image processing unit 206 performs image processing such as deleting isolated regions that are not connected to surrounding blood vessels on the extracted linear blood vessel regions, median filtering, opening processing, and shrinking processing to delete noise regions.

[0067] Furthermore, the image processing unit 206 performs binarization processing on the pixel data of the linear blood vessel regions after noise processing.

[0068] By performing the first blood vessel extraction process described above, only the regions of the linear blood vessels remain from the OCT volume data 400D, and a three-dimensional image 680L of the linear blood vessels shown in FIG. 9 is generated. The image data of the three-dimensional image 680L of the linear blood vessels is stored in the RAM 266 by the processing unit 208. The linear blood vessels shown in FIG. 9 are an example of the "first choroidal blood vessels" of the technology of the present disclosure, and the three-dimensional image 680L of the linear blood vessels is an example of the "first three-dimensional image" of the technology of the present disclosure.

[0069] In step 650, as a second preprocessing for performing the second blood vessel extraction process (bulge portion extraction) on the OCT volume data 400D, the image processing unit 206 performs binarization processing on the OCT volume data 400D. By setting the binarization threshold to a predetermined threshold value that leaves the blood vessel bulge portion, the OCT volume data 400 D has black pixels for the blood vessel bulge portions and white pixels for the other portions.

[0070] Then, in step 660, the image processing unit 206 extracts the second choroidal blood vessels that are bulge portions from the OCT volume data by deleting noise regions in the binarized OCT volume data 400D. Thereby, a second three-dimensional image is generated. The noise regions are regions where the black pixel regions are isolated or regions corresponding to thin blood vessels. In order to delete such noise regions, the image processing unit 206 performs median filtering, opening processing, or shrinking processing on the binarized OCT volume data 400D to delete the noise regions.

[0071] In step 660, further, for surface smoothing of the extracted bulging portion, the image processing unit 206 may perform segmentation processing (image processing such as dynamic contour, graph cut, or U-net) on the OCT volume data with the noise region removed. Here, the “segmentation” mentioned here refers to image processing that performs binarization processing to separate the background and foreground for the image to be analyzed.

[0072] By performing such a second blood vessel extraction process, only the region of the bulging portion remains from the OCT volume data 400D, and a three-dimensional image 680B of the blood vessels in the bulging portion shown in FIG. 9 is generated. The image data of the three-dimensional image 680B of the blood vessels in the bulging portion is stored in the RAM 266 by the processing unit 208. The blood vessels in the bulging portion shown in FIG. 9 are an example of the “second choroidal blood vessels” of the technology of the present disclosure, and the three-dimensional image 680B of the blood vessels in the bulging portion is an example of the “second three-dimensional image” of the technology of the present disclosure.

[0073] Regarding the processing of steps 630 and 640 and the processing of steps 650 and 660, either one of the processes may be executed first, or they may proceed simultaneously.

[0074] When the processing of steps 630 and 640 and the processing of steps 650 and 660 are completed, in step 670, the image processing unit 206 reads out the three-dimensional image 680L of the linear blood vessels and the three-dimensional image 680B of the bulging portion from the RAM 266. Then, the two three-dimensional images are aligned, and by calculating the logical sum of the two images, the three-dimensional image 680L of the linear blood vessels and the three-dimensional image 680B of the bulging portion are combined. Thereby, a three-dimensional image 680M of the choroidal blood vessels including the vortex vein (see also FIG. 9) is generated. The image data of the three-dimensional image 680M is stored in the RAM 266 and the storage device 254 by the processing unit 208. The three-dimensional image 680M of the choroidal blood vessels including the vortex vein is an example of the “three-dimensional image of the choroidal blood vessels” of the technology of the present disclosure.

[0075] Next, a display screen for displaying a three-dimensional image (3D image) of the choroidal blood vessels including the generated vortex veins will be described. The display screen is generated by the display control unit 204 of the server 140 based on a user's instruction, and is output as an image signal to the viewer 150 by the processing unit 208. The viewer 150 displays the display screen on the display based on the image signal.

[0076] FIG. 10 shows a first display screen 500A. As shown in FIG. 10, the first display screen 500A has an information area 502 and an image display area 504A.

[0077] The information area 502 has a patient ID display field 512, a patient name display field 514, an age display field 516, a visual acuity display field 518, a right eye / left eye display field 520, and an axial length display field 522. In each display area from the patient ID display field 512 to the axial length display field 522, the viewer 150 displays each piece of information based on the information received from the server 140.

[0078] The image display area 504A is an area for displaying an image of the eye to be examined, etc. The following display fields are provided in the image display area 504A. Specifically, there are a UWF fundus image display field 542, an OCT volume data conceptual diagram display field 544, a tomographic image display field 546, and a three-dimensional image display field 548 of the choroidal blood vessels.

[0079] A comment field may be provided in the image display area 504A. The comment field is a remarks column where an ophthalmologist who is the user can arbitrarily input the results of observation or diagnosis.

[0080] In the UWF fundus image display field 542, a UWF-SLO fundus image 542B of the fundus of the eye to be examined taken by the ophthalmic device 110 is displayed. In the UWF-SLO fundus image 542B, a range 542A indicating the position where the OCT volume data was acquired is superimposed and displayed. When there are multiple pieces of OCT volume data associated with the UWF-SLO image, multiple ranges may be superimposed and displayed, and the user may select one position from the multiple ranges. FIG. 10 shows that a range including the vortex vein in the upper right of the UWF-SLO image was scanned.

[0081] In the OCT volume data conceptual diagram display field 544, an OCT volume data conceptual diagram (three-dimensional shape) 544B is displayed. The user designates a cross-section 544A to be displayed, for example, with a mouse or the like, in order to display a cross-sectional image in the depth direction in the OCT volume data conceptual diagram (three-dimensional shape). When the cross-section 544A is designated, an enface image corresponding to the designated cross-section 544A of the OCT volume data is generated and displayed in the tomographic image display field 546 as a tomographic image 546B.

[0082] In the three-dimensional image display field 548 of the choroidal blood vessels, a three-dimensional image (3D image) 548B of the choroidal blood vessels obtained by image processing the OCT volume data is displayed. The three-dimensional image 548B can be rotated about three axes by the user's operation. Also, in the three-dimensional image 548B of the choroidal blood vessels, a cross-section 548A is superimposed and displayed at a position corresponding to the cross-section 544A of the displayed tomographic image 546B.

[0083] According to the image display area 504A of the first display screen 500A, a three-dimensional image of the choroidal blood vessels can be recognized. By scanning a range including the vortex vein, the vortex vein and the surrounding choroidal blood vessels can be displayed in a three-dimensional image, and the user can obtain more information for diagnosis.

[0084] In addition, according to the image display area 504A, the position of the OCT volume data on the UWF-SLO image can be grasped.

[0085] Furthermore, according to the image display area 504A, a cross-section of the three-dimensional image can be arbitrarily selected, and by displaying the tomographic image, the user can obtain detailed information on the choroidal blood vessels.

[0086] In addition, in the three-dimensional display of the choroidal blood vessels according to the present embodiment, a three-dimensional display of the choroidal blood vessels can be performed without using OCT-A (OCT-angiography). It is possible to generate a three-dimensional image of the choroidal blood vessels without performing a complex and computationally intensive process of taking the difference of OCT volume data to obtain motion contrast. In OCT-A, a plurality of OCT volume data at different times are required to take the difference, but in the present embodiment, a three-dimensional image of the choroidal blood vessels can be generated based on one OCT volume data without performing motion contrast extraction processing.

[0087] FIG. 11 shows a second display screen 500B. Since the second display screen 500B has the same field as the field of the first display screen 500A, the same reference numerals are used for the same fields and their descriptions are omitted, and different parts will be described.

[0088] The display screen 500B has an information area 502 and an image display area 504B. Since the image display area 504B has the same field as the image display area 504A, the same reference numerals are used for the same fields and their descriptions are omitted, and different parts will be described. Specifically, the image display area 504B is different in that it has an en-face image display field 550 instead of the tomographic image display field 546 of the image display area 504A.

[0089] In the OCT volume data conceptual diagram 544B, for example, the user specifies the cross-section 544n to be displayed in order to display a cross-sectional image (enface image) perpendicular to the depth direction. When the cross-section 544n is specified, an en-face image 550B corresponding to the cross-section 544n is generated based on the OCT volume data.

[0090] In the en-face image display field 550, the en-face image 550B corresponding to the cross-section 544n is displayed. For the en-face image 550B displayed in the en-face image display field 550, highlighting such as displaying the contours 550A of the first blood vessel and the second blood vessel extracted in steps 640 and 660 of FIG. 6 in color (for example, red) may be performed. Also, the position of the cross-section 544n may be displayed numerically (in FIG. 11, it is denoted as "nth layer").

[0091] In the three-dimensional image display field 548 of the choroidal blood vessels, a cross-section 548n corresponding to the cross-section 544n is displayed superimposed on the three-dimensional image 548B.

[0092] According to the display screen 500B of the second display screen 500B, a three-dimensional (3D) image of the selected vortex vein at the position of the vortex vein can be recognized. Furthermore, according to the display screen 500B, a three-dimensional image of the choroidal blood vessels can be recognized. By scanning the range including the vortex vein, the vortex vein and the surrounding choroidal blood vessels can be displayed in a three-dimensional image, and the user can obtain more information for diagnosis.

[0093] Also, according to the image display area 504B, the position of the OCT volume data on the UWF-SLO image can be grasped.

[0094] Furthermore, according to the image display area 504B, the enface plane of the three-dimensional image can be arbitrarily selected, and by displaying the enface image, the user can obtain detailed information regarding the depth direction of the choroidal blood vessels.

[0095] Figure 12 shows a third display screen 500C. Since the third display screen 500C has the same fields as those of the first display screen 500A, the same reference numerals are used for the same fields and their descriptions are omitted, and only the different parts will be described.

[0096] The third display screen 500C has an information area 502 and an image display area 504C. Since the image display area 504C has the same fields as those of the image display area 504A, the same reference numerals are used for the same fields and their descriptions are omitted, and only the different parts will be described.

[0097] Specifically, the image display area 504C is different from the display screens 500A and 500B in that it does not have the tomographic image display field 546 of the image display area 504A.

[0098] In addition, the image display area 504C is different in that instead of the OCT volume data conceptual diagram display field 544, it has two OCT volume data conceptual diagram display fields 544P and 544Q for recognizing OCT volume data from two different angles (a first angle and a second angle). A conceptual diagram 544PB indicating that the OCT volume data is drawn at an angle of 45 degrees is displayed in the OCT volume data conceptual diagram display field 544P. A conceptual diagram 544QB indicating that the OCT volume data is drawn as viewed from directly above is displayed in the OCT volume data conceptual diagram display field 544Q. The OCT volume data conceptual diagram display fields 544P and 544Q can be specified at any angle by a user operation.

[0099] Furthermore, the choroidal vessel stereoscopic image display field 548 in the image display area 504C has choroidal vessel stereoscopic image sections 548D1 and 548D2 for displaying choroidal vessel stereoscopic images 548D1B and 548D2B viewed from two different angles specified in the OCT volume data conceptual diagram display fields 544P and 544Q.

[0100] The two different angles (first angle and second angle) may be preset directions or may be determined by AI. Note that the number of angles is not limited to two, and may be three or more.

[0101] The choroidal vessel stereoscopic images 548D1B, 548D2B may be moved individually or in conjunction with each other in any direction selected by the user, or may be clicked to be enlarged and displayed in a separate window.

[0102] The image display area 504C of the third display screen 500C allows the user to view stereoscopic images of the choroidal vessels from multiple angles, particularly the choroidal vessels including the vortex veins, from an angle that allows the vortex veins to be viewed from the scleral side.

[0103] 13 shows a fourth display screen 500D. The fourth display screen 500D has the same fields as the third display screen 500C, so the same fields are denoted by the same reference numerals and their description is omitted, and only the differences will be described.

[0104] The fourth display screen 500D has an information area 502 and an image display area 504D. Since the image display area 504D has the same fields as the image display area 504C, the same fields are given the same reference numerals and their explanations are omitted, and only the differences will be explained.

[0105] To visualize a wider range of choroidal blood vessels, an example is shown in FIG. 13 where a plurality of OCT volume data are synthesized and the choroidal blood vessels are visualized using the synthesized OCT volume data. In the UWF-SLO image display field 542, the acquired ranges of two adjacent OCT volume data that partially overlap are superimposed and displayed on the UWF-SLO image 542B as ranges 542K and 542L.

[0106] The stereoscopic image display field 548 of the choroidal blood vessels in the image display area 504D has stereoscopic image display sections 548KL1 and 548KL2 for displaying stereoscopic images 548KL1B and 548KL2B obtained based on adjacent OCT volume data. Two stereoscopic images 548K1B and 548L1B of the choroidal blood vessels are stereoscopic images viewed from two different angles specified in the OCT volume data conceptual diagram display fields 544P and 544Q, similar to FIG. 12.

[0107] According to the image display area 504D of the fourth display screen 500D, a stereoscopic image of the choroidal blood vessels is created and displayed based on a plurality of adjacent OCT volume data. Therefore, a stereoscopic image of a wider range of choroidal blood vessels can be confirmed compared to the stereoscopic image of the choroidal blood vessels created based on one OCT volume data. Also, the user can confirm the stereoscopic image of the wider range of choroidal blood vessels from a plurality of different angles. In particular, in the stereoscopic image of the choroidal blood vessels including the vortex vein, it can be confirmed at an angle as if viewing the vortex vein from the scleral side.

[0108] FIG. 14 shows the fifth display screen 500E. Since the fifth display screen 500E has the same fields as those of the first display screen 500A, the same reference numerals are given to the same fields and their descriptions are omitted, and the different parts will be described.

[0109] The fifth display screen 500E has an information area 502 and an image display area 504E. Since the image display area 504E has the same fields as the image display area 504A, the same codes are assigned to the same fields and their descriptions are omitted, and the different parts are described.

[0110] Specifically, the image display area 504E is different in that it does not have the tomographic image display fields 544 and 546 of the image display area 504A. Also, the image display area 504E is different in that instead of the three-dimensional image display field 548 of the vortex vein in the image display area 504A, it has a three-dimensional image display field 548E of the choroidal blood vessels suitable for follow-up observation described below.

[0111] The three-dimensional image display field 548E of the choroidal blood vessels is a field that displays three-dimensional images of a plurality of choroidal blood vessels in time series by OCT volumes obtained by photographing the fundus of the same subject at different timings.

[0112] In the example shown in FIG. 14, specifically, the three-dimensional image display field 548E of the choroidal blood vessels has three three-dimensional image display sections 548E1, 548E2, and 548E3 from left to right in the order of the older fundus photographing dates. The three-dimensional image display sections 548E1, 548E2, and 548E3 have photographing date display portions 548D1, 548D2, and 548D3 that display the fundus photographing dates. Specifically, a three-dimensional image 548E1B obtained by photographing the fundus on March 12, 2021 is displayed in the three-dimensional image display section 548E1. A three-dimensional image 548E2B obtained by photographing the fundus on June 15, 2021 is displayed in the three-dimensional image display section 548E2. A three-dimensional image 548E3B obtained by photographing the fundus on September 12, 2021 is displayed in the three-dimensional image display section 548E3. The three-dimensional image display field 548E is not limited to three, and two or three or more three-dimensional images may be displayed.

[0113] The three-dimensional image display field 548E of the choroidal blood vessels has a return button 548R that gives an instruction to display a three-dimensional image with an older shooting date than the currently displayed three-dimensional image of the vortex vein, and a forward button 548F that gives an instruction to display a three-dimensional image with a newer shooting date than the currently displayed three-dimensional image. When the return button 548R is pressed, a three-dimensional image with an older shooting date than the currently displayed three-dimensional image is displayed. When the forward button 548F is pressed, a three-dimensional image with a newer shooting date than the currently displayed three-dimensional image is displayed.

[0114] According to the fifth display screen 500E, three-dimensional images of a plurality of choroidal blood vessels of the subject can be displayed in chronological order. Therefore, the user can, for example, confirm the temporal change in the thickness of the vortex vein and confirm the appropriate treatment method required at the current time.

[0115] FIG. 15 shows a sixth display screen 500F. As shown in FIG. 15, the first display screen 500F has an information area 502F and an image display area 504F.

[0116] The information area 502F has patient information display fields 502P, 502Q, 502R for displaying information of a plurality of, for example, three patients. Each of the patient information display fields 502P, 502Q, 502R has a patient number display field, a gender display field, an age display field, a right eye / left eye display field, a visual acuity display field, and a disease name display field for each patient. The patient to be displayed can be specified by the user specifying the patient ID on a patient identification screen (not shown). For example, patients having the same disease can be specified, or patients having the same gender or age can be specified. Then, the display control unit 204 of the server 140 reads out the three-dimensional image and the UWF-SLO image corresponding to the specified patient ID and generates the display screen 500F.

[0117] The image display area 504F includes image display fields 548P, 548Q, and 548R corresponding to each patient in the information area 502F. The image display field 548P displays the patient number 542PA, the UWF-SLO image 542PB, and the three-dimensional image 548PB of the choroidal blood vessels. In FIG. 15, an example is shown in which the patient number 542PA, the UWF-SLO image 542PB, and the three-dimensional image 548PB of the choroidal blood vessels are displayed in this order from the bottom of the page, but it is not limited to this, and the display positions of the respective images may be changed according to the user's settings. In addition to the patient number 542PA, the UWF-SLO image 542PB, and the three-dimensional image 548PB of the choroidal blood vessels, the image display field 548P may also display the attribute information of the patient that the user wants to compare and the fundus images of the same part (for example, around the optic disc, around the macula, etc.). Similarly, the image display field 548Q displays the patient number 542QA, the UWF-SLO image 542QB, and the three-dimensional image 548QB of the choroidal blood vessels. The image display field 548R displays the patient number 542RA, the UWF-SLO image 542RB, and the three-dimensional image 548RB of the choroidal blood vessels. Note that the information area 502F can display images of the eyes to be examined of not only three patients but also two or more patients.

[0118] According to the sixth display screen 500F, since it includes three-dimensional images of the choroidal blood vessels of a plurality of patients, the user can compare the three-dimensional images of the choroidal blood vessels of a plurality of patients without switching the screen.

[0119] The first display screen 500A to the sixth display screen 500F may be individually and selectively displayed, or may be displayed in order.

[0120] As described above, in this embodiment, the choroidal blood vessels are extracted based on the OCT volume data including the choroid, and a three-dimensional image of the choroidal blood vessels is generated, so that the choroid can be three-dimensionally visualized.

[0121] Further, in the present embodiment, a three-dimensional image of the choroidal blood vessels is generated without using OCT-A (OCT-angiography) based on the OCT volume data. Therefore, in the present embodiment, a three-dimensional image of the choroidal blood vessels can be generated without performing a complex and computationally intensive process of taking the difference of the OCT volume data and extracting the motion contrast, and the computational amount can be reduced.

[0122] In the above embodiment, the image processing (FIG. 5) is executed by the server 140, but the technology of the present disclosure is not limited thereto, and it may be executed by the ophthalmic apparatus 110, the viewer 150, or an additional image processing apparatus further provided in the network 130.

[0123] In the present disclosure, each component (apparatus, etc.) may exist alone or in two or more as long as there is no contradiction.

[0124] In each of the examples described above, the case where image processing is realized by a software configuration using a computer is illustrated, but the technology of the present disclosure is not limited thereto. For example, instead of a software configuration using a computer, image processing may be executed only by a hardware configuration such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). Part of the image processing may be executed by a software configuration, and the remaining processing may be executed by a hardware configuration.

[0125] As described above, since the technology of the present disclosure includes the case where image processing is realized by a software configuration using a computer and the case where it is not, the following technology is included.

[0126] (First technology) An acquisition unit that acquires OCT volume data including the choroid, A generation unit that extracts choroidal blood vessels based on the OCT volume data and generates a three-dimensional image of the choroidal blood vessels, An image processing apparatus comprising

[0127] (Second technology) a step of an acquisition unit acquiring OCT volume data including the choroid, a step of a generation unit extracting choroidal blood vessels based on the OCT volume data and generating a three-dimensional image of the choroidal blood vessels, An image processing method including The image processing unit 206 is an example of the "acquisition unit" and "generation unit" of the technology of the present disclosure. The following technology is proposed from the above disclosure content.

[0128] (Third technology) A computer program product for image processing, wherein the computer program product comprises a computer-readable storage medium that is not itself a transient signal, a program is stored in the computer-readable storage medium, the program causes a computer to acquire OCT volume data including the choroid, extract choroidal blood vessels based on the OCT volume data and generate a three-dimensional image of the choroidal blood vessels, and execute A computer program product. The server 140 is an example of the "computer program product" of the technology of the present disclosure.

[0129] Each of the image processings described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within the scope not departing from the gist.

[0130] All documents, patent applications, and technical standards described in this specification are incorporated herein by reference as if each individual document, patent application, and technical standard were specifically and individually indicated to be incorporated by reference.

[0131] Also, the disclosure of Japanese Patent Application No. 2021-026196 filed on February 22, 2021 is incorporated herein by reference in its entirety.

Claims

1. An image processing method performed by a processor, comprising: obtaining OCT volume data including the choroid; performing a linear blood vessel extraction process and a binarization process on the OCT volume data, extracting first choroidal blood vessels from the obtained first processed image, and generating a first three-dimensional image; performing a binarization process on the OCT volume data, extracting second choroidal blood vessels from the obtained second processed image, and generating a second three-dimensional image; generating a three-dimensional image of the choroidal blood vessels by synthesizing the first three-dimensional image and the second three-dimensional image. An image processing method comprising the above steps.

2. The OCT volume data is obtained by scanning a region including at least the vortex veins of the fundus oculi. The image processing method according to Claim 1.

3. The method further comprises the step of extracting choroidal OCT volume data of the choroidal portion from the OCT volume data, wherein the step of generating the three-dimensional image generates the three-dimensional image based on the choroidal OCT volume data. The image processing method according to Claim 1 or Claim 2, characterized by the above.

4. An image processing apparatus comprising a memory and a processor connected to the memory, wherein the processor obtains OCT volume data including the choroid; performs a linear blood vessel extraction process and a binarization process on the OCT volume data, extracts first choroidal blood vessels from the obtained first processed image, and generates a first three-dimensional image; performs a binarization process on the OCT volume data, extracts second choroidal blood vessels from the obtained second processed image, and generates a second three-dimensional image; generates a three-dimensional image of the choroidal blood vessels by synthesizing the first three-dimensional image and the second three-dimensional image. The image processing apparatus executes the above steps.

5. A program for causing a computer to obtain OCT volume data including the choroid; perform a linear blood vessel extraction process and a binarization process on the OCT volume data, extract first choroidal blood vessels from the obtained first processed image, and generate a first three-dimensional image; perform a binarization process on the OCT volume data, extract second choroidal blood vessels from the obtained second processed image, and generate a second three-dimensional image; generate a three-dimensional image of the choroidal blood vessels by synthesizing the first three-dimensional image and the second three-dimensional image. The program causes the computer to execute the above steps.

Images