Image processing method

Abstract

To analyze vortex veins from an ocular fundus image.SOLUTION: The present invention provides an image processing method to be executed by a processor, which detects positions of a plurality of vortex veins from an ocular fundus image of an eye to be examined, and calculates a distribution center of the detected positions of the plurality of vortex veins.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The technology of the present disclosure relates to an image processing method.

Background Art

[0002] It is desired to analyze the vortex veins from fundus images (U.S. Patent No. 8,636,364).

Summary of the Invention

[0003] An image processing method according to a first aspect of the technology of the present disclosure is an image processing method performed by a processor, which detects positions of a plurality of vortex veins from a fundus image of an eye to be examined and calculates a distribution center of the detected positions of the plurality of vortex veins.

[0004] An image processing apparatus according to a second aspect of the technology of the present disclosure includes a memory and a processor connected to the memory, and the processor detects positions of a plurality of vortex veins from a fundus image of an eye to be examined and calculates a distribution center of the detected positions of the plurality of vortex veins.

[0005] A program according to a third aspect of the technology of the present disclosure causes a computer to detect positions of a plurality of vortex veins from a fundus image of an eye to be examined and calculate a distribution center of the detected positions of the plurality of vortex veins.

Brief Description of the Drawings

[0006] [Figure 1] It is a block diagram of an ophthalmic system 100. [Figure 2] It is a schematic configuration diagram showing the overall configuration of an ophthalmic device 110. [Figure 3A] It is a first diagram showing a photographing range of the fundus of an eye to be examined 12 by an ophthalmic device 110. [Figure 3B] It is a second diagram showing a photographing range of the fundus of an eye to be examined 12 by an ophthalmic device 110 and is an image of the fundus obtained by the photographing. [Figure 3C] It is a third diagram showing a photographing range of the fundus of an eye to be examined 12 by an ophthalmic device 110. [Figure 4] This is a block diagram showing the configuration of server 140. [Figure 5] This is a functional block diagram of CPU 162 of server 140. [Figure 6] This is a flowchart of the image processing performed by server 140. [Figure 7] Figure 7 is a flowchart of the projection process in step 210 of Figure 6. [Figure 8] Figure 8 is a flowchart of the VV center calculation process in step 212 of Figure 6. [Figure 9] Figure 9 shows a flowchart of the feature calculation process in step 214 of Figure 6. [Figure 10A] This figure shows a color fundus image G1. [Figure 10B] This is a diagram of choroidal vascular image G2. [Figure 10C] This diagram shows how VV4 was detected from four vortex veins VV1. [Figure 11] This figure shows an eyeball model on a virtual sphere (virtual eyeball surface) with the optic nerve head ONH at longitude 0 degrees, the optic nerve head ONH, the macula M, and the positions of the five VVs Pvv1 to Pvv5 projected onto it. [Figure 12A] This diagram shows the vectors VC1 to VC5 from the center C of the sphere to the positions Pvv1 to Pvv5 of each VV, and is a view of the eyeball model from an oblique angle. [Figure 12B] This diagram shows the vectors VC1 to VC5 from the center C of the sphere to the positions Pvv1 to Pvv5 of each VV, and is a side view of the eyeball model. [Figure 12C] This diagram shows the vectors VC1 to VC5 from the center C of the sphere to the positions Pvv1 to Pvv5 of each VV, and is a top-down view of the eyeball model. [Figure 13A] The composite vector VCT, calculated by combining vectors VC1 to VC5 for each VV, is shown, and the image is a view of the eyeball model from an oblique angle. [Figure 13B] The composite vector VCT, calculated by combining vectors VC1 to VC5 for each VV, is shown, and the image is a side view of the eyeball model. [Figure 13C] It shows the composite vector VCT calculated by synthesizing vectors VC1 to VC5 to each VV, and is a view of the eyeball model from above. [Figure 14A] It shows the normalized composite vector VCTN obtained by normalizing so that the length of the composite vector VCT becomes the length of the vector VC1 (for example, 1), and is a view of the eyeball model from an oblique angle. [Figure 14B] It shows the normalized composite vector VCTN obtained by normalizing so that the length of the composite vector VCT becomes the length of the vector VC1 (for example, 1), and is a view of the eyeball model from the side. [Figure 14C] It shows the normalized composite vector VCTN obtained by normalizing so that the length of the composite vector VCT becomes the length of the vector VC1 (for example, 1), and is a view of the eyeball model from above. [Figure 15] It is a diagram explaining the concept of a feature amount indicating the positional relationship between the optic nerve head ONH and the center of VV. [Figure 16] It is a diagram showing a feature amount table for memorizing a feature amount indicating the positional relationship between the optic nerve head ONH and the center of VV. [Figure 17] It is a diagram showing the first display screen 300A for displaying the object and the feature amount. [Figure 18] It is a diagram showing the second display screen 300B for displaying the object and the feature amount.

Embodiments for Carrying Out the Invention

[0007] Hereinafter, embodiments of the technology of the present disclosure will be described in detail with reference to the drawings.

[0008] Referring to FIG. 1, the configuration of the ophthalmic system 100 will be described. As shown in FIG. 1, the ophthalmic system 100 includes an ophthalmic device 110, an axial length measuring device 120, a management server device (hereinafter referred to as the "server") 140, and an image display device (hereinafter referred to as the "viewer") 150. The ophthalmic device 110 acquires fundus images. The axial length measuring device 120 measures the axial length of a patient's eye. The server 140 stores the fundus images obtained by photographing the patient's fundus by the ophthalmic device 110 corresponding to the patient's ID. The viewer 150 displays medical information such as the fundus images acquired from the server 140.

[0009] The ophthalmic device 110, the axial length measuring device 120, the server 140, and the viewer 150 are interconnected via a network 130.

[0010] Next, referring to FIG. 2, the configuration of the ophthalmic device 110 will be described.

[0011] For convenience of explanation, a scanning laser ophthalmoscope is referred to as "SLO". Also, an optical coherence tomography is referred to as "OCT".

[0012] When the ophthalmic device 110 is installed on a horizontal plane, the horizontal direction is the "X direction", the vertical direction with respect to the horizontal plane is the "Y direction", and the direction connecting the center of the pupil of the anterior segment of the examined eye 12 and the center of the eyeball is the "Z direction". Therefore, the X direction, the Y direction, and the Z direction are perpendicular to each other.

[0013] The ophthalmic device 110 includes a photographing device 14 and a control device 16. The photographing device 14 includes an SLO unit 18, an OCT unit 20, and a photographing optical system 19, and acquires a fundus image of the fundus of the examined eye 12. Hereinafter, the two-dimensional fundus image acquired by the SLO unit 18 is referred to as an SLO image. Also, a tomographic image or an en-face image of the retina created based on the OCT data acquired by the OCT unit 20 is referred to as an OCT image.

[0014] The control unit 16 includes a computer having a CPU (Central Processing Unit) 16A, RAM (Random Access Memory) 16B, ROM (Read-Only memory) 16C, and input / output (I / O) ports 16D.

[0015] The control device 16 includes an input / display device 16E connected to the CPU 16A via an I / O port 16D. The input / display device 16E has a graphic user interface that displays an image of the eye under examination 12 and accepts various instructions from the user. A touch panel display is an example of a graphic user interface.

[0016] Furthermore, the control device 16 includes an image processor 16G connected to the I / O port 16D. The image processor 16G generates an image of the eye under examination 12 based on the data obtained by the imaging device 14. Alternatively, the image processor 16G may be omitted, and the CPU 16A may generate an image of the eye under examination 12 based on the data obtained by the imaging device 14. The control device 16 also includes a communication interface (I / F) 16F connected to the I / O port 16D. The ophthalmic device 110 is connected to the axial length measuring instrument 120, the server 140, and the viewer 150 via the communication interface (I / F) 16F and the network 130.

[0017] As described above, in Figure 2, the control device 16 of the ophthalmic device 110 is equipped with an input / display device 16E, but the technology of this disclosure is not limited thereto. For example, the control device 16 of the ophthalmic device 110 may not be equipped with an input / display device 16E, but may be equipped with a separate input / display device that is physically independent of the ophthalmic device 110. In this case, the display device includes an image processing processor unit that operates under the control of the CPU 16A of the control device 16. The image processing processor unit may display an SLO image or the like based on an image signal that the CPU 16A has instructed to output.

[0018] The imaging device 14 operates under the control of the CPU 16A of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The imaging optical system 19 includes a first optical scanner 22, a second optical scanner 24, and a wide-angle optical system 30.

[0019] The first optical scanner 22 scans the light emitted from the SLO unit 18 in two dimensions in the X and Y directions. The second optical scanner 24 scans the light emitted from the OCT unit 20 in two dimensions in the X and Y directions. The first optical scanner 22 and the second optical scanner 24 can be any optical elements capable of deflecting a light beam, such as polygon mirrors or galvanometer mirrors. A combination of these may also be used.

[0020] The wide-angle optical system 30 includes an objective optical system (not shown in Figure 2) having a common optical system 28, and a combining unit 26 that combines light from the SLO unit 18 and light from the OCT unit 20.

[0021] Furthermore, the objective optical system of the common optical system 28 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a reflective-refractory optical system combining a concave mirror and lenses. By using a wide-angle optical system using an elliptical mirror or a wide-angle lens, it becomes possible to photograph not only the central part of the fundus where the optic nerve head and macula are located, but also the retina in the equatorial region of the eyeball and the peripheral part of the fundus where the vortex veins are located.

[0022] When using a system that includes an elliptical mirror, refer to International Publication WO2016 / 103484 or This may also be a configuration using a system with an elliptical mirror as described in International Publication WO2016 / 103489. Each of the disclosures in International Publication WO2016 / 103484 and International Publication WO2016 / 103489 is incorporated herein by reference in their entirety.

[0023] The wide-angle optical system 30 enables observation of the fundus with a wide field of view (FOV) 12A. The FOV 12A indicates the range that can be captured by the imaging device 14. The FOV 12A can be expressed as the field of view angle. In this embodiment, the field of view angle can be defined by the internal illumination angle and the external illumination angle. The external illumination angle is the illumination angle of the light beam irradiated from the ophthalmic device 110 onto the eye under examination 12, defined with respect to the pupil 27. The internal illumination angle is the illumination angle of the light beam irradiated onto the fundus, defined with respect to the center O of the eyeball. The external illumination angle and the internal illumination angle are in a corresponding relationship. For example, if the external illumination angle is 120 degrees, the internal illumination angle corresponds to approximately 160 degrees. In this embodiment, the internal illumination angle is set to 200 degrees.

[0024] An internal illumination angle of 200 degrees is an example of a "predetermined value" in the technology of this disclosure.

[0025] Here, SLO fundus images obtained by imaging with an internal illumination angle of 160 degrees or more are referred to as UWF-SLO fundus images. UWF stands for UltraWide Field. Of course, it is possible to obtain SLO images that are not UWF by imaging the fundus with an internal illumination angle of less than 160 degrees.

[0026] The SLO system is implemented by the control device 16, SLO unit 18, and imaging optical system 19 shown in Figure 2. Because the SLO system is equipped with a wide-angle optical system 30, it is possible to perform fundus imaging with a wide FOV 12A, specifically, to image the region of the fundus of the eye under examination 12, from the posterior pole to beyond the equator.

[0027] Figure 3A will be used to explain the equatorial region 174. The eyeball (examined eye 12) has a diameter of approximately 24 mm. The eye is a spherical structure with its center 170. The straight line connecting its anterior pole 175 and posterior pole 176 is called the eyeball axis 172, and the lines where planes perpendicular to the eyeball axis 172 intersect the surface of the eyeball are called lines of latitude, with the longest line of latitude being the equator 174. The part of the retina and choroid corresponding to the position of the equator 174 is called the equatorial region 178. The equatorial region 178 is part of the peripheral part of the fundus.

[0028] The ophthalmic device 110 can image a region with an internal illumination angle of 200°, using the center 170 of the eyeball of the eye under examination 12 as the reference position. Note that an internal illumination angle of 200° corresponds to an external illumination angle of 167°, with the pupil of the eyeball of the eye under examination 12 as the reference point. In other words, the wide-angle optical system 80 emits laser light from the pupil with an external illumination angle of 167° and images the fundus region with an internal illumination angle of 200°.

[0029] Figure 3B shows an SLO image 179 obtained by scanning with an ophthalmic device 110 capable of scanning with an internal illumination angle of 200°. As shown in Figure 3B, the equatorial region 174 corresponds to an internal illumination angle of 180°, and in the SLO image 179, the area indicated by the dotted line 178a corresponds to the equatorial region 178. Thus, the ophthalmic device 110 can capture the peripheral fundus region from the posterior pole, including the posterior pole 176, to beyond the equatorial region 178 in a single image (in one capture or one scan). In other words, the ophthalmic device 110 can capture the fundus from the central to the peripheral region in a single image.

[0030] Figure 3C, which will be described in more detail later, shows the positional relationship between the choroid 12M and the vortex veins 12V1 and V2 in the eyeball.

[0031] In Figure 3C, the reticular pattern represents the choroidal vessels of choroid 12M. Choroidal vessels These veins circulate blood throughout the choroid. Then, blood flows out of the eyeball through multiple vortex veins present in the eye under examination 12. Figure 3C shows the superior vortex vein V1 and the inferior vortex vein V2, which are located on one side of the eyeball. Vortex veins are often located near the equator 178. Therefore, to image the vortex veins in the eye under examination 12 and the choroidal blood vessels surrounding the vortex veins, the ophthalmic device 110, which has an internal illumination angle of 200° and can scan a wide area around the fundus, is used.

[0032] The configuration of the ophthalmic apparatus 110 equipped with the wide-angle optical system 30 may be the configuration described in international application PCT / EP2017 / 075852. The disclosure of international application PCT / EP2017 / 075852 (international publication WO2018 / 069346), filed internationally on 10 October 2017, is incorporated herein by reference in its entirety.

[0033] Returning to Figure 2, the SLO unit 18 comprises multiple light sources, for example, a B-light (blue light) light source 40, a G-light (green light) light source 42, an R-light (red light) light source 44, and an IR-light (infrared light (e.g., near-infrared light)) light source 46, and optical systems 48, 50, 52, 54, 56 that reflect or transmit the light from light sources 40, 42, 44, and 46 into a single optical path. Optical systems 48, 50, and 56 are mirrors, and optical systems 52 and 54 are beam splitters. The B-light is reflected by optical system 48, transmitted through optical system 50, and reflected by optical system 54; the G-light is reflected by optical systems 50 and 54; the R-light is transmitted through optical systems 52 and 54; and the IR-light is reflected by optical systems 56 and 52, each to be guided into a single optical path.

[0034] The SLO unit 18 is configured to allow switching between combinations of light sources that emit or emit laser light of different wavelengths, such as modes that emit G light, R light, and B light, and modes that emit infrared light. In the example shown in Figure 2, there are four light sources: a B light (blue light) light source 40, a G light light source 42, an R light light source 44, and an IR light light source 46, but the technology of this disclosure is not limited thereto. For example, the SLO unit 18 may further include a white light light source and emit light in various modes, such as a mode that emits only white light.

[0035] Light incident from the SLO unit 18 into the imaging optical system 19 is scanned in the X and Y directions by the first optical scanner 22. The scanned light passes through the wide-angle optical system 30 and the pupil 27 and illuminates the posterior segment of the eye under examination 12. The reflected light reflected by the fundus passes through the wide-angle optical system 30 and the first optical scanner 22 and is incident on the SLO unit 18.

[0036] The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits other light from the posterior segment (e.g., fundus) of the eye under examination 12, and a beam splitter 58 that reflects G light and transmits other light from the light transmitted through beam splitter 64. The SLO unit 18 also includes a beam splitter 60 that reflects R light and transmits other light from the light transmitted through beam splitter 58. The SLO unit 18 also includes a beam splitter 62 that reflects IR light from the light transmitted through beam splitter 60.

[0037] The SLO unit 18 is equipped with multiple photodetectors to correspond to multiple light sources. The SLO unit 18 includes a B-light detection element 70 for detecting B-light reflected by the beam splitter 64, and a G-light detection element 72 for detecting G-light reflected by the beam splitter 58. The SLO unit 18 also includes an R-light detection element 74 for detecting R-light reflected by the beam splitter 60, and an IR-light detection element 76 for detecting IR-light reflected by the beam splitter 62.

[0038] Light incident on the SLO unit 18 via the wide-angle optical system 30 and the first optical scanner 22 (i.e., reflected light reflected by the fundus) is, in the case of B light, reflected by the beam splitter 64 and received by the B light detection element 70, and in the case of G light, transmitted through the beam splitter 64, reflected by the beam splitter 58 and received by the G light detection element 72. In the case of red light, the light passes through beam splitters 64 and 58, is reflected by beam splitter 60, and is received by red light detection element 74. In the case of irradiant light, the incident light passes through beam splitters 64, 58, and 60, is reflected by beam splitter 62, and is received by irradiant light detection element 76. The image processor 16G, operating under the control of CPU 16A, generates a UWF-SLO image using the signals detected by the blue light detection element 70, the green light detection element 72, the red light detection element 74, and the irradiant light detection element 76.

[0039] UWF-SLO images (also called UWF fundus images or original images, as described later) include UWF-SLO images obtained when the fundus is captured in green (G) color (G-color fundus images) and UWF-SLO images obtained when the fundus is captured in red (R) color (R-color fundus images). UWF-SLO images also include UWF-SLO images obtained when the fundus is captured in blue (B) color (B-color fundus images) and UWF-SLO images obtained when the fundus is captured in infrared (IR) (IR fundus images).

[0040] Furthermore, the control device 16 controls the light sources 40, 42, and 44 to emit light simultaneously. By simultaneously photographing the fundus of the eye under examination 12 with B light, G light, and R light, G-color fundus images, R-color fundus images, and B-color fundus images are obtained where each position corresponds to the others. An RGB color fundus image is obtained from the G-color fundus images, R-color fundus images, and B-color fundus images. The control device 16 controls the light sources 42 and 44 to emit light simultaneously, and by simultaneously photographing the fundus of the eye under examination 12 with G light and R light, G-color fundus images and R-color fundus images are obtained where each position corresponds to the others. An RG color fundus image is obtained from the G-color fundus images and R-color fundus images.

[0041] Specifically, UWF-SLO images include B-color fundus images, G-color fundus images, R-color fundus images, IR fundus images, RGB-color fundus images, and RG-color fundus images. Each UWF-SLO image data, along with patient information input via the input / display device 16E, is transmitted from the ophthalmic device 110 to the server 140 via the communication interface (I / F) 16F. Each UWF-SLO image data and patient information are stored in the memory 164 in correspondence. Patient information includes, for example, patient name ID, name, age, visual acuity, and distinction between right and left eye. Patient information is input by the operator via the input / display device 16E.

[0042] The OCT system is implemented by the control device 16, OCT unit 20, and imaging optical system 19 shown in Figure 2. The OCT system is equipped with a wide-angle optical system 30, enabling fundus imaging with a wide FOV 12A, similar to the acquisition of SLO fundus images described above. The OCT unit 20 includes a light source 20A, a sensor (detection element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

[0043] Light emitted from the light source 20A is split by the first optical coupler 20C. One of the split beams of light is made parallel by the collimating lens 20E and then incident on the imaging optical system 19 as measurement light. The measurement light is scanned in the X and Y directions by the second optical scanner 24. The scanning light is irradiated onto the fundus of the eye via the wide-angle optical system 30 and the pupil 27. The measurement light reflected by the fundus of the eye is incident on the OCT unit 20 via the wide-angle optical system 30 and the second optical scanner 24, and then incident on the second optical coupler 20F via the collimating lens 20E and the first optical coupler 20C.

[0044] The other beam of light emitted from the light source 20A and branched by the first optical coupler 20C is incident on the reference optical system 20D as reference light, and then, via the reference optical system 20D, is incident on the second optical coupler 20F.

[0045] These lights incident on the second optical coupler 20F, i.e., the measurement light reflected from the fundus and the reference light, interfere with each other at the second optical coupler 20F to generate interference light. The interference light is then transmitted to sensor 20B. The light is received by the sensor 20B. An image processor 16G, operating under the control of the CPU 16A, generates OCT images such as tomographic images and en-face images based on the OCT data detected by the sensor 20B. Alternatively, the image processor 16G may be omitted, and the CPU 16A may generate the OCT images based on the OCT data detected by the sensor 20B.

[0046] Here, OCT fundus images obtained with an internal illumination angle of 160 degrees or more are referred to as UWF-OCT images. Of course, OCT fundus image data can also be obtained with an internal illumination angle of less than 160 degrees.

[0047] The UWF-OCT image data, along with patient information, is transmitted from the ophthalmic device 110 to the server 140 via the communication interface (I / F) 16F. The UWF-OCT image data and patient information are stored in memory 164 in correspondence.

[0048] In this embodiment, the light source 20A is exemplified as a wavelength-swept type SS-OCT (Swept-Source OCT), but SD-OCT (Spectral-Domain OCT) is also used. Various types of OCT systems are acceptable, such as OCT (Optical Coherence Tomography) and TD-OCT (Time-Domain OCT).

[0049] Next, the axial length measuring device 120 will be described. The axial length measuring device 120 has two modes: a first mode and a second mode for measuring the axial length, which is the length of the eye in the direction of the eye axis of the eye being examined 12. In the first mode, light from a light source (not shown) is guided to the eye being examined 12, and the interference light of the reflected light from the fundus and the reflected light from the cornea is received, and the axial length is measured based on the interference signal indicating the received interference light. The second mode is a mode for measuring the axial length using ultrasound (not shown).

[0050] The axial length measuring device 120 transmits the axial length measured by the first mode or the second mode to the server 140. The axial length may be measured by both the first mode and the second mode, in which case the average of the axial lengths measured by both modes is transmitted to the server 140 as the axial length. The server 140 stores the patient's axial length corresponding to the patient name ID.

[0051] Next, the configuration of server 140 will be described with reference to Figure 4. As shown in Figure 4, server 140 comprises a control unit 160 and a display / operation unit 170. The control unit 160 includes a computer with a CPU 162, a memory 164 which is a storage device, and a communication interface (I / F) 166, etc. Image processing programs are stored in the memory 164. The display / operation unit 170 is a graphic user interface that displays images and accepts various instructions, and comprises a display 172 and an input / instruction device 174 such as a touch panel.

[0052] CPU 162 is an example of a “processor” in the technology of this disclosure. Memory 164 is an example of a “computer-readable storage medium” in the technology of this disclosure. Control unit 160 is an example of a “computer program product” in the technology of this disclosure. Server 140 is an example of an “image processing device” in the technology of this disclosure.

[0053] The configuration of viewer 150 is the same as that of server 140, so its explanation will be omitted.

[0054] Next, referring to Figure 5, various functions realized by the CPU 162 of the server 140 executing the image processing program will be explained. The image processing program includes image processing functions, display control functions, and processing functions. When the CPU 162 executes the image processing program having these functions, the CPU 162 functions as an image processing unit 182, a display control unit 184, and a processing unit 186, as shown in Figure 4.

[0055] The image processing unit 182 is an example of a "detection unit" in the technology of this disclosure.

[0056] The image processing unit 182 is an example of the "calculation unit," "projection unit," and "generation unit" of the technology of this disclosure.

[0057] Next, we will explain in detail the image processing performed by server 140 using Figure 6. The image processing shown in the flowchart of Figure 6 is achieved when the CPU 162 of server 140 executes the image processing program.

[0058] The image processing program is executed when the server 140 receives image data of fundus images (e.g., UWF fundus images) taken by the ophthalmic device 110, and generates a choroidal vascular image based on the image data of the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image) from the received fundus images. Figure 10A shows the RGB color fundus image G1 from the received fundus images. The ophthalmic device 110 sends the image data to the server 140 along with the patient's image data. The person's information (e.g., patient name, patient ID, age, and vision) is transmitted.

[0059] Choroidal vascular images are generated as follows:

[0060] First, let's explain the information contained in the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image).

[0061] The structure of the eye consists of the vitreous humor surrounded by multiple layers with different structures. These layers, from the innermost to the outermost layer on the vitreous side, include the retina, choroid, and sclera. Red light passes through the retina and reaches the choroid. Therefore, the first fundus image (red color fundus image) contains information about the blood vessels present in the retina (retinal blood vessels) and the blood vessels present in the choroid (choroidal blood vessels). In contrast, green light only reaches the retina. Therefore, the second fundus image (green color fundus image) contains information about the blood vessels present in the retina (retinal blood vessels). Thus, a choroidal blood vessel image can be obtained by extracting the retinal blood vessels from the second fundus image (green color fundus image) and removing the retinal blood vessels from the first fundus image (red color fundus image).

[0062] Next, the method for generating choroidal vascular images will be explained. The image processing unit 182 of the server 140 extracts retinal blood vessels from the second fundus image (G-color fundus image) by applying a black hat filter to the second fundus image (G-color fundus image). Next, the image processing unit 182 removes retinal blood vessels from the first fundus image (R-color fundus image) by inpainting using the retinal blood vessels extracted from the second fundus image (G-color fundus image). In other words, it uses the positional information of the retinal blood vessels extracted from the second fundus image (G-color fundus image) to fill in the retinal vascular structure of the first fundus image (R-color fundus image) with the same value as the surrounding pixels. Then, the image processing unit 182 emphasizes the choroidal blood vessels in the first fundus image (R-color fundus image) by applying adaptive histogram equalization (CLAHE, Contrast Limited Adaptive Histogram Equalization) to the image data of the first fundus image (R-color fundus image) from which the retinal blood vessels have been removed. This yields the choroidal vascular image G2 shown in Figure 10B. The generated choroidal vascular image is stored in memory 164, corresponding to the patient's information.

[0063] Furthermore, although a choroidal vascular image is generated from the first fundus image (R-color fundus image) and the second fundus image (G-color fundus image), the image processing unit 182 may then generate a choroidal vascular image using the first fundus image (R-color fundus image) or an IR fundus image captured with IR light. Regarding the method for generating a choroidal fundus image, the disclosures in Japanese Patent Application No. 2018-052246 and International Publication WO2019 / 181981A1, filed on March 20, 2018, are provided by reference in their entirety. This is incorporated into this specification.

[0064] When the image processing program starts, in step 202 of Figure 5, the processing unit 186 reads the choroidal vascular image G2 (see Figure 10B) and the G-color fundus image from memory 164 as fundus images.

[0065] In step 204, the image processing unit 182 estimates the position of the macula from the G-color fundus image. Specifically, since the macula is a dark region in the G-color fundus image, the image processing unit 182 detects the region of a predetermined number of pixels with the smallest pixel value in the read G-color fundus image as the position of the macula. The coordinates indicating the position of the macula are stored in the memory 164, corresponding to the patient's information.

[0066] In step 206, the image processing unit 182 detects the position of the optic nerve head (ONH) from the G-color fundus image. Specifically, the image processing unit 182 detects the position of the optic nerve head (ONH) in the G-color fundus image by performing pattern matching of a predetermined optic nerve head image with the G-color fundus image read out. The coordinates indicating the position of the optic nerve head (ONH) are stored in the memory 164, corresponding to the patient's information.

[0067] Incidentally, since the optic nerve head is the brightest region in the G-color fundus image, the location of the optic nerve head may be detected as the region of a predetermined number of pixels with the largest pixel value in the G-color fundus image read out as described above.

[0068] Furthermore, the choroidal vascular image is created by processing the red-colored fundus image and the green-colored fundus image as described above. Therefore, when the coordinate system of the green-colored fundus image is superimposed on the coordinate system of the choroidal vascular image, each position in the coordinate system of the green-colored fundus image is the same as each position in the coordinate system of the choroidal vascular image. Thus, each position on the choroidal vascular image that corresponds to the respective positions of the macula and optic nerve head detected from the green-colored fundus image is the respective position of the macula and optic nerve head.

[0069] Therefore, in step 204, the position of the macula may be detected from the choroidal vascular image instead of the G-color fundus image. Similarly, in step 206, the position of the optic nerve head may be detected from the choroidal fundus image instead of the G-color fundus image.

[0070] In step 208, the image processing unit 182 detects the location of vortex veins (hereinafter referred to as "VV") in the choroidal vascular image. Here, vortex veins (VV) are outflow pathways for blood flow that has entered the choroid, and multiple vortex veins exist near the posterior pole of the equatorial region of the eyeball.

[0071] The image processing unit 182 determines the blood vessel running direction for each pixel in the choroidal blood vessel image. Specifically, the image processing unit 182 repeats the following process for all pixels. That is, the image processing unit 182 sets up a region (cell) consisting of multiple surrounding pixels centered on the pixel. Then, it calculates the gradient direction of brightness for each pixel in the cell (for example, indicated by an angle between 0 degrees and less than 180 degrees, where 0 degrees is defined as the direction of a straight line (i.e., a horizontal line)) based on the brightness values ​​of the pixels surrounding the pixel to be calculated. This gradient direction calculation is performed for all pixels in the cell.

[0072] Next, to create a histogram with nine bins (for example, each bin width being 20 degrees) representing gradient directions of 0, 20, 40, 60, 80, 100, 120, 140, and 160 degrees, we count the number of pixels in the cells corresponding to the gradient direction of each bin. Each bin width in the histogram corresponds to 20 degrees, and the 0-degree bin is set to the number of pixels in the cells (i.e., the count value) for gradient directions between 0 and 10 degrees (inclusive) and between 170 and 180 degrees (inclusive). The 20-degree bin is set to the number of pixels (i.e., count value) within the cell with a gradient direction of 10 degrees or more and less than 30 degrees. Similarly, the count values ​​for the 40-degree, 60-degree, 80-degree, 100-degree, 120-degree, 140-degree, and 160-degree bins are also set. Since the histogram has 9 bins, the direction of blood vessel course for each pixel is defined by one of 9 different directions. Note that the resolution of blood vessel course direction can be increased by narrowing the bin width and increasing the number of bins. The count value in each bin (i.e., the vertical axis of the histogram) is normalized, and a histogram for the analysis point is created.

[0073] Next, the image processing unit 182 identifies the direction of the blood vessel at the analysis point from the histogram. Specifically, it identifies the bin with the smallest count value (for example, 60 degrees) and identifies 60 degrees, which is the gradient direction of the identified bin, as the direction of the blood vessel at the pixel. The reason why the gradient direction with the fewest counts is the direction of the blood vessel is as follows: The brightness gradient is small in the direction of the blood vessel, while the brightness gradient is large in other directions (for example, there is a large difference in brightness between blood vessels and non-blood vessels). Therefore, when a histogram of the brightness gradient for each pixel is created, the count value of the bin corresponding to the direction of the blood vessel will be small. Similarly, a histogram is created for each pixel in the choroidal blood vessel image and the direction of the blood vessel at each pixel is calculated. The calculated direction of the blood vessel at each pixel is stored in memory 164, corresponding to the patient's information.

[0074] The image processing unit 182 then sets initial positions for a total of L virtual particles, M vertically and N horizontally, at equal intervals on the choroidal blood vessel image. For example, if M=10 and N=50, a total of L=500 initial positions are set.

[0075] Furthermore, the image processing unit 182 acquires the direction of the blood vessel at the initial position (one of the L), moves the virtual particle a predetermined distance along the acquired direction of the blood vessel, acquires the direction of the blood vessel again at the moved position, and moves the virtual particle a predetermined distance along the acquired direction of the blood vessel. This process of moving the virtual particle a predetermined distance along the direction of the blood vessel is repeated a predetermined number of times. The above process is performed at all L positions. When the predetermined number of moves has been performed for all L virtual particles, the point where a certain number of virtual particles have gathered is detected as a VV position. Figure 10C shows how VV4 is detected from the four vortex veins VV1 to VV4.

[0076] The number of detected vortex veins (VVs) and VV location information (for example, coordinates indicating the VV location in the choroidal vascular image) are stored in memory 164, corresponding to the patient's information. VV detection may be performed using various fundus images, not just choroidal vascular images. These may include color fundus images such as red-colored and green-colored fundus images taken at various visible light wavelengths, fluorescent fundus images obtained by fluorescence imaging, two-dimensional images (en-face OCT images) generated from three-dimensional OCT volume data, and binarized images obtained by image processing of these images, as well as images with enhanced blood vessels. Even if it is just a vascular image, it may be a choroidal vascular image generated by a process that extracts choroidal vessels from the vascular image.

[0077] In step 210, the image processing unit 182 projects the optic nerve head ONH, macula M, and each VV onto the spherical surface of the eyeball model.

[0078] Figure 7 shows a flowchart of the projection process in step 210 of Figure 6. In step 221 of Figure 7, the processing unit 186 obtains the coordinates (X,Y) of the optic nerve head ONH, macula M, and VV positions in the choroidal fundus image, corresponding to the patient's information, from the memory 164.

[0079] In step 223, the image processing unit 182 checks the positions of the optic nerve head ONH, macula M, and VV. The coordinates are projected onto a virtual spherical surface (the surface of the eyeball or the surface of the eyeball model) corresponding to the fundus of the eyeball model shown in Figure 11. This eyeball model is a spherical model with the center of the eyeball at C and the radius at R (with the axial length of the eyeball being 2R). The spherical surface of this eyeball model is defined as the surface of the eyeball. The position of a point on the surface of the eyeball is determined by its latitude and longitude. The optic nerve head (ONH) is projected as the reference point (longitude 0 degrees, latitude 0 degrees), and the macula (M) is projected so that its latitude is 0 degrees. The positions of each vortex vein (VV) are also projected onto the ocular surface as Pvv1, Pvv2, Pvv3, Pvv4, and Pvv5 (if five vortex vein VVs are detected in the eye being examined), with the optic nerve head (ONH) as the reference point. Therefore, the optic disc (ONH) and macula (M) are projected onto the 0-degree latitude line, and the line segment connecting the optic disc (ONH) and macula (M) on the sphere lies on the 0-degree latitude line.

[0080] Here, the eyeball model is also defined in three-dimensional space (X, Y, Z) and is defined in the memory space of the image processing. As described above, the X direction is the horizontal direction when the ophthalmic device 110 is placed on a horizontal plane, and since the patient is positioned facing the ophthalmic device 110, the X direction is the left-right direction of the eye under examination. The Y direction is the direction perpendicular to the horizontal plane and is the up-down direction of the eye under examination. The Z direction is the direction connecting the center of the pupil of the anterior segment of the eye under examination 12 to the center of the eyeball.

[0081] Memory 164 stores a transformation formula for performing inverse stereoprojection transformation on the coordinates (X,Y) of each position on a two-dimensional fundus image (UWF fundus image) into a three-dimensional space (X,Y,Z). In this embodiment, this transformation formula transforms (i.e., projects) the position of the optic nerve head ONH to a reference point on the surface of the eyeball (where both longitude and latitude are 0 degrees), and the position of the macula M to a position at latitude 0 degrees. Furthermore, Using this transformation formula, the position (X, Y) of each VV on a two-dimensional fundus image is transformed (i.e., projected) to the three-dimensional eye surface positions Pvv1 through Pvv5.

[0082] The coordinates (X,Y,Z) of the positions of vortex veins VV1-5 on the surface of the eyeball, from Pvv1 to Pvv5, are stored in memory 164, along with the three-dimensional coordinates of the optic nerve head ONH and macula M, and are associated with the patient's identification information.

[0083] Once the process in step 223 of Figure 7 is completed, the image processing proceeds to step 212 of Figure 6.

[0084] In step 212, the image processing unit 182 calculates the distribution center points of multiple vortex veins (hereinafter referred to as VV centers). The VV centers are the distribution center points of each position of multiple VVs. The distribution center points can be determined by first finding a vector from the center point C of the eyeball model to the vortex vein position (Pvv1, etc.), and then finding a composite vector by combining multiple vectors to each VV.

[0085] Figure 8 shows a flowchart of the VV center calculation process in step 212 of Figure 6. In Figure 8, the explanation assumes that five vortex veins (VV) are detected in the eye being examined.

[0086] In step 232 of Figure 8, the processing unit 186 retrieves the coordinates (x1, y1, z1) to (x5, y5, z5) of the positions Pvv1 to Pvv5 of each VV from the memory 164, corresponding to the patient information. Here, the eyeball model uses a unit sphere with a radius of 1, which is the unit length.

[0087] In step 234, the image processing unit 182 generates vectors VC1 to VC5 from the center C of the sphere of the eyeball model to the positions Pvv1 to Pvv5 of each VV, as shown in Figures 12A to 12C, as shown in Equation 1.

number

[0088] The magnitude of each vector VC is 1, which is its unit length. Figures 12A to 12C show the unit sphere viewed from an oblique angle, a side view, and a top view, respectively.

[0089] In step 236, the image processing unit 182 calculates a composite vector VCT (see VVcomposition in Equation 2) by combining the vectors VC1 to VC5 for each VV, as shown in Figures 13A to 13C. Specifically, as shown in Equation 2, each vector VC This process involves summing the coordinate values.

number

[0090] The coordinates of point Pvvc, determined by the composite vector VCT from the center C, are determined by the sum of the X, Y, and Z coordinates of each of the positions Pvv1 to Pvv5. Point Pvvc is the distribution center in three-dimensional space for each of the positions VV1 to VV5. Point Pvvc is the position where the sum of the distances (e.g., Euclidean distances) between each of the positions VV1 to VV5 is minimized. Point Pvvc is generally located inside the sphere of the eyeball model. Figures 13A to 13 C represents the eyeball model, which is the unit sphere, viewed from an oblique angle, a side view, and a top view, respectively.

[0091] In step 238, the image processing unit 182 normalizes the composite vector VCT so that the magnitude of the composite vector becomes 1, as shown in Figures 14A to 14C. The normalized composite vector VCTN (see VVcenter in Equation 3) is then... It can be obtained as shown in Math 3.

number

[0092] Then, in step 240, the point Pvvcn, determined by the normalized composite vector VCTN from the center C, lies on the ocular surface of the eyeball model, which is the unit sphere. Point Pvvcn is the intersection of the normalized composite vector VCTN and the ocular surface of the unit sphere. Point Pvvcn is the distribution center on the sphere of the eyeball model for each of the multiple VV positions.

[0093] It can also be defined as the position where the sum of the distances between each of the multiple VV positions and the distribution center (for example, the great circle distances on the sphere of the eyeball model) is minimized. Figures 14A to 14C show the eyeball model of the unit sphere viewed from an oblique angle, a side view, and a top view, respectively.

[0094] Step 240 determines the distribution center positions of the unit sphere on the spherical surface for multiple vortex veins. Thus, the latitude and longitude of the distribution centers of the vortex veins are determined.

[0095] In step 241, the image processing unit 182 enlarges the unit sphere so that it becomes an eyeball model with an axial length of 2R. In other words, if the axial length of the eye under examination is 24 mm (i.e., 2 * 12 mm, R = 12), it is transformed to become a sphere with a radius of 12 mm.

[0096] In step 241, the image processing unit 182 further extracts the coordinates on the ocular surface of the optic nerve head ONH, macula M, vortex veins VV1-VV5, and vortex vein distribution center Pvvcn in an eyeball model that reflects the axial length of the eye being examined, or the latitude and longitude of the macula M, vortex veins, and vortex vein distribution center calculated based on the optic nerve head. The extracted coordinate or latitude and longitude information is stored in memory 164 in accordance with the patient's information. Both coordinates and latitude and longitude may be stored in memory 164.

[0097] Once the process in step 241 in Figure 8 is completed, the image processing proceeds to step 214 in Figure 6.

[0098] In step 214, the image processing unit 182 calculates at least one feature that shows the positional relationship between the position of a specific part of the eyeball model and the positions of multiple objects, including the VV center. Examples of specific parts include the optic nerve head, macula, pupillary center (i.e., the apex of the cornea), and fovea. Hereafter, the optic nerve head will be used as an example of a specific part.

[0099] Figure 9 shows a detailed flowchart of the feature calculation process in step 214 of Figure 6. In step 242 of Figure 9, the image processing unit 182 sets the variable t, which identifies the target for feature calculation, to 0, and in step 244, the image processing unit 182 increments the variable t by 1. For example, the VV center, VV1, VV2, etc. are identified by the variable t = 1, 2, 3...

[0100] In step 246, the image processing unit 182 uses the optic nerve head ONH as a reference point (for example, longitude: 0 degrees, latitude: 0 degrees) and calculates the great circle distance and angle to the target t, the longitude and latitude where the target t is located, and the longitude and latitude distance from the reference point to the target t as feature quantities of the target t. Specifically, it is as follows:

[0101] First, let's explain the case where the variable t=1, that is, the case where we calculate features focusing on the VV center.

[0102] As shown in Figure 15, the image processing unit 182 calculates the great circle distance GCL between the optic nerve head ONH and the VV center (point Pvvcn) using the formula for spherical trigonometry. A great circle is defined as the cross-section obtained by cutting a sphere through its center C in the eyeball model, and the great circle distance is the arc length of the great circle connecting two points on the spherical surface that are the targets of distance measurement (the optic nerve head ONH and the VV center (point Pvvcn)).

[0103] Next, as shown in Figure 15, the image processing unit 182 calculates the angle θ between the first line segment GCL on the great circle connecting the position of the macula M and the position of the optic nerve head ONH, and the second line segment GL connecting the position of the optic nerve head ONH and the position of the VV center Pvvcn, using conformal projection or spherical trigonometry.

[0104] The method for calculating the above distances and angles is the same as the method described in International Application PCT / JP2019 / 016653, filed internationally on April 18, 2019. The method for calculating the above distances and angles described in International Application PCT / JP2019 / 016653 (International Publication No. WO2019203310, published internationally on October 24, 2019), filed internationally on April 18, 2019, is incorporated herein by reference in its entirety.

[0105] Next, the image processing unit 182 calculates the longitude (LG=lgn) and latitude (LT=ltn) of the VV center Pvvcn, as shown in Figure 15. As described above, the eyeball model is defined in three-dimensional space, with the position of the optic nerve head OHN being set with the position of 0 degrees latitude and longitude as the reference point, and the position of the macula M being located on the line of latitude LLT at 0 degrees latitude.

[0106] The image processing unit 182 calculates the latitude (LG=lgn) and longitude (LT=ltn) of the VV center Pvvcn by transforming the coordinates (x,y,z) of the VV center Pvvcn in three-dimensional space using a predetermined transformation formula, with the position of the optic nerve head ONH (X,Y,Z)=(1,0,0). Longitude LT=arctan(y / x) Latitude LG=arctan(z / √(x 2 +y 2 )) This allows the (LG (latitude), longitude (LT)) of the VV center Pvvcn to be calculated as (lgn, ltn).

[0107] Then, as shown in Figure 15, the image processing unit 182 calculates the latitude distance TL and longitude distance GL of the VV center Pvvcn using the formula for spherical trigonometry.

[0108] The latitude distance TL of the VV center Pvvcn is the great circle distance between the VV center Pvvcn along the longitude line LLG passing through the VV center Pvvcn and the intersection point Ptg of the longitude line LLG and the latitude line LLT at latitude 0.

[0109] The longitude distance GL of the VV center Pvvcn is the great circle distance between the position of the optic disc OHN and the intersection point Ptg along the latitude line LLT at latitude 0.

[0110] Once the above features have been calculated for the object identified by the variable t, the feature calculation process proceeds to step 248.

[0111] In step 248, the image processing unit 182 determines whether the variable t is equal to the total number T of objects for which features are to be calculated.

[0112] If the variable t is not equal to the total number T, there are objects for which features have not been calculated, so the feature calculation process returns to step 244 and executes the above process (i.e., steps 244 to 248).

[0113] If the variable t=2 or later, the image processing unit 182 calculates the above distance, angle, latitude, longitude, longitude distance, and latitude distance, using the position of VV1 or later as the target instead of the VV center Pvvcn.

[0114] If the variable t is equal to the total number T, then features have been calculated for all objects for which features should be calculated. In step 250, the processing unit 186 stores each feature for each object in the feature table of memory 164, corresponding to the patient's information, as shown in Figure 16. In the example shown in Figure 16, the feature table includes items for patient ID, left / right eye identification information, and axial length corresponding to the left / right eye. A feature memory area is provided for each eye examined (left eye or right eye). The feature memory area stores type information indicating the type of object (optic nerve head, macula, vortex veins VV1-VV5, VV center, etc.), and feature information for distance, angle, latitude, longitude, longitude distance, and latitude distance. The coordinates of the optic nerve head ONH, macula M, vortex veins VV1-VV5, and VV center on the ocular surface may also be included in the feature table. Furthermore, although the above example was explained assuming five vortex veins, it goes without saying that this method is applicable to any number of vortex veins N (where N is a natural number) detected in the eye being examined. Furthermore, the amount of deviation between the determined optic disc position and the VV center position, specifically the distance between the optic disc position and the VV center position (such as the great circle distance), and a characteristic quantity determined by the direction from the optic disc to the VV center position (i.e., a vector connecting the optic disc position and the VV center position) may be determined and used as part of the characteristic quantity. In addition, the amount of deviation between the macula position and the VV center position, and the amount of deviation between the lesion position on the fundus and the VV center position may be determined in the same way and used as characteristic quantities. This amount of deviation can generally be used as a numerical indicator of the pathological condition.

[0115] Once the process in step 250 is complete, the process in step 214 in Figure 6 is also complete, and the image processing is finished.

[0116] Next, we will explain the process of displaying the distribution centers and other information in the viewer 150 after the server 140 has calculated the distribution centers of multiple vortex veins.

[0117] The display screen of the viewer 150 shows icons and buttons for instructing the generation of images described later (Figures 17 and 18). When an ophthalmologist, who is the user, wants to know the position of the VV center when diagnosing a patient, and clicks on a designated icon, the viewer 150 sends an instruction signal corresponding to the clicked icon to the server 140.

[0118] Upon receiving an instruction signal from the viewer 150, the server 140 generates an image corresponding to the instruction signal (Figures 17 and 18) and transmits the image data of the generated image to the viewer 150 via the network. The data is transmitted via 130. The viewer 150, having received image data from server 140, displays the image on the display based on the received image data. The display screen generation process on server 140 is performed by a display screen generation program running on CPU 162 (display control unit 184).

[0119] Figure 17 shows a first display screen 300A for displaying the target and feature quantities. As shown in Figure 17, the first display screen 300A has a personal information display area 302 for displaying the patient's personal information and an image display area 320.

[0120] The personal information display area 302 includes a patient ID display area 304, a patient name display area 306, an age display area 308, an axial length display area 310, a visual acuity display area 312, and a patient selection icon 314. The patient ID display area 304, patient name display area 306, age display area 308, axial length display area 310, and visual acuity display area 312 display the respective information. When the patient selection icon 314 is clicked, a list of patients is displayed on the viewer 150's display 172, allowing the user (such as an ophthalmologist) to select the patient to be analyzed.

[0121] The image display area 320 includes a date display area 322N1, a right eye information display area 324R, a left eye information display area 324L, a first eyeball model image display area 326A showing the eyeball model from an oblique angle, a second eyeball model image display area 328 showing the eyeball model from the side, and an information display area 342. The information display area 342 displays comments and notes from the user (such as an ophthalmologist) during the examination as text.

[0122] The example shown in Figure 17 displays eye model images of the right eye fundus (324R lit) of a patient identified by patient ID: 123456, taken on March 10, 2018, December 10, 2017, and September 10, 2017. Note that clicking on the date display field 322N1 and the right eye information display field 324R displays the eye model image of the right eye obtained from the image taken on March 10, 2018.

[0123] As shown in Figure 17, the first eyeball model image display area 326A displays the eyeball model viewed from an oblique angle, and the second eyeball model image display area 328 displays the eyeball model viewed from the side. The eyeball model displays the respective positions Pvv1 to Pvv5 from VV1 to VV5, the vectors VC1 to VC5 from each of VV1 to VV5, the point Pvvcn determined from the center C by the normalized composite vector VCTN, and the normalized composite vector VCTN.

[0124] When either the first eyeball model image display area 326A or the second eyeball model image display area 328 is clicked at the respective positions Pvv1 to Pvv5 of each VV1 to VV5, or at the position of point Pvvcn, the feature quantities corresponding to the clicked position are displayed. For example, when the position of point Pvvcn is clicked, the feature quantities are displayed near the position of point Pvvcn. Specifically, the great circle distance GC between the optic nerve head ONH and the VV center, the angle θ between the position of the macula M, the position of the optic nerve head ONH, and the position of the VV center Pvvcn, the latitude and longitude of the VV center Pvvcn, and the latitude distance TL and longitude distance GL of the VV center Pvvcn are displayed.

[0125] Figure 18 shows the second display screen 300B for displaying the target and feature quantities. Since the second display screen 300B is substantially the same as the first display screen 300A, the same reference numerals are used for the same display parts, their explanations are omitted, and the different display parts are explained. Note that the first display screen 300A and the second display screen 300B are switched using a toggle button (not shown).

[0126] On the second display screen 300B, the target position image display field 326B and the target and tomographic image display field 328B are displayed instead of the first eyeball model image display field 326A and the second eyeball model image display field 328.

[0127] Figure 18 shows an example where four VVs were detected.

[0128] As described above, the position of the eyeball model on the eyeball is defined by latitude and longitude, and in the feature calculation process of step 214 in Figure 6, the latitude and longitude of each VV and VV center are calculated. In the target position image display area 326B, the position of each VV and VV center is displayed based on the positional relationship of latitude and longitude.

[0129] The target and tomographic image display area 328B includes the original image display area 1032A and the VV tomographic image display area 890.

[0130] In the original image display area 1032A, the OCT image acquisition areas 872A, 872B, 872C, and 872D are displayed superimposed on the original image. As described above, in step 208 of Figure 6, the VV position is detected, and in this embodiment, OCT images are acquired for the OCT image acquisition areas 872A, 872B, 872C, and 872D centered on the detected VV position.

[0131] The OCT image display area 890 displays OCT images 892A, 892B, 872C, and 892D, which correspond to the OCT image acquisition areas 872A, 872B, 872C, and 872D, respectively, that are superimposed on the original image in the original image display area 1032A. The OCT images 892A, 892B, 892C, and 892D display the shape of vortex veins visualized as tomographic images by, for example, scanning the OCT image acquisition areas 872A, 872B, 872C, and 872D. The codes No.1, No.2, No.3, and No.4 attached to the top of the OCT images 892A, 892B, 892C, and 892D correspond to the codes No.1, No.2, No.3, and No.4 attached to the OCT image acquisition areas 872A, 872B, 872C, and 872D, respectively. Therefore, the target and tomographic image display area 328B can display the OCT image acquisition areas 872A, 872B, 872C, and 872D in correspondence with the OCT images 892A, 892B, 892C, and 892D that show the shape of the visualized vortex veins (or vortex veins and choroidal vessels connected to vortex veins).

[0132] The OCT image display area 890 displays an image showing the shape of vortex veins, but this embodiment is not limited to this. When an OCT image of the macula is acquired, a cross-sectional image of the macula is displayed in the OCT image display area 890, and when an OCT image of the optic nerve head is acquired, a cross-sectional image of the optic nerve head is displayed in the OCT image display area 890.

[0133] The OCT image display area 890 is provided with a display switching button (not shown). Clicking this button displays a pull-down menu for switching the display of OCT images 892A, 892B, 892C, and 892D. By selecting, for example, a 3D polygon, an en-face image, OCT pixels, or a blood flow visualization image from the displayed pull-down menu, the display of OCT images 892A, 892B, 892C, and 892D can be switched.

[0134] As described above, this embodiment makes it possible to obtain the VV center and support ophthalmologists in making diagnoses using the VV center.

[0135] Furthermore, in the above embodiment, multiple feature quantities indicating the positional relationship between the location of a specific part of the fundus and the location of the distribution center are calculated and displayed, thereby further supporting ophthalmologists' diagnoses using the VV center. For example, the distance between the optic nerve head and the VV center is calculated and displayed as a feature quantity, allowing ophthalmologists to understand the deviation (distance) from the general position of the VV center (e.g., the position of the optic nerve head OHN).

[0136] In the embodiments described above, the latitude and longitude of the vortex veins on the spherical surface of an eyeball model, where the position of 0 degrees latitude and longitude corresponds to the position of a specific part of the fundus, as well as the latitudinal and longitudinal distances between the specific part and the vortex veins, can be obtained. This can support ophthalmologists in making diagnoses using the latitude and longitude of the vortex veins and the latitudinal and longitudinal distances between the specific part and the vortex veins.

[0137] In the embodiments described above, each distance and each line segment is a great circle distance, but the technology of this disclosure is not limited thereto. Instead of or together with the great circle distance, Euclidean distance (straight-line distance) may be calculated.

[0138] In the above embodiment, the VV center is determined by normalizing the composite vector obtained by combining the vectors to each VV position, but the technology of this disclosure is not limited to this.

[0139] That is, for example, firstly, the position of each VV in the eyeball model can be identified by latitude and longitude, the average value of the latitude and longitude of each VV position can be calculated, and the VV center of the position (latitude, longitude) on the surface of the eyeball model can be determined from the calculated average latitude and average longitude.

[0140] Secondly, in the eyeball model, for each of the multiple candidate center positions expected to be the VV center, the sum of the great circle distances to each VV position is calculated, and the candidate center position that minimizes the sum of great circle distances is identified using a Monte Carlo method or similar. This candidate center position that minimizes the sum of great circle distances may then be designated as the VV center.

[0141] Thirdly, in order to determine the VV center, instead of using the positions of all detected VVs, only the positions of VVs located in likely locations among the detected VVs may be used. For example, if 10 VVs are detected, only 9 may be used instead of all of them. The 9 are obtained as a result of selecting those that are likely to be VVs, and as a result of selecting those that are likely, there may be 8 or 7. Those that are likely can be selected, for example, by excluding VVs whose distance from neighboring VVs is less than a predetermined value, or by excluding VVs whose number of connected choroidal vessels is less than or equal to a predetermined value. In this case, when calculating the vectors up to the positions of the 10 VVs and combining the vectors, the vectors up to the positions of the 9 VVs located in likely locations may be combined, or only the vectors up to the positions of the 9 VVs located in likely locations may be calculated and the calculated vectors may be combined.

[0142] In the above embodiment, the VV center is used as the target for feature calculation and display, but the technology of this disclosure is not limited thereto. For example, the distribution center in the eyeball model (3D space) of each VV may be targeted.

[0143] In the first embodiment, the VV center features (distance, angle, latitude, longitude, longitude distance, and latitude distance) are calculated with the optic disc as the reference point (latitude and longitude each set to 0 degrees), but the technology of this disclosure is not limited thereto. For example, the VV center features (e.g., distance, angle, latitude, longitude, longitude distance, and latitude distance) may be calculated with the macula M as the reference point (latitude and longitude each set to 0 degrees). The positions of each point mentioned above are determined using latitude and longitude, but they may also be determined using polar coordinates.

[0144] In the above embodiment, an example was described in which a fundus image with an internal light illumination angle of approximately 200 degrees is acquired using the ophthalmic device 110. The technology of this disclosure is not limited thereto, and can also be used with an internal illumination angle of 100 degrees or less. The technology of this disclosure may be applied to fundus images taken with ophthalmic equipment, or to montage images created by combining multiple fundus images.

[0145] In the above embodiment, fundus images are captured using an ophthalmic device 110 equipped with an SLO imaging unit. However, fundus images obtained using a fundus camera capable of capturing choroidal blood vessels may also be used, or the technology of this disclosure may be applied to images obtained by OCT angiography.

[0146] In the above embodiment, the management server 140 executes the image processing program. The technology of this disclosure is not limited thereto. For example, the ophthalmic device 110, the image viewer 150, and another image processing device provided in the network 130 may execute the image processing program. When the ophthalmic device 110 executes the image processing program, the image processing program is stored in the ROM 16C. When the image viewer 150 executes the image processing program, the image processing program is stored in the memory 164 of the image viewer 150. When another image processing device executes the image processing program, the image processing program is stored in the memory of the other image processing device.

[0147] In the above embodiment, an ophthalmic system 100 comprising an ophthalmic device 110, an axial length measuring device 120, a management server 140, and an image viewer 150 was described as an example, but the technology of this disclosure is not limited thereto. For example, as a first example, the axial length measuring device 120 may be omitted, and the ophthalmic device 110 may further have the functions of the axial length measuring device 120. As a second example, the ophthalmic device 110 may further have the functions of at least one of the management server 140 and the image viewer 150. For example, if the ophthalmic device 110 has the functions of the management server 140, the management server 140 can be omitted. In this case, the image processing program is executed by the ophthalmic device 110 or the image viewer 150. Also, if the ophthalmic device 110 has the functions of the image viewer 150, the image viewer 150 can be omitted. As a third example, the management server 140 may be omitted, and the image viewer 150 may perform the functions of the management server 140. In this disclosure, each component (device, etc.) may exist as one or more, as long as it does not create a contradiction.

[0148] Furthermore, while the above embodiments and their modifications illustrate cases where data processing is realized by a computer-based software configuration, the technology of this disclosure is not limited thereto. For example, instead of a computer-based software configuration, data processing may be performed solely by a hardware configuration such as an FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit). Alternatively, some of the data processing may be performed by a software configuration, and the remaining processing may be performed by a hardware configuration.

[0149] Thus, the technology disclosed herein includes cases where image processing is implemented using computer-based software and cases where it is not, and therefore includes the following technologies.

[0150] (First technology) A detection unit that detects the positions of multiple vortex veins from the fundus image of the eye being examined, A calculation unit that calculates the distribution center of the positions of the detected plurality of vortex veins, An image processing device equipped with the following features.

[0151] (Second technology) The detection unit detects the locations of multiple vortex veins from the fundus image of the eye being examined. The calculation unit calculates the distribution center of the detected plurality of vortex veins. Image processing methods.

[0152] (Third technology) A detection unit that detects the positions of multiple vortex veins and the optic nerve head from the fundus image of the eye being examined, A projection unit that projects the plurality of vortex veins and the optic nerve head onto the surface of the eyeball model, A generation unit that generates multiple vectors from the center of the eyeball model to the positions of each vortex vein on the surface of the eyeball, A calculation unit that calculates a composite vector obtained by combining the aforementioned multiple vectors, An image processing device equipped with the following features.

[0153] (The fourth technology) The detection unit detects the locations of multiple vortex veins and the optic nerve head from the fundus image of the eye being examined. The projection unit projects the plurality of vortex veins and the optic nerve head onto the surface of the eyeball model. The generation unit generates multiple vectors from the center of the eyeball model to the positions of each vortex vein on the surface of the eyeball, The calculation unit calculates a composite vector by combining the multiple vectors. Image processing methods.

[0154] Based on the information disclosed above, the following technologies are proposed.

[0155] (The fifth technology) A computer program product for image processing, The aforementioned computer program product includes a computer-readable storage medium that is not itself a temporary signal, The aforementioned computer-readable storage medium stores a program. The aforementioned program, On the computer, The locations of multiple vortex veins are detected from the fundus image of the eye being examined. The distribution center of the locations of the detected plurality of vortex veins is calculated. A computer program product that performs a task.

[0156] (The sixth technology) A computer program product for image processing, The aforementioned computer program product includes a computer-readable storage medium that is not itself a temporary signal, The aforementioned computer-readable storage medium stores a program. The aforementioned program, On the computer, From the fundus image of the eye being examined, the positions of multiple vortex veins and the optic nerve head are detected. The plurality of vortex veins and the optic nerve head are projected onto the surface of the eyeball model, Multiple vectors are generated from the center of the eyeball model to the positions of each vortex vein on the surface of the eyeball, The composite vector obtained by combining the aforementioned multiple vectors is calculated. A computer program product that performs a task.

[0157] The image processing methods described above are merely examples. Therefore, it goes without saying that you may remove unnecessary steps, add new steps, or change the processing order, as long as you do not deviate from the main purpose.

[0158] All documents and patent applications described herein (including Japanese application: Patent Application No. 2019-220285), Technical standards are incorporated herein by reference in the same way as when individual documents, patent applications, and technical standards are specifically and individually incorporated by reference.

Claims

1. From fundus images of the eye being examined, the locations of the optic nerve head, macula, and multiple vortex veins are detected, Projecting the optic disc, the macula, and the multiple vortex veins onto an eyeball model, On the spherical surface of the eyeball model, the positions of the optic nerve head, the macula, and the multiple vortex veins are determined from the center point of the eyeball model, On the aforementioned spherical surface, determine the angle between a first line segment on the great circle connecting the optic nerve head and the macula, and a second line segment connecting the optic nerve head and the distribution center points of the plurality of vortex veins, or connecting the macula and the distribution center points. Image processing methods including [specific details omitted].

2. This includes determining the positional displacement between the optic nerve head and the distribution center point, or between the macula and the distribution center point, on the aforementioned spherical surface. The image processing method according to claim 1.

Images