Image processing method, image processing device, and program

Abstract

Provided is an image processing method that is executed by a processor, the image processing method comprising: a step for generating, from a two-dimensional tomographic image of a predetermined region on a fundus of an eye being examined, front images at positions different in the depth direction on a per-depth position basis; a step for extracting a choroid blood vessel from each of the generated front images and detecting the center line of the choroid blood vessel; and a step for estimating a three-dimensional choroid blood vessel and the center line of the three-dimensional choroid blood vessel on the basis of the plurality of front images in which the center lines have been detected in the step for detecting the center lines.

Description

Image processing method, image processing device, and program

[0001] The present disclosure relates to an image processing method, an image processing device, and a program.

[0002] U.S. Patent No. 8,356,901 discloses a technique for analyzing vortex veins from fundus images, and Japanese Patent Application Laid-Open No. 2015-131 discloses a technique for quantifying choroidal blood vessels from measurement data obtained by optical coherence tomography.

[0003] A first aspect of the present disclosure is an image processing method performed by a processor, the image processing method including: generating front images at different depth positions for each depth position from two-dimensional tomographic images obtained from a predetermined region on the fundus of the subject's eye; extracting choroidal blood vessels from each of the generated front images and detecting centerlines of the choroidal blood vessels; and estimating three-dimensional choroidal blood vessels and the centerlines of the three-dimensional choroidal blood vessels based on the multiple front images in which the centerlines have been detected in the detecting step.

[0004] A second aspect of the present disclosure is an image processing device including: a generation unit that generates front images at different depth positions from two-dimensional tomographic images obtained from a predetermined region on the fundus of the subject's eye; a detection unit that extracts choroidal blood vessels from each of the generated front images and detects center lines of the choroidal blood vessels; and an estimation unit that estimates three-dimensional choroidal blood vessels and the center lines of the three-dimensional choroidal blood vessels based on the multiple front images in which the center lines have been detected by the detection unit.

[0005] A third aspect of the present disclosure is a program that causes a computer to execute the following steps: generating front images at different depth positions from two-dimensional tomographic images obtained from a predetermined region on the fundus of the subject's eye, for each depth position; extracting choroidal blood vessels from each of the generated front images and detecting center lines of the choroidal blood vessels; and estimating three-dimensional choroidal blood vessels and the center lines of the three-dimensional choroidal blood vessels based on the multiple front images in which the center lines have been detected in the detecting step.

[0006] 1 is a schematic diagram of an ophthalmologic system according to an embodiment; FIG. 2 is a schematic diagram of an ophthalmologic apparatus according to an embodiment; FIG. 3 is a schematic diagram of a server; FIG. 4 is an explanatory diagram of functions realized by an image processing program in a CPU of the server; FIG. 5 is a flowchart showing an example of the flow of image processing by the server; FIG. 6 is a flowchart showing an example of the flow of fluorescein fundus angiography analysis processing; FIG. 7 is a diagram showing an example of a fundus image of choroidal blood vessels including choroidal arteries; FIG. 8 is an example of a fundus image showing fundus images before and after angiography in chronological order; FIG. 9 is a diagram showing an example of an image obtained by polar coordinate transformation of a fundus image; FIG. 10 is a conceptual diagram showing an example of setting an area in a fundus image; FIG. 11 is a diagram showing an example of an image obtained by inverse transformation from an image in a polar coordinate system to an image in a Cartesian coordinate system; FIG. 12 is a diagram showing an example of a display screen; FIG. 13 is a flowchart showing an example of the flow of OCT analysis processing; FIG. 14 is a conceptual diagram of OCT volume data; FIG. 15 is a conceptual diagram showing a process until choroidal blood vessels are extracted; FIG. 16 is a flowchart showing an example of the flow of processing to extract the center position of choroidal blood vessels; FIG. 17 is a conceptual diagram showing a process until the center line of choroidal blood vessels is derived; FIG. 18 is a conceptual diagram of an image obtained by combining multiple en-face images; FIG. 19 is a flowchart showing an example of the flow of processing to detect second feature amounts of choroidal blood vessels. 1 is a diagram showing the relationship of an analysis region to an image of choroidal blood vessels. FIG. 1 is a conceptual diagram relating to setting multiple analysis regions. FIG. 2 is a conceptual diagram of an analysis region relative to choroidal blood vessels. FIG. 3 is a conceptual diagram showing the state of choroidal blood vessels in the choroid using multiple analysis regions. FIG. 4 is a conceptual diagram showing an example of an analysis region relative to a choroidal region. FIG. 5 is a diagram showing an example of a display screen. FIG. 6 is a diagram showing an example of a display screen. FIG. 7 is a diagram showing an example of a display screen. FIG. 8 is an explanatory diagram relating to providing an image of choroidal blood vessels separately from a centerline. FIG. 9 is a flowchart showing an example of a processing flow according to an embodiment. FIG. 10 is a schematic diagram showing a cross section of an eye. FIG. 11 is a conceptual diagram relating to setting a region to be analyzed. FIG. 12 is a conceptual diagram showing setting a region to be analyzed. FIG. 13 is a conceptual diagram showing how a specific region is excluded from the analysis region. FIG. 14 is a conceptual diagram showing a specific region not to be analyzed.

[0007] Hereinafter, embodiments of the technology of the present disclosure will be described in detail with reference to the drawings. Note that components and processes that perform the same actions and functions are given the same reference numerals throughout the drawings, and redundant explanations may be omitted as appropriate. Also, explanations of configurations that are not directly related to the present disclosure or well-known configurations may be omitted. Also, the dimensional ratios in the drawings are exaggerated for the sake of explanation and may differ from the actual ratios. Furthermore, each drawing is merely a schematic illustration to allow a sufficient understanding of the technology of the present disclosure. Therefore, the technology of the present disclosure is not limited to the illustrated examples.

[0008] First Embodiment Fig. 1 shows a schematic configuration of an ophthalmologic system 100. As shown in Fig. 1, the ophthalmologic system 100 includes an ophthalmologic apparatus 110, a server apparatus (hereinafter referred to as "server") 140, and a display device (hereinafter referred to as "viewer") 150. The ophthalmologic apparatus 110 acquires fundus images. The server 140 stores, in association with patient IDs, a plurality of fundus images obtained by photographing the funduses of a plurality of patients using the ophthalmologic apparatus 110 and axial lengths measured by an axial length measuring device (not shown). The viewer 150 displays the fundus images and analysis results acquired by the server 140.

[0009] The server 140 is an example of the "image processing device" of the present disclosure.

[0010] The ophthalmic apparatus 110, the server 140, and the viewer 150 are connected to each other via a network 130. The network 130 may be any network such as a LAN, a WAN, the Internet, or a wide area Ethernet network. For example, if the ophthalmic system 100 is established in a single hospital, a LAN may be used as the network 130.

[0011] The viewer 150 is a client in a client-server system, and multiple viewers 150 are connected via a network. Multiple servers 140 may also be connected via a network to ensure system redundancy. Alternatively, if the ophthalmic apparatus 110 has an image processing function and the image viewing function of the viewer 150, the ophthalmic apparatus 110 can acquire, process, and view fundus images in a standalone state. If the server 140 has the image viewing function of the viewer 150, the configuration of the ophthalmic apparatus 110 and the server 140 can acquire, process, and view fundus images.

[0012] In addition, other ophthalmic equipment (examination equipment such as visual field measurement and intraocular pressure measurement) and a diagnostic support device that performs image analysis using AI (Artificial Intelligence) may be connected to the ophthalmic device 110, the server 140, and the viewer 150 via the network 130.

[0013] Next, the configuration of the ophthalmologic apparatus 110 will be described with reference to FIG.

[0014] For ease of explanation, a scanning laser ophthalmoscope will be referred to as an "SLO," and an optical coherence tomography will be referred to as an "OCT."

[0015] When the ophthalmic apparatus 110 is placed on a horizontal plane, the horizontal direction is defined as the "X direction," the vertical direction relative to the horizontal plane is defined as the "Y direction," and the direction connecting the center of the pupil of the anterior segment of the subject's eye 12 and the center of the eyeball is defined as the "Z direction." Therefore, the X direction, Y direction, and Z direction are perpendicular to each other.

[0016] The ophthalmologic apparatus 110 includes an imaging device 14 and a control device 16. The imaging device 14 is equipped with an SLO unit 18 and an OCT unit 20, and acquires a fundus image of the subject's eye 12. Hereinafter, a two-dimensional fundus image acquired by the SLO unit 18 will be referred to as an SLO image. Furthermore, a tomographic image or an en-face image of the retina created based on OCT data acquired by the OCT unit 20 will be referred to as an OCT image.

[0017] The control device 16 comprises a computer having a CPU (Central Processing Unit) 16A, a RAM (Random Access Memory) 16B, a ROM (Read-Only Memory) 16C, and an input / output (I / O) port 16D.

[0018] The control device 16 includes an input / display device 16E connected to the CPU 16A via an I / O port 16D. The input / display device 16E has a graphic user interface that displays an image of the subject's eye 12 and receives various instructions from the user. An example of the graphic user interface is a touch panel display.

[0019] The control device 16 also includes an image processor 17 connected to an I / O port 16D. The image processor 17 generates an image of the subject's eye 12 based on data obtained by the photographing device 14. The control device 16 is connected to a network 130 via a communication interface (I / F) 16F.

[0020] As described above, in FIG. 2 , the control device 16 of the ophthalmic apparatus 110 includes the input / display device 16E, but the present disclosure is not limited to this. For example, the control device 16 of the ophthalmic apparatus 110 may not include the input / display device 16E, but may instead include an input / display device that is physically independent from the ophthalmic apparatus 110. In this case, the display device includes an image processing unit that operates under the control of a display control unit 208 (see FIG. 4 ) of the CPU 16A of the control device 16. The image processing unit may display an SLO image or the like based on an image signal instructed to be output by the display control unit 208.

[0021] The imaging device 14 operates under the control of the CPU 16A of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The imaging optical system 19 includes an optical scanner 22 and a wide-angle optical system 30.

[0022] The optical scanner 22 performs two-dimensional scanning in the X and Y directions with the light emitted from the SLO unit 18. The optical scanner 22 may be any optical element that can deflect a light beam, such as a polygon mirror or a galvanometer mirror, or a combination thereof.

[0023] The wide-angle optical system 30 combines the light from the SLO unit 18 and the light from the OCT unit 20 .

[0024] The wide-angle optical system 30 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a catadioptric optical system combining concave mirrors and lenses. By using a wide-angle optical system using an elliptical mirror or a wide-angle lens, it becomes possible to photograph the retina in the peripheral part of the fundus as well as the center of the fundus.

[0025] When a system including an elliptical mirror is used, the system using the elliptical mirror described in International Publication WO2016 / 103484 or International Publication WO2016 / 103489 may be used, the disclosures of which are each incorporated herein by reference in their entirety.

[0026] The wide-angle optical system 30 enables observation of the fundus over a wide field of view (FOV) 12A. The FOV 12A indicates the range that can be photographed by the imaging device 14. The FOV 12A can be expressed as a field of view. In this embodiment, the field of view can be defined by an internal illumination angle and an external illumination angle. The external illumination angle is the illumination angle of the light beam irradiated from the ophthalmic device 110 to the subject's eye 12, determined with the pupil 27 as the reference point. The internal illumination angle is the illumination angle of the light beam irradiated to the fundus, determined with the center O of the eyeball as the reference point. The external illumination angle and the internal illumination angle correspond to each other. For example, if the external illumination angle is 120 degrees, the internal illumination angle corresponds to approximately 160 degrees. In this embodiment, the internal illumination angle is 200 degrees.

[0027] Here, an SLO fundus image captured at an internal illumination angle of 160 degrees or more is referred to as a UWF-SLO fundus image. UWF stands for Ultra Wide Field. The wide-angle optical system 30, which provides an ultra-wide field of view (FOV) of the fundus, can capture an image of the area extending from the posterior pole of the fundus of the subject's eye 12 beyond the equator, enabling the imaging of structures present in the peripheral area of ​​the fundus.

[0028] The ophthalmologic apparatus 110 can capture an image of an area 12A with an internal illumination angle of 200°, with the center O of the eyeball of the subject's eye 12 as the reference position. Note that the internal illumination angle of 200° corresponds to an external illumination angle of 110° with the pupil of the eyeball of the subject's eye 12 as the reference. In other words, the wide-angle optical system 30 irradiates the laser light from the pupil at an angle of view with an external illumination angle of 110°, and captures an image of a fundus area of ​​200° with an internal illumination angle.

[0029] 2, the SLO system is realized by a control device 16, an SLO unit 18, and an imaging optical system 19. The SLO system includes a wide-angle optical system 30, and therefore enables fundus imaging with a wide FOV 12A.

[0030] The SLO unit 18 includes a light source 40 for B light (blue light), a light source 42 for G light (green light), a light source 44 for R light (red light), and a light source 46 for IR light (infrared light (e.g., near-infrared light)), as well as optical systems 48, 50, 52, 54, and 56 that reflect or transmit the light from the light sources 40, 42, 44, and 46 and guide them into a single optical path. The optical systems 48 and 56 are mirrors, and the optical systems 50, 52, and 54 are beam splitters. The B light is reflected by the optical system 48, passes through the optical system 50, and is reflected by the optical system 54; the G light is reflected by the optical systems 50 and 54; the R light is transmitted through the optical systems 52 and 54; and the IR light is reflected by the optical systems 52 and 56 and is each guided into a single optical path.

[0031] The SLO unit 18 is configured to be switchable between a light source that emits laser light of different wavelengths or a combination of light sources that emit light, such as a mode that emits R light and G light and a mode that emits infrared light. In the example shown in Fig. 2, the SLO unit 18 includes four light sources: a B light source 40, a G light source 42, an R light source 44, and an IR light source 46; however, the present disclosure is not limited to this. For example, the SLO unit 18 may further include a white light source and emit light in various modes, such as a mode that emits G light, R light, and B light, or a mode that emits only white light.

[0032] Light incident on the imaging optical system 19 from the SLO unit 18 is scanned in the X and Y directions by the optical scanner 22. The scanned light passes through the wide-angle optical system 30 and the pupil 27 and is then irradiated onto the fundus. The light reflected by the fundus passes through the wide-angle optical system 30 and the optical scanner 22 and is incident on the SLO unit 18.

[0033] The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits all light except B light from the posterior segment (fundus) of the eye 12, and a beam splitter 58 that reflects G light and transmits all light except G light from the light that has passed through the beam splitter 64. The SLO unit 18 also includes a beam splitter 60 that reflects R light and transmits all light except R light from the light that has passed through the beam splitter 58. The SLO unit 18 also includes a beam splitter 62 that reflects IR light from the light that has passed through the beam splitter 60. The SLO unit 18 also includes a B light detection element 70 that detects B light reflected by the beam splitter 64, a G light detection element 72 that detects G light reflected by the beam splitter 58, an R light detection element 74 that detects R light reflected by the beam splitter 60, and an IR light detection element 76 that detects IR light reflected by the beam splitter 62.

[0034] Light (light reflected by the fundus) incident on the SLO unit 18 via the wide-angle optical system 30 and the optical scanner 22 is reflected by the beam splitter 64 and received by the B light detection element 70 in the case of B light, and is reflected by the beam splitter 58 and received by the G light detection element 72 in the case of G light. The incident light is transmitted through the beam splitter 58 in the case of R light, reflected by the beam splitter 60, and received by the R light detection element 74. The incident light is transmitted through the beam splitters 58 and 60 in the case of IR light, reflected by the beam splitter 62, and received by the IR light detection element 76. The image processor 17, which operates under the control of the CPU 16A, generates a UWF-SLO image using signals detected by the B light detection element 70, the G light detection element 72, the R light detection element 74, and the IR light detection element 76.

[0035] A UWF-SLO image generated using a signal detected by the B light detecting element 70 is referred to as a B-UWF-SLO image (B-color fundus image). A UWF-SLO image generated using a signal detected by the G light detecting element 72 is referred to as a G-UWF-SLO image (G-color fundus image). A UWF-SLO image generated using a signal detected by the R light detecting element 74 is referred to as an R-UWF-SLO image (R-color fundus image). A UWF-SLO image generated using a signal detected by the IR light detecting element 76 is referred to as an IR-UWF-SLO image (IR fundus image). UWF-SLO images include R-color fundus images, G-color fundus images, B-color fundus images, and even IR fundus images. Fluorescent UWF-SLO images obtained by capturing fluorescence are also included.

[0036] The control device 16 also controls the light sources 40, 42, and 44 to emit light simultaneously. By simultaneously capturing images of the fundus of the subject's eye 12 using B, G, and R light, a G-color fundus image, a R-color fundus image, and a B-color fundus image, each of which corresponds to a different position, are obtained. An RGB color fundus image is obtained from the G-color fundus image, the R-color fundus image, and the B-color fundus image. By simultaneously controlling the light sources 42 and 44 to emit light simultaneously and simultaneously capturing images of the fundus of the subject's eye 12 using G and R light, a G-color fundus image and a R-color fundus image, each of which corresponds to a different position, are obtained. An RG color fundus image is obtained from the G-color fundus image and the R-color fundus image. A full-color fundus image may also be generated using the G-color fundus image, the R-color fundus image, and the B-color fundus image.

[0037] The wide-angle optical system 30 makes the field of view (FOV) of the fundus an ultra-wide angle, and can capture an image of the area from the posterior pole of the fundus of the subject's eye 12 beyond the equator.

[0038] Image data of the SLO image is sent from the ophthalmologic apparatus 110 to the server 140 via the communication interface 16F and stored in the storage device 254 (FIG. 3).

[0039] The OCT system is realized by the control device 16, the OCT unit 20, and the imaging optical system 19. The OCT system includes a wide-angle optical system 30, which enables OCT imaging of the peripheral portion of the fundus, similar to the above-described SLO fundus image capture. That is, the wide-angle optical system 30, which provides an ultra-wide field of view (FOV) of the fundus, enables OCT imaging of the region extending from the posterior pole of the fundus to beyond the equator of the subject's eye 12. OCT data of structures present in the peripheral portion of the fundus, such as the choroidal artery, can be acquired, and tomographic images of the choroidal blood vessels, such as the choroidal artery, and 3D structures of the choroidal blood vessels, such as the choroidal artery, can be obtained by image processing the OCT data.

[0040] The OCT unit 20 includes a light source 20A, a sensor (detecting element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

[0041] Light emitted from the light source 20A is branched by the first optical coupler 20C. One of the branched beams is collimated by the collimating lens 20E as measurement light and then enters the imaging optical system 19. The measurement light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus. The measurement light reflected by the fundus passes through the wide-angle optical system 30 and enters the OCT unit 20, and then passes through the collimating lens 20E and the first optical coupler 20C and enters the second optical coupler 20F.

[0042] The other light beam emitted from the light source 20A and branched by the first optical coupler 20C is incident as reference light on the reference optical system 20D, passes through the reference optical system 20D, and enters the second optical coupler 20F.

[0043] The light beams incident on the second optical coupler 20F, i.e., the measurement light beam reflected by the fundus and the reference light beam, interfere with each other at the second optical coupler 20F to generate interference light. The interference light beam is received by the sensor 20B. The image processor 17, which operates under the control of the image processing unit 206 (see FIG. 4), generates OCT data detected by the sensor 20B. The image processor 17 can also generate OCT images, such as tomographic images and en-face images, based on the OCT data.

[0044] By using the wide-angle optical system 30, the ophthalmic apparatus 110 can scan an area 12A with an internal illumination angle of 200°. That is, by controlling the optical scanner 22, OCT imaging of a predetermined range is performed. The ophthalmic apparatus 110 can generate OCT data by the OCT imaging.

[0045] Therefore, the ophthalmologic apparatus 110 can generate OCT images, such as tomographic images (B-scan images) of the fundus, OCT volume data, and en-face images (frontal images generated based on the OCT volume data) that are cross sections of the OCT volume data. Needless to say, the OCT images include OCT images of the center of the fundus (the posterior pole of the eyeball where the macula, optic disc, etc. are present).

[0046] The OCT data (or image data of the OCT image) is sent from the ophthalmologic apparatus 110 to the server 140 via the communication interface 16F and stored in the storage device 254 described in FIG.

[0047] In this embodiment, the light source 20A is exemplified as a wavelength-swept type SS-OCT (Swept-Source OCT), but various types of OCT systems may also be used, such as SD-OCT (Spectral-Domain OCT) and TD-OCT (Time-Domain OCT).

[0048] Next, the configuration of the electrical system of the server 140 will be described with reference to FIG. 3 . As shown in FIG. 3 , the server 140 includes a computer main unit 252. The computer main unit 252 has a CPU 262, a RAM 266, a ROM 264, and an input / output (I / O) port 268. The input / output (I / O) port 268 is connected to a storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I / F) 258. The storage device 254 is configured, for example, with a non-volatile memory. The input / output (I / O) port 268 is connected to the network 130 via the communication interface (I / F) 258. Therefore, the server 140 can communicate with the ophthalmologic apparatus 110 and the viewer 150.

[0049] The ROM 264 or the storage device 254 stores an image processing program.

[0050] The ROM 264 or the storage device 254 is an example of a "memory" in the present disclosure. The CPU 262 is an example of a "processor" in the present disclosure. The image processing program is an example of a "program" in the present disclosure.

[0051] The server 140 stores each piece of data received from the ophthalmologic apparatus 110 in the storage device 254 .

[0052] Next, various functions realized by the CPU 262 of the server 140 executing the image processing program will be described with reference to Fig. 4. As shown in Fig. 4, the image processing program executed by the CPU 262 has an imaging control function, an image processing function, and a display control function. When the CPU 262 executes the image processing program having these functions, the CPU 262 functions as an imaging control unit 204, an image processing unit 206, and a display control unit 208. The image processing unit 206 is an example of an "acquisition unit," a "detection unit," an "extraction unit," a "generation unit," and an "estimation unit" of the present disclosure.

[0053] Next, the image processing executed by the server 140 will be described in detail with reference to Fig. 5. The image processing (image processing method) shown in Fig. 5 is realized by the CPU 262 of the server 140 executing an image processing program.

[0054] The image processing program is executed when an instruction to start analysis is received, for example, when an instruction to start analysis is received by an operator operating the keyboard 255K of the management server 140 or when an instruction to start analysis is received by the management server 140 from the viewer 150. Note that the image processing program may be executed when the management server 140 receives a captured fundus image from the ophthalmologic device 110.

[0055] In step S10, the imaging control unit 204 executes an initial process for the analysis process, which will be described later. In the initial process, various parameters are initially set. The initial process also includes a process of acquiring information indicating the type of analysis process. In this embodiment, the information indicating the type of analysis process is information indicating either fluorescein angiography analysis or OCT analysis. Note that step S10 may also include a process in which the imaging control unit 204 instructs the ophthalmologic apparatus 110 to capture an SLO image using the SLO unit 18 and an OCT image using the OCT unit 20 to capture a fundus image.

[0056] In step S20, the image processing unit 206 determines whether the information indicating the type of analysis processing indicates fluorescein angiography analysis, and if the determination is positive, the processing proceeds to step S30, and if the determination is negative, the processing proceeds to step S40.

[0057] In step S30, the image processing unit 206 executes fluorescein angiography analysis processing, and in step S50, the display control unit 208 outputs analysis data of the analysis results, thereby terminating this processing routine. Meanwhile, in step S40, the image processing unit 206 executes OCT analysis processing, and in step S50, the display control unit 208 outputs analysis data of the analysis results, thereby terminating this processing routine. Note that the image processing unit 206 may execute the processing of step S40 after step S30, or step S40 after step S30. In these cases, in step S50, the display control unit may output analysis data of the analysis results of both the fluorescein angiography analysis and the OCT analysis processing, thereby terminating this processing routine.

[0058] Next, the fluorescein fundus angiography analysis process will be described with reference to FIG.

[0059] The fluorescein fundus angiography analysis process is a process for analyzing the state of choroidal blood vessels, including choroidal arteries, using a contrast agent such as indocyanine green.

[0060] In step S102, the image processing unit 206 acquires a fundus image. Specifically, an early fundus image after administration of a contrast agent is acquired from the storage device 254. That is, among fundus images captured in a time series, the fundus image (e.g., an SLO image) captured within a predetermined time range from the time a predetermined time has elapsed since administration of the contrast agent until a separately predetermined time has elapsed is acquired as the early fundus image. The predetermined time range may be a time range experimentally determined as the initial period during which the contrast agent flows through the choroidal vessels, or a predetermined time range within which the contrast agent is estimated to flow through the choroidal vessels. Alternatively, multiple images captured in a time series may be classified into early images (the first half of the image) and late images (the second half of the image), and the early fundus image may be selected from the classified early images.

[0061] Fig. 7 shows an example of a fundus image of choroidal vessels, including choroidal arteries, based on an early stage UWF-SLO image of fluorescence. Fig. 8 shows an example of fundus images showing fundus images before and after contrast enhancement in time series. As shown in Fig. 8, choroidal vessels are not visible in fundus image Gt0 taken before contrast enhancement is initiated. However, in the early stage when the contrast enhancement begins to flow through the choroidal vessels, most of the contrast enhancement flows through the choroidal arteries, and fluorescence appears primarily in the choroidal arteries in fundus image Gt1. Furthermore, in fundus image Gt2 taken after the contrast enhancement has filled the choroidal vessels, fluorescence appears in almost all blood vessels, including arteries and veins, in the choroid.

[0062] Next, in step S104 of FIG. 6 , the image processing unit 206 performs brightness adjustment. This brightness adjustment is a process for enhancing choroidal vessels (e.g., choroidal arteries) that appear in the fluorescent UWF-SLO image. The process of enhancing choroidal vessels is a process for increasing the ratio between the brightness of the background image and the brightness of the vessel image compared to the ratio before brightness adjustment. For example, this process includes normalizing the brightness of the difference between the maximum and minimum brightness values ​​to a predetermined brightness range or adjusting it as contrast. Other examples include adjustments such as eliminating the distribution of minimum brightness values ​​as noise, assigning a bias brightness value to the brightness value, or multiplying the brightness value by a predetermined coefficient. In other words, the process of step S104 corresponds to adjusting the brightness value so as to increase the dynamic range of the choroidal vessel image in the fundus image.

[0063] In step S106, the image processing unit 206 performs a first coordinate transformation. The first coordinate transformation is a polar coordinate transformation that transforms coordinates in a Cartesian coordinate system into coordinates in a polar coordinate system. The origin in the polar coordinate transformation may be set at a predetermined position such as the center of the fundus image, a position indicated by an operator, a position determined by a fundus structure such as a position between the macula and the optic disc, or the position on the fundus image where the brightness value is maximum.

[0064] In step S108, the image processing unit 206 performs a filter process on the fundus image that has been subjected to the polar coordinate transformation as the first coordinate transformation. The filter process can be an image filter that enhances an image in which brightness is continuous in a predetermined direction, and in this embodiment, Gabor filtering is applied.

[0065] The choroidal artery runs radially from a predetermined position toward the periphery. Therefore, the choroidal artery is considered to include a linear image with continuous brightness in a predetermined direction from the predetermined position toward the periphery. Therefore, the fundus image is transformed into polar coordinates, and an image with continuous brightness in the predetermined direction is extracted by filter processing. This makes it possible to emphasize the choroidal artery.

[0066] 9A is an example of an image obtained by polar coordinate conversion and filtering of a fundus image captured by the ophthalmologic apparatus 110. As shown in FIG. 9A, blood vessels running in a predetermined direction corresponding to the choroidal arteries are emphasized.

[0067] Next, in step S110 of Fig. 6, the image processing unit 206 performs a second coordinate transformation. The second coordinate transformation is an orthogonal coordinate transformation that transforms coordinates in the polar coordinate system into coordinates in the orthogonal coordinate system. By this orthogonal coordinate transformation, the image in which the blood vessels running in a predetermined direction corresponding to the choroidal artery are emphasized is inversely transformed from the polar coordinate system to an image in the orthogonal coordinate system.

[0068] Fig. 10 is a diagram showing a fundus image Gt1A as an example of an image obtained by inversely converting an image in a polar coordinate system into an image in a Cartesian coordinate system. As shown in Fig. 10, the image running in a predetermined direction corresponding to the choroidal artery is emphasized compared to the image shown in Fig. 7. That is, the choroidal artery is clearly formed as an image in the fundus image taken early after the administration of a contrast agent.

[0069] Next, in step S112 of FIG. 6 , the image processing unit 206 detects a first feature amount of the choroidal blood vessels. The first feature amount of the choroidal blood vessels is information indicating the degree to which the image of the choroidal artery appears in the fundus image taken early after the administration of the contrast agent. For example, the image of the choroidal artery is detected on the fundus image that has been inversely transformed into an image in a Cartesian coordinate system. The image of the choroidal artery can be detected by measuring pixels that exceed a predetermined brightness value. The ratio of pixels constituting the detected image of the choroidal artery to the pixels in the entire fundus image is set as the first feature amount. This first feature amount makes it possible to quantify the appearance of the choroidal artery in the fundus image taken early after the administration of the contrast agent.

[0070] In step S114, the image processing unit 206 stores the fundus image and data including the first feature amount described above. Specifically, the image data representing the fundus image inversely converted in step S110 and the data representing the first feature amount detected in step S112 are stored in the RAM 266 or the storage device 254, and this processing routine ends.

[0071] Although the above description deals with the case where a photographed fundus image is subjected to polar coordinate transformation, filtering, and Cartesian coordinate transformation, the above-described processing may be performed by setting a portion of the photographed fundus image as the image to be processed. For example, an image included in an area surrounded by curves and lines in the shape of a circle, ellipse, or polygon of a predetermined size or a size specified by an operator on the photographed fundus image may be extracted as the image to be processed. Furthermore, the fundus image may include predetermined structures present on the fundus, such as the macula and optic disc.

[0072] The stored data (step S114) is output by the display control unit 208 as analysis data.

[0073] Next, the output of analysis data based on the stored data will be described. The analysis data is included in a display screen for displaying an image (2D image) of the choroidal blood vessels, including the choroidal arteries. The display screen is generated by the display control unit 208 of the server 140 based on a user instruction and output as an image signal to the viewer 150. The viewer 150 displays the display screen on the display based on the image signal.

[0074] A display screen 500A is shown in Fig. 11. As shown in Fig. 11, the display screen 500A has an information area 502 and an image display area 504A.

[0075] The information area 502 has a patient ID display field 512, a patient name display field 514, an age display field 516, a visual acuity display field 518, a right eye / left eye display field 520, and an axial length display field 522. In each display area from the patient ID display field 512 to the axial length display field 522, the viewer 150 displays the respective information based on the information received from the server 140.

[0076] The image display area 504A is an area that mainly displays an image of the subject's eye, etc. The image display area 504A is provided with display fields for displaying the fundus image Gt1 in the state (early stage) in which the contrast agent has just started to flow through the choroidal blood vessels, and the fundus image Gt1A in which the choroidal artery is clearly formed as an image. Although not shown in the figure, the image display area 504A may be provided with fields that function as a remarks column for displaying the patient's treatment history and for allowing the ophthalmologist, who is the operator, to arbitrarily input the results of his or her observations and diagnosis.

[0077] FIG. 11 shows that the display screen 500A includes an early-stage fundus image Gt1 as a UWF-SLO image and a fundus image Gt1A in which the choroidal artery has been clearly image-processed as an image showing the analysis results.

[0078] As described above, by performing image processing including fluorescein angiography analysis processing, choroidal arteries are extracted based on an image showing choroidal blood vessels, and an image is generated in which the choroidal arteries are emphasized, making it possible to visualize the choroidal arteries in an image of the choroid photographed by fluorescein angiography.

[0079] In the above, a first feature amount of the choroidal blood vessels is detected, and this first feature amount makes it possible to quantify the state of the choroidal arteries that appear in fundus images taken early after administration of a contrast agent.

[0080] In the above, we have described a case where a first coordinate transformation is performed on the fundus image in step S104, a filtering process is performed in step S108, and then a second coordinate transformation is performed in step S110. However, in the case of fluorescent fundus angiography analysis, the first coordinate transformation in step S106 and the second coordinate transformation in step S110 may be omitted.

[0081] The processing of step S106 and step S108 when step S110 is omitted will be described. FIG. 9B is a conceptual diagram illustrating the setting of regions for each predetermined central angle centered on a predetermined position on a fundus image captured by the ophthalmologic apparatus 110. As shown in FIG. 9B , the image processing unit 206 sets regions for each predetermined central angle centered on a predetermined position on the fundus image, and performs a filter process to enhance an image with continuous brightness in a predetermined direction for each region. The filter process is performed for each of a plurality of predetermined directions extending from a predetermined central position of the fundus image toward the periphery in the target region. By combining the obtained images with different enhancement directions at a predetermined mixing ratio, an image with continuous brightness in a direction extending from a predetermined central position of the fundus image toward the periphery in the target region can be extracted. By performing the filter process on all set regions, it is possible to obtain a fundus image in which the choroidal arteries extending radially from the predetermined position are enhanced without performing polar coordinate transformation.

[0082] In the above description, a case where a fluorescein fundus angiography analysis process is performed on a fluorescein fundus image using a contrast agent has been described. However, the technology disclosed herein is not limited to performing the fluorescein fundus angiography analysis process on a fluorescein fundus image using a contrast agent. For example, the fluorescein fundus angiography analysis process may be performed on an image in which blood vessels are visualized without using a contrast agent, such as an OCTA image captured by OCT angiography.

[0083] Next, the OCT analysis process will be described with reference to Fig. 12. The OCT analysis process is a process for analyzing the state of choroidal blood vessels, including choroidal arteries, using OCT images.

[0084] In step S202, the image processing unit 206 acquires OCT volume data including the choroid corresponding to the fundus image from the storage device 254. Next, in step S204, the image processing unit 206 performs preprocessing such as blurring to remove noise components, and then in step S206, performs extraction processing of choroidal blood vessels. Gaussian blurring, which eliminates the influence of speckle noise, can be used as the blurring processing.

[0085] 13 , the OCT volume data 400 is obtained by OCT imaging of the subject's eye using the ophthalmologic apparatus 110 and is of a predetermined area, for example, a rectangular area of ​​6 mm × 6 mm. A plurality of planes at different depths are set in the OCT volume data 400. From the OCT volume data 400, a region where choroidal blood vessels are predicted to exist is extracted. This extraction process can extract, as the choroidal region, a surface (bottom surface 400E) of a region deeper than the retinal pigment epithelium (hereinafter referred to as the RPE layer) (a region farther from the RPE layer as viewed from the center of the eyeball) from a surface a predetermined number of pixels below, for example, 10 pixels below, the RPE layer.

[0086] The image processing unit 206 then removes noise components (preprocessing) and generates multiple en-face images corresponding to each of the multiple planes that have been set. The generated en-face images corresponding to each plane are stored in the RAM 266 by the image processing unit 206. In this manner, the image processing unit 206 generates and stores en-face images. The en-face images may be generated from the pixel values ​​of pixels present on the corresponding plane, or may be generated by extracting a group of pixels in the shallow direction and a group of pixels in the deep direction that include the corresponding plane from the OCT volume data 400 and calculating the pixel value as the average or median brightness value of these pixel groups. Image processing such as noise removal may be used to calculate the pixel values.

[0087] Instead of using the plane 10 pixels below the RPE layer as the reference, for example, a plane 10 pixels below the Bruch's membrane located directly below the RPE layer may be used. To specify the position 10 pixels below, the plane may be 10 pixels below in the direction of the A-scan when the OCT volume data was generated. The number of pixels defining the plane is not limited to 10 pixels, and any number of pixels may be set. Alternatively, the plane may be defined in terms of a length such as millimeters or micrometers instead of the number of pixels.

[0088] The choroidal vessel extraction process can be a line extraction process that can extract vessels while reflecting their vascular shapes. Therefore, the image processing unit 206 extracts choroidal vessels from the OCT volume data 400D by performing a line extraction process on the pre-processed OCT volume data 400D. Specifically, the image processing unit 206 performs image processing using, for example, an eigenvalue filter or a Gabor filter to extract linear vessel regions from the OCT volume data 400D. In the OCT volume data 400D, vascular regions are represented by low-brightness pixels (dark pixels), and regions with consecutive low-brightness pixels remain as vascular portions.

[0089] In step S208, the image processing unit 206 performs field synthesis to synthesize the extracted multiple choroid result images, derives an image of an area larger than the area obtained by a single OCT imaging, and stores the visualized image in the RAM 266. Specifically, the above-described processing is performed on each of the different fields of view, i.e., multiple different areas, of the region of interest obtained by the OCT imaging, and synthesizes the resulting images.

[0090] FIG. 14 conceptually illustrates the process up to choroidal vessel extraction. In the example shown in FIG. 14, in order to make the area in which choroidal vessels are visualized larger than the area (rectangular area of ​​a predetermined area) obtained by a single OCT scan, multiple (three) areas are used, with at least some overlapping areas. OCT volume data 400-1, 400-2, and 400-3 obtained by OCT scan of each of the multiple (three) areas are acquired from the storage device 254, and after preprocessing (step S204), choroidal vessels are extracted (step S206). Then, the choroidal vessel images MG1, MG2, and MG3 are combined to obtain a choroidal vessel image MG-A. It is preferable to combine the images using en-face images generated from the choroidal vessel images MG1, MG2, and MG3. To combine en-face images with the same depth, the choroidal vessels are combined on the choroidal vessel image MG-A. In this way, by three-dimensionally combining choroidal vessels extracted from multiple different regions obtained by OCT imaging, it is possible to visualize choroidal vessels over a larger range than the range of choroidal vessels obtained by a single OCT imaging. If the range the operator wishes to observe is within the range of choroidal vessels obtained by a single OCT imaging, step S208 may be skipped, and step S210 may be performed after step S206. The choroidal vessel images MG1, MG2, and MG3 may be combined, for example, by the following method. The binarized images of the choroidal vessel images MG1, MG2, and MG3 are divided into small regions using, for example, the existing SLIC method. For overlapping regions when the choroidal images are combined, the choroidal vessel region on the choroidal vessel image MG-A may be identified using a logical OR. In other words, if a vascular region is identified in either of the overlapping choroidal vessel images, it is also identified as a vascular region in the combined choroidal vessel image MG-A. Furthermore, a logical sum is not necessary. For example, for the overlapping region, the choroidal vessel image with the larger lumen volume may be identified as the vascular region in the choroidal vessel image MG-A. Furthermore, each of the choroidal vessel images MG1, MG2, and MG3 is not limited to a binary image, and may be a grayscale image.

[0091] Next, in step S210 of FIG. 12 , the image processing unit 206 executes a process for extracting the center position of choroidal blood vessels. The process for extracting the center position of choroidal blood vessels is a process for deriving a center line passing through the center of the choroidal blood vessel. In this embodiment, the center line of the blood vessel is a representative line indicating the direction in which the blood vessel runs. Note that as long as the running direction of the blood vessel is known, the blood vessel center line may be a line passing through a position slightly shifted from the center of the choroidal blood vessel. Note that the center position extraction process is also called a skeletonization process or an image thinning process, and refers to the process of converting an image into a line drawing.

[0092] Here, the process of extracting the center position of the choroidal blood vessels will be described with reference to Figures 15 and 16. In this embodiment, for the choroidal blood vessels, a center line shown in two dimensions is derived from each en-face image, and a center line shown in three dimensions is derived from information on the extracted multiple two-dimensional center lines.

[0093] In step S222 shown in FIG. 15 , the image processing unit 206 acquires an image for extracting the center position. The image for extracting the center position is a two-dimensional image including an image of the choroidal blood vessels. Specifically, one en-face image may be extracted from the en-face images generated from the OCT volume data 400, or a two-dimensional image generated by image processing multiple en-face images may be applied. For example, as shown in FIG. 16 , one en-face image included in the choroidal blood vessel image MG-A is acquired as the image MG-a for extracting the center position.

[0094] In step S224, the image processing unit 206 executes a center position extraction process to derive a two-dimensional center line using the image acquired in step S222. Specifically, a line segment representing the two-dimensional center line is derived for the acquired two-dimensional image (e.g., a single en-face image). As an example of the two-dimensional center position extraction process, it is possible to repeatedly expand and contract an image representing the choroidal vascular region until a line segment is obtained, and to use the finally obtained line segment (e.g., a group of pixels in which one pixel is continuous in the vascular direction) as the two-dimensional center line SK. For example, as shown in FIG. 16, for an image MG-a, which is a single en-face image, a representative line along which the choroidal vessels run (shown as a dotted line in FIG. 16) is derived as the center line SK of the choroidal vessels.

[0095] In step S226, the image processing unit 206 estimates a depth direction position (Z coordinate value) from the two-dimensional center line SK derived in step S224 and performs three-dimensional center position extraction processing. Specifically, the image processing unit 206 assigns a depth direction Z coordinate value to the two-dimensionally expressed center line SK to derive a three-dimensional center line SK. For example, the choroidal vessels including the two-dimensional center line SK are located at a depth corresponding to the extraction position of the en-face image. In this case, assuming that the blood vessels are cylindrical, the cross section at the three-dimensional center line SK is a cross section positioned evenly on the retina side and the sclera side. Therefore, the extraction position of the en-face image is assigned as an estimated Z coordinate value to derive the three-dimensional center line SK. The Z coordinate value estimation processing may be performed by applying morphology processing to data created by extracting choroidal vessels in the depth direction (Z direction) in an arbitrary XY plane along the center line, i.e., data indicating the cross-sectional shape of the choroidal vessels. The Z coordinate value estimation process may also be performed using a graph shortest path search process, which extracts choroidal blood vessels from an arbitrary XY plane along the center line only at branch points where the extracted center line branches into multiple center lines and at the end points of the center line, estimating Z coordinate values, and then connecting the estimated Z coordinate values ​​by the shortest distance to estimate the Z coordinate value of the center line.

[0096] In step S228, the image processing unit 206 stores data indicating the three-dimensional center line derived in step S226 in the RAM 266 or the storage device 254, and ends the process. In Fig. 16, a conceptual image in which the three-dimensional center line SK derived as described above is superimposed on the choroidal vessel image MG-A is shown as choroidal vessel image MG-Ax.

[0097] The above describes a case where one en-face image is extracted from en-face images generated from OCT volume data 400 and a three-dimensional centerline is derived (Z coordinate value is estimated). The technology disclosed herein is not limited to extracting and using one en-face image. For example, an image generated by combining multiple en-face images may be used as a single en-face image.

[0098] 17 shows a conceptual diagram relating to the application of an image obtained by combining multiple en-face images as a single en-face image. In FIG. 17, a conceptual image MGv is shown in which the brightness of multiple en-face images generated from OCT volume data 400 is averaged. Also shown is a single en-face image MGs extracted from the multiple en-face images. A composite image MGc is shown, which is generated by combining multiple en-face images. A graph image MGg is shown showing the relationship between depth and the area occupied by choroidal vessels in the multiple en-face images.

[0099] As shown in FIG. 17 , compared to the conceptual image MGv, the extracted en-face image MGs contains missing portions of choroidal blood vessels. Therefore, the accuracy of the obtained center line SK may be reduced. This is because the choroidal blood vessels meander in the depth direction (Z direction). For example, if the choroidal blood vessels run in the +Z direction relative to the Z direction (running upward, running toward the center of the eyeball), only a portion of the blood vessels will be extracted from the en-face image at a specific depth position. Furthermore, the choroidal blood vessels will be extracted from the en-face image at a position shallower than the specific depth position. Thus, a single en-face image MGs may contain missing portions of the choroidal blood vessels. Therefore, a composite image MGc, which is a composite of multiple en-face images, is used as a single en-face image. As shown in the graph image MGg, the composite image MGc may be an en-face image in which the area of ​​the choroidal vessels exceeds a predetermined value (e.g., a predetermined number of en-face images from the maximum area). By using a composite image of multiple en-face images as a single en-face image, it is possible to faithfully represent the center line SK of each vessel covering the choroidal vessels while estimating the Z-direction running direction of the choroidal vessels. It is not necessary to generate the composite image MGc from multiple en-face images. A center position extraction process is performed on each of multiple en-face images acquired at different depth positions. The three-dimensional center line SK may be derived by combining information on the center line extracted from each en-face image and the Z coordinate value. Furthermore, the circularity / ellipticity of the vessel cross section may be calculated using the en-face images at each depth position. The circularity / ellipticity is calculated from the depth information of the en-face image and the area occupied by the blood vessel on the en-face image. If the area occupied changes in proportion to the change in depth, the shape will be circular, and if the change in area occupied is greater or smaller than the change in depth, the shape will be elliptical.

[0100] Upon completion of the center position extraction process, the image processing unit 206 proceeds to step S212 shown in FIG. 12 . In step S212, the image processing unit 206 executes a process of detecting second feature amounts of choroidal blood vessels. The process of detecting second feature amounts of choroidal blood vessels is a process of detecting information indicating characteristics related to the shape of choroidal blood vessels. In this embodiment, as described below, the cross-sectional area and blood vessel diameter of choroidal blood vessels are detected as examples of the second feature amounts of choroidal blood vessels. Note that the step of detecting the second feature amounts of choroidal blood vessels may use a fundus image on which center position extraction process has not been performed. In this case, step S212 may be executed after step S206 or after step S208.

[0101] Here, the process of detecting the second feature amount of the choroidal blood vessels will be described with reference to FIG.

[0102] 18, the image processing unit 206 acquires an image of the choroidal vessels whose central positions have been extracted (see image MG-Ax shown in FIG. 16). Next, in step S234, the image processing unit 206 sets a plurality of analysis regions for the acquired image of the choroidal vessels.

[0103] The analysis region is defined as a region extending from a first region representing a first cross-section at a position a first distance away from a predetermined position determined relative to the choroidal region to a second region representing a second cross-section at a position a second distance away from the predetermined position, the second distance being different from the first distance. For example, each of the first and second cross-sections may be a cross-section having a curved line centered at the predetermined position as its bottom contour. In a specific example, as shown in FIGS. 19 and 20 , the analysis region is defined as a concentric region (a cylindrical region including the depth direction) obtained by dividing the choroidal region in the depth direction of the choroidal region using concentric circles centered at a predetermined position O as its contour. FIG. 19 shows a diagram of an analysis region defined by multiple concentric circles centered at a predetermined position O on a plan view of the choroidal region. FIG. 20 shows a perspective view of analysis regions AN1, AN2, and AN3 obtained by dividing a flat choroidal region by cylindrical contours having multiple concentric circles centered at the predetermined position O as their bottom contours.

[0104] The predetermined position O when setting the analysis region may be set manually, such as by an operator setting a position designated as a region of interest while checking an image of the choroidal blood vessels. For example, the analysis region may be set so as to perpendicularly cross the blood vessels through which the dilation of the choroidal blood vessels runs. The center of a predetermined structure on the fundus, such as the center of the dilation, or a predetermined position may also be set.

[0105] 20 , the analysis regions are set as follows: analysis region AN1, which is obtained by dividing the choroidal region into circles with radii R1 and R2 (>R1) as their outlines, centered at a predetermined position O; analysis region AN2, which is contoured by circles with radii R2 and R3 (>R2); and analysis region AN3, which is contoured by circles with radii R3 and R4 (>R3). In this way, a plurality of analysis regions are set outward from the predetermined position O in a direction intersecting the depth direction of the choroidal region, or a plurality of analysis regions are set outward from the outer side toward the predetermined position O. Note that adjacent analysis regions AN1, AN2, and AN3 may be partially overlapped, or may be set to be spaced apart at a predetermined interval.

[0106] The analysis region is not limited to a concentric region. For example, it may be an ellipse centered at a predetermined position, an oval centered at a predetermined position, or an arc centered at a predetermined position. In other words, the analysis region may be set so as to be superimposed on the choroidal blood vessels on the image. Furthermore, when forming the concentric region, the analysis region is not limited to being set in a cylindrical shape. For example, it may be a region separated into a plate shape by a curve, such as a part of a concentric sphere.

[0107] 18 , the image processing unit 206 derives second feature values ​​for each of the set analysis regions. In this embodiment, the second feature values ​​of the choroidal vessels are detected by deriving the cross-sectional areas and diameters of the choroidal vessels. Specifically, the image processing unit 206 derives the average diameters and cross-sectional areas of the choroidal vessels using the volumes and lengths of the choroidal vessels.

[0108] FIG. 21 shows a conceptual diagram of an analysis region AN. In the example shown in FIG. 21, the analysis region AN includes a first vascular region BL1 having a first center line SK1 and a second vascular region BL2 having a second center line SK2. In this analysis region AN, the number of pixels in the first vascular region BL1 is calculated and defined as the volume V1 of the first vascular region BL1. The number of pixels along the first center line SK1 is also calculated and defined as the vascular length L1 of the first vascular region BL1. Similarly, the number of pixels in the second vascular region BL2 is calculated and defined as the volume V2 of the second vascular region BL1, and the number of pixels along the first center line SK1 is calculated and defined as the vascular length L2 of the second vascular region BL2. The average cross-sectional area Sa can be derived by dividing the total volume (V) of the vascular regions within the analysis region AN by the total vascular length (L). Sa = (V) / (L) For example, (V) = V1 + V2, (L) = L1 + L2

[0109] Furthermore, assuming that the cross section of the blood vessel is circular, the average blood vessel diameter ra within the analysis region AN can be made to correspond to the radius rb of a circle having a common area with the derived average cross-sectional area Sa: Sa = π (ra) ２ That is, (ra) ２ = (r b ) ２ =Sa / π The average blood vessel diameter ra within the analysis region AN may be calculated using the above-mentioned circularity / ellipticity.

[0110] The average cross-sectional area Sa of the vascular region within the analysis region AN and the average vascular diameter ra, which corresponds to a circular cross section, are derived for each analysis region AN as second feature quantities. The average cross-sectional area Sa and the average vascular diameter ra derived as second feature quantities within the analysis region AN are examples of physical quantities related to the shape of choroidal vessels of the present disclosure. Examples of the second feature quantity include average vascular length and skeleton density (the number of pixels of the center line relative to the number of pixels per unit area in the image or the entire image).

[0111] Next, in step S238, the image processing unit 206 stores data indicating the average cross-sectional area Sa and the average vascular diameter ra, which are the second feature amounts of the analysis region derived in step S236, in the RAM 266 or the storage device 254, and then ends the process. Note that, as the second feature amounts, data indicating the volume V1 and the vascular length L1 of the first vascular region BL1 and the volume V2 and the vascular length L2 of the second vascular region BL2 may be stored in the RAM 266 or the storage device 254. Note that, in step S238, not only the second feature amounts but also position information of the first vascular region BL1 and the second vascular region BL2 relative to the predetermined position O may be stored in the RAM 266 or the storage device 254.

[0112] By setting the above-described multiple analysis regions as different regions, such as concentric circular regions, the operator can observe the state of the choroidal blood vessels in the choroid (e.g., shape distribution, etc.) by displaying the multiple analysis regions in a predetermined order as shown in Fig. 22. In addition, for example, it is possible to detect the tapering / expansion of the choroidal blood vessels as the distance from a predetermined position O increases relative to the average cross-sectional area Sa and average blood vessel diameter ra, the branching positions / number of branches of the choroidal blood vessels, the joining positions / number of joinings of the choroidal blood vessels, and the tortuosity of the choroidal blood vessels in the depth direction (Z direction) along their course.

[0113] Although the above description has been given of a case where multiple analysis regions are set, the technology of the present disclosure is not limited to this. For example, as shown in Fig. 23 , a choroidal region extending up to a predetermined radius Rr from the above-described predetermined position O as the center may be set as the analysis region.

[0114] The stored data (step S214) is output by the display control unit 208 as analysis data.

[0115] Next, the output of analysis data from the saved data will be described. The analysis data is included in a display screen for displaying the analysis results of the OCT analysis. The display screen is generated by the display control unit 208 of the server 140 based on a user instruction and output as an image signal to the viewer 150. The viewer 150 displays the display screen on the display based on the image signal.

[0116] A display screen 500B is shown in Fig. 24. As shown in Fig. 24, the display screen 500B has an information area 502 similar to the display screen 500A, and an image display area 504B.

[0117] The image display area 504B is an area for displaying the analysis results of the above-described OCT analysis processing, etc. The image display area 504B can include a diagram of the analysis region ( FIG. 19 ) with multiple concentric circles centered at a predetermined position O on the plan view of the above-described choroidal region.

[0118] As another example of the output of the analysis data described above, it is possible to apply a modification in which various visualized display images are included on the display screen. In a first modification, as shown in Figure 25, a cross-sectional view at an arbitrary position on the choroidal vessel image (e.g., image MG-Ax shown in Figure 16) of the analysis result can be applied as a diagram of the analysis region.

[0119] The display screen 500C of the first modification has an information area 502 similar to that of the display screen 500A, and image display areas 504Ca, 504Cx, 504Cy, and 504Cz.

[0120] The image display area 504Ca is an area for displaying an image MG-Ax ( FIG. 16 ) of choroidal blood vessels as an analysis result of the OCT analysis process described above. The image display area 504Ca includes a movable frame surface Wa for providing a cross-sectional view of the choroidal blood vessels in the XY plane at any Z coordinate value. The image display area 504Cx is an area for displaying a cross-sectional view of the choroidal blood vessels in the XY plane at any Z coordinate value in conjunction with the movement of the frame surface Wa. Similarly, the image display area 504Ca includes a movable frame surface Wb for providing a cross-sectional view of the choroidal blood vessels in the XZ plane at any Y coordinate value. The image display area 504Cy is an area for displaying a cross-sectional view of the choroidal blood vessels in the XZ plane at any Y coordinate value in conjunction with the movement of the frame surface Wb. The image display area 504Cy also includes a movable frame surface Wc for providing a cross-sectional view of the choroidal blood vessels in the YZ plane at any X coordinate value. The image display area 504Cz is an area that displays a cross section of the image MG-Ax of the choroidal blood vessels in the YZ plane at an arbitrary X coordinate value in conjunction with the movement of the frame surface Wc.

[0121] In this way, in the first modified example, a cross-sectional view at any position on the image of the choroidal blood vessels can be visualized and provided, so that the operator can check the image of the choroidal blood vessels at any position on the choroid.

[0122] In the second modification, as shown in FIG. 26, information showing the analysis results of choroidal blood vessels at any position can be applied in conjunction with the diagram.

[0123] The second modified display screen 500D has an information area 502 similar to that of the display screen 500A, and an image display area 504D.

[0124] The image display area 504D is an area for displaying the image MG-Ax ( FIG. 16 ) of the choroidal vessels as the analysis results of the OCT analysis process described above, etc. When the image processing unit 206 receives a command for an arbitrary position P in the image MG-Ax of the choroidal vessels, information about the cross section of the choroidal vessels at the position P is displayed based on the analysis area AN.

[0125] In this way, in the second modified example, it is possible to provide information about the cross section of the blood vessels at any position while providing an image of the choroidal blood vessels, so that the operator can confirm information about the cross section of the choroidal blood vessels at any specified position.

[0126] In the third modification, the display format of the choroidal vascular image MG-Ax ( FIG. 16 ) resulting from the choroidal vascular analysis can be changed and applied (not shown). For example, the choroidal vascular image can be provided in a display format in which the color of each layer is changed according to the depth position. Furthermore, the choroidal vascular image can be provided in a display format in which it is enlarged or reduced in at least one of the specified X, Y, and Z axes. In this way, by making the display format of the choroidal vascular image changeable, it is possible to provide, as requested, areas that the operator wishes to focus on or not focus on.

[0127] In the fourth modification, as shown in FIG. 27, an image of the choroidal vessels and the center line can be separately provided.

[0128] By separating the image of the choroidal vessels from the centerline, it is possible to detect branches of the centerline, i.e., the divided blood vessels. This allows the operator to check the divided choroidal vessels in the choroid by changing the display format, such as by changing the color, for each divided blood vessel.

[0129] As described above, by performing image processing including OCT analysis processing, an image showing the state of the choroidal blood vessels is provided, making it possible to visualize the choroidal blood vessels in various forms in an image of the choroid photographed by OCT.

[0130] Second Embodiment Next, a second embodiment will be described. Since the second embodiment has substantially the same configuration as the first embodiment, the same parts are denoted by the same reference numerals and detailed description thereof will be omitted. In the first embodiment, the analysis region is set based on a plurality of concentric regions centered at a predetermined position. In the second embodiment, the analysis region is set in consideration of the corresponding region of the choroidal blood vessels. Below, differences from the above embodiment will be mainly described.

[0131] 28 shows a process for detecting second feature values ​​of choroidal blood vessels in consideration of corresponding regions of the choroidal blood vessels according to the present embodiment. In this embodiment, the process shown in FIG. 28 is executed instead of the process shown in FIG. 18 in the above embodiment.

[0132] The above-mentioned analysis region may be affected by the lumen ratio (for example, lumen ratio = size of (blood vessel) / size of (blood vessel + interstitium)). The lumen ratio may be used to check the degree of blood vessel expansion. Note that the lumen ratio may also use the ratio of interstitium to lumen size as an indicator, but it is not limited to the above as long as it indicates the degree of blood vessel and interstitium. Therefore, it is preferable that the analysis region be a region including blood vessels.

[0133] First, in step S302, the image processing unit 206 acquires an image of the choroidal vessels in the same manner as in step S232 shown in Fig. 18 (see, for example, image MG-Ax shown in Fig. 16). Next, in step S304, the image processing unit 206 sets an analysis target region prior to setting multiple analysis regions.

[0134] The analysis target region is a process for determining a single analysis region to be analyzed when analyzing an image of choroidal blood vessels. The process for setting the analysis target region includes a process for fitting the boundary of the analysis target region to the shape of the choroidal blood vessels. This fitting process is an example of a process for setting a minimum polygon that encompasses the pixels of the acquired image (i.e., the choroidal blood vessel image) or a polygon that is equal to or smaller than a predetermined threshold, such as well-known convex hull processing and concave hull processing. The process for setting the analysis target region can set regions in each of the three-dimensional X, Y, and Z directions. In the Z direction, regions showing anatomical features of the eye and regions such as blood vessel regions can be set. In addition, in the X and Y directions, regions with a predetermined radius centered at a predetermined position and regions such as blood vessel regions can be set. Regions may also be set in the X, Y, and Z directions, with the blood vessel region as the target.

[0135] First, the setting of the analysis region in the Z direction will be described. FIG. 29 is a schematic diagram showing a cross section of an eye. As shown in FIG. 29 , the fundus includes the choroid containing blood vessels and has, in order in the depth direction, a first anatomical feature Z10 (e.g., the RPE), a shallowest part Z12 of the blood vessel, a deepest part Z14 of the blood vessel, and a second anatomical feature Z16 (e.g., the boundary between the sclera and the choroid). The analysis region in the Z direction can be set by determining the start point and end point using these locations. Specifically, when setting the analysis region in the Z direction, the start point in the depth direction can be any of the first anatomical feature Z10 (e.g., the RPE), the shallowest part Z12 of the blood vessel, and a predetermined fixed position. Furthermore, the end point in the depth direction can be any of the deepest part Z14 of the blood vessel, the second anatomical feature Z16 (e.g., the boundary between the sclera and the choroid), and a predetermined fixed position.

[0136] Next, the setting of the analysis target region in the XY direction will be described. FIG. 30 is a conceptual diagram illustrating the setting of the analysis target region in the XY direction. To set the analysis target region in the XY direction, a fitting process is performed on the acquired choroidal blood vessel image MG-B. This fitting process can be performed by either setting a region with a predetermined shape as its contour on the choroidal blood vessel image MG-B, or setting a region that includes the choroidal blood vessel image. Specifically, as an example of a predetermined shape as a contour, a region with an arc of radius Rxy centered at a predetermined position O as its contour Cr can be set as the analysis target region. As another example, convex hull processing can be performed to set a polygon that encompasses the pixels of the choroidal blood vessel image, and a region with contour Cv can be set as the analysis target region. Furthermore, the above-described process using radius Rxy and convex hull processing can be combined to set a region with contour Crv as the analysis target region. Note that convex hull processing is a well-known process, and therefore a detailed description thereof will be omitted.

[0137] In the above, convex hull processing was applied as an example of processing to set a region to include a choroidal blood vessel image. However, as shown in FIG. 31 , concave hull processing may also be applied. In concave hull processing, a process is performed to set a polygon that encompasses the pixels of the choroidal blood vessel image, and a region with a contour Cc1 can be set as the analysis target region. In concave hull processing, a curve obtained by performing segmentation processing such as active contour extraction or graph cutting, or an envelope obtained using a curve such as a Bézier curve, can be set as the analysis target region with a contour Cc2. Note that concave hull processing is a well-known process, and therefore a detailed description thereof will be omitted. In addition, while the above-described processes of applying convex hull processing and concave hull processing, etc., have been shown to set the analysis target region, the technology of the present disclosure is not limited to these processes.

[0138] The above describes the setting of the analysis target area in the XY and Z directions. However, by combining these, the analysis target area in the XY and Z directions can be set. For example, a two-dimensional analysis target area in the XY directions can be set, and the analysis range can be expanded in the Z direction by a predetermined distance to set the analysis target area in the XY and Z directions. The analysis target area in the XY and Z directions may also be set as an overlapping area between the expanded XY and Z directions and the analysis target area in the Z direction. Furthermore, a two-dimensional analysis target area in the XY directions may be set, and then the analysis target area in the XY and Z directions may be expanded only in the Z direction to set the analysis target area in the XY and Z directions. Furthermore, the analysis target area in the XY and Z directions may also be set at once. For example, a three-dimensional analysis target area may be set by performing convex hull and concave hull processing on a three-dimensional area. The analysis target area may also be set using machine learning, for example, a learning model trained to input multiple two-dimensional images, three-dimensional images, etc., and output an analysis target area.

[0139] After the setting of the analysis target region is completed, in step S306 shown in Fig. 28, a plurality of analysis regions are set in the same manner as in step S236 (Fig. 18). Note that in step S306, each of the plurality of analysis regions may be set by performing image processing such as the convex hull and concave hull described above, or only a portion of the analysis regions may be set.

[0140] However, analysis processing of regions where choroidal blood vessels are not present or regions that contribute little to analysis of choroidal blood vessels increases the processing load. For example, when analyzing blood vessels extending from vortex veins (VVs) to the posterior pole, analysis processing of regions where blood vessels are not present and ampullae results in excessive processing. For this reason, it is preferable to exclude regions where choroidal blood vessels are not present or regions that contribute little to analysis of choroidal blood vessels from the analysis region. Therefore, in this embodiment, when setting multiple analysis regions, it is possible to exclude specific regions from the analysis region.

[0141] FIG. 32 is a conceptual diagram illustrating the exclusion of a specific region from the analysis region. The example shown in FIG. 32 illustrates an example in which a region without blood vessels is designated as a specific region outside the analysis region. Specifically, when the analysis region is divided into the choroid region in the XY direction of the choroid using contours such as the concentric circles described above, a region without a predetermined number of N or more different blood vessels in at least one of a predetermined region, a region within a predetermined distance, and a region within a predetermined angle is designated as a region outside the analysis region BN. This region BN is excluded from the analysis region. For example, a region of a specified size may be defined at a specified position on a two-dimensional or three-dimensional image, and the region without N or more blood vessels may be designated as region BN. The specified position and size may be preset or may be set by the user. When a contour is used, the region BN may be designated as a region within a specified distance on the inner or outer contour where N or more blood vessels are not present. Alternatively, the region of a predetermined shape defined by the contour may be moved a predetermined distance or rotated by a predetermined angle, and the region where N or more blood vessels do not exist during the movement and rotation may be defined as region BN. Alternatively, the region corresponding to the movement and rotation until N or more blood vessels exist may be defined as region BN. The example in FIG. 32 shows an example in which a partial region ABN, which is a predetermined angle from the center of the concentric circle to the contour, is rotated by a predetermined angle around the center point of the concentric circle, and the region where N or more blood vessels do not exist is defined as region BN. The partial region ABN may have any predetermined shape and is not limited to the example shown in FIG. 32. Furthermore, the partial region ABN is not limited to rotational movement, but may be movement in any of the X, Y, and Z directions, or a combination of these movements.

[0142] Next, a case where a bulging portion is set as an example of a specific region to be excluded from the analysis region will be described. The following process can be applied to set the specific region.

[0143] The first process is a process of setting a corresponding region of the dilation derived by image processing as a specific region. Note that the first process may also set the specific region using machine learning, for example, a learning model trained to input multiple two-dimensional images, three-dimensional images, etc., and output the corresponding region of the dilation. FIG. 33 is a conceptual diagram illustrating the specific region derived in the first process. In the first process, image processing such as known morphology processing is performed on blood vessels in a choroid image to remove specific blood vessels and derive the corresponding region of the dilation. The specific blood vessels are blood vessels that are relatively thin compared to the dilation, such as blood vessels with a diameter less than a specific value, or blood vessels that are thinner than the other blood vessels among multiple blood vessels. The derived corresponding region of the dilation is set as the specific region. In FIG. 33, the corresponding region of the dilation is indicated by diagonal lines.

[0144] The second process derives a central curve (skeleton) for the dilation derived by image processing, and then defines a region of blood vessels connected to the central curve of the dilation as a new specific region. Figure 34 is a conceptual diagram illustrating the specific region derived in the second process. In the second process, for example, a central curve SKv (skeleton) for the dilation derived by image processing in the first process is derived, and a region including the region of blood vessels connected to the central curve SKv of the dilation is defined as the specific region. In Figure 34, the region belonging to the central curve SKv of the dilation is indicated by diagonal lines, and the region of the connected blood vessels is indicated by dotted lines. The above-mentioned dilation central curve SKv (skeleton) can be derived by determining the central curve in advance, such as in step S210 (Figure 12), and then selecting the central curve within the dilation derived in the first process from the central curve. Note that the curve SKv is not limited to being selected from the central curves determined in advance. For example, if there are multiple curves within the ampulla, multiple central curves may be used to select from the multiple curves present within the ampulla, and the curve SKv may be derived to match the size of the ampulla derived in the first process.

[0145] The third process is a process of deriving blood vessels extending in a predetermined direction (e.g., a deep direction) from the above-mentioned analysis center point (e.g., a predetermined position O) and setting the area belonging to the blood vessels as the corresponding area of ​​the dilation portion as a specific area.

[0146] Once the setting of the analysis area is complete, the image processing unit 206 derives the second feature in step S308 of FIG. 28, similar to steps S236 and 238 (FIG. 18), and in step S310, stores the derived data in RAM 266 or the storage device 254, thereby completing the processing.

[0147] By excluding the specific region set as described above from the analysis region, the computational load during analysis can be reduced. Furthermore, by excluding the ampulla from the analysis target, the user can check the image taking into account the direction of blood flow.

[0148] Although the above describes a case where a specific region is set in the region of the dilation, the technology of the present disclosure is not limited to the size of the dilation. For example, a region of a predetermined width may be enlarged or reduced from the region of the dilation. Furthermore, when using blood vessels connected to the dilation, a region may be set that is extended or shortened by a predetermined distance.

[0149] (Other embodiments) In the above embodiment, image processing is performed by the server 140, but the present disclosure is not limited to this, and image processing may be performed by the ophthalmic device 110, the viewer 150, or an additional image processing device further provided on the network 130.

[0150] In the present disclosure, each component (device, etc.) may be present in one or more instances, unless a contradiction arises.

[0151] In the above-described examples, image processing is implemented by a software configuration using a computer. However, the present disclosure is not limited to this, and at least a portion of the processing may be implemented by a hardware configuration. Furthermore, while the above description uses a CPU as an example of a general-purpose processor, the term "processor" refers to a processor in a broad sense and includes general-purpose processors (e.g., a CPU (Central Processing Unit), etc.) and dedicated processors (e.g., a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a programmable logic device, etc.). Therefore, image processing may be performed solely by a hardware configuration, or some of the image processing may be performed by a software configuration and the remaining processing may be performed by a hardware configuration.

[0152] Furthermore, the operations of the above-mentioned processors may not only be performed by a single processor, but may also be performed by multiple processors working together, or may be performed by multiple processors located in physically separate locations working together.

[0153] Furthermore, in order to cause a computer to execute the above-described processing, a program in which the above-described processing is written in computer-processable code may be stored on a storage medium such as an optical disk and distributed.

[0154] While the technology of the present disclosure has been described above using embodiments, the image processing described above is merely an example, and the technical scope of the present disclosure is not limited to the scope described in the above embodiments. Therefore, various modifications or improvements can be made to the above embodiments, such as deleting unnecessary processes, adding new processes, or changing the order of processes, without departing from the spirit of the present disclosure, and such modifications or improvements are also included in the technical scope of the present disclosure.

[0155] All documents, patent applications, and technical standards described herein are incorporated by reference herein to the same extent as if each individual document, patent application, and technical standard were specifically and individually indicated to be incorporated by reference. In addition, the disclosures of Japanese Patent Application Nos. 2023-064528, filed April 11, 2023, and 2023-116319, filed July 14, 2023, are incorporated by reference in their entirety.

Claims

1. An image processing method performed by a processor, comprising: generating front images at different depth positions from two-dimensional tomographic images obtained from a predetermined region on the fundus of the subject's eye; Extracting choroidal vessels from each of the generated frontal images and detecting centerlines of the choroidal vessels; a step of estimating three-dimensional choroidal vessels and center lines of the three-dimensional choroidal vessels based on a plurality of front images in which the center lines have been detected in the detecting step; An image processing method comprising:

2. The generating step includes: generating the front images from a plurality of the two-dimensional tomographic images obtained from a plurality of predetermined regions that partially overlap each other within a plane intersecting the depth direction; The detecting step includes: synthesizing the choroidal vessels extracted from each of the front images, and detecting centerlines of the choroidal vessels from the synthesized front image; 2. The image processing method according to claim 1, further comprising:

3. For the three-dimensional choroidal region including the three-dimensional choroidal blood vessels for which the center line has been detected in the detecting step, a step of setting, as an analysis region, a region between a first region representing a first cross section at a position that is a first predetermined distance away from a predetermined position in the choroidal region, and a second region representing a second cross section at a position that is a second predetermined distance away from the predetermined position, the second predetermined distance being greater than the first predetermined distance; Analyzing a physical quantity related to the shape of the choroidal blood vessels for the analysis region; The image processing method of claim 1 further comprising:

4. Each of the first cross section and the second cross section is a cross section whose bottom surface contour is a curve centered on the predetermined position. The image processing method according to claim 3 .

5. The curve is one of a circle, an ellipse, an oval, and an arc. The image processing method according to claim 4.

6. The analysis region is a cylindrical region. The image processing method according to claim 3 .

7. The analysis region is a spherical region centered at the predetermined position. The image processing method according to claim 3 .

8. The physical quantity related to the shape of the choroidal blood vessels includes any one of the volume, blood vessel length, average blood vessel length, and skeleton density of the choroidal blood vessels in the analysis region. The image processing method according to claim 3 .

9. The analysis region is set in a direction intersecting the depth direction of the choroid region from the predetermined position toward the outside, or from the outside toward the predetermined position. The image processing method according to claim 3 .

10. A step of outputting data of the processing result by any one of the steps. The image processing method of claim 3 further comprising:

11. A generation unit that generates front images at different depth positions from two-dimensional tomographic images obtained from a predetermined region on the fundus of the subject's eye, for each depth position; a detection unit that extracts choroidal blood vessels from each of the generated front images and detects centerlines of the choroidal blood vessels; an estimation unit that estimates three-dimensional choroidal vessels and center lines of the three-dimensional choroidal vessels based on a plurality of front images in which the center lines are detected by the detection unit; An image processing device comprising:

12. An image processing method performed by a processor, comprising: A step of setting, from an image of choroidal blood vessels of the test eye, a region between a first region showing a first cross section at a position that is a first predetermined distance away from a predetermined position on the image with respect to a choroidal blood vessel region, and a second region showing a second cross section at a position that is a second predetermined distance away from the predetermined position, the second predetermined distance being greater than the first predetermined distance, as an analysis region; Analyzing a physical quantity related to the shape of the choroidal blood vessels for the analysis region; An image processing method comprising:

13. Each of the first cross section and the second cross section is a cross section having a curve centered at the predetermined position as a contour of a bottom surface. The image processing method according to claim 12.

14. The curve is one of a circle, an ellipse, an oval, and an arc. The image processing method according to claim 13.

15. The analysis region is a cylindrical region. The image processing method according to claim 12.

16. The analysis region is a spherical region centered at the predetermined position. The image processing method according to claim 12.

17. The physical quantity related to the shape of the choroidal vessels includes any one of the volume, vessel length, average vessel length, and skeleton density of the choroidal vessels in the analysis region. The image processing method according to claim 12.

18. The analysis region is set in a direction intersecting the depth direction of the choroid region from the predetermined position toward the outside, or from the outside toward the predetermined position. The image processing method according to claim 12.

19. outputting data resulting from the processing performed by any one of the steps; The image processing method of claim 12 further comprising: