Image processing method, image processing apparatus, and program

Abstract

Visualizing blood vessels based on volume data from the examined eye is desired. [Solution] An image processing method performed by a processor, comprising: acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes with different depths based on the OCT volume data; deriving image feature quantities in each of the plurality of en-face images; and determining, based on each of the image feature quantities, a boundary between en-face images where the image feature quantities indicate the switching between the presence or absence of choroidal blood vessels.

Description

Technical Field

[0001] The present disclosure relates to an image processing method, an image processing apparatus, and a program.

Background Art

[0002] U.S. Patent No. 10238281 discloses a technique for generating volume data of an eye to be examined using an optical coherence tomography. Conventionally, it has been desired to visualize blood vessels based on the volume data of the eye to be examined.

Summary of the Invention

[0003] A first aspect is an image processing method performed by a processor, the method including: obtaining OCT volume data including a choroid; generating a plurality of en-face images corresponding to a plurality of surfaces having different depths based on the OCT volume data; deriving an image feature amount in each of the plurality of en-face images; and specifying, based on each of the image feature amounts, a boundary between en-face images in which the image feature amount indicates a change in the presence or absence of choroidal blood vessels.

[0004] A second aspect is an image processing apparatus including a processor, wherein the processor executes: obtaining OCT volume data including a choroid; generating a plurality of en-face images corresponding to a plurality of surfaces having different depths based on the OCT volume data; deriving an image feature amount in each of the plurality of en-face images; and specifying, based on each of the image feature amounts, a boundary between en-face images in which the image feature amount indicates a change in the presence or absence of choroidal blood vessels.

[0005] The third aspect is a program for image processing, which causes a processor to perform the following steps: acquire OCT volume data including the choroid; generate a plurality of en-face images corresponding to a plurality of planes with different depths based on the OCT volume data; derive image feature quantities in each of the plurality of en-face images; and, based on each of the image feature quantities, identify as a boundary between en-face images where the image feature quantities indicate the switching between the presence or absence of choroidal blood vessels. [Brief explanation of the drawing]

[0006] [Figure 1] This is a schematic diagram of the ophthalmic system according to the embodiment. [Figure 2] This is a schematic diagram of the ophthalmic apparatus according to the present invention. [Figure 3] This is a schematic diagram of the server configuration. [Figure 4] This is a diagram illustrating the functions implemented by image processing programs on the server's CPU. [Figure 5] This flowchart shows an example of the image processing flow by the server. [Figure 6] This is an explanatory diagram regarding image processing applied to an image. [Figure 7] This is an explanatory diagram of how image features change depending on the presence or absence of vascular components. [Figure 8] This figure shows the characteristics of the standard deviation for multiple en-face images in OCT volume data. [Figure 9] A flowchart illustrating an example of the process for obtaining the boundary for the presence or absence of vascular components. [Figure 10] This is a flowchart illustrating an example of the image formation process for choroidal blood vessels. [Figure 11] This flowchart shows an example of the third image processing flow using the third blood vessel extraction process. [Figure 12] This is a schematic diagram showing the relationship between the eyeball and the position of the vortex veins. [Figure 13]This figure shows the relationship between OCT volume data and en-face images. [Figure 14] This figure shows an example of a fundus image of choroidal blood vessels, including vortex veins. [Figure 15] This is a conceptual diagram of a three-dimensional image of a vortex vein. [Figure 16] This figure shows an example of a three-dimensional image of choroidal vessels around vortex veins. [Figure 17] This figure shows an example of a display screen using a three-dimensional image of vortex veins. [Modes for carrying out the invention]

[0007] Hereinafter, an ophthalmic system 100 according to an embodiment of this disclosure will be described with reference to the drawings. Figure 1 shows a schematic configuration of the ophthalmology system 100. As shown in Figure 1, the ophthalmology system 100 comprises an ophthalmology device 110, a server device (hereinafter referred to as "server") 140, and a display device (hereinafter referred to as "viewer") 150. The ophthalmology device 110 acquires fundus images. The server 140 stores multiple fundus images obtained by the ophthalmology device 110 capturing the funduses of multiple patients, and the axial length measured by an axial length measuring device (not shown), corresponding to the patient ID. The viewer 150 displays the fundus images and analysis results acquired by the server 140.

[0008] Server 140 is an example of the “image processing device” in this disclosure.

[0009] The ophthalmic device 110, server 140, and viewer 150 are interconnected via network 130. Network 130 can be any network, such as a LAN, WAN, the Internet, or a wide-area Ethernet network. For example, if the ophthalmic system 100 is built in a single hospital, a LAN can be used for network 130.

[0010] The viewer 150 is a client in a client-server system, and multiple units are connected via the network. The server 140 may also be connected via the network in multiple units to ensure system redundancy. Alternatively, if the ophthalmic device 110 has image processing capabilities and the viewer 150 has image viewing capabilities, the ophthalmic device 110 can acquire, process, and view fundus images in a standalone state. Furthermore, if the server 140 has image viewing capabilities, the configuration of the ophthalmic device 110 and the server 140 enables the acquisition, processing, and viewing of fundus images.

[0011] Furthermore, other ophthalmic devices (such as visual field measurement and intraocular pressure measurement equipment) and diagnostic support devices that perform image analysis using AI (Artificial Intelligence) may be connected to the ophthalmic device 110, server 140, and viewer 150 via the network 130.

[0012] Next, the configuration of the ophthalmic device 110 will be explained with reference to Figure 2.

[0013] For the sake of clarity, we will refer to the Scanning Laser Ophthalmoscope as "SLO" and the Optical Coherence Tomography as "OCT".

[0014] When the ophthalmic device 110 is placed on a horizontal plane, the horizontal direction is defined as the "X direction," the direction perpendicular to the horizontal plane is defined as the "Y direction," and the direction connecting the center of the pupil of the anterior segment of the eye under examination 12 to the center of the eyeball is defined as the "Z direction." Therefore, the X, Y, and Z directions are perpendicular to each other.

[0015] The ophthalmic device 110 includes a photographing device 14 and a control device 16. The photographing device 14 includes an SLO unit 18 and an OCT unit 20, and acquires fundus images of the subject eye 12. Hereinafter, the two-dimensional fundus image acquired by the SLO unit 18 is referred to as an SLO image. Also, the tomographic image or en-face image of the retina created based on the OCT data acquired by the OCT unit 20 is referred to as an OCT image.

[0016] The control device 16 includes a computer having a CPU (Central Processing Unit) 16A, a RAM (Random Access Memory) 16B, a ROM (Read-Only Memory) 16C, and an input / output (I / O) port 16D.

[0017] The control device 16 includes an input / output display device 16E connected to the CPU 16A via the I / O port 16D. The input / output display device 16E has a graphic user interface for displaying an image of the subject eye 12 and receiving various instructions from the user. Examples of the graphic user interface include a touch panel display.

[0018] The control device 16 also includes an image processor 17 connected to the I / O port 16D. The image processor 17 generates an image of the subject eye 12 based on the data obtained by the photographing device 14. The control device 16 is connected to a network 130 via a communication interface (I / F) 16F.

[0019] As described above, in Figure 2, the control device 16 of the ophthalmic device 110 is equipped with an input / display device 16E, but the disclosure is not limited thereto. For example, the control device 16 of the ophthalmic device 110 may not be equipped with an input / display device 16E, but may be equipped with a separate input / display device that is physically independent of the ophthalmic device 110. In this case, the display device includes an image processing processor unit that operates under the control of the display control unit 204 of the CPU 16A of the control device 16. The image processing processor unit may display an SLO image or the like based on an image signal that the display control unit 204 has instructed to output.

[0020] The imaging device 14 operates under the control of the CPU 16A of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The imaging optical system 19 includes an optical scanner 22 and a wide-angle optical system 30.

[0021] The optical scanner 22 scans the light emitted from the SLO unit 18 in two dimensions, in the X and Y directions. The optical scanner 22 can be any optical element capable of deflecting the light beam, such as a polygon mirror or a galvanometer mirror. A combination of these may also be used.

[0022] The wide-angle optical system 30 combines light from the SLO unit 18 and light from the OCT unit 20.

[0023] The wide-angle optical system 30 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a reflective-refractive optical system combining a concave mirror and a lens. By using a wide-angle optical system using an elliptical mirror or a wide-angle lens, it becomes possible to photograph not only the central part of the fundus but also the peripheral part of the fundus.

[0024] When using a system that includes an elliptical mirror, a configuration using an elliptical mirror as described in International Publication WO2016 / 103484 or International Publication WO2016 / 103489 is also acceptable. Each of the disclosures in International Publication WO2016 / 103484 and International Publication WO2016 / 103489 is incorporated herein by reference in its entirety.

[0025] The wide-angle optical system 30 enables observation of the fundus with a wide field of view (FOV) 12A. The FOV 12A indicates the range that can be captured by the imaging device 14. The FOV 12A can be expressed as the field of view angle. In this embodiment, the field of view angle can be defined by the internal illumination angle and the external illumination angle. The external illumination angle is the illumination angle of the light beam irradiated from the ophthalmic device 110 onto the eye under examination 12, defined with respect to the pupil 27. The internal illumination angle is the illumination angle of the light beam irradiated onto the fundus, defined with respect to the center O of the eyeball. The external illumination angle and the internal illumination angle are in a corresponding relationship. For example, if the external illumination angle is 120 degrees, the internal illumination angle corresponds to approximately 160 degrees. In this embodiment, the internal illumination angle is set to 200 degrees.

[0026] Here, SLO fundus images obtained by capturing with an internal illumination angle of 160 degrees or more are referred to as UWF-SLO fundus images. UWF stands for UltraWide Field. The wide-angle optical system 30, which sets the field of view (FOV) of the fundus to an ultra-wide angle, can capture the region from the posterior pole to beyond the equator of the fundus of the eye under examination 12, and can capture structures present in the peripheral part of the fundus, such as vortex veins.

[0027] The ophthalmic device 110 can image a region 12A with an internal illumination angle of 200°, using the center O of the eyeball of the eye being examined 12 as the reference position. Note that an internal illumination angle of 200° corresponds to an external illumination angle of 110°, with the pupil of the eyeball of the eye being examined 12 as the reference point. In other words, the wide-angle optical system 30 emits laser light from the pupil with an external illumination angle of 110° and images the fundus region with an internal illumination angle of 200°.

[0028] The SLO system is implemented by the control device 16, SLO unit 18, and imaging optical system 19 shown in Figure 2. Because the SLO system includes a wide-angle optical system 30, it enables fundus imaging with a wide FOV 12A.

[0029] The SLO unit 18 includes a light source 40 for B light (blue light), a light source 42 for G light (green light), a light source 44 for R light (red light), and a light source 46 for IR light (infrared light (e.g., near-infrared light)), and optical systems 48, 50, 52, 54, and 56 that reflect or transmit the light from the light sources 40, 42, 44, and 46 into a single optical path. Optical systems 48 and 56 are mirrors, and optical systems 50, 52, and 54 are beam splitters. The B light is reflected by optical system 48, transmitted through optical system 50, and reflected by optical system 54; the G light is reflected by optical systems 50 and 54; the R light is transmitted through optical systems 52 and 54; and the IR light is reflected by optical systems 52 and 56, each leading into a single optical path.

[0030] The SLO unit 18 is configured to allow switching between combinations of light sources that emit or emit laser light of different wavelengths, such as a mode that emits R-light and G-light, and a mode that emits infrared light. In the example shown in Figure 2, there are four light sources: a B-light light source 40, a G-light light source 42, an R-light light source 44, and an IR-light light source 46, but the disclosure is not limited thereto. For example, the SLO unit 18 may further include a white light light source and emit light in various modes, such as a mode that emits G-light, R-light, and B-light, or a mode that emits only white light.

[0031] Light incident from the SLO unit 18 into the imaging optical system 19 is scanned in the X and Y directions by the optical scanner 22. The scanned light passes through the wide-angle optical system 30 and the pupil 27 and illuminates the fundus of the eye. The reflected light reflected by the fundus of the eye passes through the wide-angle optical system 30 and the optical scanner 22 and is incident on the SLO unit 18.

[0032] The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits other light from the posterior segment (fundus) of the eye under examination 12, and a beam splitter 58 that reflects G light and transmits other light from the light transmitted through the beam splitter 64. The SLO unit 18 also includes a beam splitter 60 that reflects R light and transmits other light from the light transmitted through the beam splitter 58. The SLO unit 18 also includes a beam splitter 62 that reflects IR light from the light transmitted through the beam splitter 60. The SLO unit 18 includes a B light detection element 70 that detects B light reflected by the beam splitter 64, a G light detection element 72 that detects G light reflected by the beam splitter 58, an R light detection element 74 that detects R light reflected by the beam splitter 60, and an IR light detection element 76 that detects IR light reflected by the beam splitter 62.

[0033] Light incident on the SLO unit 18 via the wide-angle optical system 30 and optical scanner 22 (reflected light reflected by the fundus) is reflected by the beam splitter 64 and received by the B-light detection element 70 in the case of B-light, and reflected by the beam splitter 58 and received by the G-light detection element 72 in the case of G-light. In the case of R-light, the incident light passes through the beam splitter 58, is reflected by the beam splitter 60 and received by the R-light detection element 74. In the case of IR-light, the incident light passes through the beam splitters 58 and 60, is reflected by the beam splitter 62 and received by the IR-light detection element 76. The image processor 17, operating under the control of the CPU 16A, generates a UWF-SLO image using the signals detected by the B-light detection element 70, the G-light detection element 72, the R-light detection element 74, and the IR-light detection element 76.

[0034] UWF-SLO images generated using the signal detected by the B-colored light detector 70 are called B-UWF-SLO images (B-colored fundus images). UWF-SLO images generated using the signal detected by the G-colored light detector 72 are called G-UWF-SLO images (G-colored fundus images). UWF-SLO images generated using the signal detected by the R-colored light detector 74 are called R-UWF-SLO images (R-colored fundus images). UWF-SLO images generated using the signal detected by the IR-colored light detector 76 are called IR-UWF-SLO images (IR fundus images). UWF-SLO images include these R-colored fundus images, G-colored fundus images, B-colored fundus images, and IR fundus images. UWF-SLO images of fluorescence captured by imaging fluorescence are also included.

[0035] Furthermore, the control device 16 controls the light sources 40, 42, and 44 to emit light simultaneously. By simultaneously photographing the fundus of the eye under examination 12 with B light, G light, and R light, G-color fundus images, R-color fundus images, and B-color fundus images are obtained where each position corresponds to the others. An RGB color fundus image is obtained from the G-color fundus images, R-color fundus images, and B-color fundus images. The control device 16 controls the light sources 42 and 44 to emit light simultaneously, and by simultaneously photographing the fundus of the eye under examination 12 with G light and R light, G-color fundus images and R-color fundus images are obtained where each position corresponds to the others. An RG color fundus image is obtained from the G-color fundus images and R-color fundus images. Alternatively, a full-color fundus image may be generated using the G-color fundus image, R-color fundus image, and B-color fundus image.

[0036] The wide-angle optical system 30 provides an ultra-wide field of view (FOV) of the fundus, allowing imaging of the region from the posterior pole to beyond the equator of the fundus of the eye under examination 12.

[0037] The OCT system is implemented by the control device 16, OCT unit 20, and imaging optical system 19 shown in Figure 2. Because the OCT system is equipped with a wide-angle optical system 30, it enables OCT imaging of the peripheral portion of the fundus, similar to the acquisition of SLO fundus images described above. Specifically, the wide-angle optical system 30, which provides an ultra-wide field of view (FOV) of the fundus, allows for OCT imaging of the area from the posterior pole to beyond the equator 178 of the fundus of the eye under examination 12. OCT data of structures present in the peripheral portion of the fundus, such as vortex veins, can be acquired, and tomographic images of vortex veins and, by image processing the OCT data, the 3D structure of vortex veins can be obtained.

[0038] The OCT unit 20 includes a light source 20A, a sensor (detection element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

[0039] Light emitted from the light source 20A is split by the first optical coupler 20C. One of the split beams of light is made parallel by the collimating lens 20E and then incident on the imaging optical system 19 as measurement light. The measurement light is directed onto the fundus of the eye via the wide-angle optical system 30 and the pupil 27. The measurement light reflected by the fundus of the eye is incident on the OCT unit 20 via the wide-angle optical system 30 and then incident on the second optical coupler 20F via the collimating lens 20E and the first optical coupler 20C.

[0040] The other beam of light emitted from the light source 20A and branched by the first optical coupler 20C is incident on the reference optical system 20D as reference light, and then, via the reference optical system 20D, is incident on the second optical coupler 20F.

[0041] These lights incident on the second optical coupler 20F, i.e., the measurement light reflected from the fundus and the reference light, interfere with each other at the second optical coupler 20F to generate interference light. The interference light is received by the sensor 20B. The image processor 17, operating under the control of the image processing unit 206, generates OCT data detected by the sensor 20B. Based on this OCT data, the image processor 17 can also generate OCT images such as tomographic images and en-face images.

[0042] Here, the OCT unit 20 can scan a predetermined range (for example, a rectangular range of 6 mm x 6 mm) in a single OCT scan. This predetermined range is not limited to 6 mm x 6 mm; it may also be a square range of 12 mm x 12 mm or 23 mm x 23 mm, or a rectangular range such as 14 mm x 9 mm or 6 mm x 3.5 mm, and can be any rectangular range. Alternatively, it may be a range of circles with diameters of 6 mm, 12 mm, 23 mm, etc.

[0043] By using the wide-angle optical system 30, the ophthalmic device 110 can scan an area 12A with an internal illumination angle of 200°. In other words, by controlling the optical scanner 22, OCT imaging is performed on a predetermined range including vortex veins. The ophthalmic device 110 can generate OCT data from this OCT imaging.

[0044] Therefore, the ophthalmic device 110 can generate OCT images, including tomographic images of the fundus including vortex veins (B-scan images), OCT volume data including vortex veins, and en-face images (frontal images generated based on OCT volume data), which are cross-sections of the OCT volume data. Needless to say, the OCT images include OCT images of the central part of the fundus (the posterior pole of the eyeball where the macula and optic nerve head are located).

[0045] OCT data (or image data of OCT images) is sent from the ophthalmic device 110 to the server 140 via the communication interface 16F and stored in the storage device 254.

[0046] In this embodiment, the light source 20A is exemplified as a wavelength-swept type SS-OCT (Swept-Source OCT), but various other types of OCT systems such as SD-OCT (Spectral-Domain OCT) and TD-OCT (Time-Domain OCT) may also be used.

[0047] Next, the electrical system configuration of server 140 will be described with reference to Figure 3. As shown in Figure 3, server 140 includes a computer unit 252. The computer unit 252 has a CPU 262, RAM 266, ROM 264, and input / output (I / O) ports 268. A storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I / F) 258 are connected to the input / output (I / O) ports 268. The storage device 254 is composed of, for example, non-volatile memory. The input / output (I / O) ports 268 are connected to the network 130 via the communication interface (I / F) 258. Thus, server 140 can communicate with the ophthalmic device 110 and the viewer 150.

[0048] The ROM 264 or storage device 254 stores an image processing program.

[0049] ROM 264 or storage device 254 is an example of “memory” in this disclosure. CPU 262 is an example of “processor” in this disclosure. Image processing program is an example of “program” in this disclosure.

[0050] Server 140 stores each piece of data received from the ophthalmic device 110 in the storage device 254.

[0051] This section describes the various functions realized by the CPU 262 of server 140 executing an image processing program. As shown in Figure 4, the image processing program executed by CPU 262 includes display control functions, image processing functions, and processing functions. By executing this image processing program with these functions, CPU 262 functions as a display control unit 204, an image processing unit 206, and a processing unit 208.

[0052] Next, we will explain the main flowchart of image processing by server 140 using Figure 5. The image processing (image processing method) shown in Figure 5 is realized when the CPU 262 of server 140 executes the image processing program.

[0053] First, in step S10, the image processing unit 206 acquires a fundus image from the storage device 254. This fundus image includes data related to the vortex veins to be displayed in 3D, based on user instructions.

[0054] Next, in step S20, the image processing unit 206 acquires OCT volume data, including the choroid, corresponding to the fundus image, from the storage device 254.

[0055] When OCT volume data is acquired, the image processing unit 206 executes a vascular component presence / absence boundary acquisition process (details described later) in step S22 to acquire the boundary of the presence or absence of choroidal blood vessels.

[0056] In the next step S30, the image processing unit 206 extracts choroidal vessels based on the OCT volume data and performs choroidal vessel image formation processing (details described later) to generate a three-dimensional image (3D image) of vortex venous vessels.

[0057] Once a three-dimensional image (3D image) of the vortex venous vessels is generated, in step S40, the processing unit 208 outputs the generated three-dimensional image (3D image) of the vortex venous vessels, specifically saving it to the RAM 266 or storage device 254, and terminates the image processing.

[0058] Here, based on user instructions, the display control unit 204 generates a display screen containing a three-dimensional image of the vortex veins (an example of the display screen is shown in Figure 17, described later). The generated display screen is output as an image signal to the viewer 150 by the processing unit 208. The display screen is then displayed on the viewer 150's display.

[0059] Here, the positional relationship between the choroid 12M and the vortex veins 12V1 and V2 in the eyeball will be explained using Figure 12. In Figure 12, the reticular pattern represents the choroidal vessels of the choroid 12M. These choroidal vessels circulate blood throughout the choroid. Blood then flows out of the eyeball through vortex veins, of which there are multiple (usually four to six) in the eye under examination 12. Figure 12 shows the superior vortex vein 12V1 and the inferior vortex vein 12V2, both located on one side of the eyeball. Vortex veins are often located near the equator. Therefore, imaging the vortex veins in the eye under examination 12 and the choroidal vessels surrounding them is performed using, for example, an ophthalmic device 110 capable of scanning with an internal illumination angle of 200°.

[0060] First, the image processing unit 206 acquires a fundus image (step S10) and identifies the vortex veins (VV) to be displayed in 3D. Here, as an example, a UWF-SLO image is acquired from the storage device 254 as a UWF fundus image. Next, the image processing unit 206 creates a choroidal vascular image, which is a binarized image, from the acquired UWF-SLO image. Then, it identifies the area indicated by the user as the vortex veins to be displayed in 3D.

[0061] Figure 14 is a fundus image of choroidal vessels, including vortex veins. The fundus image shown in Figure 14 is an example of a choroidal vessel image, which is a binarized image created from a UWF-SLO image. As shown in Figure 14, the choroidal vessel image is a binarized image in which pixels corresponding to choroidal vessels and vortex veins are white, and pixels in other areas are black.

[0062] Figure 14 is also an image 302 showing the presence of choroidal vessels connected to vortex veins. Image 302 shows a case where vortex vein 310V1, which is an image of the superior vortex vein 12V1 included in the user-indicated area 310A, is identified as a vortex vein (VV) to be displayed in 3D, and the area containing choroidal vessels is identified.

[0063] Choroidal vascular images, including vortex veins (VVs), are generated by image processing of image data from R-UWF-SLO images (red-color fundus images) acquired with red light (laser light with a wavelength of 630-660 nm) and G-UWF-SLO images (green-color fundus images) acquired with green light (laser light with a wavelength of 500-550 nm). Specifically, choroidal vascular images are generated by image processing that extracts retinal blood vessels from the green-color fundus image, removes retinal blood vessels from the red-color fundus image, and enhances choroidal blood vessels. The method for generating choroidal vascular images is disclosed in international publication WO2019 / 181981, which is incorporated herein by reference in its entirety.

[0064] Furthermore, while the above describes a case in which vortex veins to be displayed in 3D are identified based on user instructions, this disclosure is not limited to this. The position of vortex veins to be displayed in 3D may be determined by manual detection or automatic detection. For example, in the case of manual detection, the user can visually inspect the displayed choroidal vessels and detect the indicated position. In the case of automatic detection, for example, choroidal vessels may be extracted from a choroidal vessel image, the direction of movement (vascular course) of each choroidal vessel may be estimated, and the position of the vortex veins may be estimated based on the locations where the choroidal vessels converge.

[0065] Incidentally, when performing image formation of choroidal vessels, it is sometimes required to extract each of the choroidal vessels that exceed a predetermined diameter (hereinafter referred to as "thick vessels") and those that are smaller than a predetermined diameter (hereinafter referred to as "thin vessels"). Since thin vessels have lower image contrast than thick vessels, it is difficult to extract thin vessels as a continuous linear structure if the same image processing is applied to both thick and thin vessels. For this reason, it is conceivable to extract thick and thin vessels using separate image processing. Details of these processes for extracting thick and thin vessels using separate image processing will be described later. However, in the process of extracting thin vessels, there is a risk that noisy images may be extracted as thin vessels, and it may be determined that vessels exist even in areas where no vessels do not exist. Therefore, the positional accuracy of the boundary of the presence or absence of vascular components (e.g., the sclera) related to the presence of vessels decreases.

[0066] As shown in Figure 13, the OCT volume data 400 is obtained by performing OCT imaging on one of the multiple vortex veins VV present in the eye under examination using the ophthalmic device 110, and is the OCT volume data 400 of a predetermined area including the vortex vein VV, for example, a rectangular area of ​​6 mm × 6 mm. For the OCT volume data 400, N surfaces with different depths are set, from the first surface f401 to the Nth surface f40N. The OCT volume data 400 may also be obtained by performing OCT imaging on each of the multiple vortex veins VV present in the eye under examination using the ophthalmic device 110.

[0067] In this embodiment, the OCT volume data 400D is described using OCT volume data 400, which includes vortex veins and choroidal vessels surrounding those vortex veins, as an example. In this case, choroidal vessels refer to vortex veins and choroidal vessels surrounding those vortex veins.

[0068] Figure 6 shows an example of image processing applied to images of choroidal blood vessels. Figure 6 shows the results of applying image processing to large blood vessels and to small blood vessels to en-face images of areas where no blood vessels are present in the choroidal blood vessel images. As shown in Figure 6, when noise components are present in the en-face image f40K of a region where no blood vessels exist, the noise components are removed from the image f40KL2, which is obtained by extracting large blood vessels (image f40KL1) and then binarizing it. On the other hand, the noise components remain in the image f40KS2, which is obtained by extracting small blood vessels (image f40KS1) and then binarizing it. Therefore, when extracting small blood vessels, noise may be extracted as small blood vessels, and it may be determined that blood vessels exist even in regions where no blood vessels actually exist.

[0069] Images containing vascular components and images with residual noise components will have different image features. Examples of applicable image features include the standard deviation of image brightness, the trend of change in the standard deviation, and the entropy of image brightness. For the standard deviation of image brightness, the standard deviation of each element in the en-face image can be used. For the trend of change in the standard deviation, a feature represented by the differential value of the characteristic curve of the standard deviations of multiple en-face images can be used. For the entropy of image brightness, a physical quantity relating to the sum of the brightness of pixels in the en-face image can be used as a feature. In this embodiment, the case in which the standard deviation of image brightness is applied as an image feature will be described.

[0070] Figure 7 shows an example of how image features (in this case, standard deviation) change depending on the presence or absence of vascular components. An example of image processing applied to an image of choroidal blood vessels is also shown.

[0071] First, for the en-face image f40H of the region where blood vessels exist (with blood vessel components) as an image of choroidal blood vessels, the standard deviation of the image f40HS1 obtained by performing image processing is examined. The standard deviation value in the image f40HS1 corresponds to the distribution width TH1 in the characteristics of signal intensity and frequency. The signal intensity indicates a physical quantity representing the brightness of the image in the image f40HS1, and the frequency indicates the frequency with which the physical quantity appears in the image f40HS1. Similarly, in the image f40KS1 obtained by performing image processing on the en-face image f40K of the region where no blood vessels exist (without blood vessel components), the standard deviation value corresponds to the distribution width TH2. In terms of width, for the width TH1 indicating the standard deviation value of the en-face image f40H with blood vessel components, the en-face image f40KS1 without blood vessel components has a width TH2 (<TH1) indicating a standard deviation value smaller than TH1. This means that as the number of blood vessel components decreases, the standard deviation value tends to decrease. Therefore, by predetermining a boundary determination value indicating the presence or absence of blood vessels, that is, the switching between the presence and absence of blood vessel components, it becomes possible to define the boundary between the presence and absence of blood vessel components. The boundary determination value is determined by the standard deviation value with a width TH0 (TH2≦TH0<TH1) that is smaller than the width TH1 and larger than or equal to the width TH2. Thus, the en-face image of the surface (layer) with a standard deviation value larger than the standard deviation value indicated by the width TH0 can be determined to have blood vessel components, and the en-face image of the surface (layer) with a standard deviation value smaller than or equal to the standard deviation value indicated by the width TH0 can be determined to have no blood vessel components.

[0072] The above-described boundary determination value can be derived in advance. Fig. 8 shows the characteristics of the standard deviation for a plurality of en-face images in the OCT volume data 400. As shown in Fig. 8, the characteristics of the standard deviation reach a maximum value Hu at the u-th surface and then gradually decrease, converging to a minimum value Hv at the v-th surface as moving from the first surface f401 to the N-th surface f40N. Therefore, a value smaller than the maximum value Hu and equal to or greater than the minimum value Hv can be defined as the boundary determination value Ho. This boundary determination value Ho is likely to be a value close to the minimum value Hv where the standard deviation converges, and it is also possible to reflect the results of pre-measured values. Note that the minimum value Hv may also be used as the boundary determination value.

[0073] Furthermore, from the perspective of the change in the characteristics of the standard deviation, it is also possible to apply the slope w, which represents the derivative of the characteristic curve of the standard deviation.

[0074] Therefore, in this embodiment, a vascular component presence / absence boundary acquisition process is performed to acquire boundaries related to the presence or absence of choroidal blood vessels using image features based on the OCT volume data. Next, the process of acquiring the boundary for the presence or absence of vascular components (step S22) will be explained in detail using Figure 9. The CPU 262 of server 140 executes the image processing program, thereby realizing the image processing (image processing method) shown in the flowchart of Figure 9.

[0075] Specifically, in step S220, the image processing unit 206 acquires OCT volume data 400, which is OCT data, for the process of acquiring the boundary of the presence or absence of vascular components. The OCT volume data 400 is set to have N surfaces with different depths, from the first surface f401 to the Nth surface f40N.

[0076] In step S221, the image processing unit 206 sets parameter n to 1. This parameter n indicates the number of en-face images (number of faces, number of layers).

[0077] In step S222, the image processing unit 206 analyzes the OCT volume data 400 and, for example, sets a first plane from the retinal pigment epithelium (RPE) layer in the OCT volume data 400. The first plane may be set to a predetermined number of pixels below the RPE layer, for example, 10 pixels below. The image processing unit 206 can identify the RPE layer 400R as the reference plane for the first plane f401. The RPE layer 400R can be identified by performing a predetermined segmentation process on the OCT volume data 400. Alternatively, the RPE layer may be identified as the most bright layer in the OCT volume data 400.

[0078] Setting the plane 10 pixels below the RPE layer as the first plane is effective for generating an en-face image of the region where choroidal blood vessels exist, as the region deeper than the RPE layer (the region further from the RPE layer when viewed from the center of the eyeball) is the choroid region. The first plane is not limited to this setting of 10 pixels below the RPE layer; for example, the first plane may be set to the plane 10 pixels below Bruch's membrane, which is located immediately below the RPE layer. Bruch's membrane is also identified by performing a different predetermined segmentation process on the OCT volume data 400 than that for the RPE layer. Note that to identify the position 10 pixels below, it may be set to 10 pixels below the A-scan direction when the OCT volume data was generated.

[0079] Furthermore, the first surface is not limited to being defined as the plane 10 pixels below the RPE layer or Bruch's membrane, but may be set to any number of pixels. Alternatively, it may be defined by length, such as millimeters or nanometers, rather than by the number of pixels. Also, a spherical surface maintaining a certain distance from the pupil or the center of the eyeball may be defined as the reference surface.

[0080] In step S223, the image processing unit 206 generates a first en-face image corresponding to the set first surface. This en-face image may be generated from the pixel values ​​of pixels present on the first surface, or the pixel values ​​may be obtained by extracting the pixel groups in the shallow direction and the pixel groups in the deep direction, including the first surface, from the OCT volume data 400 and taking the average or median of the brightness values ​​of these pixel groups. When determining the pixel values, image processing such as noise reduction may be used. The first en-face image corresponding to the generated first surface is saved to the RAM 266 by the processing unit 208.

[0081] In step S224, the image processing unit 206 derives image features for the nth (in this case, the first) en-face image. Here, the standard deviation value for the first en-face image is derived. The standard deviation value is derived using the pixel values ​​of pixels present in the en-face image. When deriving these image features, the applicable range of layers may be defined. For example, it is possible to perform a process to determine a predetermined layer range as the range for which image features are derived, and then derive image features for the determined layer range. The predetermined layer range can be a layer range where the depth at which the boundary exists has been empirically confirmed (for example, a layer range from 80 to 120 layers).

[0082] In step S225, the image processing unit 206 determines the boundary between the presence and absence of blood vessels by using the boundary determination value Ho to determine whether the standard deviation value corresponds to the boundary determination value Ho. The boundary between the presence and absence of blood vessels is determined by using either an en-face image where no blood vessel components exist, or adjacent en-face images where a switch in the presence or absence of blood vessels has occurred.

[0083] In step S226, the image processing unit 206 determines whether a boundary has been detected based on the determination result of the presence or absence of blood vessels. If the determination is positive, the process proceeds to step S229; otherwise, the process proceeds to step S227.

[0084] When the boundary of the presence or absence of blood vessels is determined and the process moves to step S229, the image processing unit 206 saves information indicating the boundary of the presence or absence of blood vessels. Specifically, in step S229, the processing unit 208 saves the determined en-face image position, or the position between adjacent en-face images, to the RAM 266 or storage device 254, and terminates the process.

[0085] Meanwhile, in step S227, the image processing unit 206 increments the parameter n (n=n+1), sets the nth plane in step S228, and returns to step S223.

[0086] In this way, the image processing unit 206 repeats the loop from step S223 to step S228 until the parameter n reaches its maximum number N.

[0087] By performing the image processing shown in Figure 9, the image processing unit 206 can identify the boundary between areas with and without blood vessels. By superimposing this boundary onto the choroidal vessel image, the boundary between the blood vessel image and the noise image, for example, the portion corresponding to the sclera, can be visualized.

[0088] Next, the image formation process for choroidal vessels that generates a three-dimensional image of vortex veins (VV) in step S30 will be explained in detail using Figure 10.

[0089] In step S31 of Figure 10, the image processing unit 206 extracts the region corresponding to the choroid from the OCT volume data 400 (see Figure 13) acquired in step S20, and based on the extracted region, extracts (acquires) OCT volume data for the choroidal portion.

[0090] Specifically, the image processing unit 206 acquires OCT volume data for choroidal vessel extraction. The acquisition of OCT volume data may involve extracting a portion of the scanned OCT volume data that includes vortex veins and the choroidal vessels surrounding those vortex veins. For example, OCT volume data 400D of the region below the RPE layer may be extracted. Alternatively, OCT volume data 400D of the region determined to have vascular components in the vascular component presence / absence boundary acquisition process described above may be extracted.

[0091] Next, in step S32, the image processing unit 206 performs a first blood vessel extraction process (ampulla extraction) using the OCT volume data 400D. The first blood vessel extraction process is a process to extract the choroidal blood vessels that form the ampulla, which is the first blood vessel (hereinafter referred to as the ampulla).

[0092] The image processing unit 206 performs a binarization process on the OCT volume data 400D as a preprocessing step for the first blood vessel extraction process (ampulla extraction), and then performs a noise reduction process. To remove noise regions, the image processing unit 206 applies a median filter, opening process, or contraction process to the binarized OCT volume data 400D to remove noise regions.

[0093] Next, the image processing unit 206 performs segmentation processing (such as dynamic contouring, graph cuts, or U-net image processing) on ​​the OCT volume data from which the noise region has been removed, in order to smooth the surface of the extracted enlarged area. "Segmentation" refers to image processing that performs binarization to separate the background and foreground from the image being analyzed.

[0094] By performing this first blood vessel extraction process, only the ampulla region remains from the OCT volume data 400D, and a three-dimensional image 680B of the ampulla blood vessels shown in Figure 15 is generated. The image data of the three-dimensional image 680B of the ampulla blood vessels is stored in RAM 266 by the processing unit 208.

[0095] Furthermore, in step S33 shown in Figure 10, the image processing unit 206 performs a second blood vessel extraction process (extraction of large blood vessels) using the OCT volume data 400D. The second blood vessel extraction process is a process that extracts choroidal blood vessels (large blood vessels) that are large, linear second blood vessels extending from the ampulla and exceed a predetermined threshold, i.e., a predetermined diameter. In the second blood vessel extraction process (extraction of large blood vessels), linear second blood vessels extending from the ampulla are extracted. These large blood vessels mainly represent blood vessels located in the Haller layer.

[0096] Furthermore, the predetermined threshold (i.e., the predetermined diameter) can be a value predetermined to leave blood vessels with a diameter of several hundred μm as large blood vessels. Also, the threshold for leaving thin blood vessels, as described later, may be a value less than the several hundred μm diameter predetermined for leaving thick blood vessels, or it may be a value smaller than the value predetermined for leaving thick blood vessels. For example, it is possible to use a value predetermined to leave blood vessels with a diameter of several tens of μm as thin blood vessels.

[0097] The image processing unit 206 performs image processing to pre-process the OCT volume data 400D. One example of pre-processing is blurring, such as noise reduction. This blurring can be performed to eliminate the effects of speckle noise and to extract linear vessels that accurately reflect the shape of the blood vessels. Examples of speckle noise reduction include Gaussian blurring.

[0098] Next, the image processing unit 206 extracts the second choroidal vessel, which is a thick linear portion, from the OCT volume data 400D by performing a line extraction process (extraction of thick linear vessels) on the pre-processed OCT volume data 400D. In the extraction process of the second choroidal vessel, image processing such as an eigenvalue filter and a Gabor filter is performed to extract the region of the linear vessel from the OCT volume data 400D.

[0099] The image processing unit 206 performs a binarization process on the OCT volume data 400D, and then performs image processing on the binarized linear blood vessel region, such as deleting isolated regions not connected to surrounding blood vessels, median filtering, opening, and contraction, to remove discrete minute regions.

[0100] Through the image processing described above, a second three-dimensional image of the second choroidal vessel, which is a large blood vessel, is generated.

[0101] By performing the second blood vessel extraction process described above, only the regions of large blood vessels remain from the OCT volume data 400D, and a 3D image of the large blood vessels 680L is generated, as shown in Figure 15. The image data of the 3D image of the large blood vessels 680L is stored in RAM 266 by the processing unit 208.

[0102] Furthermore, Figure 16 shows an example of a three-dimensional image of choroidal vessels around vortex vein VV obtained by the image processing described above (Figure 5). By performing the second blood vessel extraction process described above, only the regions of large blood vessels remain from the OCT volume data 400D, and a 3D image of the large blood vessels 681L shown in Figure 16 is generated. The image data of this 3D image of the large blood vessels 681L is also saved to RAM 266 by the processing unit 208.

[0103] The image processing unit 206 aligns the three-dimensional image 680B of the ampulla and the three-dimensional image 680L of the linear vessels, and performs a logical OR operation on both images to synthesize the three-dimensional image 680L of the linear vessels and the three-dimensional image 680B of the ampulla. This makes it possible to generate a three-dimensional image 680M (Figure 15) of the choroidal vessels, including the vortex veins, which are large vessels. In the process of extracting large vessels as described above, thin vessels smaller than the predetermined diameter may be removed.

[0104] Incidentally, when observing vortex veins, it is important to observe not only the large vessels located in the Haller layer, but also the small vessels mainly located in the Sattler layer. For example, analysis of small vessels in the Sattler layer is effective in diagnosing pachychoroid diseases. Therefore, this disclosure includes a process for extracting choroidal vessels (small vessels) that are thin, linear third vessels extending from the ampulla, which are below a predetermined threshold, i.e., below a predetermined diameter.

[0105] Specifically, in step S34 shown in Figure 10, the image processing unit 206 performs a third blood vessel extraction process (extraction of thin blood vessels) using the OCT volume data 400D. The third blood vessel extraction process is a process that extracts choroidal blood vessels (thin blood vessels) that are thin, linear third blood vessels extending from the ampulla, and that are below a predetermined threshold, i.e., a predetermined diameter. In the third blood vessel extraction process (extraction of thin blood vessels), linear third blood vessels extending from the ampulla are extracted. These thin blood vessels mainly represent blood vessels located in the Sattler layer. In the third blood vessel extraction process (extraction of thin blood vessels), the third image processing shown in Figure 11 is performed.

[0106] In the process of extracting a third blood vessel, which is a thin blood vessel, the image processing unit 206 applies preprocessing for thin blood vessels, including first and second preprocessing, to the OCT volume data 400D. First, in step S341 shown in Figure 11, image processing is performed to apply first preprocessing to the OCT volume data 400D. An example of first preprocessing is blurring, which is an example of noise reduction.

[0107] In the next step, S342, the image processing unit 206 performs image processing to apply a second preprocessing step to the OCT volume data 400D that has undergone the first preprocessing step. One example of the second preprocessing step is contrast enhancement. Contrast enhancement is effective when extracting thin blood vessels. Contrast enhancement is a process that increases the contrast of the image compared to before processing, that is, it increases the difference between light and dark. For example, the difference between the maximum and minimum values ​​of brightness (e.g., luminance) is increased from the difference value before processing to a predetermined value. This predetermined value can be set as appropriate.

[0108] When the image with the contrast enhancement processing described above is binarized, the thin blood vessels appear as continuous lines, reducing the separation of these continuous thin blood vessels.

[0109] In step S342, the image processing unit 206 can perform image processing, for example, using an eigenvalue filter or a Gabor filter, to extract the region of linear vessels, which are thin blood vessels, from the OCT volume data 400D.

[0110] Next, in step S343 shown in Figure 11, the image processing unit 206 performs image processing to apply binarization to the OCT volume data 400D that has undergone contrast enhancement processing. Specifically, by setting the binarization threshold to a predetermined threshold that leaves thin blood vessels, the OCT volume data D becomes such that thin blood vessels are black pixels and the rest are white pixels.

[0111] Furthermore, in step S344, the image processing unit 206 removes discrete minute regions from the binarized image (region including thin blood vessels). Here, for example, image processing is performed to remove speckle noise and isolated regions that are estimated to be not continuous with the surrounding blood vessels and are separated by a predetermined distance, thereby removing discrete minute regions.

[0112] In the next step S345, the image processing unit 206 performs a micro-region consolidation process on the OCT volume data 400D from which minute regions have been removed as a post-processing step, thereby extracting the third choroidal vessels, which are thin linear vessels, from the OCT volume data 400D. Specifically, the image processing unit 206 extracts the third choroidal vessels, which are thin vessels, from the OCT volume data 400D by performing image processing, such as morphogenetic processing including closing processing, and connecting the discretely detected thin vessels. Specifically, it connects the third choroidal vessels within a predetermined distance. The image after this micro-region consolidation process appears as a continuous line even for thin vessels with large curvature, and the separation of continuous thin vessels can be reduced.

[0113] Furthermore, in step S346, the image processing unit 206 performs segmentation processing (image processing such as dynamic contouring, graph cuts, or U-net) on the OCT volume data to which the above-mentioned fine regions are connected, in order to smooth the surface of the extracted thin blood vessels. In other words, it performs processing to separate the background and foreground from the image to be analyzed.

[0114] Through the image processing described above, a third three-dimensional image of the third choroidal vessel, which is a thin blood vessel, is generated.

[0115] By performing the third blood vessel extraction process described above, only the regions of thin blood vessels remain from the OCT volume data 400D, and the 3D image 681S of the thin blood vessels shown in Figure 16 is generated. The image data of the 3D image 681S of the thin blood vessels is stored in RAM 266 by the processing unit 208.

[0116] The processes in steps S32, S33, and S34 are not limited to the order described above; any of the processes may be executed first, or they may be performed simultaneously in parallel.

[0117] Once steps S32, S33, and S34 are completed, in step S35, the image processing unit 206 reads the 3D images of the ampulla, the large blood vessels, and the small blood vessels from the RAM 266. Then, by aligning these 3D images and calculating the logical OR of each image, the 3D images of the ampulla, the large blood vessels, and the small blood vessels are synthesized. This generates a 3D image 381M of the choroidal vessels including the vortex veins (see Figure 16). The image data of the 3D image 681M is saved by the processing unit 208 to the RAM 266 or storage device 254.

[0118] Furthermore, the information indicating the presence or absence of blood vessels, obtained through the blood vessel component presence / absence boundary acquisition process described above (Figure 9), is also read from RAM 266 and incorporated into the synthesized 3D image. The image data of the 3D image 681M, in which the information indicating the presence or absence of blood vessels has been incorporated, is stored in RAM 266 or storage device 254 by the processing unit 208.

[0119] The following describes a display screen for displaying a three-dimensional image (3D image) of the choroidal vessels, including the generated vortex veins. This display screen is generated by the display control unit 204 of the server 140 based on user instructions, and output as an image signal to the viewer 150 by the processing unit 208. The viewer 150 displays the display screen on the display based on this image signal.

[0120] Figure 17 shows the display screen 500A. As shown in Figure 17, the display screen 500A has an information area 502 and an image display area 504A. The image display area 504A includes a comment field 506 for displaying the patient's treatment history.

[0121] The information area 502 includes a patient ID display field 512, a patient name display field 514, an age display field 516, a visual acuity display field 518, a right eye / left eye display field 520, and an axial length display field 522. In each display area from the patient ID display field 512 to the axial length display field 522, the viewer 150 displays the respective information based on the information received from the server 140.

[0122] The image display area 504A is primarily for displaying images of the eye being examined. The image display area 504A is provided with the following display fields, specifically including a UWF fundus image display field 542 and a choroidal vessel 3D image display field 548. Although not shown in the illustration, the image display area 504A can also display an OCT volume data conceptual diagram display field and a tomographic image display field 546 overlaid on it.

[0123] The comment field 506 included in the image display area 504A functions as a remarks field where the patient's treatment history can be displayed, and the user, an ophthalmologist, can optionally input observation results and diagnostic results.

[0124] The UWF fundus image display field 542 displays a UWF-SLO fundus image 542B of the fundus of the eye examined, taken with the ophthalmic device 110. A range 542A indicating the location where OCT volume data was acquired is superimposed on the UWF-SLO fundus image 542B. If there are multiple OCT volume data associated with the UWF-SLO image, multiple ranges may be superimposed, and the user may select one location from the multiple ranges. Figure 17 shows that the range including the upper right vortex vein of the UWF-SLO image has been scanned.

[0125] The choroidal vessel stereoscopic image display field 548 displays a stereoscopic image (3D image) 548B of the choroidal vessels obtained by image processing of OCT volume data. The stereoscopic image 548B can be rotated in three axes by user operation. Furthermore, the choroidal vessel stereoscopic image 548B can display images of the second choroidal vessels (3D images of thick vessels) and the third choroidal vessels (3D images of thin vessels) extending from the ampulla 548X in different display formats. In Figure 17, the 3D image 548L of the thick vessels extending from the ampulla 548X is shown with a solid line, and the 3D image 548S of the thin vessels is shown with a dotted line. In addition, the 3D image 548L of the thick vessels and the 3D image 548S of the thin vessels may be displayed in different colors, or the background (fill) of the images may be made different.

[0126] Furthermore, the choroidal vessel 3D image display field 548 overlays the boundary indicating the presence or absence of choroidal vessels, which was obtained through the vascular component presence / absence boundary acquisition process described above. Figure 17 shows an example where the layer boundary 548P is displayed with a thick solid line. This boundary 548P is the boundary related to the presence or absence of choroidal vessels, allowing for the identification of areas containing vascular components and enabling accurate treatment of patients. It also enables high-precision quantitative measurements of blood vessel depth and other parameters.

[0127] According to the image display area 504A of the display screen 500A, a three-dimensional image of choroidal vessels, including large and small blood vessels, can be viewed. By scanning the area including vortex veins, the choroidal vessels, including the vortex veins and the surrounding large and small blood vessels, can be displayed in three dimensions. Furthermore, by superimposing the boundary of the presence or absence of choroidal vessels, the user can obtain more information for diagnosis.

[0128] As described above, in this embodiment, since the boundary indicating the presence or absence of choroidal blood vessels can be obtained based on OCT volume data including the choroid, it becomes possible to visualize the boundary indicating the presence or absence of choroidal blood vessels in three dimensions together with the choroidal blood vessels.

[0129] The above describes the method of identifying boundaries using image features, but it is not limited to image features that change depending on the presence or absence of vascular components. For example, it is also possible to identify boundaries using information about choroidal blood vessels. Choroidal blood vessels gradually become thinner as the layer deepens. In this disclosure, it is also possible to identify boundaries by using information about the depth of the layer in the fundus and information about the diameter of the choroidal blood vessels at that depth as auxiliary information. Specifically, it is possible to detect the thickness of the choroidal blood vessels, or the degree to which the thickness of the choroidal blood vessels changes in the depth direction, and identify boundaries based on the thickness or degree and a predetermined threshold. For example, when using information about the thickness of the choroidal blood vessels and a threshold, a threshold indicating the thickness corresponding to the boundary can be predetermined, and layers where the thickness of the choroidal blood vessels is less than or equal to the threshold can be identified as boundaries. Also, when using information about the degree and a threshold, a threshold indicating the degree of change corresponding to the boundary can be predetermined, and layers where the degree of change in the thickness of the choroidal blood vessels is less than or equal to the threshold can be identified as boundaries.

[0130] In the above embodiment, image processing (Figure 5) is performed by the server 140, but the disclosure is not limited thereto, and it may also be performed by the ophthalmic device 110, the viewer 150, or an additional image processing device further provided in the network 130.

[0131] In this disclosure, each component (device, etc.) may exist as one or more, as long as it does not create a contradiction.

[0132] The examples described above illustrate cases where image processing is implemented using a software configuration with a computer. However, this disclosure is not limited to these cases, and at least some of the processing may be implemented using a hardware configuration. Furthermore, although a CPU was used as an example of a general-purpose processor above, the term "processor" refers to a broader term that includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). Therefore, image processing may be performed solely by a hardware configuration, or some of the image processing may be performed by a software configuration and the remaining processing by a hardware configuration.

[0133] Furthermore, the processor operation described above may not be performed by a single processor, but may also be performed by multiple processors working together, or by multiple processors located in physically separate locations.

[0134] Furthermore, in order to have a computer execute the above-mentioned processes, a program in which the above-mentioned processes are written in code that can be processed by a computer may be stored on a storage medium such as an optical disc and distributed.

[0135] Thus, this disclosure includes cases where image processing is implemented using computer-based software configurations and cases where it is not, and therefore includes the following technologies.

[0136] (First technology) An acquisition unit that acquires OCT volume data including the choroid, A generation unit that generates multiple en-face images corresponding to multiple surfaces with different depths based on the aforementioned OCT volume data, A derivation unit for deriving image feature quantities for each of the plurality of en-face images, Based on each of the aforementioned image features, a determination unit determines a boundary between en-face images that indicate the switching between the presence or absence of choroidal blood vessels, An image processing device equipped with the following features.

[0137] (Second technology) The acquisition unit performs the step of acquiring OCT volume data including the choroid, The generation unit generates multiple en-face images corresponding to multiple surfaces with different depths based on the OCT volume data, The derivation unit performs the steps of deriving image feature quantities for each of the plurality of en-face images, The determination unit determines, based on each of the image features, that the boundary between en-face images indicating the switching between the presence or absence of choroidal blood vessels is defined as the boundary between the image features, Image processing methods including [specific details omitted]. The image processing unit 206 is an example of the "acquisition unit," "generation unit," "derivation unit," and "determination unit" of this disclosure. Based on the information disclosed above, the following technologies are proposed.

[0138] (Third technology) A computer program product for image processing, The aforementioned computer program product includes a computer-readable storage medium that is not itself a temporary signal, The aforementioned computer-readable storage medium stores a program. The aforementioned program, In the processor, Steps include acquiring OCT volume data including the choroid, The steps include generating multiple en-face images corresponding to multiple surfaces with different depths based on the aforementioned OCT volume data, The steps include: deriving image features from each of the aforementioned multiple en-face images; Based on each of the aforementioned image features, the step of determining a boundary between en-face images that indicate the switching between the presence or absence of choroidal blood vessels, To process Computer program products. Server 140 is an example of a “computer program product” in this disclosure.

[0139] Although the technology of this disclosure has been described above using embodiments, the image processing described above is merely an example, and the technical scope of this disclosure is not limited to the scope described in the embodiments above. Therefore, various modifications or improvements can be made to the above embodiments, such as deleting unnecessary processing, adding new processing, or changing the processing order, without departing from the spirit of the invention, and such modified or improved forms are also included in the technical scope of this disclosure.

[0140] Furthermore, the disclosure of Japanese Patent Application No. 2022-066636 is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards described herein are incorporated herein by reference in the same way as if the incorporation of each individual document, patent application, and technical standard were specifically and individually described.

Claims

1. A method of image processing performed by a processor, The steps include acquiring OCT volume data including the choroid, The steps include generating multiple en-face images corresponding to multiple surfaces with different depths based on the OCT volume data, The steps include: calculating the standard deviation of brightness as an image feature in each of the plurality of en-face images, and deriving the trend of change of the image feature among the plurality of en-face images; Based on the aforementioned trend of change in the image features, the step of identifying a boundary layer corresponding to the position of the en-face image showing the transition between the presence and absence of choroidal blood vessels, Image processing methods, including those mentioned above.

2. The step of deriving the aforementioned trend of change is to show the trend of change by the derivative of the characteristic curve of the standard deviation of the brightness of the plurality of en-face images. The image processing method according to claim 1.

3. The step of identifying the layer corresponding to the aforementioned position is: Based on the aforementioned trend of change of the image features and a predetermined threshold for the aforementioned trend of change of the image features, the switching of the presence or absence of the choroidal blood vessels is indicated. The image processing method according to claim 1.

4. The step of identifying the aforementioned boundary is: The process includes the step of identifying a layer corresponding to the position of an en-face image where the standard deviations converge, based on the standard deviations of each of the plurality of en-face images. The image processing method according to claim 1.

5. The step of identifying the boundary involves determining a threshold for the standard deviation that determines the boundary, based on the layer corresponding to the position of the en-face image where the standard deviation converges. The image processing method according to claim 3.

6. The steps include extracting choroidal blood vessels from each of the aforementioned multiple en-face images, The method further includes the step of detecting the degree to which the diameter of the extracted choroidal blood vessel changes in correspondence with depth, The step of identifying the boundary includes identifying the boundary based on a predetermined threshold and the degree to which the diameter of the choroidal vessel changes in the depth direction. The image processing method according to claim 1.

7. The step of deriving the image features involves deriving the image features from only some of the multiple en-face images that have been generated. The image processing method according to claim 1.

8. The step of obtaining the OCT volume data involves scanning a region of the fundus that includes at least vortex veins to obtain the OCT volume data. The image processing method according to any one of claims 1 to 7.

9. A method of image processing performed by a processor, The steps include acquiring OCT volume data including the choroid, The steps include generating multiple en-face images corresponding to multiple surfaces with different depths based on the OCT volume data, The steps include: calculating the standard deviation of brightness as an image feature in each of the plurality of en-face images, and deriving the trend of change of the image feature among the plurality of en-face images; Based on the aforementioned trend of change in the image features, the step of identifying the boundary between en-face images that show the transition between the presence or absence of choroidal blood vessels, Image processing methods, including those mentioned above.

10. In an image processing device equipped with a processor, The aforementioned processor, The steps include acquiring OCT volume data including the choroid, The steps include generating multiple en-face images corresponding to multiple surfaces with different depths based on the OCT volume data, The steps include: calculating the standard deviation of brightness as an image feature in each of the plurality of en-face images, and deriving the trend of change of the image feature among the plurality of en-face images; Based on the aforementioned trend of change in the image features, the step of identifying a boundary layer corresponding to the position of the en-face image showing the transition between the presence and absence of choroidal blood vessels, An image processing device that performs this operation.

11. The step of deriving the aforementioned trend of change is to show the trend of change by the derivative of the characteristic curve of the standard deviation of the brightness of the plurality of en-face images. The image processing apparatus according to claim 10.

12. The step of identifying the layer corresponding to the aforementioned position is: Based on the aforementioned trend of change of the image features and a predetermined threshold for the aforementioned trend of change of the image features, the switching of the presence or absence of the choroidal blood vessels is indicated. The image processing apparatus according to claim 10.

13. In an image processing device equipped with a processor, The aforementioned processor, The steps include acquiring OCT volume data including the choroid, The steps include generating multiple en-face images corresponding to multiple surfaces with different depths based on the OCT volume data, The steps include: calculating the standard deviation of brightness as an image feature in each of the plurality of en-face images, and deriving the trend of change of the image feature among the plurality of en-face images; Based on the aforementioned trend of change in the image features, the step of identifying the boundary between en-face images that show the transition between the presence or absence of choroidal blood vessels, An image processing device that performs this operation.

14. A program for image processing, In the processor, The steps include acquiring OCT volume data including the choroid, The steps include generating multiple en-face images corresponding to multiple surfaces with different depths based on the OCT volume data, The steps include: calculating the standard deviation of brightness as an image feature in each of the plurality of en-face images, and deriving the trend of change of the image feature among the plurality of en-face images; Based on the aforementioned trend of change in the image features, the step of identifying a boundary layer corresponding to the position of the en-face image showing the transition between the presence and absence of choroidal blood vessels, A program that processes [something].

15. The step of deriving the aforementioned trend of change is to show the trend of change by the derivative of the characteristic curve of the standard deviation of the brightness of the plurality of en-face images. The program according to claim 14.

16. The step of identifying the layer corresponding to the aforementioned position is: Based on the aforementioned trend of change of the image features and a predetermined threshold for the aforementioned trend of change of the image features, the switching of the presence or absence of the choroidal blood vessels is indicated. The program according to claim 14.

17. A program for image processing, In the processor, The steps include acquiring OCT volume data including the choroid, The steps include generating multiple en-face images corresponding to multiple surfaces with different depths based on the OCT volume data, The steps include: calculating the standard deviation of brightness as an image feature in each of the plurality of en-face images, and deriving the trend of change of the image feature among the plurality of en-face images; Based on the aforementioned trend of change in the image features, the step of identifying the boundary between en-face images that show the transition between the presence or absence of choroidal blood vessels, A program that processes [something].

Images