Image processing method, image processing device, and program

Abstract

Provided is an image processing method that is performed by a processor, the image processing method comprising: a step for acquiring OCT volume data including a choroid; a step for generating, on the basis of the OCT volume data, a plurality of en-face images corresponding to a plurality of planes that differ in depth; a step for deriving image features for each of the plurality of en-face images; and a step for determining, as boundaries, lines between en-face images that show changes in the presence or absence of choroidal blood vessels through the image features, on the basis of each of the image features.

Description

[Technical Field]

[0001] The present disclosure relates to an image processing method, an image processing device, and a program. [Background technology]

[0002] U.S. Patent No. 10,238,281 discloses a technique for generating volume data of a subject's eye using an optical coherence tomography. Conventionally, it has been desired to visualize blood vessels based on the volume data of the subject's eye. Summary of the Invention

[0003] The first aspect is an image processing method performed by a processor, the image processing method including the steps of: acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; deriving image features for each of the plurality of en-face images; and identifying, based on each of the image features, a boundary between the en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels.

[0004] A second aspect is an image processing device including a processor, wherein the processor executes the steps of acquiring OCT volume data including the choroid, generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data, deriving image features for each of the plurality of en-face images, and identifying, based on each of the image features, a boundary between the en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels.

[0005] The third aspect is a program for performing image processing, which causes a processor to perform the following steps: acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; deriving image features for each of the plurality of en-face images; and identifying, based on each of the image features, a boundary between the en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a schematic configuration diagram of an ophthalmologic system according to an embodiment. [Figure 2] 1 is a schematic configuration diagram of an ophthalmologic apparatus according to an embodiment. [Figure 3] FIG. 2 is a schematic configuration diagram of a server. [Figure 4] FIG. 2 is an explanatory diagram of functions realized by an image processing program in a CPU of a server. [Figure 5] 10 is a flowchart illustrating an example of the flow of image processing by the server. [Figure 6] FIG. 2 is an explanatory diagram relating to image processing performed on an image. [Figure 7] FIG. 10 is an explanatory diagram of image feature amounts that change depending on the presence or absence of blood vessel components. [Figure 8] FIG. 10 is a diagram showing the characteristics of standard deviation for multiple en-face images in OCT volume data. [Figure 9] 10 is a flowchart showing an example of the flow of a blood vessel component presence / absence boundary acquisition process. [Figure 10] 10 is a flowchart showing an example of the flow of an image formation process of choroidal blood vessels. [Figure 11] 10 is a flowchart showing an example of the flow of third image processing by third blood vessel extraction processing. [Figure 12] FIG. 1 is a schematic diagram showing the relationship between the eyeball and the positions of vortex veins. [Figure 13]FIG. 1 is a diagram showing the relationship between OCT volume data and an en-face image. [Figure 14] FIG. 1 is a diagram showing an example of a fundus image of choroidal blood vessels including vortex veins. [Figure 15] FIG. 1 is a conceptual diagram of a three-dimensional image of a vortex vein. [Figure 16] FIG. 10 is a diagram showing an example of a stereoscopic image of choroidal blood vessels around a vortex vein. [Figure 17] FIG. 10 is a diagram showing an example of a display screen using a stereoscopic image of vortex veins. DETAILED DESCRIPTION OF THE INVENTION

[0007] An ophthalmologic system 100 according to an embodiment of the present disclosure will be described below with reference to the drawings. FIG. 1 shows a schematic configuration of an ophthalmologic system 100. As shown in FIG. 1, the ophthalmologic system 100 includes an ophthalmologic apparatus 110, a server apparatus (hereinafter referred to as "server") 140, and a display device (hereinafter referred to as "viewer") 150. The ophthalmologic apparatus 110 acquires fundus images. The server 140 stores, in association with patient IDs, a plurality of fundus images obtained by photographing the funduses of a plurality of patients using the ophthalmologic apparatus 110 and axial lengths measured by an axial length measuring device (not shown). The viewer 150 displays the fundus images and analysis results acquired by the server 140.

[0008] The server 140 is an example of the "image processing device" of the present disclosure.

[0009] The ophthalmologic apparatus 110, the server 140, and the viewer 150 are connected to each other via a network 130. The network 130 may be any network such as a LAN, a WAN, the Internet, or a wide area Ethernet network. For example, if the ophthalmologic system 100 is established in a single hospital, a LAN may be used as the network 130.

[0010] The viewer 150 is a client in a client-server system, and a plurality of viewers 150 are connected via a network. A plurality of servers 140 may also be connected via a network to ensure system redundancy. Alternatively, if the ophthalmic apparatus 110 has an image processing function and an image viewing function of the viewer 150, the ophthalmic apparatus 110 can acquire, process, and view fundus images in a standalone state. Alternatively, if the server 140 has an image viewing function of the viewer 150, the configuration of the ophthalmic apparatus 110 and the server 140 can acquire, process, and view fundus images.

[0011] In addition, other ophthalmic devices (examination devices for visual field measurement, intraocular pressure measurement, etc.) and diagnostic support devices that perform image analysis using AI (Artificial Intelligence) may be connected to the ophthalmic device 110, the server 140, and the viewer 150 via the network 130.

[0012] Next, the configuration of the ophthalmologic apparatus 110 will be described with reference to FIG.

[0013] For ease of explanation, Scanning Laser Ophthalmoscope will be referred to as "SLO" and Optical Coherence Tomography will be referred to as "OCT."

[0014] When the ophthalmologic apparatus 110 is placed on a horizontal plane, the horizontal direction is defined as the "X direction," the vertical direction relative to the horizontal plane is defined as the "Y direction," and the direction connecting the center of the pupil of the anterior segment of the subject's eye 12 and the center of the eyeball is defined as the "Z direction." Therefore, the X direction, Y direction, and Z direction are perpendicular to each other.

[0015] The ophthalmologic apparatus 110 includes an imaging device 14 and a control device 16. The imaging device 14 is equipped with an SLO unit 18 and an OCT unit 20, and acquires a fundus image of the subject's eye 12. Hereinafter, a two-dimensional fundus image acquired by the SLO unit 18 will be referred to as an SLO image. Also, a tomographic image or a front image (en-face image) of the retina created based on OCT data acquired by the OCT unit 20 will be referred to as an OCT image.

[0016] The control device 16 comprises a computer having a CPU (Central Processing Unit) 16A, a RAM (Random Access Memory) 16B, a ROM (Read-Only Memory) 16C, and an input / output (I / O) port 16D.

[0017] The control device 16 includes an input / display device 16E connected to the CPU 16A via an I / O port 16D. The input / display device 16E has a graphic user interface that displays an image of the subject's eye 12 and receives various instructions from the user. An example of the graphic user interface is a touch panel display.

[0018] The control device 16 also includes an image processor 17 connected to an I / O port 16D. The image processor 17 generates an image of the subject's eye 12 based on data obtained by the photographing device 14. The control device 16 is connected to a network 130 via a communication interface (I / F) 16F.

[0019] 2, the control device 16 of the ophthalmic apparatus 110 includes the input / display device 16E, but the present disclosure is not limited to this. For example, the control device 16 of the ophthalmic apparatus 110 may not include the input / display device 16E, but may include a separate input / display device that is physically independent from the ophthalmic apparatus 110. In this case, the display device includes an image processing processor unit that operates under the control of the display control unit 204 of the CPU 16A of the control device 16. The image processing processor unit may display an SLO image or the like based on an image signal instructed to be output by the display control unit 204.

[0020] The image capturing device 14 operates under the control of the CPU 16A of the control device 16. The image capturing device 14 includes an SLO unit 18, an image capturing optical system 19, and an OCT unit 20. The image capturing optical system 19 includes an optical scanner 22 and a wide-angle optical system 30.

[0021] The optical scanner 22 performs two-dimensional scanning in the X and Y directions with the light emitted from the SLO unit 18. The optical scanner 22 may be any optical element that can deflect a light beam, such as a polygon mirror or a galvanometer mirror, or a combination thereof.

[0022] The wide-angle optical system 30 combines the light from the SLO unit 18 and the light from the OCT unit 20 .

[0023] The wide-angle optical system 30 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a catadioptric system combining concave mirrors and lenses. By using a wide-angle optical system using an elliptical mirror or a wide-angle lens, it becomes possible to photograph the retina in the peripheral part of the fundus as well as the center of the fundus.

[0024] When a system including an elliptical mirror is used, the system using the elliptical mirror described in International Publication WO2016 / 103484 or International Publication WO2016 / 103489 may be used. The disclosures of International Publication WO2016 / 103484 and International Publication WO2016 / 103489 are each incorporated herein by reference in their entirety.

[0025] The wide-angle optical system 30 enables observation of the fundus over a wide field of view (FOV) 12A. The FOV 12A indicates the range that can be photographed by the imaging device 14. The FOV 12A can be expressed as a field of view. In this embodiment, the field of view can be defined by an internal illumination angle and an external illumination angle. The external illumination angle is the illumination angle of the light beam irradiated from the ophthalmic device 110 to the subject's eye 12, determined with the pupil 27 as the reference. The internal illumination angle is the illumination angle of the light beam irradiated to the fundus, determined with the center O of the eyeball as the reference. The external illumination angle and the internal illumination angle correspond to each other. For example, if the external illumination angle is 120 degrees, the internal illumination angle corresponds to approximately 160 degrees. In this embodiment, the internal illumination angle is 200 degrees.

[0026] Here, an SLO fundus image captured at an internal illumination angle of 160 degrees or more is referred to as a UWF-SLO fundus image. UWF stands for Ultra Wide Field. The wide-angle optical system 30, which provides an ultra-wide field of view (FOV) of the fundus, can capture images of the area from the posterior pole of the fundus of the subject's eye 12 beyond the equator, enabling the capture of structures present in the peripheral area of ​​the fundus, such as vortex veins.

[0027] The ophthalmologic apparatus 110 can capture an image of an area 12A with an internal illumination angle of 200°, with the center O of the eyeball of the subject's eye 12 as the reference position. The internal illumination angle of 200° corresponds to an external illumination angle of 110° with the pupil of the subject's eye 12 as the reference. In other words, the wide-angle optical system 30 irradiates laser light from the pupil at an angle of view of an external illumination angle of 110°, and captures an image of a fundus area of ​​200° with an internal illumination angle.

[0028] The SLO system is realized by a control device 16, an SLO unit 18, and an imaging optical system 19 shown in Fig. 2. The SLO system includes a wide-angle optical system 30, and therefore enables fundus imaging with a wide FOV 12A.

[0029] The SLO unit 18 includes a light source 40 for B light (blue light), a light source 42 for G light (green light), a light source 44 for R light (red light), and a light source 46 for IR light (infrared light (e.g., near-infrared light)), as well as optical systems 48, 50, 52, 54, and 56 that reflect or transmit the light from the light sources 40, 42, 44, and 46 and guide them into a single optical path. The optical systems 48 and 56 are mirrors, and the optical systems 50, 52, and 54 are beam splitters. The B light is reflected by the optical system 48, passes through the optical system 50, and is reflected by the optical system 54; the G light is reflected by the optical systems 50 and 54; the R light is transmitted through the optical systems 52 and 54; and the IR light is reflected by the optical systems 52 and 56 and is each guided into a single optical path.

[0030] The SLO unit 18 is configured to be switchable between a light source that emits laser light of different wavelengths, or a combination of light sources that emit light, such as a mode that emits R light and G light and a mode that emits infrared light. In the example shown in FIG. 2 , four light sources are provided: a light source 40 for B light, a light source 42 for G light, a light source 44 for R light, and a light source 46 for IR light, but the present disclosure is not limited to this. For example, the SLO unit 18 may further include a light source of white light, and may emit light in various modes, such as a mode that emits G light, R light, and B light, or a mode that emits only white light.

[0031] Light incident on the photographing optical system 19 from the SLO unit 18 is scanned in the X and Y directions by the optical scanner 22. The scanning light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus. The light reflected by the fundus passes through the wide-angle optical system 30 and the optical scanner 22 and is incident on the SLO unit 18.

[0032] The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits all light except B light from the posterior segment (fundus) of the subject's eye 12, and a beam splitter 58 that reflects G light and transmits all light except G light from the light that has passed through the beam splitter 64. The SLO unit 18 includes a beam splitter 60 that reflects R light and transmits all light except R light from the light that has passed through the beam splitter 58. The SLO unit 18 includes a beam splitter 62 that reflects IR light from the light that has passed through the beam splitter 60. The SLO unit 18 includes a B light detecting element 70 that detects B light reflected by the beam splitter 64, a G light detecting element 72 that detects G light reflected by the beam splitter 58, an R light detecting element 74 that detects R light reflected by the beam splitter 60, and an IR light detecting element 76 that detects IR light reflected by the beam splitter 62.

[0033] Light (reflected light reflected by the fundus) incident on the SLO unit 18 via the wide-angle optical system 30 and the optical scanner 22 is reflected by the beam splitter 64 and received by the B light detection element 70 in the case of B light, and is reflected by the beam splitter 58 and received by the G light detection element 72 in the case of G light. The incident light is transmitted through the beam splitter 58 in the case of R light, reflected by the beam splitter 60, and received by the R light detection element 74. The incident light is transmitted through the beam splitters 58 and 60 in the case of IR light, reflected by the beam splitter 62, and received by the IR light detection element 76. The image processor 17, which operates under the control of the CPU 16A, generates a UWF-SLO image using signals detected by the B light detection element 70, the G light detection element 72, the R light detection element 74, and the IR light detection element 76.

[0034] A UWF-SLO image generated using the signal detected by the B light detecting element 70 is referred to as a B-UWF-SLO image (B-color fundus image). A UWF-SLO image generated using the signal detected by the G light detecting element 72 is referred to as a G-UWF-SLO image (G-color fundus image). A UWF-SLO image generated using the signal detected by the R light detecting element 74 is referred to as an R-UWF-SLO image (R-color fundus image). A UWF-SLO image generated using the signal detected by the IR light detecting element 76 is referred to as an IR-UWF-SLO image (IR fundus image). UWF-SLO images include R-color fundus images, G-color fundus images, B-color fundus images, and even IR fundus images. Fluorescent UWF-SLO images captured using fluorescence are also included.

[0035] The control device 16 also controls the light sources 40, 42, and 44 to emit light simultaneously. By simultaneously photographing the fundus of the subject's eye 12 with B light, G light, and R light, a G-color fundus image, a R-color fundus image, and a B-color fundus image, each of which corresponds to a different position, are obtained. An RGB color fundus image is obtained from the G-color fundus image, the R-color fundus image, and the B-color fundus image. The control device 16 also controls the light sources 42 and 44 to emit light simultaneously, and by simultaneously photographing the fundus of the subject's eye 12 with G light and R light, a G-color fundus image and a R-color fundus image, each of which corresponds to a different position, are obtained. An RG color fundus image is obtained from the G-color fundus image and the R-color fundus image. A full-color fundus image may also be generated using the G-color fundus image, the R-color fundus image, and the B-color fundus image.

[0036] The wide-angle optical system 30 makes the field of view (FOV) of the fundus an ultra-wide angle, and can capture an image of the area from the posterior pole of the fundus of the subject's eye 12 beyond the equator.

[0037] The OCT system is realized by the control device 16, OCT unit 20, and imaging optical system 19 shown in FIG. 2. The OCT system includes a wide-angle optical system 30, which enables OCT imaging of the peripheral part of the fundus, similar to the above-described SLO fundus image capture. In other words, the wide-angle optical system 30, which provides an ultra-wide field of view (FOV) of the fundus, enables OCT imaging of the area from the posterior pole of the fundus of the subject's eye 12 beyond the equator 178. OCT data of structures present in the peripheral part of the fundus, such as vortex veins, can be acquired, and tomographic images of the vortex veins and the 3D structure of the vortex veins can be obtained by image processing the OCT data.

[0038] The OCT unit 20 includes a light source 20A, a sensor (detecting element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

[0039] Light emitted from the light source 20A is branched by the first optical coupler 20C. One of the branched beams is collimated by the collimating lens 20E as measurement light and then enters the imaging optical system 19. The measurement light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus. The measurement light reflected by the fundus passes through the wide-angle optical system 30 and enters the OCT unit 20, and then passes through the collimating lens 20E and the first optical coupler 20C and enters the second optical coupler 20F.

[0040] The other light beam emitted from the light source 20A and branched by the first optical coupler 20C is incident as reference light on the reference optical system 20D, passes through the reference optical system 20D, and then enters the second optical coupler 20F.

[0041] The light beams incident on the second optical coupler 20F, i.e., the measurement light beam reflected from the fundus and the reference light beam, interfere with each other to generate interference light. The interference light beam is received by the sensor 20B. The image processor 17, which operates under the control of the image processing unit 206, generates OCT data detected by the sensor 20B. The image processor 17 can also generate OCT images, such as tomographic images and en-face images, based on the OCT data.

[0042] Here, the OCT unit 20 can scan a predetermined range (for example, a rectangular range of 6 mm x 6 mm) in one OCT imaging session. The predetermined range is not limited to 6 mm x 6 mm, but may be a square range of 12 mm x 12 mm or 23 mm x 23 mm, or a rectangular range such as 14 mm x 9 mm or 6 mm x 3.5 mm, or any other rectangular range. It may also be a circular range with a diameter of 6 mm, 12 mm, or 23 mm.

[0043] By using the wide-angle optical system 30, the ophthalmic apparatus 110 can scan the area 12A with an internal illumination angle of 200°. That is, by controlling the optical scanner 22, OCT imaging of a predetermined range including vortex veins is performed. The ophthalmic apparatus 110 can generate OCT data by the OCT imaging.

[0044] Therefore, the ophthalmologic apparatus 110 can generate OCT images, such as a tomographic image (B-scan image) of the fundus including the vortex vein, OCT volume data including the vortex vein, and an en-face image (a frontal image generated based on the OCT volume data) that is a cross section of the OCT volume data. Needless to say, the OCT image also includes an OCT image of the center of the fundus (the posterior pole of the eyeball where the macula, optic disc, etc. are present).

[0045] The OCT data (or image data of the OCT image) is sent from the ophthalmic apparatus 110 to the server 140 via the communication interface 16F and stored in the storage device 254.

[0046] In this embodiment, the light source 20A is exemplified as a wavelength-swept type SS-OCT (Swept-Source OCT), but various types of OCT systems may also be used, such as SD-OCT (Spectral-Domain OCT) and TD-OCT (Time-Domain OCT).

[0047] Next, the configuration of the electrical system of the server 140 will be described with reference to Fig. 3. As shown in Fig. 3, the server 140 includes a computer main body 252. The computer main body 252 has a CPU 262, a RAM 266, a ROM 264, and an input / output (I / O) port 268. The input / output (I / O) port 268 is connected to a storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I / F) 258. The storage device 254 is configured, for example, with a non-volatile memory. The input / output (I / O) port 268 is connected to the network 130 via the communication interface (I / F) 258. Therefore, the server 140 can communicate with the ophthalmologic apparatus 110 and the viewer 150.

[0048] The ROM 264 or the storage device 254 stores an image processing program.

[0049] The ROM 264 or the storage device 254 is an example of a "memory" in the present disclosure. The CPU 262 is an example of a "processor" in the present disclosure. The image processing program is an example of a "program" in the present disclosure.

[0050] The server 140 stores each piece of data received from the ophthalmic apparatus 110 in the storage device 254 .

[0051] The following describes various functions that are realized by the CPU 262 of the server 140 executing the image processing program. As shown in Fig. 4, the image processing program executed by the CPU 262 has a display control function, an image processing function, and a processing function. By the CPU 262 executing the image processing program having these functions, the CPU 262 functions as the display control unit 204, the image processing unit 206, and the processing unit 208.

[0052] Next, a main flowchart of image processing by server 140 will be described with reference to Fig. 5. CPU 262 of server 140 executes an image processing program, thereby realizing the image processing (image processing method) shown in Fig. 5.

[0053] First, in step S10, the image processing unit 206 acquires a fundus image from the storage device 254. The fundus image includes data related to the vortex vein to be displayed three-dimensionally, based on a user's instruction.

[0054] Next, in step S20, the image processing unit 206 acquires, from the storage device 254, OCT volume data including the choroid corresponding to the fundus image.

[0055] When the OCT volume data is acquired, the image processing unit 206 executes a vascular component presence / absence boundary acquisition process (details of which will be described later) in step S22 to acquire the boundary between the presence and absence of choroidal blood vessels.

[0056] In the next step S30, the image processing unit 206 extracts choroidal blood vessels based on the OCT volume data and executes image formation processing of the choroidal blood vessels (described in detail later) to generate a stereoscopic image (3D image) of the vortex vein blood vessels.

[0057] Once the stereoscopic image (3D image) of the vortex vein blood vessels is generated, in step S40, the processing unit 208 outputs the generated stereoscopic image (3D image) of the vortex vein blood vessels, specifically, stores it in the RAM 266 or the storage device 254, and terminates the image processing.

[0058] Here, a display screen (an example of a display screen is shown in FIG. 17, which will be described later) containing a three-dimensional image of vortex veins is generated by the display control unit 204 based on a user instruction. The generated display screen is output as an image signal by the processing unit 208 to the viewer 150. The display screen is displayed on the display of the viewer 150.

[0059] Here, the positional relationship between the choroid 12M and the vortex veins 12V1 and V2 in the eyeball will be described with reference to FIG. In FIG. 12, the mesh-like pattern represents the choroidal blood vessels of the choroid 12M. The choroidal blood vessels circulate blood throughout the entire choroid. Blood flows out of the eyeball through multiple (usually four to six) vortex veins present in the subject's eye 12. FIG. 12 shows an upper vortex vein 12V1 and an lower vortex vein 12V2 present on one side of the eyeball. Vortex veins are often present near the equator. Therefore, to photograph the vortex veins present in the subject's eye 12 and the choroidal blood vessels around the vortex veins, for example, an ophthalmic apparatus 110 capable of scanning at an internal illumination angle of 200° is used.

[0060] First, the image processing unit 206 acquires a fundus image (step S10) and identifies a vortex vein (VV) to be displayed in three dimensions. Here, as an example, a UWF-SLO image is acquired from the storage device 254 as a UWF fundus image. Next, the image processing unit 206 creates a choroidal vessel image, which is a binarized image, from the acquired UWF-SLO image. Then, the image processing unit 206 identifies a region designated by the user as a vortex vein to be displayed in three dimensions.

[0061] Fig. 14 is a fundus image of choroidal blood vessels including vortex veins. The fundus image shown in Fig. 14 is an example of a choroidal blood vessel image, which is a binarized image created from a UWF-SLO image. As shown in Fig. 14, the choroidal blood vessel image is a binarized image in which pixels corresponding to choroidal blood vessels and vortex veins are white and pixels in other areas are black.

[0062] 14 is an image 302 showing the presence of choroidal blood vessels connected to a vortex vein. Image 302 shows a case where a vortex vein 310V1, which is an image of a superior vortex vein 12V1 included in a user-specified area 310A, is identified as a vortex vein (VV) to be displayed in three dimensions, and a region including choroidal blood vessels is identified.

[0063] A choroidal vascular image including vortex veins (VVs) is generated by processing image data of an R-UWF-SLO image (a red-color fundus image) captured with red light (laser light with a wavelength of 630 to 660 nm) and a G-UWF-SLO image (a green-color fundus image) captured with green light (laser light with a wavelength of 500 to 550 nm). Specifically, the choroidal vascular image is generated by extracting retinal blood vessels from the green fundus image, removing retinal blood vessels from the red fundus image, and performing image processing to enhance the choroidal blood vessels. Regarding the method for generating a choroidal vascular image, the disclosure of International Publication WO 2019 / 181981 is incorporated herein by reference in its entirety.

[0064] Although the above describes a case where a vortex vein to be displayed in three dimensions is identified by a user's instruction, the present disclosure is not limited to this. The position of the vortex vein to be displayed in three dimensions may be detected manually or automatically. For example, in the case of manual detection, the position indicated by the user's visual inspection of the displayed choroidal vessels may be detected. In the case of automatic detection, for example, the choroidal vessels may be extracted from a choroidal vessel image, the movement direction of each choroidal vessel (the direction of blood vessel travel) may be estimated, and the position of the vortex vein may be estimated based on the position where the choroidal vessels converge.

[0065] When forming an image of choroidal blood vessels, it may be necessary to extract choroidal blood vessels with diameters exceeding a predetermined value (hereinafter referred to as "thick blood vessels") and those with diameters less than the predetermined value (hereinafter referred to as "thin blood vessels"). Because thin blood vessels have lower image contrast than thick blood vessels, applying a common image process to thick and thin blood vessels makes it difficult to extract thin blood vessels as continuous line structures. Therefore, it is conceivable to extract thick and thin blood vessels using separate image processes. The details of the process of extracting thick and thin blood vessels using separate image processes will be described later. However, in the process of extracting thin blood vessels, there is a risk that noise images may be extracted as thin blood vessels, and blood vessels may be determined to exist even in areas where no blood vessels exist. This reduces the accuracy of the position of the boundary (e.g., the sclera) between the presence and absence of blood vessel components.

[0066] 13 , the OCT volume data 400 is OCT volume data 400 of a predetermined area, for example, a rectangular region of 6 mm×6 mm, including the vortex vein VV, obtained by OCT imaging of one of the multiple vortex veins VV present in the subject's eye using the ophthalmic apparatus 110. N planes at different depths, from a first plane f401 to an Nth plane f40N, are set for the OCT volume data 400. The OCT volume data 400 may be obtained by OCT imaging of each of the multiple vortex veins VV present in the subject's eye using the ophthalmic apparatus 110.

[0067] In this embodiment, the OCT volume data 400 including vortex veins and choroidal blood vessels around the vortex veins will be described as an example of the OCT volume data 400D. In this case, the choroidal blood vessels refer to the vortex veins and the choroidal blood vessels around the vortex veins.

[0068] An example of image processing performed on an image of choroidal blood vessels is shown in Figure 6. Figure 6 shows the results of performing image processing for thick blood vessels and image processing for thin blood vessels on an en-face image of an area where blood vessels do not exist as an image of choroidal blood vessels. As shown in Figure 6, when an image containing noise components is present in the en-face image f40K in an area where blood vessels do not exist, a process for extracting large blood vessels (image f40KL1) is performed, and then binarization is performed to produce an image f40KL2 in which the noise components have been removed. On the other hand, a process for extracting small blood vessels (image f40KS1) is performed, and then binarization is performed to produce an image f40KS2 in which the noise components remain. Therefore, when a process for extracting small blood vessels is performed, the noise components may be extracted as small blood vessels, and blood vessels may be determined to exist even in areas where no blood vessels exist.

[0069] Image features differ between an image containing vascular components and an image containing residual noise components. Examples of image features include the standard deviation of image brightness, the trend in the standard deviation, and the entropy of image brightness. The standard deviation of image brightness can be calculated using the standard deviation of each en-face image. The trend in the standard deviation can be calculated using a feature represented by the differential value of the characteristic curve of the standard deviation of multiple en-face images. The entropy of image brightness can be calculated using a physical quantity related to the sum of the brightnesses of the pixels in the en-face image. In this embodiment, a case where the standard deviation of image brightness is used as the image feature will be described.

[0070] 7 shows an example of image feature amounts (here, standard deviations) that change depending on the presence or absence of blood vessel components. An example of the case where image processing is performed on an image of choroidal blood vessels is shown.

[0071] First, for the en-face image f40H of the area where blood vessels exist (with blood vessel components) as an image of choroidal blood vessels, the standard deviation of the image f40HS1 obtained by image processing is examined. The standard deviation value in the image f40HS1 corresponds to the distribution width TH1 in the characteristics of signal intensity and frequency. The signal intensity indicates a physical quantity representing the brightness of the image in the image f40HS1, and the frequency indicates the frequency with which the physical quantity appears in the image f40HS1. Similarly, in the image f40KS1 obtained by image processing the en-face image f40K of the area where no blood vessels exist (without blood vessel components), the standard deviation value corresponds to the distribution width TH2. In terms of width, for the width TH1 indicating the standard deviation value of the en-face image f40H with blood vessel components, the en-face image f40KS1 without blood vessel components has a width TH2 (<TH1) indicating a standard deviation value smaller than the width TH1. This means that as the blood vessel components decrease, the standard deviation value tends to decrease. Therefore, by determining in advance a boundary determination value indicating the presence or absence of blood vessels, that is, the switching of the presence or absence of blood vessel components, it becomes possible to define the boundary of the presence or absence of blood vessel components. The boundary determination value is determined by the standard deviation value with a width TH0 (TH2≦TH0<TH1) that is smaller than the width TH1 and larger than or equal to the width TH2. Thus, the en-face image of the surface (layer) with a standard deviation value larger than the standard deviation value indicated by the width TH0 can be determined to have blood vessel components, and the en-face image of the surface (layer) with a standard deviation value smaller than or equal to the standard deviation value indicated by the width TH0 can be determined to have no blood vessel components.

[0072] The above-described boundary determination value can be derived in advance. Fig. 8 shows the characteristics of the standard deviation for a plurality of en-face images in the OCT volume data 400. As shown in Fig. 8, the characteristics of the standard deviation reach a maximum value Hu at the u-th surface and then gradually decrease, converging to a minimum value Hv at the v-th surface, from the first surface f401 to the N-th surface f40N. Therefore, a boundary determination value Ho can be defined as a value smaller than the maximum value Hu and greater than or equal to the minimum value Hv of the standard deviation. This boundary determination value Ho is likely to be a value close to the minimum value Hv to which the standard deviation converges, and it is also possible to reflect the results of pre-measured values. Note that the minimum value Hv may also be used as the boundary determination value.

[0073] In addition, from the viewpoint of the characteristic change of the standard deviation, it is also possible to apply the slope w that indicates the differential value of the characteristic curve of the standard deviation.

[0074] Therefore, in this embodiment, a vascular component presence / absence boundary acquisition process is executed based on the OCT volume data, using image feature amounts to acquire a boundary regarding the presence or absence of choroidal blood vessels. Next, the vascular component presence / absence boundary acquisition process (step S22) will be described in detail with reference to Fig. 9. The CPU 262 of the server 140 executes an image processing program to realize the image processing (image processing method) shown in the flowchart of Fig. 9.

[0075] Specifically, in step S220, the image processing unit 206 acquires OCT volume data 400, which is OCT data, for processing to obtain the boundary between the presence and absence of blood vessel components. N planes at different depths, from a first plane f401 to an Nth plane f40N, are set in the OCT volume data 400.

[0076] In step S221, the image processing unit 206 sets a parameter n to 1. The parameter n is a parameter indicating the number of en-face images (number of faces, number of layers).

[0077] In step S222, the image processing unit 206 analyzes the OCT volume data 400 and sets a first plane from, for example, the retinal pigment epithelium cell layer (hereinafter referred to as the RPE layer) in the OCT volume data 400. The first plane may be set as a plane a predetermined number of pixels below the RPE layer, for example, 10 pixels below. The image processing unit 206 can identify the RPE layer 400R as the reference plane as the first plane f401. The RPE layer 400R can be identified by performing a predetermined segmentation process on the OCT volume data 400. Alternatively, the RPE layer may be identified by determining the most luminous layer in the OCT volume data 400 as the RPE layer.

[0078] Setting the plane 10 pixels below the RPE layer as the first plane is effective for generating an en-face image of the area where choroidal blood vessels exist, since the area deeper than the RPE layer (the area farther from the RPE layer when viewed from the center of the eyeball) is the choroidal area. Setting the plane 10 pixels below the RPE layer as the first plane is not limited to this. For example, the plane 10 pixels below the Bruch's membrane, which is located immediately below the RPE layer, may be set as the first plane. Bruch's membrane is also identified by performing a predetermined segmentation process on the OCT volume data 400 that is different from that for the RPE layer. To identify the position 10 pixels below, the plane may be set 10 pixels below the A-scan direction when the OCT volume data was generated.

[0079] Furthermore, the first plane is not limited to being a plane 10 pixels below the RPE layer or Bruch's membrane, and may be set to any number of pixels. Furthermore, instead of being defined by the number of pixels, it may be defined by a length such as millimeters or nanometers. Furthermore, a spherical surface at a certain distance from the pupil or the center of the eyeball may be defined as the reference plane.

[0080] In step S223, the image processing unit 206 generates a first en-face image corresponding to the set first plane. The en-face image may be generated from the pixel values ​​of pixels present on the first plane, or a group of shallow pixels and a group of deep pixels including the first plane may be extracted from the OCT volume data 400, and pixel values ​​may be calculated as the average or median brightness value of these pixel groups. Image processing such as noise removal may be used to calculate the pixel values. The generated first en-face image corresponding to the first plane is stored in the RAM 266 by the processing unit 208.

[0081] In step S224, the image processing unit 206 derives image features for the nth en-face image (here, the first surface). Here, a standard deviation value for the en-face image of the first surface is derived. The standard deviation value is derived using pixel values ​​of pixels present in the en-face image. When deriving the image features, a layer application range may be determined. For example, a process may be performed to determine a predetermined layer range as the range for deriving image features, and image features may be derived for the determined layer range. The predetermined layer range may be a layer range whose depth at which a boundary exists has been empirically confirmed (e.g., a layer range from layer 80 to layer 120).

[0082] In step S225, the image processing unit 206 determines the boundary between the presence and absence of blood vessels by using the boundary determination value Ho to determine whether the standard deviation value corresponds to the boundary determination value Ho. The boundary between the presence and absence of blood vessels is determined as an en-face image in which no blood vessel components exist, or between adjacent en-face images in which a change in the presence or absence of blood vessels has occurred.

[0083] In step S226, the image processing unit 206 determines whether a boundary has been detected based on the determination result of the boundary between the presence and absence of blood vessels, and if the determination is positive, proceeds to step S229, and if the determination is negative, proceeds to step S227.

[0084] When the boundary between the presence and absence of blood vessels is determined and the process proceeds to step S229, the image processing unit 206 stores information indicating the boundary between the presence and absence of blood vessels. Specifically, in step S229, the processing unit 208 stores the determined position of the en-face image or the position between adjacent en-face images in the RAM 266 or the storage device 254, and then ends the process.

[0085] On the other hand, in step S227, the image processing unit 206 increments the parameter n (n=n+1), sets the nth plane in step S228, and returns the process to step S223.

[0086] In this way, the image processing unit 206 repeats the loop from step S223 to step S228 until the parameter n reaches the maximum number N.

[0087] By performing the image processing shown in FIG. 9 using the image processing unit 206, it becomes possible to identify the boundary between the presence and absence of blood vessels, and by superimposing this boundary on the choroidal vessel image, it becomes possible to visualize the boundary between the blood vessel image and the noise image, for example, the part corresponding to the sclera.

[0088] Next, the image formation process of choroidal blood vessels in step S30 for generating a stereoscopic image of vortex veins (VV) will be described in detail with reference to FIG.

[0089] In step S31 of FIG. 10, the image processing unit 206 extracts a region corresponding to the choroid from the OCT volume data 400 (see FIG. 13) acquired in step S20, and extracts (acquires) OCT volume data of the choroid based on the extracted region.

[0090] Specifically, the image processing unit 206 acquires OCT volume data for choroidal vessel extraction. The acquisition of OCT volume data may involve extracting a portion of the OCT volume data scanned to include vortex veins and choroidal vessels surrounding the vortex veins. For example, OCT volume data 400D of the region below the RPE layer may be extracted. Alternatively, OCT volume data 400D of the region determined to contain vascular components in the vascular component presence / absence boundary acquisition process described above may be extracted.

[0091] Next, in step S32, the image processing unit 206 executes a first blood vessel extraction process (ampullary portion extraction) using the OCT volume data 400D. The first blood vessel extraction process is a process for extracting choroidal blood vessels (hereinafter referred to as ampulla) that form the ampulla, which is the first blood vessel.

[0092] As preprocessing for the first blood vessel extraction process (ampulla extraction), the image processing unit 206 performs binarization processing on the OCT volume data 400D, and then performs noise removal processing. To remove noise regions, the image processing unit 206 performs median filtering, opening processing, or contraction processing on the binarized OCT volume data 400D to remove the noise regions.

[0093] Next, the image processing unit 206 performs segmentation processing (image processing such as active contouring, graph cut, or U-net) on the OCT volume data from which the noise regions have been deleted in order to smooth the surface of the extracted bulging portion. Note that "segmentation" refers to image processing that performs binarization processing to separate the background and foreground of the image to be analyzed.

[0094] By performing such first blood vessel extraction processing, only the region of the dilation remains from the OCT volume data 400D, and a three-dimensional image 680B of the blood vessels of the dilation is generated as shown in Fig. 15. Image data of the three-dimensional image 680B of the blood vessels of the dilation is stored in the RAM 266 by the processing unit 208.

[0095] In addition, in step S33 shown in Fig. 10, the image processing unit 206 executes a second blood vessel extraction process (thick blood vessel extraction) using the OCT volume data 400D. The second blood vessel extraction process is a process for extracting choroidal blood vessels (thick blood vessels) that are thick linear second blood vessels growing from the dilation and exceed a predetermined threshold, i.e., a predetermined diameter. In the second blood vessel extraction process (thick blood vessel extraction), linear second blood vessels growing from the dilation are extracted. The thick blood vessels mainly indicate blood vessels located in the Haller layer.

[0096] The predetermined threshold (i.e., predetermined diameter) can be a value that is predetermined so that blood vessels with a diameter of several hundred microns are left as thick blood vessels. The threshold value that is determined so that thin blood vessels are left as thick blood vessels, which will be described later, can be a value that is less than a diameter of several hundred microns that is determined so that blood vessels are left as thick blood vessels, or a value that is smaller than a value that is predetermined so that blood vessels are left as thick blood vessels. For example, a value that is predetermined so that blood vessels with a diameter of several tens of microns are left as thin blood vessels can be used.

[0097] The image processing unit 206 executes image processing to perform preprocessing on the OCT volume data 400D. An example of the preprocessing is blurring, such as noise removal. The blurring can be performed by eliminating the influence of speckle noise and extracting linear blood vessels that accurately reflect the blood vessel shapes. Examples of speckle noise processing include Gaussian blurring.

[0098] Next, the image processing unit 206 performs line extraction processing (thick linear blood vessel extraction) on the pre-processed OCT volume data 400D to extract second choroidal blood vessels, which are thick linear portions, from the OCT volume data 400D. In the extraction processing of the second choroidal blood vessels, for example, image processing using an eigenvalue filter, a Gabor filter, or the like is performed to extract linear blood vessel regions from the OCT volume data 400D.

[0099] The image processing unit 206 performs binarization processing on the OCT volume data 400D, and performs image processing such as median filter processing, opening processing, and contraction processing on the binarized linear blood vessel regions to remove isolated regions that are not connected to surrounding blood vessels, and to remove discrete microregions.

[0100] By the above image processing, a second stereoscopic image of the second choroidal vessels, which are thick blood vessels, is generated.

[0101] By performing the second blood vessel extraction process described above, only the thick blood vessel regions remain from the OCT volume data 400D, and a three-dimensional image 680L of the thick blood vessels shown in Fig. 15 is generated. The image data of the three-dimensional image 680L of the thick blood vessels is stored in the RAM 266 by the processing unit 208.

[0102] FIG. 16 shows an example of a stereoscopic image of the choroidal blood vessels around the vortex vein VV obtained by the image processing described above (FIG. 5). By performing the second blood vessel extraction process described above, only the thick blood vessel regions remain from the OCT volume data 400D, and a three-dimensional image 681L of the thick blood vessels is generated as shown in Fig. 16. Image data of this three-dimensional image 681L of the thick blood vessels is also stored in the RAM 266 by the processing unit 208.

[0103] The image processing unit 206 aligns the three-dimensional image 680B of the ampulla and the three-dimensional image 680L of the linear blood vessels, and then performs a logical OR operation on both images to synthesize the three-dimensional image 680L of the linear blood vessels and the three-dimensional image 680B of the ampulla. This makes it possible to generate a three-dimensional image 680M (FIG. 15) of the choroidal blood vessels, including vortex veins, which are thick blood vessels. In the process of extracting the thick blood vessels described above, thin blood vessels smaller than the predetermined diameter may be removed.

[0104] When observing vortex veins, it is important to observe not only the large blood vessels located in the Haller layer but also the small blood vessels located mainly in the Sattler layer. For example, analyzing the small blood vessels in the Sattler layer is effective for diagnosing pachychoroidal diseases. Therefore, the present disclosure includes a process for extracting choroidal vessels (small blood vessels) with a diameter equal to or smaller than a predetermined threshold, i.e., a predetermined diameter, which are the thin, linear third blood vessels extending from the ampulla.

[0105] Specifically, in step S34 shown in FIG. 10, the image processing unit 206 executes a third blood vessel extraction process (thin blood vessel extraction) using the OCT volume data 400D. The third blood vessel extraction process is a process for extracting choroidal blood vessels (thin blood vessels) that are thin, linear third blood vessels extending from the dilatation area and have a diameter equal to or smaller than a predetermined threshold, i.e., a predetermined diameter. In the third blood vessel extraction process (thin blood vessel extraction), linear third blood vessels extending from the dilatation area are extracted. The thin blood vessels mainly indicate blood vessels located in the Sattler layer. In the third blood vessel extraction process (thin blood vessel extraction), the third image processing shown in FIG. 11 is executed.

[0106] In the process of extracting the third blood vessels, which are thin blood vessels, the image processing unit 206 performs preprocessing for thin blood vessels, including first preprocessing and second preprocessing, on the OCT volume data 400D. First, in step S341 shown in Fig. 11, image processing is executed to perform the first preprocessing on the OCT volume data 400D. An example of the first preprocessing is blurring, which is an example of processing for removing noise.

[0107] In the next step S342, the image processing unit 206 executes image processing to perform second pre-processing on the OCT volume data 400D that has been subjected to the first pre-processing. Contrast enhancement processing is an example of the second pre-processing. Contrast enhancement processing is effective when extracting thin blood vessels. Contrast enhancement processing increases the contrast of the image compared to before processing, that is, increases the difference between light and dark. For example, the difference between the maximum and minimum values ​​of the degree of brightness (e.g., luminance) is increased by a predetermined value from the difference value before processing. The predetermined value can be set appropriately.

[0108] When an image that has undergone the above-described contrast enhancement processing is binarized, thin blood vessels appear as continuous lines, making it possible to reduce the separation of continuous thin blood vessels.

[0109] In step S342, the image processing unit 206 performs image processing using, for example, an eigenvalue filter, a Gabor filter, or the like, and is able to extract regions of linear blood vessels, which are thin blood vessels, from the OCT volume data 400D.

[0110] Next, the image processing unit 206 executes image processing to perform binarization on the contrast-enhanced OCT volume data 400D in step S343 shown in Fig. 11. Specifically, by setting the binarization threshold to a predetermined threshold that leaves thin blood vessels, thin blood vessels become black pixels and other parts become white pixels in the OCT volume data D.

[0111] Furthermore, in step S344, the image processing unit 206 removes discrete minute regions from the binarized image (regions including thin blood vessels). Here, for example, image processing such as removing speckle noise and regions isolated at a predetermined distance that are estimated not to be continuous with the surrounding blood vessels is performed to remove the discrete minute regions.

[0112] In the next step S345, the image processing unit 206 performs post-processing on the OCT volume data 400D from which the microregions have been removed, performing microregion connection processing to extract third choroidal vessels, which are thin blood vessels in thin linear portions, from the OCT volume data 400D. Specifically, the image processing unit 206 performs image processing using, for example, morphology processing such as closing processing to connect discretely detected thin blood vessels, thereby extracting third choroidal vessels, which are thin blood vessels, from the OCT volume data 400D. Specifically, the image processing unit 206 connects third choroidal vessels within a predetermined distance. In the image that has undergone this microregion connection processing, even thin blood vessels with large curvatures appear as continuous lines, making it possible to reduce the separation of continuous thin blood vessels.

[0113] In step S346, the image processing unit 206 performs segmentation processing (image processing such as active contouring, graph cut, or U-net) on the OCT volume data to which the fine regions are connected in order to smooth the surfaces of the extracted thin blood vessels. That is, the image to be analyzed is processed to separate the background from the foreground.

[0114] By the above image processing, a third three-dimensional image of the third choroidal vessels, which are thin blood vessels, is generated.

[0115] By performing the third blood vessel extraction process described above, only the thin blood vessel regions remain from the OCT volume data 400D, and a three-dimensional image 681S of the thin blood vessels shown in Fig. 16 is generated. The image data of the three-dimensional image 681S of the thin blood vessels is stored in the RAM 266 by the processing unit 208.

[0116] The processing of steps S32, S33, and S34 is not limited to the processing order described above, and any one of the processing may be executed first, or the processing may be executed simultaneously in parallel.

[0117] After the processing of steps S32, S33, and S34 is completed, in step S35, the image processing unit 206 reads out the three-dimensional image of the ampulla, the three-dimensional image of the large blood vessels, and the three-dimensional image of the small blood vessels from the RAM 266. Then, by aligning these three-dimensional images and calculating the logical sum of each image, the three-dimensional image of the ampulla, the three-dimensional image of the large blood vessels, and the three-dimensional image of the small blood vessels are synthesized. As a result, a three-dimensional image 381M (see FIG. 16 ) of the choroidal blood vessels including the vortex veins is generated. Image data of the three-dimensional image 681M is stored in the RAM 266 or the storage device 254 by the processing unit 208.

[0118] Furthermore, information indicating the boundary between the presence and absence of blood vessels obtained by the above-described blood vessel component presence / absence boundary acquisition process (FIG. 9) is also read from the RAM 266 and combined into the combined three-dimensional image. Image data of the three-dimensional image 681M in which information indicating the boundary between the presence and absence of blood vessels is constructed is stored in the RAM 266 or the storage device 254 by the processing unit 208.

[0119] The following describes a display screen for displaying the generated stereoscopic image (3D image) of choroidal vessels including vortex veins. The display screen is generated by the display control unit 204 of the server 140 based on a user's instruction, and is output as an image signal to the viewer 150 by the processing unit 208. The viewer 150 displays the display screen on a display based on the image signal.

[0120] A display screen 500A is shown in Figure 17. As shown in Figure 17, the display screen 500A has an information area 502 and an image display area 504A. The image display area 504A includes a comment field 506 that displays the patient's medical history.

[0121] The information area 502 has a patient ID display field 512, a patient name display field 514, an age display field 516, a visual acuity display field 518, a right eye / left eye display field 520, and an axial length display field 522. In each display area from the patient ID display field 512 to the axial length display field 522, the viewer 150 displays the respective information based on the information received from the server 140.

[0122] The image display area 504A is an area that mainly displays an image of the eye to be examined, etc. The image display area 504A is provided with the following display fields, specifically, a UWF fundus image display field 542 and a choroidal blood vessel stereoscopic image display field 548. Although not shown in the figure, the image display area 504A can also display an OCT volume data conceptual diagram display field and a tomographic image display field 546 in a superimposed manner.

[0123] A comment field 506 included in the image display area 504A displays the patient's medical history and functions as a remarks field in which the user, an ophthalmologist, can arbitrarily input the results of his or her observations and diagnosis.

[0124] The UWF fundus image display field 542 displays a UWF-SLO fundus image 542B captured by the ophthalmologic apparatus 110 of the fundus of the subject's eye. A range 542A indicating the position where the OCT volume data was acquired is superimposed on the UWF-SLO fundus image 542B. If there are multiple OCT volume data associated with the UWF-SLO image, the multiple ranges may be superimposed and the user may select one position from the multiple ranges. FIG. 17 shows that the range including the vortex vein in the upper right corner of the UWF-SLO image was scanned.

[0125] The choroidal vessel stereoscopic image display field 548 displays a stereoscopic image (3D image) 548B of the choroidal vessels obtained by image processing the OCT volume data. The stereoscopic image 548B can be rotated around three axes by a user operation. Furthermore, the choroidal vessel stereoscopic image 548B can display an image of the second choroidal vessels (a stereoscopic image of the thick blood vessels) and an image of the third choroidal vessels (a stereoscopic image of the thin blood vessels) extending from the ampulla 548X in different display formats. In FIG. 17, a stereoscopic image 548L of the thick blood vessels extending from the ampulla 548X is shown by a solid line, and a stereoscopic image 548S of the thin blood vessels is shown by a dotted line. Furthermore, the stereoscopic image 548L of the thick blood vessels and the stereoscopic image 548S of the thin blood vessels may be displayed in different colors, or the background (fill-in) of the image may be different.

[0126] In addition, the choroidal vessel stereoscopic image display field 548 displays a superimposed boundary indicating the presence or absence of choroidal vessels, which is acquired by the above-described vascular component presence / absence boundary acquisition process. Fig. 17 shows an example in which a layer boundary 548P is displayed using a thick solid line. This boundary 548P indicates the presence or absence of choroidal vessels, allowing the user to identify areas containing vascular components and provide appropriate treatment for the patient. It also enables the user to perform quantitative measurements of vessel depth and the like with high precision.

[0127] The image display area 504A of the display screen 500A allows a user to view a stereoscopic image of the choroidal vessels, including both thick and thin blood vessels. By scanning an area including a vortex vein, the vortex vein and the surrounding choroidal vessels, including both thick and thin blood vessels, can be displayed in a stereoscopic image. Furthermore, by superimposing the boundary between the presence and absence of choroidal vessels, the user can obtain more information for diagnosis.

[0128] As described above, in this embodiment, the boundary indicating the presence or absence of choroidal blood vessels can be obtained based on OCT volume data including the choroid, and therefore, the boundary indicating the presence or absence of choroidal blood vessels can be visualized three-dimensionally together with the choroidal blood vessels.

[0129] Although the above describes the case where a boundary is identified using image features, the image features are not limited to those that change depending on the presence or absence of vascular components. For example, it is also possible to identify a boundary using information about choroidal blood vessels. Choroidal blood vessels gradually thin as the layer deepens. In the present disclosure, it is also possible to identify a boundary by supplementing information about the depth of a layer in the fundus and information about the diameter of the choroidal blood vessels at that depth. Specifically, it is possible to detect the thickness of the choroidal blood vessels or the degree to which the thickness of the choroidal blood vessels changes in the depth direction, and identify the boundary based on the thickness or degree and a predetermined threshold. For example, when using information about the thickness of the choroidal blood vessels and a threshold, it is possible to predetermine a threshold indicating the thickness corresponding to the boundary, and identify a layer where the thickness of the choroidal blood vessels is equal to or less than the threshold. Furthermore, when using information about the degree and the threshold, it is possible to predetermine a threshold indicating the degree of change corresponding to the boundary, and identify a layer where the degree of change in the thickness of the choroidal blood vessels is equal to or less than the threshold.

[0130] In the above embodiment, the image processing (FIG. 5) is performed by the server 140, but the present disclosure is not limited to this and may be performed by the ophthalmic device 110, the viewer 150, or an additional image processing device further provided on the network 130.

[0131] In the present disclosure, each component (device, etc.) may be present in one or more instances, unless a contradiction arises.

[0132] In the above-described examples, image processing is performed using a software configuration that utilizes a computer. However, the present disclosure is not limited to this configuration, and at least a portion of the processing may be performed using a hardware configuration. Furthermore, while the above description uses a CPU as an example of a general-purpose processor, the term "processor" refers to a processor in a broad sense and includes general-purpose processors (e.g., a CPU (Central Processing Unit), etc.) and dedicated processors (e.g., a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a programmable logic device, etc.). Therefore, image processing may be performed solely using a hardware configuration, or a portion of the image processing may be performed using a software configuration and the remaining processing may be performed using a hardware configuration.

[0133] Furthermore, the operations of the above-mentioned processors may not only be performed by a single processor, but may also be performed by multiple processors working together, or may be performed by multiple processors located in physically separate locations working together.

[0134] Furthermore, in order to cause a computer to execute the above-described processing, a program in which the above-described processing is written in computer-processable code may be stored on a storage medium such as an optical disk and distributed.

[0135] As such, the present disclosure includes both cases in which image processing is realized by a software configuration using a computer and cases in which it is not realized, and therefore includes the following techniques.

[0136] (First Technology) an acquisition unit for acquiring OCT volume data including the choroid; a generation unit that generates a plurality of en-face images corresponding to a plurality of planes having different depths based on the OCT volume data; a derivation unit that derives image features for each of the plurality of en-face images; A determination unit that determines, based on each of the image features, a boundary between en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels; An image processing device comprising:

[0137] (Second Technology) An acquisition unit acquires OCT volume data including the choroid; generating, by a generating unit, a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; a derivation unit deriving image features for each of the plurality of en-face images; a determining unit determining, based on each of the image features, a boundary between en-face images in which the image features indicate a transition between the presence and absence of choroidal blood vessels; An image processing method comprising: The image processing unit 206 is an example of an "acquisition unit," a "generation unit," a derivation unit, and a determination unit of the present disclosure. Based on the above disclosure, the following technology is proposed.

[0138] (Third Technology) 1. A computer program product for image processing, comprising: the computer program product comprises a computer-readable storage medium that is not itself a transitory signal; The computer-readable storage medium stores a program, The program The processor acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; deriving image features for each of the plurality of en-face images; determining a boundary between en-face images where the image features indicate a transition between the presence and absence of choroidal vessels based on each of the image features; Let the Computer program products. Server 140 is an example of a "computer program product" of the present disclosure.

[0139] While the technology of the present disclosure has been described above using embodiments, the image processing described above is merely an example, and the technical scope of the present disclosure is not limited to the scope described in the above embodiments. Therefore, various modifications or improvements can be made to the above embodiments, such as deleting unnecessary processes, adding new processes, or changing the order of processes, without departing from the spirit of the present disclosure, and such modifications or improvements are also included in the technical scope of the present disclosure.

[0140] The disclosure of Japanese Patent Application No. 2022-066636 is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually indicated to be incorporated by reference.

Claims

1. An image processing method performed by a processor, comprising: acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; calculating a standard deviation of brightness for each of the plurality of en-face images as an image feature, and deriving a change tendency of the image feature among the plurality of en-face images; Identifying a boundary between en-face images that indicates a transition between the presence and absence of choroidal vessels based on the change tendency of the image feature amount and a predetermined threshold value of the change tendency of the image feature amount; An image processing method comprising:

2. In the step of deriving the change tendency, the change tendency is represented by a differential value of a characteristic curve of a standard deviation of the brightness of the plurality of en-face images. The image processing method according to claim 1 .

3. The step of identifying the boundary includes: and identifying, based on a standard deviation of each of the plurality of en-face images, a layer corresponding to a position of the en-face image where the standard deviation converges as the boundary. The image processing method according to claim 1 .

4. The step of identifying the boundary includes determining a standard deviation threshold value for determining the boundary based on a layer corresponding to a position of the en-face image where the standard deviation converges. The image processing method according to claim 3 .

5. extracting choroidal vessels from each of the plurality of en-face images; Detecting a degree of change in the thickness of the extracted choroidal vessels in the depth direction, The step of identifying the boundary includes a step of identifying the boundary based on a predetermined threshold and a degree of change in thickness of the choroidal blood vessels in the depth direction. The image processing method according to claim 1 .

6. The step of deriving the image feature amount derives the image feature amount from only a portion of the en-face images among the generated en-face images. The image processing method according to claim 1 .

7. the step of acquiring the OCT volume data includes scanning a region of the fundus including at least a vortex vein to obtain the OCT volume data; The image processing method according to any one of claims 1 to 6.

8. In an image processing device having a processor, The processor: acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; calculating a standard deviation of brightness for each of the plurality of en-face images as an image feature, and deriving a change tendency of the image feature among the plurality of en-face images; Identifying a boundary between en-face images that indicates a transition between the presence and absence of choroidal vessels based on the change tendency of the image feature amount and a predetermined threshold value of the change tendency of the image feature amount; An image processing device that performs the above.

9. In the step of deriving the change tendency, the change tendency is represented by a differential value of a characteristic curve of a standard deviation of the brightness of the plurality of en-face images. The image processing device according to claim 8 .

10. A program for performing image processing, The processor acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; calculating a standard deviation of brightness for each of the plurality of en-face images as an image feature, and deriving a change tendency of the image feature among the plurality of en-face images; Identifying a boundary between en-face images that indicates a transition between the presence and absence of choroidal vessels based on the change tendency of the image feature amount and a predetermined threshold value of the change tendency of the image feature amount; A program that processes the following.

11. In the step of deriving the change tendency, the change tendency is represented by a differential value of a characteristic curve of a standard deviation of the brightness of the plurality of en-face images. The program according to claim 10.

Images