Image processing method, image processing device, and program

Abstract

Provided is an image processing method that is performed by a processor, the image processing method comprising: a step for acquiring OCT volume data including a choroid; a step for generating, on the basis of the OCT volume data, a plurality of en-face images corresponding to a plurality of planes that differ in depth; a step for deriving image features for each of the plurality of en-face images; and a step for determining, as boundaries, lines between en-face images that show changes in the presence or absence of choroidal blood vessels through the image features, on the basis of each of the image features.

Description

Image processing method, image processing device, and program

[0001] The present disclosure relates to an image processing method, an image processing device, and a program.

[0002] U.S. Patent No. 10,238,281 discloses a technique for generating volume data of a subject's eye using an optical coherence tomography device. It has been desired to visualize blood vessels based on the volume data of the subject's eye.

[0003] The first aspect is an image processing method performed by a processor, the image processing method including the steps of: acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes having different depths based on the OCT volume data; deriving image features for each of the plurality of en-face images; and identifying, based on each of the image features, a boundary between the en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels.

[0004] A second aspect is an image processing device including a processor, wherein the processor executes the steps of acquiring OCT volume data including the choroid, generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data, deriving image features for each of the plurality of en-face images, and identifying, based on each of the image features, a boundary between the en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels.

[0005] A third aspect is a program for performing image processing, which causes a processor to perform the following steps: acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; deriving image features for each of the plurality of en-face images; and identifying, based on each of the image features, a boundary between en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels.

[0006] 1 is a schematic diagram of an ophthalmologic system according to an embodiment. FIG. 1 is a schematic diagram of an ophthalmologic apparatus according to an embodiment. FIG. 1 is a schematic diagram of a server. FIG. 2 is an explanatory diagram of functions realized by an image processing program in a CPU of the server. FIG. 2 is a flowchart showing an example of the flow of image processing by the server. FIG. 3 is an explanatory diagram related to image processing performed on an image. FIG. 4 is an explanatory diagram of image features that change depending on the presence or absence of vascular components. FIG. 5 is a diagram showing the characteristics of standard deviation for a plurality of en-face images in OCT volume data. FIG. 6 is a flowchart showing an example of the flow of vascular component presence / absence boundary acquisition processing. FIG. 7 is a flowchart showing an example of the flow of choroidal vessel image formation processing. FIG. 8 is a flowchart showing an example of third image processing by third vessel extraction processing. FIG. 9 is a schematic diagram showing the relationship between the positions of the eyeball and vortex veins. FIG. 10 is a diagram showing the relationship between OCT volume data and en-face images. FIG. 11 is a diagram showing an example of a fundus image of choroidal vessels including vortex veins. FIG. 12 is a conceptual diagram of a stereoscopic image of vortex veins. FIG. 13 is a diagram showing an example of a stereoscopic image of choroidal vessels around vortex veins. FIG. 14 is a diagram showing an example of a display screen using a stereoscopic image of vortex veins.

[0007] An ophthalmologic system 100 according to an embodiment of the present disclosure will be described below with reference to the drawings. FIG. 1 shows a schematic configuration of the ophthalmologic system 100. As shown in FIG. 1, the ophthalmologic system 100 includes an ophthalmologic apparatus 110, a server apparatus (hereinafter referred to as the "server") 140, and a display device (hereinafter referred to as the "viewer") 150. The ophthalmologic apparatus 110 acquires fundus images. The server 140 stores, in association with patient IDs, multiple fundus images obtained by photographing the funduses of multiple patients using the ophthalmologic apparatus 110 and axial lengths measured by an axial length measuring device (not shown). The viewer 150 displays the fundus images and analysis results acquired by the server 140.

[0008] The server 140 is an example of the "image processing device" of the present disclosure.

[0009] The ophthalmic apparatus 110, the server 140, and the viewer 150 are connected to each other via a network 130. The network 130 may be any network such as a LAN, a WAN, the Internet, or a wide area Ethernet network. For example, if the ophthalmic system 100 is established in a single hospital, a LAN may be used as the network 130.

[0010] The viewer 150 is a client in a client-server system, and multiple viewers 150 are connected via a network. Multiple servers 140 may also be connected via a network to ensure system redundancy. Alternatively, if the ophthalmic apparatus 110 has an image processing function and the image viewing function of the viewer 150, the ophthalmic apparatus 110 can acquire, process, and view fundus images in a standalone state. If the server 140 has the image viewing function of the viewer 150, the configuration of the ophthalmic apparatus 110 and the server 140 can acquire, process, and view fundus images.

[0011] In addition, other ophthalmic equipment (examination equipment such as visual field measurement and intraocular pressure measurement) and a diagnostic support device that performs image analysis using AI (Artificial Intelligence) may be connected to the ophthalmic device 110, the server 140, and the viewer 150 via the network 130.

[0012] Next, the configuration of the ophthalmologic apparatus 110 will be described with reference to FIG.

[0013] For ease of explanation, a scanning laser ophthalmoscope will be referred to as an "SLO," and an optical coherence tomography will be referred to as an "OCT."

[0014] When the ophthalmic apparatus 110 is placed on a horizontal plane, the horizontal direction is defined as the "X direction," the vertical direction relative to the horizontal plane is defined as the "Y direction," and the direction connecting the center of the pupil of the anterior segment of the subject's eye 12 and the center of the eyeball is defined as the "Z direction." Therefore, the X direction, Y direction, and Z direction are perpendicular to each other.

[0015] The ophthalmologic apparatus 110 includes an imaging device 14 and a control device 16. The imaging device 14 is equipped with an SLO unit 18 and an OCT unit 20, and acquires a fundus image of the subject's eye 12. Hereinafter, a two-dimensional fundus image acquired by the SLO unit 18 will be referred to as an SLO image. Furthermore, a tomographic image or an en-face image of the retina created based on OCT data acquired by the OCT unit 20 will be referred to as an OCT image.

[0016] The control device 16 comprises a computer having a CPU (Central Processing Unit) 16A, a RAM (Random Access Memory) 16B, a ROM (Read-Only Memory) 16C, and an input / output (I / O) port 16D.

[0017] The control device 16 includes an input / display device 16E connected to the CPU 16A via an I / O port 16D. The input / display device 16E has a graphic user interface that displays an image of the subject's eye 12 and receives various instructions from the user. An example of the graphic user interface is a touch panel display.

[0018] The control device 16 also includes an image processor 17 connected to an I / O port 16D. The image processor 17 generates an image of the subject's eye 12 based on data obtained by the photographing device 14. The control device 16 is connected to a network 130 via a communication interface (I / F) 16F.

[0019] 2 , the control device 16 of the ophthalmic apparatus 110 includes the input / display device 16E, but the present disclosure is not limited to this. For example, the control device 16 of the ophthalmic apparatus 110 may not include the input / display device 16E, but may instead include a separate input / display device that is physically independent from the ophthalmic apparatus 110. In this case, the display device includes an image processing unit that operates under the control of the display control unit 204 of the CPU 16A of the control device 16. The image processing unit may display an SLO image or the like based on an image signal instructed to be output by the display control unit 204.

[0020] The imaging device 14 operates under the control of the CPU 16A of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The imaging optical system 19 includes an optical scanner 22 and a wide-angle optical system 30.

[0021] The optical scanner 22 performs two-dimensional scanning in the X and Y directions with the light emitted from the SLO unit 18. The optical scanner 22 may be any optical element that can deflect a light beam, such as a polygon mirror or a galvanometer mirror, or a combination thereof.

[0022] The wide-angle optical system 30 combines the light from the SLO unit 18 and the light from the OCT unit 20 .

[0023] The wide-angle optical system 30 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a catadioptric optical system combining concave mirrors and lenses. By using a wide-angle optical system using an elliptical mirror or a wide-angle lens, it becomes possible to photograph the retina in the peripheral part of the fundus as well as the center of the fundus.

[0024] When a system including an elliptical mirror is used, the system using the elliptical mirror described in International Publication WO2016 / 103484 or International Publication WO2016 / 103489 may be used, the disclosures of which are each incorporated herein by reference in their entirety.

[0025] The wide-angle optical system 30 enables observation of the fundus over a wide field of view (FOV) 12A. The FOV 12A indicates the range that can be photographed by the imaging device 14. The FOV 12A can be expressed as a field of view. In this embodiment, the field of view can be defined by an internal illumination angle and an external illumination angle. The external illumination angle is the illumination angle of the light beam irradiated from the ophthalmic device 110 to the subject's eye 12, determined with the pupil 27 as the reference point. The internal illumination angle is the illumination angle of the light beam irradiated to the fundus, determined with the center O of the eyeball as the reference point. The external illumination angle and the internal illumination angle correspond to each other. For example, if the external illumination angle is 120 degrees, the internal illumination angle corresponds to approximately 160 degrees. In this embodiment, the internal illumination angle is 200 degrees.

[0026] Here, an SLO fundus image captured at an internal illumination angle and a field of view of 160 degrees or more is referred to as a UWF-SLO fundus image. UWF stands for Ultra Wide Field. The wide-angle optical system 30, which provides an ultra-wide field of view (FOV) of the fundus, can capture an image of the area extending from the posterior pole of the fundus of the subject's eye 12 beyond the equator, enabling the image of structures present in the peripheral area of ​​the fundus, such as vortex veins.

[0027] The ophthalmologic apparatus 110 can capture an image of an area 12A with an internal illumination angle of 200°, with the center O of the eyeball of the subject's eye 12 as the reference position. Note that the internal illumination angle of 200° corresponds to an external illumination angle of 110° with the pupil of the eyeball of the subject's eye 12 as the reference. In other words, the wide-angle optical system 30 irradiates the laser light from the pupil at an angle of view with an external illumination angle of 110°, and captures an image of a fundus area of ​​200° with an internal illumination angle.

[0028] 2, the SLO system is realized by a control device 16, an SLO unit 18, and an imaging optical system 19. The SLO system includes a wide-angle optical system 30, and therefore enables fundus imaging with a wide FOV 12A.

[0029] The SLO unit 18 includes a light source 40 for B light (blue light), a light source 42 for G light (green light), a light source 44 for R light (red light), and a light source 46 for IR light (infrared light (e.g., near-infrared light)), as well as optical systems 48, 50, 52, 54, and 56 that reflect or transmit the light from the light sources 40, 42, 44, and 46 and guide them into a single optical path. The optical systems 48 and 56 are mirrors, and the optical systems 50, 52, and 54 are beam splitters. The B light is reflected by the optical system 48, passes through the optical system 50, and is reflected by the optical system 54; the G light is reflected by the optical systems 50 and 54; the R light is transmitted through the optical systems 52 and 54; and the IR light is reflected by the optical systems 52 and 56 and is each guided into a single optical path.

[0030] The SLO unit 18 is configured to be switchable between a light source that emits laser light of different wavelengths or a combination of light sources that emit light, such as a mode that emits R light and G light and a mode that emits infrared light. In the example shown in Fig. 2, the SLO unit 18 includes four light sources: a B light source 40, a G light source 42, an R light source 44, and an IR light source 46; however, the present disclosure is not limited to this. For example, the SLO unit 18 may further include a white light source and emit light in various modes, such as a mode that emits G light, R light, and B light, or a mode that emits only white light.

[0031] Light incident on the imaging optical system 19 from the SLO unit 18 is scanned in the X and Y directions by the optical scanner 22. The scanned light passes through the wide-angle optical system 30 and the pupil 27 and is then irradiated onto the fundus. The light reflected by the fundus passes through the wide-angle optical system 30 and the optical scanner 22 and is incident on the SLO unit 18.

[0032] The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits all light except B light from the posterior segment (fundus) of the eye 12, and a beam splitter 58 that reflects G light and transmits all light except G light from the light that has passed through the beam splitter 64. The SLO unit 18 also includes a beam splitter 60 that reflects R light and transmits all light except R light from the light that has passed through the beam splitter 58. The SLO unit 18 also includes a beam splitter 62 that reflects IR light from the light that has passed through the beam splitter 60. The SLO unit 18 also includes a B light detection element 70 that detects B light reflected by the beam splitter 64, a G light detection element 72 that detects G light reflected by the beam splitter 58, an R light detection element 74 that detects R light reflected by the beam splitter 60, and an IR light detection element 76 that detects IR light reflected by the beam splitter 62.

[0033] Light (light reflected by the fundus) incident on the SLO unit 18 via the wide-angle optical system 30 and the optical scanner 22 is reflected by the beam splitter 64 and received by the B light detection element 70 in the case of B light, and is reflected by the beam splitter 58 and received by the G light detection element 72 in the case of G light. The incident light is transmitted through the beam splitter 58 in the case of R light, reflected by the beam splitter 60, and received by the R light detection element 74. The incident light is transmitted through the beam splitters 58 and 60 in the case of IR light, reflected by the beam splitter 62, and received by the IR light detection element 76. The image processor 17, which operates under the control of the CPU 16A, generates a UWF-SLO image using signals detected by the B light detection element 70, the G light detection element 72, the R light detection element 74, and the IR light detection element 76.

[0034] A UWF-SLO image generated using a signal detected by the B light detecting element 70 is referred to as a B-UWF-SLO image (B-color fundus image). A UWF-SLO image generated using a signal detected by the G light detecting element 72 is referred to as a G-UWF-SLO image (G-color fundus image). A UWF-SLO image generated using a signal detected by the R light detecting element 74 is referred to as an R-UWF-SLO image (R-color fundus image). A UWF-SLO image generated using a signal detected by the IR light detecting element 76 is referred to as an IR-UWF-SLO image (IR fundus image). UWF-SLO images include R-color fundus images, G-color fundus images, B-color fundus images, and even IR fundus images. Fluorescent UWF-SLO images obtained by capturing fluorescence are also included.

[0035] The control device 16 also controls the light sources 40, 42, and 44 to emit light simultaneously. By simultaneously capturing images of the fundus of the subject's eye 12 using B, G, and R light, a G-color fundus image, a R-color fundus image, and a B-color fundus image, each of which corresponds to a different position, are obtained. An RGB color fundus image is obtained from the G-color fundus image, the R-color fundus image, and the B-color fundus image. By simultaneously controlling the light sources 42 and 44 to emit light simultaneously and simultaneously capturing images of the fundus of the subject's eye 12 using G and R light, a G-color fundus image and a R-color fundus image, each of which corresponds to a different position, are obtained. An RG color fundus image is obtained from the G-color fundus image and the R-color fundus image. A full-color fundus image may also be generated using the G-color fundus image, the R-color fundus image, and the B-color fundus image.

[0036] The wide-angle optical system 30 makes the field of view (FOV) of the fundus an ultra-wide angle, and can capture an image of the area from the posterior pole of the fundus of the subject's eye 12 beyond the equator.

[0037] The OCT system is realized by the control device 16, OCT unit 20, and imaging optical system 19 shown in FIG. 2 . The OCT system includes a wide-angle optical system 30, which enables OCT imaging of the peripheral portion of the fundus, similar to the above-described SLO fundus image capture. That is, the wide-angle optical system 30, which provides an ultra-wide fundus field of view (FOV), enables OCT imaging of the region extending from the posterior pole of the fundus of the subject's eye 12 beyond the equator 178. OCT data of structures present in the peripheral portion of the fundus, such as vortex veins, can be acquired, and tomographic images of the vortex veins and the 3D structure of the vortex veins can be obtained by image processing the OCT data.

[0038] The OCT unit 20 includes a light source 20A, a sensor (detecting element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

[0039] Light emitted from the light source 20A is branched by the first optical coupler 20C. One of the branched beams is collimated by the collimating lens 20E as measurement light and then enters the imaging optical system 19. The measurement light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus. The measurement light reflected by the fundus passes through the wide-angle optical system 30 and enters the OCT unit 20, and then passes through the collimating lens 20E and the first optical coupler 20C and enters the second optical coupler 20F.

[0040] The other light beam emitted from the light source 20A and branched by the first optical coupler 20C is incident as reference light on the reference optical system 20D, passes through the reference optical system 20D, and enters the second optical coupler 20F.

[0041] The light beams incident on the second optical coupler 20F, i.e., the measurement light beam reflected by the fundus and the reference light beam, interfere with each other at the second optical coupler 20F to generate interference light. The interference light beam is received by the sensor 20B. The image processor 17, which operates under the control of the image processing unit 206, generates OCT data detected by the sensor 20B. The image processor 17 can also generate OCT images, such as tomographic images and en-face images, based on the OCT data.

[0042] Here, the OCT unit 20 can scan a predetermined range (e.g., a rectangular range of 6 mm x 6 mm) in one OCT imaging session. The predetermined range is not limited to 6 mm x 6 mm, but may be a square range of 12 mm x 12 mm or 23 mm x 23 mm, or a rectangular range such as 14 mm x 9 mm or 6 mm x 3.5 mm, or any other rectangular range. It may also be a circular range with a diameter of 6 mm, 12 mm, or 23 mm.

[0043] By using the wide-angle optical system 30, the ophthalmic apparatus 110 can scan the area 12A with an internal illumination angle of 200°. That is, by controlling the optical scanner 22, OCT imaging of a predetermined range including vortex veins is performed. The ophthalmic apparatus 110 can generate OCT data through this OCT imaging.

[0044] Therefore, the ophthalmologic apparatus 110 can generate OCT images, such as tomographic images (B-scan images) of the fundus including vortex veins, OCT volume data including vortex veins, and en-face images (frontal images generated based on the OCT volume data) that are cross sections of the OCT volume data. It goes without saying that the OCT images include OCT images of the center of the fundus (the posterior pole of the eyeball where the macula, optic disc, etc. are present).

[0045] The OCT data (or image data of the OCT image) is sent from the ophthalmologic apparatus 110 to the server 140 via the communication interface 16F and stored in the storage device 254.

[0046] In this embodiment, the light source 20A is exemplified as a wavelength-swept type SS-OCT (Swept-Source OCT), but various types of OCT systems may also be used, such as SD-OCT (Spectral-Domain OCT) and TD-OCT (Time-Domain OCT).

[0047] Next, the configuration of the electrical system of the server 140 will be described with reference to FIG. 3 . As shown in FIG. 3 , the server 140 includes a computer main unit 252. The computer main unit 252 has a CPU 262, a RAM 266, a ROM 264, and an input / output (I / O) port 268. The input / output (I / O) port 268 is connected to a storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I / F) 258. The storage device 254 is configured, for example, with a non-volatile memory. The input / output (I / O) port 268 is connected to the network 130 via the communication interface (I / F) 258. Therefore, the server 140 can communicate with the ophthalmologic apparatus 110 and the viewer 150.

[0048] The ROM 264 or the storage device 254 stores an image processing program.

[0049] The ROM 264 or the storage device 254 is an example of a "memory" in the present disclosure. The CPU 262 is an example of a "processor" in the present disclosure. The image processing program is an example of a "program" in the present disclosure.

[0050] The server 140 stores each piece of data received from the ophthalmologic apparatus 110 in the storage device 254 .

[0051] The following describes various functions that are realized by the CPU 262 of the server 140 executing the image processing program. As shown in Fig. 4, the image processing program executed by the CPU 262 has a display control function, an image processing function, and a processing function. By the CPU 262 executing the image processing program having these functions, the CPU 262 functions as the display control unit 204, the image processing unit 206, and the processing unit 208.

[0052] Next, a main flowchart of image processing by the server 140 will be described with reference to Fig. 5. The CPU 262 of the server 140 executes an image processing program to realize the image processing (image processing method) shown in Fig. 5.

[0053] First, in step S10, the image processing unit 206 acquires a fundus image from the storage device 254. The fundus image includes data related to the vortex vein to be displayed in three dimensions, based on a user's instruction.

[0054] Next, in step S20, the image processing unit 206 acquires OCT volume data including the choroid corresponding to the fundus image from the storage device 254.

[0055] After acquiring the OCT volume data, the image processing unit 206 executes a vascular component presence / absence boundary acquisition process (details of which will be described later) in step S22 to acquire a boundary indicating the presence or absence of choroidal blood vessels.

[0056] In the next step S30, the image processing unit 206 extracts choroidal blood vessels based on the OCT volume data and executes image formation processing of the choroidal blood vessels (details of which will be described later) to generate a stereoscopic image (3D image) of the vortex vein blood vessels.

[0057] Once the three-dimensional image (3D image) of the vortex vein blood vessels is generated, in step S40, the processing unit 208 outputs the generated three-dimensional image (3D image) of the vortex vein blood vessels, specifically, stores it in the RAM 266 or the storage device 254, and terminates image processing.

[0058] Here, a display screen (an example of a display screen is shown in FIG. 17 , which will be described later) containing a three-dimensional image of the vortex veins is generated by the display control unit 204 based on a user instruction. The generated display screen is output as an image signal by the processing unit 208 to the viewer 150. The display screen is displayed on the display of the viewer 150.

[0059] The positional relationship between the choroid 12M and the vortex veins 12V1 and V2 in the eyeball will now be described with reference to FIG. 12 . In FIG. 12 , the mesh-like pattern represents the choroidal blood vessels of the choroid 12M. The choroidal blood vessels circulate blood throughout the entire choroid. Blood flows out of the eyeball through multiple (usually four to six) vortex veins present in the subject's eye 12. FIG. 12 shows the superior vortex vein 12V1 and the inferior vortex vein 12V2 present on one side of the eyeball. Vortex veins are often located near the equator. Therefore, to photograph the vortex veins present in the subject's eye 12 and the choroidal blood vessels around the vortex veins, an ophthalmic apparatus 110 capable of scanning at an internal illumination angle of 200° is used, for example.

[0060] First, the image processing unit 206 acquires a fundus image (step S10) and identifies the vortex vein (VV) to be displayed in three dimensions. Here, as an example, a UWF-SLO image is acquired from the storage device 254 as a UWF fundus image. Next, the image processing unit 206 creates a choroidal vessel image, which is a binarized image, from the acquired UWF-SLO image. Then, the image processing unit 206 identifies the area designated by the user as the vortex vein to be displayed in three dimensions.

[0061] Fig. 14 is a fundus image of choroidal vessels including vortex veins. The fundus image shown in Fig. 14 is an example of a choroidal vessel image, which is a binarized image created from a UWF-SLO image. As shown in Fig. 14, the choroidal vessel image is a binarized image in which pixels corresponding to choroidal vessels and vortex veins are white and pixels in other areas are black.

[0062] 14 shows an image 302 indicating the presence of choroidal blood vessels connected to a vortex vein. The image 302 shows a case in which a vortex vein 310V1, which is an image of an upper vortex vein 12V1 included in a user-specified area 310A, is identified as a vortex vein (VV) to be displayed in three dimensions, and a region including the choroidal blood vessels is identified.

[0063] A choroidal vessel image including vortex veins (VVs) is generated by processing image data of an R-UWF-SLO image (R-color fundus image) captured with red light (laser light with a wavelength of 630 to 660 nm) and a G-UWF-SLO image (G-color fundus image) captured with green light (laser light with a wavelength of 500 to 550 nm). Specifically, the choroidal vessel image is generated by extracting retinal vessels from the G-color fundus image, removing retinal vessels from the R-color fundus image, and performing image processing to enhance the choroidal vessels. The disclosure of International Publication WO 2019 / 181981 regarding a method for generating a choroidal vessel image is incorporated herein by reference in its entirety.

[0064] Although the above describes a case in which a vortex vein to be displayed in three dimensions is identified by a user's instruction, the present disclosure is not limited to this. The position of the vortex vein to be displayed in three dimensions may be detected manually or automatically. For example, in the case of manual detection, the position indicated by the user's visual inspection of the displayed choroidal vessels may be detected. In the case of automatic detection, for example, the choroidal vessels may be extracted from a choroidal vessel image, the movement direction of each choroidal vessel (the direction of blood vessel travel) may be estimated, and the position of the vortex vein may be estimated based on the position where the choroidal vessels converge.

[0065] When forming an image of choroidal blood vessels, it may be necessary to extract both choroidal blood vessels exceeding a predetermined diameter (hereinafter referred to as "thick blood vessels") and those smaller than the predetermined diameter (hereinafter referred to as "thin blood vessels"). Because thin blood vessels have lower image contrast than thick blood vessels, applying a common image processing method to thick and thin blood vessels makes it difficult to extract thin blood vessels as continuous line structures. Therefore, it is possible to extract thick and thin blood vessels using separate image processing methods. The details of the process of extracting thick and thin blood vessels using separate image processing methods will be described later. However, in the process of extracting thin blood vessels, there is a risk that noise images may be extracted as thin blood vessels, leading to the determination that blood vessels exist even in areas where no blood vessels exist. This reduces the accuracy of the location of the boundary (e.g., the sclera) between the presence and absence of blood vessel components.

[0066] 13 , the OCT volume data 400 is OCT volume data 400 of a predetermined area, for example, a rectangular area of ​​6 mm × 6 mm, including the vortex vein VV, obtained by OCT imaging of one of the multiple vortex veins VV present in the subject's eye using the ophthalmic apparatus 110. N planes at different depths, from a first plane f401 to an Nth plane f40N, are set for the OCT volume data 400. The OCT volume data 400 may be obtained by OCT imaging of each of the multiple vortex veins VV present in the subject's eye using the ophthalmic apparatus 110.

[0067] In this embodiment, the OCT volume data 400 including vortex veins and choroidal blood vessels around the vortex veins will be described as an example of the OCT volume data 400D. In this case, the choroidal blood vessels refer to the vortex veins and the choroidal blood vessels around the vortex veins.

[0068] FIG. 6 shows an example of image processing performed on an image of choroidal vessels. FIG. 6 shows the results of performing image processing for thick blood vessels and image processing for thin blood vessels on an en-face image of a region of choroidal vessels where no blood vessels are present. As shown in FIG. 6, if an image containing noise components is present in the en-face image f40K of the region where no blood vessels are present, the image f40KL1 is subjected to processing to extract thick blood vessels, and then binarized to produce an image f40KL2, from which the noise components have been removed. On the other hand, the image f40KS2 is subjected to processing to extract thin blood vessels (image f40KS1), and then binarized to produce an image f40KS2, from which the noise components remain. Therefore, when processing to extract thin blood vessels is performed, the noise image may be extracted as thin blood vessels, and it may be determined that blood vessels exist even in regions where no blood vessels exist.

[0069] An image containing vascular components differs in image feature quantity from an image containing residual noise components. Examples of image feature quantities that can be applied include the standard deviation of image brightness, the trend in the standard deviation, and the entropy of image brightness. The standard deviation of image brightness can be calculated using the standard deviation of each en-face image. Furthermore, the trend in the standard deviation can be calculated using a feature quantity represented by the differential value of the characteristic curve of the standard deviation of multiple en-face images. The entropy of image brightness can be calculated using a physical quantity related to the sum of the brightnesses of the pixels in the en-face image. In this embodiment, a case where the standard deviation of image brightness is used as the image feature quantity will be described.

[0070] 7 shows an example of image feature amounts (here, standard deviations) that change depending on the presence or absence of blood vessel components. An example of a case where image processing is performed on an image of choroidal blood vessels is shown.

[0071] First, we will consider the standard deviation of image f40HS1, which is an image of choroidal vessels obtained by image processing of en-face image f40H, a region where blood vessels are present (vascular components present). The standard deviation value in image f40HS1 corresponds to the distribution width TH1 in the characteristics of signal intensity and frequency. Signal intensity indicates a physical quantity that represents the brightness of the image in image f40HS1, and frequency indicates the frequency with which that physical quantity appears in image f40HS1. Similarly, in image f40KS1, which is an image processing performed on en-face image f40K, a region where blood vessels are not present (no vascular components), the standard deviation value corresponds to the distribution width TH2. In terms of width, the width TH1 indicates the standard deviation value of en-face image f40H, which has vascular components, while the width TH2 (<TH1) indicates a standard deviation value smaller than width TH1 in en-face image f40KS1, which does not have vascular components. This means that the standard deviation value tends to decrease as the number of vascular components decreases. Therefore, by predetermining a boundary determination value that indicates the presence or absence of blood vessels, i.e., the transition between the presence and absence of vascular components, it is possible to determine the boundary between the presence and absence of vascular components. The boundary determination value determines a standard deviation value according to a width TH0 (TH2≦TH0<TH1) that is smaller than the width TH1 and larger than or equal to the width TH2. Therefore, an en-face image of a surface (layer) with a standard deviation value larger than the standard deviation value indicated by the width TH0 can be determined to have vascular components, and an en-face image of a surface (layer) with a standard deviation value smaller than or equal to the standard deviation value indicated by the width TH0 can be determined to have no vascular components.

[0072] The above-mentioned boundary determination value can be derived in advance. FIG. 8 shows the characteristics of the standard deviation for multiple en-face images in the OCT volume data 400. As shown in FIG. 8, the standard deviation characteristic, from the first surface f401 to the Nth surface f40N, reaches a maximum value Hu at the u-th surface, then gradually decreases, and converges to a minimum value Hv at the v-th surface. Therefore, a standard deviation value that is smaller than the maximum value Hu and equal to or greater than the minimum value Hv can be set as the boundary determination value Ho. This boundary determination value Ho is likely to be close to the minimum value Hv to which the standard deviation converges, and it is also possible to reflect the results of measurements made in advance. The minimum value Hv may also be used as the boundary determination value.

[0073] In addition, from the viewpoint of the characteristic change of the standard deviation, it is also possible to apply the slope w that indicates the differential value of the characteristic curve of the standard deviation.

[0074] Therefore, in this embodiment, a vascular component presence / absence boundary acquisition process is performed to acquire a boundary regarding the presence or absence of choroidal blood vessels using image feature amounts based on the OCT volume data. Next, the vascular component presence / absence boundary acquisition process (step S22) will be described in detail with reference to Figure 9. The CPU 262 of the server 140 executes an image processing program to realize the image processing (image processing method) shown in the flowchart of Figure 9.

[0075] Specifically, in step S220, the image processing unit 206 acquires OCT volume data 400, which is OCT data, for use in processing to obtain the boundary between the presence and absence of blood vessel components. N planes at different depths, from a first plane f401 to an Nth plane f40N, are set in the OCT volume data 400.

[0076] In step S221, the image processing unit 206 sets a parameter n to 1. The parameter n is a parameter indicating the number of en-face images (number of faces, number of layers).

[0077] In step S222, the image processing unit 206 analyzes the OCT volume data 400 and sets a first plane from, for example, the retinal pigment epithelium cell layer (hereinafter referred to as the RPE layer) in the OCT volume data 400. The first plane may be set as a plane a predetermined number of pixels below the RPE layer, for example, 10 pixels below. The image processing unit 206 can identify the RPE layer 400R as the reference plane as the first plane f401. The RPE layer 400R can be identified by performing a predetermined segmentation process on the OCT volume data 400. Alternatively, the RPE layer may be identified by determining the brightest layer in the OCT volume data 400 as the RPE layer.

[0078] Setting the plane 10 pixels below the RPE layer as the first plane is effective for generating an en-face image of the region where choroidal blood vessels exist, since the region deeper than the RPE layer (the region farther from the RPE layer when viewed from the center of the eyeball) is the choroid region. Setting the plane 10 pixels below the RPE layer as the first plane is not limited to this. For example, the plane 10 pixels below Bruch's membrane, which exists immediately below the RPE layer, may be set as the first plane. Bruch's membrane is also identified by performing a predetermined segmentation process on the OCT volume data 400 that is different from that for the RPE layer. Note that, to identify the position 10 pixels below, the plane may be set 10 pixels below in the direction of the A-scan when the OCT volume data was generated.

[0079] Furthermore, the first plane is not limited to being specified as a plane 10 pixels below the RPE layer or Bruch's membrane, and may be set to any number of pixels. Furthermore, instead of being defined by the number of pixels, it may be defined by a length such as millimeters or nanometers. Furthermore, a spherical surface at a certain distance from the pupil or the center of the eyeball may be defined as the reference plane.

[0080] In step S223, the image processing unit 206 generates a first en-face image corresponding to the set first plane. The en-face image may be generated from the pixel values ​​of pixels present on the first plane, or a group of pixels in the shallow direction and a group of pixels in the deep direction including the first plane may be extracted from the OCT volume data 400, and the pixel values ​​may be calculated as the average or median brightness values ​​of these pixel groups. Image processing such as noise removal may be used to calculate the pixel values. The generated first en-face image corresponding to the first plane is stored in RAM 266 by the processing unit 208.

[0081] In step S224, the image processing unit 206 derives image features for the nth en-face image (here, the first surface). Here, a standard deviation value for the en-face image of the first surface is derived. The standard deviation value is derived using pixel values ​​of pixels present in the en-face image. When deriving the image features, a layer application range may be determined. For example, a process is performed to determine a predetermined layer range as the range for deriving image features, and image features can be derived for the determined layer range. For the predetermined layer range, a layer range (e.g., a layer range from layer 80 to layer 120) where the depth at which a boundary exists can be empirically confirmed can be applied.

[0082] In step S225, the image processing unit 206 determines the boundary between the presence and absence of blood vessels by using the boundary determination value Ho to determine whether the standard deviation value corresponds to the boundary determination value Ho. The boundary between the presence and absence of blood vessels is determined as an en-face image in which no blood vessel components exist, or as a boundary between adjacent en-face images in which a change in the presence or absence of blood vessels has occurred.

[0083] In step S226, the image processing unit 206 determines whether a boundary has been detected based on the judgment result of the boundary between the presence and absence of blood vessels, and if the judgment is positive, the processing proceeds to step S229, and if the judgment is negative, the processing proceeds to step S227.

[0084] When the boundary between the presence and absence of blood vessels is determined and the process proceeds to step S229, the image processing unit 206 stores information indicating the boundary between the presence and absence of blood vessels. Specifically, in step S229, the processing unit 208 stores the determined position of the en-face image or the position between adjacent en-face images in the RAM 266 or the storage device 254, and then ends the process.

[0085] On the other hand, in step S227, the image processing unit 206 increments the parameter n (n=n+1), sets the nth plane in step S228, and returns the process to step S223.

[0086] In this way, the image processing unit 206 repeats the loop from step S223 to step S228 until the parameter n reaches the maximum number N.

[0087] By performing the image processing shown in FIG. 9 using the image processing unit 206, it becomes possible to identify the boundary between the presence and absence of blood vessels, and by superimposing this boundary on the choroidal vessel image, it becomes possible to visualize the boundary between the blood vessel image and the noise image, for example, the part corresponding to the sclera.

[0088] Next, the image formation process of choroidal vessels in step S30 for generating a stereoscopic image of vortex veins (VV) will be described in detail with reference to FIG.

[0089] In step S31 of FIG. 10 , the image processing unit 206 extracts a region corresponding to the choroid from the OCT volume data 400 (see FIG. 13 ) acquired in step S20, and extracts (acquires) OCT volume data of the choroid based on the extracted region.

[0090] Specifically, the image processing unit 206 acquires OCT volume data for choroidal vessel extraction. The acquisition of OCT volume data may involve extracting a portion of the OCT volume data scanned to include vortex veins and choroidal vessels surrounding the vortex veins. For example, OCT volume data 400D of a region below the RPE layer may be extracted. Alternatively, OCT volume data 400D of a region determined to contain vascular components in the vascular component presence / absence boundary acquisition process described above may be extracted.

[0091] Next, in step S32, the image processing unit 206 executes a first blood vessel extraction process (ampulla extraction) using the OCT volume data 400D. The first blood vessel extraction process is a process for extracting choroidal blood vessels (hereinafter referred to as ampulla) that form the ampulla, which is the first blood vessel.

[0092] As preprocessing for the first blood vessel extraction process (dilation extraction), the image processing unit 206 performs binarization processing on the OCT volume data 400D, and then performs noise removal processing. To remove noise regions, the image processing unit 206 performs median filtering, opening processing, contraction processing, or the like on the binarized OCT volume data 400D to remove the noise regions.

[0093] Next, the image processing unit 206 performs segmentation processing (image processing such as active contouring, graph cut, or U-net) on the OCT volume data from which the noise regions have been deleted in order to smooth the surface of the extracted bulging portion. Note that "segmentation" refers to image processing that performs binarization processing to separate the background and foreground of the image to be analyzed.

[0094] By performing this first blood vessel extraction process, only the dilation region remains from the OCT volume data 400D, and a three-dimensional image 680B of the blood vessels in the dilation region is generated as shown in Fig. 15. Image data of the three-dimensional image 680B of the blood vessels in the dilation region is stored in the RAM 266 by the processing unit 208.

[0095] 10, the image processing unit 206 executes a second blood vessel extraction process (thick blood vessel extraction) using the OCT volume data 400D. The second blood vessel extraction process is a process for extracting choroidal blood vessels (thick blood vessels) that are thick, linear second blood vessels extending from the dilation and exceed a predetermined threshold, i.e., a predetermined diameter. In the second blood vessel extraction process (thick blood vessel extraction), linear second blood vessels extending from the dilation are extracted. The thick blood vessels mainly indicate blood vessels located in the Haller layer.

[0096] The predetermined threshold (i.e., the predetermined diameter) may be a value determined in advance so that blood vessels with a diameter of several hundred microns are left as thick blood vessels. The threshold determined to leave thin blood vessels (described later) may be a value less than a diameter of several hundred microns determined to leave blood vessels as thick blood vessels, or may be a value smaller than a value determined in advance so that blood vessels are left as thick blood vessels. For example, a value determined in advance so that blood vessels with a diameter of several tens of microns are left as thin blood vessels may be used.

[0097] The image processing unit 206 executes image processing to perform preprocessing on the OCT volume data 400D. An example of the preprocessing is blurring, such as noise removal. The blurring can be performed by eliminating the influence of speckle noise and extracting linear blood vessels that accurately reflect the blood vessel shapes. Examples of speckle noise processing include Gaussian blurring.

[0098] Next, the image processing unit 206 performs line extraction processing (thick linear blood vessel extraction) on the pre-processed OCT volume data 400D to extract second choroidal blood vessels, which are thick linear portions, from the OCT volume data 400D. In the extraction processing of the second choroidal blood vessels, for example, image processing using an eigenvalue filter, a Gabor filter, or the like is performed to extract linear blood vessel regions from the OCT volume data 400D.

[0099] The image processing unit 206 performs a binarization process on the OCT volume data 400D, and performs image processing such as a process to remove isolated areas that are not connected to surrounding blood vessels, a median filter process, an opening process, and a contraction process on the binarized linear blood vessel areas, thereby removing discrete micro-areas.

[0100] By the above image processing, a second stereoscopic image of the second choroidal vessels, which are thick blood vessels, is generated.

[0101] By performing the second blood vessel extraction process described above, only the regions of the large blood vessels remain from the OCT volume data 400D, and a three-dimensional image 680L of the large blood vessels shown in Fig. 15 is generated. The image data of the three-dimensional image 680L of the large blood vessels is stored in the RAM 266 by the processing unit 208.

[0102] 16 shows an example of a three-dimensional image of choroidal vessels around a vortex vein VV obtained by the image processing described above (FIG. 5). By performing the second blood vessel extraction process described above, only the region of the large blood vessels remains from the OCT volume data 400D, and a three-dimensional image 681L of the large blood vessels shown in FIG. 16 is generated. Image data of this three-dimensional image 681L of the large blood vessels is also stored in the RAM 266 by the processing unit 208.

[0103] The image processing unit 206 aligns the three-dimensional image 680B of the dilation and the three-dimensional image 680L of the linear blood vessels and performs a logical OR operation on both images to synthesize the three-dimensional image 680L of the linear blood vessels and the three-dimensional image 680B of the dilation. This makes it possible to generate a three-dimensional image 680M ( FIG. 15 ) of the choroidal blood vessels, including vortex veins, which are thick blood vessels. In the process of extracting the thick blood vessels described above, thin blood vessels smaller than the predetermined diameter may be removed.

[0104] When observing vortex veins, it is important to observe not only the large blood vessels located in the Haller layer but also the small blood vessels located mainly in the Sattler layer. For example, analyzing the small blood vessels in the Sattler layer is effective for diagnosing pachychoroidal diseases. Therefore, the present disclosure includes a process for extracting choroidal vessels (small blood vessels) that are thin, linear third blood vessels extending from the ampulla and have a diameter equal to or smaller than a predetermined threshold, i.e., a predetermined value.

[0105] Specifically, in step S34 shown in FIG. 10 , the image processing unit 206 executes a third blood vessel extraction process (thin blood vessel extraction) using the OCT volume data 400D. The third blood vessel extraction process is a process for extracting thin, linear third blood vessels (thin blood vessels) extending from the dilation, which are choroidal blood vessels (thin blood vessels) having a diameter equal to or smaller than a predetermined threshold, i.e., a predetermined predetermined diameter. In the third blood vessel extraction process (thin blood vessel extraction), linear third blood vessels extending from the dilation are extracted. The thin blood vessels mainly indicate blood vessels located in the Sattler layer. In the third blood vessel extraction process (thin blood vessel extraction), the third image processing shown in FIG. 11 is executed.

[0106] In the process of extracting the third blood vessels, which are thin blood vessels, the image processing unit 206 performs preprocessing for thin blood vessels, including first preprocessing and second preprocessing, on the OCT volume data 400D. First, in step S341 shown in Fig. 11, image processing is performed to perform the first preprocessing on the OCT volume data 400D. An example of the first preprocessing is blurring, which is an example of processing for removing noise.

[0107] In the next step S342, the image processing unit 206 executes image processing to perform second preprocessing on the OCT volume data 400D that has been subjected to the first preprocessing. Contrast enhancement processing is an example of the second preprocessing. Contrast enhancement processing is effective when extracting thin blood vessels. Contrast enhancement processing increases the contrast of the image compared to before processing, that is, increases the difference between light and dark. For example, the difference between the maximum and minimum values ​​of the degree of brightness (e.g., luminance) is increased by a predetermined value from the difference value before processing. The predetermined value can be set as appropriate.

[0108] When an image that has undergone the above-described contrast enhancement processing is binarized, thin blood vessels appear as continuous lines, making it possible to reduce the separation of continuous thin blood vessels.

[0109] In step S342, the image processing unit 206 performs image processing using, for example, an eigenvalue filter, a Gabor filter, or the like, and is able to extract areas of linear blood vessels, which are thin blood vessels, from the OCT volume data 400D.

[0110] 11, the image processing unit 206 executes image processing to perform binarization on the contrast-enhanced OCT volume data 400D. Specifically, by setting the binarization threshold to a predetermined threshold that leaves thin blood vessels, thin blood vessels become black pixels and other parts become white pixels in the OCT volume data D.

[0111] Furthermore, in step S344, the image processing unit 206 removes discrete minute regions from the binarized image (regions including thin blood vessels). Here, for example, image processing such as removing speckle noise and regions that are isolated at a predetermined distance and are estimated not to be continuous with the surrounding blood vessels is performed to remove the discrete minute regions.

[0112] In the next step S345, as post-processing, the image processing unit 206 performs a microregion connection process on the OCT volume data 400D from which the microregions have been removed, thereby extracting third choroidal vessels, which are thin blood vessels in thin linear portions, from the OCT volume data 400D. Specifically, the image processing unit 206 performs image processing using, for example, a morphology process such as a closing process, to connect discretely detected thin blood vessels, thereby extracting third choroidal vessels, which are thin blood vessels, from the OCT volume data 400D. Specifically, the image processing unit 206 connects third choroidal vessels within a predetermined distance. In the image subjected to the microregion connection process, even thin blood vessels with large curvature portions appear as continuous lines, making it possible to reduce the separation of continuous thin blood vessels.

[0113] In step S346, the image processing unit 206 performs segmentation processing (image processing such as active contouring, graph cut, or U-net) on the OCT volume data to which the fine regions are connected in order to smooth the surfaces of the extracted thin blood vessels. That is, the image to be analyzed is processed to separate the background from the foreground.

[0114] By the above image processing, a third stereoscopic image of the third choroidal vessels, which are thin blood vessels, is generated.

[0115] By performing the third blood vessel extraction process described above, only the thin blood vessel regions remain from the OCT volume data 400D, and a three-dimensional image 681S of the thin blood vessels shown in Fig. 16 is generated. Image data of the three-dimensional image 681S of the thin blood vessels is stored in the RAM 266 by the processing unit 208.

[0116] The processing of steps S32, S33, and S34 is not limited to the processing order described above, and any one of the processing may be executed first, or the processing may be executed simultaneously in parallel.

[0117] After steps S32, S33, and S34 are completed, in step S35, the image processing unit 206 reads out the three-dimensional image of the dilation, the three-dimensional image of the large blood vessels, and the three-dimensional image of the small blood vessels from the RAM 266. These three-dimensional images are then aligned and the logical sum of each image is calculated to synthesize the three-dimensional image of the dilation, the three-dimensional image of the large blood vessels, and the three-dimensional image of the small blood vessels. This generates a three-dimensional image 381M (see FIG. 16 ) of the choroidal vessels, including the vortex veins. The image data of the three-dimensional image 681M is stored in the RAM 266 or the storage device 254 by the processing unit 208.

[0118] Furthermore, information indicating the boundary between the presence and absence of blood vessels obtained by the above-described blood vessel component presence / absence boundary acquisition process ( FIG. 9 ) is also read from the RAM 266 and combined into the combined three-dimensional image. Image data of the three-dimensional image 681M, in which information indicating the boundary between the presence and absence of blood vessels is constructed, is stored in the RAM 266 or the storage device 254 by the processing unit 208.

[0119] The following describes a display screen for displaying the generated stereoscopic image (3D image) of choroidal vessels including vortex veins. The display screen is generated by the display control unit 204 of the server 140 based on a user instruction, and is output as an image signal to the viewer 150 by the processing unit 208. The viewer 150 displays the display screen on the display based on the image signal.

[0120] A display screen 500A is shown in Figure 17. As shown in Figure 17, the display screen 500A has an information area 502 and an image display area 504A. The image display area 504A includes a comment field 506 that displays the patient's medical history.

[0121] The information area 502 has a patient ID display field 512, a patient name display field 514, an age display field 516, a visual acuity display field 518, a right eye / left eye display field 520, and an axial length display field 522. In each display area from the patient ID display field 512 to the axial length display field 522, the viewer 150 displays the respective information based on the information received from the server 140.

[0122] The image display area 504A is an area that mainly displays an image of the subject's eye, etc. The image display area 504A is provided with the following display fields, specifically, a UWF fundus image display field 542 and a choroidal blood vessel stereoscopic image display field 548. Although not shown, the image display area 504A can also display an OCT volume data conceptual diagram display field and a tomographic image display field 546 in a superimposed manner.

[0123] The comment field 506 included in the image display area 504A functions as a remarks field in which the patient's medical history is displayed and the ophthalmologist user can optionally input the results of his or her observations and diagnosis.

[0124] The UWF fundus image display field 542 displays a UWF-SLO fundus image 542B obtained by photographing the fundus of the subject's eye using the ophthalmologic apparatus 110. A range 542A indicating the position where the OCT volume data was acquired is superimposed on the UWF-SLO fundus image 542B. If there are multiple pieces of OCT volume data associated with the UWF-SLO image, the multiple ranges may be superimposed and displayed, allowing the user to select one position from the multiple ranges. FIG. 17 shows that the range including the vortex vein in the upper right corner of the UWF-SLO image has been scanned.

[0125] The choroidal vessel stereoscopic image display field 548 displays a stereoscopic image (3D image) 548B of the choroidal vessels obtained by image processing of the OCT volume data. The stereoscopic image 548B can be rotated around three axes by a user operation. Furthermore, the choroidal vessel stereoscopic image 548B can display an image of the second choroidal vessels (a stereoscopic image of the large blood vessels) and an image of the third choroidal vessels (a stereoscopic image of the small blood vessels) extending from the dilation 548X in different display formats. In FIG. 17 , the stereoscopic image 548L of the large blood vessels extending from the dilation 548X is shown with solid lines, and the stereoscopic image 548S of the small blood vessels is shown with dotted lines. Furthermore, the stereoscopic image 548L of the large blood vessels and the stereoscopic image 548S of the small blood vessels may be displayed in different colors, or the background (fill-in) of the image may be different.

[0126] In addition, the choroidal vessel stereoscopic image display field 548 displays a superimposed boundary indicating the presence or absence of choroidal vessels, which is acquired by the vascular component presence / absence boundary acquisition process described above. Figure 17 shows an example in which a layer boundary 548P is displayed using a thick solid line. This boundary 548P indicates the presence or absence of choroidal vessels, allowing the user to identify areas containing vascular components and provide appropriate treatment for the patient. It also enables the user to perform quantitative measurements, such as blood vessel depth, with high accuracy.

[0127] The image display area 504A of the display screen 500A allows a user to view a stereoscopic image of the choroidal vessels, including thick and thin blood vessels. By scanning an area including a vortex vein, the vortex vein and the surrounding choroidal vessels, including thick and thin blood vessels, can be displayed in a stereoscopic image. Furthermore, by superimposing the boundary between the presence and absence of choroidal vessels, the user can obtain more information for diagnosis.

[0128] As described above, in this embodiment, the boundary indicating the presence or absence of choroidal blood vessels can be obtained based on OCT volume data including the choroid, and therefore it is possible to three-dimensionally visualize the boundary indicating the presence or absence of choroidal blood vessels together with the choroidal blood vessels.

[0129] Although the above describes the case where a boundary is identified using image features, the image features are not limited to those that change depending on the presence or absence of vascular components. For example, it is also possible to identify a boundary using information about choroidal blood vessels. Choroidal blood vessels gradually thin as the layer deepens. In the present disclosure, it is also possible to identify a boundary by supplementing information about the depth of a layer in the fundus and information about the diameter of the choroidal blood vessels at that depth. Specifically, it is possible to detect the thickness of the choroidal blood vessels or the degree to which the thickness of the choroidal blood vessels changes in the depth direction, and identify the boundary based on the thickness or degree and a predetermined threshold. For example, when using information about the thickness of the choroidal blood vessels and a threshold, it is possible to predetermine a threshold indicating the thickness corresponding to the boundary, and identify a layer where the thickness of the choroidal blood vessels is equal to or less than the threshold. Furthermore, when using information about the degree and the threshold, it is possible to predetermine a threshold indicating the degree of change corresponding to the boundary, and identify a layer where the degree of change in the thickness of the choroidal blood vessels is equal to or less than the threshold.

[0130] In the above embodiment, image processing (FIG. 5) is performed by the server 140, but the present disclosure is not limited to this and may be performed by the ophthalmic device 110, the viewer 150, or an additional image processing device further provided on the network 130.

[0131] In the present disclosure, each component (device, etc.) may be present in one or more instances, unless a contradiction arises.

[0132] In the above-described examples, image processing is implemented by a software configuration using a computer. However, the present disclosure is not limited to this, and at least a portion of the processing may be implemented by a hardware configuration. Furthermore, while the above description uses a CPU as an example of a general-purpose processor, the term "processor" refers to a processor in a broad sense and includes general-purpose processors (e.g., a CPU (Central Processing Unit), etc.) and dedicated processors (e.g., a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a programmable logic device, etc.). Therefore, image processing may be performed solely by a hardware configuration, or some of the image processing may be performed by a software configuration and the remaining processing may be performed by a hardware configuration.

[0133] Furthermore, the operations of the above-mentioned processors may not only be performed by a single processor, but may also be performed by multiple processors working together, or may be performed by multiple processors located in physically separate locations working together.

[0134] Furthermore, in order to cause a computer to execute the above-described processing, a program in which the above-described processing is written in computer-processable code may be stored on a storage medium such as an optical disk and distributed.

[0135] As such, the present disclosure includes both cases in which image processing is realized by a software configuration using a computer and cases in which it is not realized, and therefore includes the following techniques.

[0136] (First Technology) An image processing device comprising: an acquisition unit that acquires OCT volume data including the choroid; a generation unit that generates a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; a derivation unit that derives image features in each of the plurality of en-face images; and a determination unit that determines, based on each of the image features, a boundary between en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels.

[0137] (Second Technology) An image processing method including: an acquisition unit acquiring OCT volume data including the choroid; a generation unit generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; a derivation unit deriving image features for each of the plurality of en-face images; and a determination unit determining, based on each of the image features, a boundary between en-face images where the image features indicate a transition between the presence and absence of choroidal vessels. The image processing unit 206 is an example of the "acquisition unit," "generation unit," derivation unit, and determination unit of the present disclosure. The following technology is proposed based on the above disclosure.

[0138] (Third Technology) A computer program product for image processing, the computer program product including a computer-readable storage medium that is not itself a temporary signal, the computer-readable storage medium storing a program, the program causing a processor to perform the following steps: acquiring OCT volume data including the choroid, generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data, deriving image features for each of the plurality of en-face images, and determining, based on each of the image features, a boundary between the en-face images where the image features indicate a transition between the presence and absence of choroidal vessels. Server 140 is an example of a "computer program product" of the present disclosure.

[0139] While the technology of the present disclosure has been described above using embodiments, the image processing described above is merely an example, and the technical scope of the present disclosure is not limited to the scope described in the above embodiments. Therefore, various modifications or improvements can be made to the above embodiments, such as deleting unnecessary processes, adding new processes, or changing the order of processes, without departing from the spirit of the present disclosure, and such modifications or improvements are also included in the technical scope of the present disclosure.

[0140] The disclosure of Japanese Patent Application No. 2022-066636 is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually indicated to be incorporated by reference.

Claims

1. An image processing method performed by a processor, comprising: acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; deriving image features for each of the plurality of en-face images; identifying a boundary between en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels based on each of the image features; An image processing method comprising:

2. The step of deriving the image feature amount includes: calculating a standard deviation of brightness for each of the plurality of en-face images; The step of identifying the boundary includes: based on a standard deviation of each of the plurality of en-face images, identifying a layer corresponding to a position of the en-face images where the standard deviation converges as the boundary; The image processing method according to claim 1 .

3. The step of identifying the boundary includes determining a standard deviation threshold value that determines the boundary based on a layer corresponding to a position of the en-face image where the standard deviation converges. The image processing method according to claim 2 .

4. The step of deriving the image feature amount includes: calculating a standard deviation of brightness for each of the plurality of en-face images; The step of identifying the boundary includes: identifying a layer corresponding to a position of an en-face image showing a standard deviation of a predetermined threshold as the boundary based on a standard deviation of each of the plurality of en-face images; The image processing method according to claim 1 .

5. extracting choroidal vessels from each of the plurality of en-face images; Detecting a degree of change in thickness of the extracted choroidal vessels in a depth direction, The step of identifying the boundary includes a step of identifying the boundary based on the threshold value and the degree. The image processing method according to claim 4 .

6. The step of deriving the image feature amount includes: calculating a standard deviation of brightness for each of the plurality of en-face images; Deriving a trend of change in standard deviation among the plurality of en-face images as the image feature amount; Including, The step of determining the boundary comprises: determining, as the boundary, a layer corresponding to a position of an en-face image showing a change trend of a standard deviation of a predetermined threshold value based on a change trend of the standard deviation of each of the plurality of en-face images; The image processing method according to claim 1 .

7. The step of deriving the image feature amount includes: computing an entropy of image brightness for each of the plurality of en-face images; The step of determining the boundary comprises: determining, based on the entropy of each of the plurality of en-face images, a layer corresponding to a position of an en-face image exhibiting an entropy of a predetermined threshold as the boundary; The image processing method according to claim 1 .

8. The image processing method according to claim 1 , wherein the step of deriving image features derives the image features from only a portion of the generated en-face images.

9. The image processing method according to claim 1 , wherein the step of acquiring the OCT volume data includes scanning a region of the fundus including at least a vortex vein to obtain the OCT volume data.

10. In an image processing device having a processor, The processor, acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; deriving image features for each of the plurality of en-face images; identifying a boundary between en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels based on each of the image features; An image processing device that performs the above.

11. A program for performing image processing, The processor: acquiring OCT volume data including the choroid; generating a plurality of en-face images corresponding to a plurality of planes at different depths based on the OCT volume data; deriving image features for each of the plurality of en-face images; identifying a boundary between en-face images where the image features indicate a transition between the presence and absence of choroidal blood vessels based on each of the image features; A program that processes the following: