Image processing method, image processing program, and image processing device

Abstract

SOLUTION: An image processing method includes: a step for acquiring an ocular fundus image of an eye to be examined; a step for extracting a candidate area, which is a candidate for a non-perfusion area, from a posterior pole area, which is a posterior pole part of the ocular fundus of the ocular fundus image; a step for acquiring a tomographic image of the eye to be examined; a step for extracting a predetermined area on the ocular fundus image that imaging light when the ocular fundus image is taken does not reach on the basis of the tomographic image; and a step for excluding the predetermined area from the candidate area, and identifying the non-perfusion area from the ocular fundus image.EFFECT: A piece of information is provided when a doctor performs diagnosis, determines and confirms a progress state and a treatment effect for diabetic retinopathy, diabetic macular edema, retina vein occlusion, etc. A piece of information is also provided when a doctor performs checking on early diagnosis.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The technology of the present disclosure relates to an image processing method, an image processing program, and an image processing device. [Background technology]

[0002] Patent Document 1 discloses a method for extracting an area where an abnormality occurs by analyzing a tomographic image of the fundus. It is desirable to be able to confirm an abnormality by analyzing the fundus image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2015 / 0366452 Summary of the Invention

[0004] The first aspect of the technology of the present disclosure is acquiring a fundus image of the subject's eye; extracting a candidate region that is a non-perfusion region from a posterior pole region that is a posterior pole portion of the fundus from the fundus image; acquiring a tomographic image of the subject's eye; extracting a predetermined area on the fundus image that is not reached by imaging light when photographing the fundus image based on the tomographic image; identifying a non-perfusion region from the fundus image by excluding the predetermined region from the candidate region; An image processing method including:

[0005] A second aspect of the technology of the present disclosure is 1 is an image processing program that causes a computer to execute the image processing method of the first aspect.

[0006] A third aspect of the technology of the present disclosure is a fundus image acquisition unit for acquiring a fundus image of the subject's eye; a tomographic image acquisition unit for acquiring a tomographic image of the subject's eye; a first extraction unit that extracts a candidate region that is a non-perfusion region from a posterior pole region that is a posterior pole portion of the fundus from the fundus image; a second extraction unit that extracts a predetermined area on the fundus image that is not reached by imaging light when the fundus image is captured, based on the tomographic image; an identifying unit that identifies a non-perfusion region from the fundus image by excluding the predetermined region from the candidate region; The image processing device is provided with: [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram of an ophthalmologic system according to an embodiment. [Figure 2] 1 is a schematic diagram illustrating the overall configuration of an ophthalmologic apparatus according to an embodiment. [Figure 3] FIG. 2 is a block diagram illustrating the configuration of an electrical system of a management server according to an embodiment. [Figure 4] FIG. 2 is a block diagram of the electrical configuration of the image viewer according to the embodiment. [Figure 5] FIG. 2 is a functional block diagram of a CPU of a management server according to an embodiment. [Figure 6] FIG. 2 is a functional block diagram of a CPU of the image viewer according to the embodiment. [Figure 7] 10 is a flowchart of an image processing program executed by the management server according to the embodiment. [Figure 8] FIG. 2 is a diagram showing a fundus region in a fundus image according to the embodiment; [Figure 9] 10 is a flowchart showing the flow of processing for detecting a non-perfused region in the posterior pole of the fundus according to an embodiment. [Figure 10] An explanatory diagram of the process of detecting non-perfused areas in the posterior pole of the fundus in an embodiment, where (A) shows the primary candidate, (B) shows the primary candidate to be excluded, and (C) shows the identified non-perfused area in the posterior pole of the fundus. [Figure 11] 10 is a flowchart showing the flow of processing for detecting a non-perfusion region around the fundus according to an embodiment. [Figure 12] FIG. 2 is a diagram showing a display screen displayed on a display of an image viewer according to the embodiment. [Figure 13] FIG. 2 is a diagram showing a display screen displayed on a display of an image viewer according to the embodiment. [Figure 14] 10 is a flowchart showing a processing flow according to a first modified example. [Figure 15] 10 is a flowchart showing a processing flow according to a second modified example. [Figure 16] 13 is a flowchart showing a process flow according to an eighth modified example. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the technology of the present disclosure will be described in detail with reference to the drawings.

[0009] The configuration of an ophthalmologic system 100 will be described with reference to Fig. 1. As shown in Fig. 1, the ophthalmologic system 100 includes an ophthalmologic apparatus 110, a laser treatment apparatus 120, a management server apparatus (hereinafter referred to as "management server") 140, and an image display apparatus (hereinafter referred to as "image viewer") 150.

[0010] The ophthalmologic apparatus 110 acquires fundus images and tomographic images. The laser treatment apparatus 120 is an apparatus that assists in the treatment of a lesion in a patient's eye 12. Examples of the laser treatment apparatus 120 include a treatment instrument such as a laser photocoagulation apparatus that irradiates a lesion in the fundus of the patient with laser light to photocoagulate the irradiated area in order to suppress the progression of disease. The laser treatment apparatus 120 transmits information regarding the treatment performed on the eye 12 to the management server 140. For example, when a specific area of ​​the retina of the eye 12 is treated, the position of the specific area, the treatment time, and the treatment method are transmitted to the management server 140 as treatment information.

[0011] The management server 140 stores a plurality of fundus images obtained by photographing the fundus of a plurality of patients using the ophthalmologic apparatus 110, in association with the patient's ID. The management server 140 also detects a non-perfusion area (NPA) from a specified fundus image. The image viewer 150 displays the analysis results of the fundus image by the management server 140, such as an image of the estimated non-perfusion area (NPA).

[0012] The nonperfused area (NPA) is an area in the fundus where there is little or no blood flow due to blockage of the retinal capillary bed, etc. It is also an area where retinal ischemia occurs due to perfusion disorder.

[0013] The ophthalmologic apparatus 110 , the laser treatment apparatus 120 , the management server 140 , and the image viewer 150 are connected to one another via a network 160 .

[0014] 1 , the ophthalmology system 100 includes the laser treatment device 120, but the technology of the present disclosure is not limited to this. For example, the ophthalmology system 100 may replace the laser treatment device 120 with a measuring device such as a perimetry device that measures the visual field of a patient or an axial length measuring device that measures the axial length of the subject's eye 12. Furthermore, such a measuring device may be further added and connected to the network 130.

[0015] The management server 140 is an example of an "image processing device" of the technology of the present disclosure. The image viewer 150 is an example of an "image display device" of the technology of the present disclosure.

[0016] For the sake of convenience, the term "scanning laser ophthalmoscope" will be used hereinafter. Optical Coherence Tomography (OCT) is used in conjunction with the SLO (Scanning Lane Ophthalmoscope).

[0017] Next, the configuration of the ophthalmologic apparatus 110 will be described with reference to FIG.

[0018] When the ophthalmologic apparatus 110 is placed on a horizontal plane, the horizontal direction is defined as the "X direction," the vertical direction relative to the horizontal plane is defined as the "Y direction," and the direction connecting the center of the pupil of the anterior segment of the subject's eye 12 and the center of the eyeball is defined as the "Z direction." Therefore, the X direction, Y direction, and Z direction are perpendicular to each other.

[0019] The ophthalmologic apparatus 110 includes an imaging device 14 and a control device 16. The imaging device 14 is equipped with an SLO unit 18 and an OCT unit 20, and acquires a fundus image of the fundus of the subject's eye 12. Hereinafter, a two-dimensional fundus image acquired by the SLO unit 18 will be referred to as an SLO image. Also, a tomographic image or a front image (en-face image) of the retina created based on OCT data acquired by the OCT unit 20 will be referred to as an OCT image.

[0020] The control device 16 comprises a computer having a central processing unit (CPU) 16A, random access memory (RAM) 16B, read-only memory (ROM) 16C, and input / output (I / O) ports 16D.

[0021] The control device 16 includes an input / display device 16E connected to the CPU 16A via an I / O port 16D. The input / display device 16E has a graphic user interface that displays an image of the subject's eye 12 and receives various instructions from the user. An example of the graphic user interface is a touch panel display.

[0022] The control device 16 also includes an image processing device 17 connected to the I / O port 16D. The image processing device 17 generates an image of the subject's eye 12 based on data obtained by the photographing device 14. The control device 16 is connected to a network 130 via a communication interface (not shown).

[0023] 2, the control device 16 of the ophthalmic apparatus 110 includes the input / display device 16E, but the technology of the present disclosure is not limited to this. For example, the control device 16 of the ophthalmic apparatus 110 may not include the input / display device 16E, but may include a separate input / display device that is physically independent from the ophthalmic apparatus 110. In this case, the display device includes an image processing processor unit that operates under the control of the display control unit 204 of the CPU 16A of the control device 16. The image processing processor unit may display an SLO image or the like based on an image signal instructed to be output by the display control unit 204.

[0024] The imaging device 14 operates under the control of an imaging control unit 202 of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The imaging optical system 19 includes a first optical scanner 22, a second optical scanner 24, and a wide-angle optical system 30.

[0025] The first optical scanner 22 performs two-dimensional scanning in the X and Y directions with the light emitted from the SLO unit 18. The second optical scanner 24 performs two-dimensional scanning in the X and Y directions with the light emitted from the OCT unit 20. The first optical scanner 22 and the second optical scanner The optical element 24 may be any optical element capable of deflecting a light beam, such as a polygon mirror or a galvanometer mirror, or a combination thereof.

[0026] The wide-angle optical system 30 includes an objective optical system (not shown in FIG. 2) having a common optical system 28 , and a combining unit 26 that combines the light from the SLO unit 18 and the light from the OCT unit 20 .

[0027] The objective optical system of the common optical system 28 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a catadioptric system combining concave mirrors and lenses. By using a wide-angle optical system using an elliptical mirror or a wide-angle lens, it becomes possible to photograph not only the central part of the fundus (the posterior pole of the fundus) but also the retina in the peripheral part of the fundus.

[0028] When a system including an elliptical mirror is used, the system using the elliptical mirror described in International Publication WO2016 / 103484 or International Publication WO2016 / 103489 may be used. The disclosures of International Publication WO2016 / 103484 and International Publication WO2016 / 103489 are each incorporated herein by reference in their entirety.

[0029] The wide-angle optical system 30 enables observation of the fundus over a wide field of view (FOV) 12A. The FOV 12A indicates the range that can be photographed by the imaging device 14. The FOV 12A can be expressed as a field of view. In this embodiment, the field of view can be defined by an internal illumination angle and an external illumination angle. The external illumination angle is the illumination angle of the light beam irradiated from the ophthalmic device 110 to the subject's eye 12, determined with the pupil 27 as the reference. The internal illumination angle is the illumination angle of the light beam irradiated to the fundus F, determined with the eyeball center O as the reference. The external illumination angle and the internal illumination angle correspond to each other. For example, if the external illumination angle is 120 degrees, the internal illumination angle corresponds to approximately 160 degrees. In this embodiment, the internal illumination angle is 200 degrees.

[0030] Here, an SLO fundus image captured at an internal illumination angle of 160 degrees or more is referred to as a UWF-SLO fundus image. UWF stands for Ultra Wide Field.

[0031] The SLO system is realized by a control device 16, an SLO unit 18, and an imaging optical system 19 shown in Fig. 2. The SLO system includes a wide-angle optical system 30, and therefore enables fundus imaging with a wide FOV 12A.

[0032] The SLO unit 18 includes a B (blue light) light source 40, a G (green light) light source 42, an R (red light) light source 44, and an IR (infrared (e.g., near-infrared) light) light source 46, as well as optical systems 48, 50, 52, 54, and 56 that reflect or transmit the light from the light sources 40, 42, 44, and 46 and guide them into a single optical path. The optical systems 48, 50, and 56 are mirrors, and the optical systems 52 and 54 are beam splitters. The B light is reflected by the optical system 48, passes through the optical system 50, and is reflected by the optical system 54; the G light is reflected by the optical systems 50 and 54; the R light is transmitted through the optical systems 52 and 54; and the IR light is reflected by the optical systems 52 and 56 and is each guided into a single optical path.

[0033] The SLO unit 18 is configured to be switchable between a light source that emits laser light of different wavelengths, or a combination of light sources that emit light, such as a mode that emits R light and G light and a mode that emits infrared light. In the example shown in FIG. 2, three light sources are provided: a light source 42 for G light, a light source 44 for R light, and a light source 46 for IR light; however, the technology of the present disclosure is not limited to this. For example, the SLO unit 18 may further include a light source for B light (blue light) and a light source for white light, and may emit light in various modes, such as a mode that emits G light, R light, and B light, or a mode that emits only white light.

[0034] Light incident on the photographing optical system 19 from the SLO unit 18 is scanned in the X and Y directions by the first optical scanner 22. The scanning light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus. The light reflected by the fundus passes through the wide-angle optical system 30 and the first optical scanner 22 and is incident on the SLO unit 18.

[0035] The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits all light except B light from the posterior segment (fundus) of the subject's eye 12, and a beam splitter 58 that reflects G light and transmits all light except G light from the light that has passed through the beam splitter 64. The SLO unit 18 includes a beam splitter 60 that reflects R light and transmits all light except R light from the light that has passed through the beam splitter 58. The SLO unit 18 includes a beam splitter 62 that reflects IR light from the light that has passed through the beam splitter 60. The SLO unit 18 includes a B light detecting element 70 that detects B light reflected by the beam splitter 64, a G light detecting element 72 that detects G light reflected by the beam splitter 58, an R light detecting element 74 that detects R light reflected by the beam splitter 60, and an IR light detecting element 76 that detects IR light reflected by the beam splitter 62.

[0036] Light (reflected light reflected by the fundus) incident on the SLO unit 18 via the wide-angle optical system 30 and the first optical scanner 22 is reflected by the beam splitter 64 and received by the B light detection element 70 in the case of B light, and is reflected by the beam splitter 58 and received by the G light detection element 72 in the case of G light. The incident light is transmitted through the beam splitter 58 in the case of R light, reflected by the beam splitter 60, and received by the R light detection element 74. The incident light is transmitted through the beam splitters 58 and 60, reflected by the beam splitter 62, and received by the IR light detection element 76 in the case of IR light. The image processing device 17, which operates under the control of the CPU 16A, generates a UWF-SLO image using signals detected by the B light detection element 70, the G light detection element 72, the R light detection element 74, and the IR light detection element 76.

[0037] UWF-SLO images include UWF-SLO images (G-color fundus images) obtained by photographing the fundus in G color, UWF-SLO images (R-color fundus images) obtained by photographing the fundus in R color, UWF-SLO images (B-color fundus images) obtained by photographing the fundus in B color, and UWF-SLO images (IR fundus images) obtained by photographing the fundus in IR color.

[0038] The control device 16 also controls the light sources 40, 42, 44 to emit light simultaneously. By simultaneously photographing the fundus of the subject's eye 12 with B light, G light, and R light, a G-color fundus image, a R-color fundus image, and a B-color fundus image, each of which corresponds to each other, are obtained. An RGB color fundus image is obtained from the G-color fundus image, the R-color fundus image, and the B-color fundus image. The control device 16 controls the light sources 42, 44 to emit light simultaneously, and by simultaneously photographing the fundus of the subject's eye 12 with G light and R light, a G-color fundus image and a R-color fundus image, each of which corresponds to each other, are obtained. An RG color fundus image is obtained from the G-color fundus image and the R-color fundus image.

[0039] UWF-SLO images also include UWF-SLO fluorescence images, which are photographed using a contrast agent.

[0040] The image data of the B color fundus image, the G color fundus image, the R color fundus image, the IR fundus image, the RGB color fundus image, the RG color fundus image, and the UWF-SLO fluorescence image are sent from the ophthalmic device 110 to the management server 140 via a communication IF not shown.

[0041] The OCT system is realized by the control device 16, the OCT unit 20, and the imaging optical system 19 shown in FIG. 2. Since the OCT system includes the wide-angle optical system 30, the above-mentioned S Similar to capturing an LO fundus image, fundus imaging is possible with a wide FOV 12A. The OCT unit 20 includes a light source 20A, a sensor (detecting element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

[0042] Light emitted from the light source 20A is branched by the first optical coupler 20C. One of the branched beams is collimated by the collimating lens 20E as measurement light and then enters the imaging optical system 19. The measurement light is scanned in the X and Y directions by the second optical scanner 24. The scanning light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus. The measurement light reflected by the fundus passes through the wide-angle optical system 30 and the second optical scanner 24 and enters the OCT unit 20, and then enters the second optical coupler 20F via the collimating lens 20E and the first optical coupler 20C.

[0043] The other light beam emitted from the light source 20A and branched by the first optical coupler 20C is incident as reference light on the reference optical system 20D, passes through the reference optical system 20D, and then enters the second optical coupler 20F.

[0044] The light beams incident on the second optical coupler 20F, i.e., the measurement light beam reflected from the fundus and the reference light beam, interfere with each other to generate interference light. The interference light beam is received by the sensor 20B. The image processing device 17, which operates under the control of the image processing controller 206, generates OCT images such as tomographic images and en-face images based on the OCT data detected by the sensor 20B.

[0045] The OCT unit 20 can obtain OCT data of OCT images, which are tomographic images of the subject's eye 12. As examples of OCT images, a one-dimensional OCT image is an A-scan image obtained by performing a so-called A-scan with the ophthalmic apparatus 110, a two-dimensional OCT image is a B-scan image obtained by performing a so-called B-scan with the ophthalmic apparatus 110, and a three-dimensional OCT image is a C-scan image obtained by performing a so-called C-scan with the ophthalmic apparatus 110.

[0046] Here, an OCT fundus image obtained by capturing an image at an internal illumination angle of 160 degrees or more is referred to as a UWF-OCT image.

[0047] Image data of the UWF-OCT image is sent from the ophthalmologic apparatus 110 to the management server 140 via a communication IF (not shown) and stored in the storage device 254 .

[0048] In this embodiment, the light source 20A is exemplified as a wavelength-swept type SS-OCT (Swept-Source OCT), but it may also be a wavelength-swept type SD-OCT (Spectral-Domain OCT). The OCT system may be of various types, such as CT (Time-Domain OCT) or TD-OCT (Time-Domain OCT).

[0049] Next, the configuration of the electrical system of the management server 140 will be described with reference to FIG. 3. As shown in FIG. 3, the management server 140 includes a computer main unit 252. The computer main unit 252 has a CPU 262, a RAM 266, a ROM 264, and an input / output (I / O) port 268. The input / output (I / O) port 268 is connected to a storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I / F) 258. The storage device 254 is configured, for example, with a non-volatile memory. The input / output (I / O) port 268 is connected to the network 130 via the communication interface (I / F) 258. Therefore, the management server 140 can communicate with the ophthalmic device 110, the laser treatment device 120, and the image viewer 150.

[0050] The management server 140 stores the data received from the ophthalmic apparatus 110 and the laser treatment apparatus 120 in the storage device 254 .

[0051] Next, the configuration of the electrical system of the image viewer 150 will be described with reference to FIG. 4. As shown in FIG. 4, the image viewer 150 includes a computer main unit 152. The computer main unit 152 has a CPU 162, a RAM 166, a ROM 164, and an input / output (I / O) port 168. The input / output (I / O) port 168 is connected to the storage device 154, the display 156, the mouse 155M, the keyboard 155K, and a communication interface (I / F) 158. The storage device 154 is configured, for example, with a non-volatile memory. The input / output (I / O) port 168 is connected to the network 130 via the communication interface (I / F) 158. Therefore, the image viewer 150 can communicate with the ophthalmologic apparatus 110 and the management server 140.

[0052] Next, various functions realized by the CPU 262 of the management server 140 executing the image processing program will be described with reference to Fig. 5. As shown in Fig. 5, the image processing program has a display control function, an image processing control function, and a processing function. When the CPU 262 executes the image processing program having these functions, the CPU 262 functions as a display control unit 204, an image processing control unit 206, and a processing unit 208, as shown in Fig. 5.

[0053] Next, various functions realized by the CPU 162 of the image viewer 150 executing the image processing program will be described with reference to Fig. 6. As shown in Fig. 6, the image processing program has a display control function, an image processing control function, and a processing function. When the CPU 162 executes the image processing program having these functions, the CPU 162 functions as the display control unit 104, the image processing control unit 106, and the processing unit 108, as shown in Fig. 6.

[0054] The image processing control unit 206 is an example of a "fundus image acquisition unit," a "first non-perfusion region extraction unit," and a "second non-perfusion region extraction unit" of the technology of the present disclosure. The image processing control unit 106 is an example of the "acquisition unit" of the technology of the present disclosure, and the display 156 is an example of the "display unit" of the technology of the present disclosure.

[0055] Next, the image processing by the management server 140 will be described in detail with reference to Fig. 7. The image processing shown in the flowchart of Fig. 7 is realized by the CPU 262 of the management server 140 executing an image processing program. The image processing shown in Fig. 7 is an example of an image processing method of the technology of the present disclosure. Also, the display processing for displaying an image obtained by the image processing shown in Fig. 7 is an example of an image display method of the technology of the present disclosure.

[0056] Here, it is assumed that UWF-SLO image processing and UWF-OCT image processing are executed, and the data obtained by the execution of the processing (for example, image data of the UWF-SLO image and image data of the UWF-OCT image) is stored in the memory device 254.

[0057] In this embodiment, as an example of a UWF-SLO image for detecting NPA, a UWF-SLO image obtained by fluorescein angiography (FA) is used. The case where an FA image (hereinafter referred to as an FA image) is used will be described. The FA image is suitable for detecting non-perfusion areas because it captures retinal capillaries with high resolution. Image data of this FA image is sent from the ophthalmologic apparatus 110 to the management server 140 via a communication IF (not shown) and stored in the storage device 254. In addition, fluorescent fundus imaging using indocyanine green (ICG) as a contrast agent UWF-SLO images taken using indocyanine green angiography (IA) (hereinafter referred to as IA images) can also be used. In FA images and IA images, blood vessels in the fundus appear white.

[0058] 7, the image processing control unit 206 functioning in the management server 140 acquires an FA image from the storage device 254. The FA image is captured by the ophthalmologic apparatus 110 and stored in the storage device 254.

[0059] In the next step S104, the image processing control unit 206 detects the fundus region of the subject's eye 12 from the acquired FA image. In this step S104, peripheral parts of the subject's eye 12, such as the patient's eyelashes and eyelids, and areas in which elements constituting the ophthalmologic apparatus 110 are reflected are removed, and the area after removal is extracted as the fundus region, thereby detecting the fundus region of the subject's eye 12. This fundus region extraction process can be realized by performing image processing that combines well-known processes such as binarization and morphology processing. Figure 8 shows an image from which the fundus region has been extracted after the process of step S104 has been performed. The dashed line in Figure 8 is a line that schematically indicates the edge of the fundus region, and there is no dashed line in the image after the process of step S104 has been performed.

[0060] Next, the image processing control unit 206 detects a non-perfused area (NPA) from the detected fundus area.

[0061] In this embodiment, due to differences in the distribution of retinal blood vessels between the posterior pole of the fundus and its surrounding peripheral portion, it is preferable to perform image processing according to the location of the fundus region of the subject's eye 12. Therefore, in this embodiment, image processing is performed to classify and visualize non-perfusion areas (NPAs) in the fundus region into non-perfusion areas in the posterior pole of the fundus (hereinafter referred to as posterior pole NPAs) and non-perfusion areas in the peripheral portion of the fundus (hereinafter referred to as UWF-NPAs). The posterior pole NPAs are searched for in the fundus region where retinal blood vessels are dense, while the UWF-NPAs are searched for in the peripheral portion of the fundus. Therefore, in this embodiment, processing optimal for detecting posterior pole NPAs (processing shown in FIG. 9 or FIG. 16, which will be described later) and processing optimal for detecting UWF-NPAs (processing shown in FIG. 11, which will be described later) are performed on the FA image.

[0062] Specifically, in step S106, the image processing control unit 206 detects posterior pole NPA in the fundus region of the subject's eye 12, and in step S108, performs image processing of posterior pole NPA to identify erroneously detected posterior pole NPA from among the detected posterior pole NPA. Simultaneously, in step S110, the image processing control unit 206 performs UWF-NPA image processing to detect UWF-NPA in the fundus region of the subject's eye 12. Here, erroneously detected posterior pole NPA refers to a region that is detected as a posterior pole NPA but is unlikely to be a posterior pole NPA.

[0063] The posterior pole of the fundus of the subject eye 12 is an example of the "first region" of the technology of the present disclosure, and the posterior pole NPA is an example of the "first non-perfusion region" of the technology of the present disclosure. The fundus periphery, which is the periphery of the posterior pole of the fundus of the subject eye 12, is an example of the "second region" of the technology of the present disclosure, and the UWF-NPA is an example of the "second non-perfusion region" of the technology of the present disclosure. The erroneously detected posterior pole NPA, which is an area unlikely to be a posterior pole NPA, is an example of the "third non-perfusion region" of the technology of the present disclosure.

[0064] Note that step S106, step S108, and step S110 may be executed either first or simultaneously. After completing steps S106, step S108, and step S110, the image processing control unit 206 executes image processing to generate a display screen in step S112. As will be described in detail later, the generated display screen displays the outline of the posterior pole NPA so that the position of the posterior pole NPA can be easily recognized in the FA image, and the outline of the UWF-NPA so that the position of the UWF-NPA can be easily recognized, superimposed on the FA image. In step 114, the processing unit 208 transmits the generated display image of the display screen to the image viewer 150.

[0065] Next, image processing of the posterior pole NPA will be described with reference to FIGS. 9, in step S304, the image processing control unit 206 performs enhancement image processing on the acquired FA image to enhance the blood vessel portion. This is processing to highlight blood vessels including capillaries, and is processing to accurately estimate the posterior pole NPA.

[0066] The enhancement image processing can be various methods such as enhancement processing using image histograms, such as histogram averaging or adaptive histogram equalization (CLAHE (Contrast Limited Adaptive Histogram Equalization)), contrast conversion processing based on gradation conversion, frequency enhancement processing of specific frequency bands such as unsharp masking, deconvolution processing such as Wiener filtering, and morphology processing that enhances the shape of blood vessels, but histogram averaging or adaptive histogram equalization is preferred. Blood vessels are enhanced by enhancement image processing.

[0067] Next, the image processing control unit 206 estimates multiple posterior pole NPAs from the FA image in which blood vessels are enhanced. Specifically, in step S306, the image processing control unit 206 selects primary candidates for the posterior pole NPAs. More specifically, multiple pixels that are darker than a first darkness are extracted from the FA image in which blood vessels are enhanced, and one or more areas in which the dark pixels that are darker than the first darkness are continuous and have an area equal to or greater than a predetermined area are selected as primary candidates for the posterior pole NPAs.

[0068] Here, a pixel that is darker than a first darkness refers to a pixel whose pixel value is equal to or less than a first predetermined value. Note that, for example, a luminance value representing lightness may be used as the pixel value, but a value representing at least one of saturation and hue may be used instead of or in addition to the luminance value.

[0069] Next, the image processing control unit 206 executes the image processing in steps S308 and S310.

[0070] In step S308, the image processing control unit 206 selects only dark candidates from one or more primary candidates for the posterior pole NPA based on the average pixel value within each candidate region. Specifically, the image processing control unit 206 calculates the average pixel value within each of the one or more primary candidates for the posterior pole NPA, and selects one or more candidates for which the calculated average value is smaller than a second predetermined value as dark regions. The second predetermined value is a value smaller than the first predetermined value by a predetermined amount. In other words, from the primary candidates for the first darkness, only candidates that are dark regions darker than a second darkness that is darker than the first darkness (candidates with a predetermined average pixel value or less) are extracted and designated as the first secondary candidates.

[0071] In step S310, the image processing control unit 206 narrows down the multiple primary candidates for the posterior pole NPA to only those regions along the blood vessels. Specifically, the image processing control unit 206 first (1) extracts blood vessels. The blood vessels are extracted using methods such as morphological processing or binarization based on pixel values. The extracted region is called a vascular region. Next, the image processing control unit 206 (2) calculates the distance between the vascular region and the periphery of one or more primary candidates or each region group of the candidate group for the posterior pole NPA using a method such as distance transformation, and selects regions where the distance is within a certain range. Here, being within a certain range means a first range that is greater than a first predetermined distance and less than a second predetermined distance that is greater than the first predetermined distance (i.e., along a running blood vessel).

[0072] In this way, in step S310, the image processing control unit 206 extracts, from the primary candidates, an area whose distance to the blood vessel is equal to or less than the first distance, as a second secondary candidate. Note that the second secondary candidate may be an area within a certain range from the end of the blood vessel.

[0073] Either step S308 or step S310 may be executed first, or they may be executed simultaneously. After the processes of step S308 and step S310 are completed, the image processing control unit 206 executes image processing shown in step S312.

[0074] In step S312, the image processing control unit 206 performs integration processing to integrate the first secondary candidate and the second secondary candidate. Specifically, an area that is both the first secondary candidate (plurality of dark areas) and the second secondary candidate (plurality of areas along blood vessels) is extracted and identified as the posterior pole NPA.

[0075] Fig. 10 is a simplified representation of the processing results from steps S306 to S312, and is a schematic diagram showing an enlarged portion of the FA image. Fig. 10(A) shows a blood vessel 400 and four primary candidates 406, 408, 410, and 412 for the posterior pole NPA after the processing of step S306. For convenience of explanation, Fig. 10 shows the blood vessel 400 with a black line. Fig. 10(B) shows primary candidates 406A, 408A, and 410A with solid lines as examples of primary candidates estimated by the processing of steps S308 and S310. Fig. 10(B) also shows primary candidate 412 with a dotted line as an example of a primary candidate to be excluded. Figure 10(C) shows the estimated candidates 406NPA, 408NPA, and 410NPA, which were narrowed down from the primary candidates to secondary candidates that are common to the first and second secondary candidates, as identified posterior pole NPAs.

[0076] 9, the image processing control unit 206 extracts the contour of the posterior pole NPA from the FA image. The contour of the posterior pole NPA is an image to be superimposed on the FA image so that the position of the posterior pole NPA can be easily recognized on the FA image. In this manner, image processing for the posterior pole NPA is performed.

[0077] Next, the process of identifying erroneous detections related to the posterior pole NPA in step S108 shown in FIG. 7 will be described.

[0078] The region detected as a posterior pole NPA as described above (step S106) may be a region that is not a posterior pole NPA (hereinafter referred to as a non-NPA). Examples of erroneously detected non-NPA include photocoagulation spots, soft exudates, and spots caused by cataracts that prevent imaging light from reaching the retina.

[0079] Therefore, in step S108 of Fig. 7, the image processing control unit 206 identifies whether the posterior pole NPA detected in step S106 is a posterior pole NPA that has been erroneously detected. Specifically, the following discrimination method is used to identify whether the detected posterior pole NPA is a posterior pole NPA that has been erroneously detected.

[0080] In the first discrimination method, image data of known photocoagulation spots is used to perform image processing using an image filter that detects photocoagulation spots, and posterior pole NPAs are discriminated from photocoagulation spots. If a photocoagulation spot is discriminated, the posterior pole NPA discriminated as a photocoagulation spot is classified as a non-NPA. Note that the image filter used in the first discrimination method is not limited to detecting photocoagulation spots, and it is also possible to use an image filter that detects areas such as soft exudates and spots caused by cataracts that prevent the imaging light from reaching the retina.

[0081] The second discrimination method uses artificial intelligence that has machine-learned image data of known photocoagulation spots to distinguish between posterior pole NPA and photocoagulation spots, and if a photocoagulation spot is identified, the posterior pole NPA identified as a photocoagulation spot is classified as a non-NPA. Note that the second discrimination method is not limited to machine-learning image data of photocoagulation spots, but may also use machine-learned image data showing areas such as soft exudates and spots caused by cataracts that prevent imaging light from reaching the retina.

[0082] The third discrimination method involves acquiring data indicative of non-NPAs from other imaging equipment and using the acquired data to distinguish non-NPAs from detected posterior pole NPAs. For example, in the case of soft exudates, the position of the soft exudates identified in the OCT B-scan image is identified. Posterior pole NPA at the identified position is excluded as a soft exudate. Furthermore, in the case of photocoagulation plaque, the position of laser irradiation by the laser treatment device 120 is identified. Posterior pole NPA at the identified laser irradiation position is excluded as a photocoagulation plaque. The data indicating non-NPA from other imaging equipment used in these third discrimination methods is an example of data for identifying non-NPA, and is not limited to this, and any data for identifying non-NPA may be used.

[0083] In this way, by identifying whether a detected posterior pole NPA is an erroneously detected posterior pole NPA, it is possible to improve the detection accuracy of posterior pole NPA. Furthermore, by eliminating erroneously detected posterior pole NPA, it becomes possible to extract only posterior pole NPA, which is a non-perfusion region that requires photocoagulation treatment.

[0084] Next, image processing of UWF-NPA will be described with reference to Fig. 11. UWF-NPA is an area located in the peripheral part of the fundus where no retinal blood vessels exist. 11, in step S400, the image processing control unit 206 performs enhancement image processing on the acquired FA image to enhance the blood vessel portion. This is processing to highlight blood vessels including capillaries, and is processing to accurately estimate UWF-NPA.

[0085] In step S402, the image processing control unit 206 performs blood vessel binarization processing to binarize the blood vessel-enhanced image in which the blood vessel portions have been emphasized. In step S404, the image processing control unit 206 performs distance image creation processing to create a distance image using the blood vessel binarized image in which the blood vessel portions have been binarized. The distance image is an image in which the brightness increases as the distance from the edge of the line segments (corresponding to the blood vessel portions) in the binarized image increases.

[0086] In step S406, the image processing control unit 206 performs a binarization process to binarize the distance image. In this binarization process, the distance image is binarized, and the area around the fundus, which is the area around the posterior pole of the fundus, is converted into a white area in the binarized distance image (although a white area may remain in part of the posterior pole as well).

[0087] In step S408, the image processing control unit 206 performs a process to remove regions with a predetermined number of pixels or less. This process converts white regions with a predetermined number of pixels or less from the binarized distance image, which is a binarized distance image, into black regions. Specifically, multiple white pixels are extracted from the binarized distance image, and one or more regions with an area of ​​continuous white pixels less than a predetermined area are converted into regions with continuous black pixels. The binarized distance image includes a blood vessel portion in the posterior pole of the fundus, which is a white region, and a region around the fundus, which is the area surrounding the posterior pole of the fundus. By predetermining the number of pixels corresponding to the size, e.g., width, of the blood vessel portion in the posterior pole of the fundus as a constant number of pixels, the blood vessel portion in the posterior pole of the fundus in the binarized distance image can be converted into a black region. As a result, the white region around the fundus remains in the binarized distance image.

[0088] In step S410, the image processing control unit 206 performs the binary image processing after removing white areas with a certain number of pixels or less. The contour of the UWF-NPA is extracted by extracting the contour of the white area remaining in the FA image. The contour of the UWF-NPA is an image that is superimposed on the FA image to make it easy to recognize the position of the UWF-NPA on the FA image. In this manner, image processing for UWF-NPA is performed.

[0089] Next, a method for displaying the non-perfused area (NPA) will be described in more detail with reference to a screen 500 of the display 156 of the image viewer 150 shown in FIGS. 12 and 13 are examples of images relating to the fundus image of the subject's eye 12 displayed by executing the "image display method" of the technique of the present disclosure.

[0090] When the operator inputs the ID of the patient whose fundus image is to be observed, the image viewer 150 commands the management server 140 to output patient information. The management server 140 reads out the patient information corresponding to the patient ID. The management server 140 then reads out the FA image and performs the image processing described with reference to FIG. 7 . The management server 140 then stores the contour image of the fundus region, the contour image of the posterior pole NPA, the contour image of the UWF-NPA, and the contour image of the erroneously detected posterior pole NPA corresponding to the obtained FA image in the storage device 154 of the management server. The management server 140 also generates an image in which the image-processed contour image of the fundus region, the contour image of the posterior pole NPA, the contour image of the UWF-NPA, and the contour image of the erroneously detected posterior pole NPA are superimposed on the FA image, generates a display screen for the image viewer 150, and transmits the image data of the display screen to the image viewer 150. The display control unit 104 of the image viewer 150 controls the display of an image based on the image data for the display screen from the management server 140 on the display 156.

[0091] When an operator instructs the display format of the outline image (described later), the image viewer 150 generates a display screen according to the instruction and issues a command to the management server 140 to output image data of the display screen. The processing unit 108 of the image viewer 150 receives image data of the display screen corresponding to the instructed display format of the outline image, and the display control unit 104 controls the display of the image on the display 156.

[0092] 12, the display content of the non-perfused area (NPA) on the display 156 of the image viewer 150 is shown on a screen 500. The screen 500 has a patient information display field 502, a fundus image display field 504, and an option instruction display field 506.

[0093] The patient information display field 502 has a patient ID display field 502A, a patient name display field 502B, an age display field 502C, a target eyeball display field 502D, a photography date and time display field 502E, an axial length display field 502F, and a visual acuity display field 502G. The image viewer 150 acquires the patient ID, patient name, age, target eyeball, photography date and time, axial length, and visual acuity stored in the management server 140. The image viewer 150 displays the acquired patient ID, patient name, age, target eyeball, photography date and time, axial length, and visual acuity in the patient ID display field 502A, patient name display field 502B, target eyeball display field 502D, photography date and time display field 502E, axial length display field 502F, and visual acuity display field 502G, respectively.

[0094] The image viewer 150 displays an FA image of the patient in a fundus image display field 504 on the screen 500. In addition, the image viewer 150 displays a contour image of a non-perfused area (NPA) superimposed on the FA image to facilitate observation and diagnosis of the fundus of the subject's eye 12.

[0095] 12 shows a display form in which a contour image 504A of the fundus region in the FA image, contour images 504B and 504C of the posterior pole NPA, and a contour image 504D of the UWF-NPA are displayed. In the example shown in FIG. 13, a display form in which a contour image 504A of the fundus region in the FA image is not displayed, and contour images 504B and 504C of the posterior pole NPA, a contour image 504D of the UWF-NPA, and a posterior pole NPA 504E that has been erroneously detected are displayed. do.

[0096] An operator of the image viewer 150 (e.g., a doctor) may wish to change the display format of the image displayed in the fundus image display field 504. Therefore, the option instruction display field 506 has instruction buttons for selecting and instructing the display format of the image displayed in the fundus image display field 504, and a display field for displaying the instruction results.

[0097] 12, the option instruction display field 506 has an instruction button 506A for instructing, in a pull-down format, whether or not to display an outline image of the fundus region in the FA image and the display color, and a display field 506B for displaying the instruction result. The display form of the outline image of the fundus region displayed in the fundus image display field 504 is changed by an operator (e.g., a doctor) of the image viewer 150 selecting the instruction button 506A. For example, the operator operates an input means such as the mouse 155M to specify whether or not to display the outline image of the fundus region and the display color. When the whether or not to display the outline image of the fundus region and the display color are specified, the image viewer 150 displays the outline image of the fundus region with the specified whether or not to display it and the specified display color.

[0098] The option instruction display field 506 also has an instruction button 506C for specifying, in a pull-down format, whether or not to display the contour image of the posterior pole NPA and the display color, and a display field 506D for displaying the instruction result. The display format of the contour image of the posterior pole NPA displayed in the fundus image display field 504 is changed when the operator of the image viewer 150 selects or instructs the instruction button 506C. For example, the operator operates an input means such as the mouse 155M to specify whether or not to display the contour image of the posterior pole NPA and the display color. When the whether or not to display the contour image of the posterior pole NPA and the display color are specified, the image viewer 150 displays the contour image of the posterior pole NPA with the specified whether or not to display it and the specified display color.

[0099] The option instruction display field 506 also has an instruction button 506E for specifying, in a pull-down format, whether or not to display the outline image of the UWF-NPA and the display color, and a display field 506F for displaying the result of the instruction. The display format of the outline image of the UWF-NPA displayed in the fundus image display field 504 is changed by the operator of the image viewer 150 selecting the instruction button 506E. For example, the operator operates an input means such as the mouse 155M to specify whether or not to display the outline image of the UWF-NPA and the display color. When the whether or not to display the outline image of the UWF-NPA and the display color are specified, the image viewer 150 displays the outline image of the UWF-NPA with the specified whether or not to display it and the specified display color.

[0100] The option instruction display field 506 also has an instruction button 506G for specifying, in a pull-down format, whether or not to display and the display color of a contour image of a erroneously detected posterior pole NPA, which is a region that has been detected as a posterior pole NPA but is unlikely to be a posterior pole NPA, and a display field 506H for displaying the instruction result. The display format of the contour image of the erroneously detected posterior pole NPA displayed in the fundus image display field 504 is changed by an operator of the image viewer 150 selecting the instruction button 506G. For example, the operator operates an input device such as the mouse 155M to specify whether or not to display and the display color of the contour image of the erroneously detected posterior pole NPA. When the whether or not to display and the display color of the contour image of the erroneously detected posterior pole NPA are specified, the image viewer 150 displays the contour image of the erroneously detected posterior pole NPA with the specified whether or not to display and the specified display color.

[0101] The image viewer 150 acquires the FA image, the contour image of the fundus region, the contour image of the posterior pole NPA, the contour image of the UWF-NPA, and the contour image of the erroneously detected posterior pole NPA from the management server 140. Then, the contour image according to the operator's instruction is superimposed on the FA image and displayed. Note that the FA image may include a non-perfusion area (NPA) or the like. The display image on which any contour image is superimposed and displayed may be generated by either the management server 140 or the image viewer 150 .

[0102] 12 and 13. For example, the thickness of the lines of the contour image and the type of line, such as a dotted line or a solid line, may be changed. This allows the operator to display the contour image in different colors, change the type of line of the contour image, or change both the color and the type of line of the contour image so that the operator can easily recognize the differences between the fundus region, posterior pole NPA, UWF-NPA, and erroneously detected posterior pole NPA.

[0103] The posterior pole NPA, which is displayed in the fundus image display area 504 on the screen 500 by the image viewer 150 and identified by the contour image of the posterior pole NPA, is one of the pieces of information that doctors use to diagnose, determine the progression of, and confirm the effectiveness of treatment for diabetic retinopathy, diabetic macular edema, retinal vein occlusion, etc. Furthermore, UWF-NPA identified by the contour image of UWF-NPA can be one of the pieces of information that doctors can use to confirm early diagnoses, such as "preproliferative diabetic retinopathy," "proliferative diabetic retinopathy," "branch retinal vein occlusion," "Coats disease," and "non-infectious uveitis," which develop from the peripheral part of the fundus or for which observation of the fundus is useful for confirming the diagnosis. Furthermore, detecting posterior pole NPA can be useful in determining the effectiveness of treatments such as photocoagulation and whether additional surgery is necessary. Furthermore, visualization of UWF-NPA can be useful for early diagnosis of conditions such as diabetic retinopathy, understanding the progression of the disease, and quantitatively understanding the effects of drug treatments such as anti-VEGF (Vascular Endothelial Growth Factor) drugs (anti-angiogenic drugs) and blood pressure control.

[0104] Next, various modifications of the technique of the present disclosure will be described.

[0105] <First Modification> In the above embodiment, the image processing control unit 206 executes image processing for the posterior pole NPA (steps S106 and S108) and image processing for the UWF-NPA (step S110), but the technology of the present disclosure is not limited to this. For example, as shown in Fig. 14, the image processing control unit 206 may execute only image processing for the UWF-NPA. The processes shown in Fig. 14 are the same as those described above, and therefore detailed description thereof will be omitted.

[0106] The first modified example includes the following technical content.

[0107] (1) An image processing method, acquiring a fundus image; extracting an unperfused area (UWF-NPA) in an area including a periphery of the fundus area from the fundus image; An image processing method comprising:

[0108] (2) An image display method, acquiring information about a fundus image and an unperfused area (UWF-NPA) in an area including the periphery of the fundus image extracted from the fundus image; a step of superimposing and displaying the non-perfusion region on the fundus image; An image display method including:

[0109] (3) An image processing program, An image processing program that causes a computer to execute the image processing method described in (1).

[0110] (4) An image display program, An image display program that causes a computer to execute the image display method described in (2).

[0111] (5) An image processing device, a fundus image acquisition unit that acquires a fundus image; a non-perfusion area extraction unit that extracts a non-perfusion area (UWF-NPA) in an area including a periphery of the fundus area from the fundus image; An image processing device comprising:

[0112] (6) An image display device, an acquisition unit that acquires a fundus image and information about an unperfused area (UWF-NPA) in an area including the periphery of the fundus area extracted from the fundus image; a display unit that displays the non-perfusion region superimposed on the fundus image; An image display device comprising:

[0113] <Second Modification> In the above embodiment, the image processing control unit 206 performs image processing for the posterior pole NPA and image processing for the UWF-NPA, and in the first modified example, the image processing control unit 206 performs only image processing for the UWF-NPA, but the technology of the present disclosure is not limited to this. For example, as shown in Fig. 15, the image processing control unit 206 may perform only image processing for the posterior pole NPA. Since each process shown in Fig. 15 is the same as that described above, detailed description will be omitted.

[0114] The second modified example includes the following technical content.

[0115] (7) An image processing method, acquiring a fundus image; Extracting a non-perfused area (NPA) from the fundus image in a region including the center of the fundus; An image processing method comprising:

[0116] (8) The image processing method according to (7), extracting regions that may be falsely detected from the extracted non-perfusion regions; excluding the region that may be a false positive from the extracted non-perfusion region; The image processing method further comprises:

[0117] (9) An image display method, obtaining a fundus image and information about a non-perfused area (NPA) in a region including a center of the fundus region extracted from the fundus image; a step of superimposing and displaying the non-perfusion region on the fundus image; An image display method including:

[0118] (10) An image processing program, An image processing program that causes a computer to execute the image processing method according to (7) or (8).

[0119] (11) An image display program, An image display program that causes a computer to execute the image display method described in (9).

[0120] (12) An image processing device, a fundus image acquisition unit that acquires a fundus image; a non-perfusion area extracting unit that extracts a non-perfusion area (NPA) in a region including a posterior pole of the fundus from the fundus image; An image processing device comprising:

[0121] (13) An image display device, an acquisition unit that acquires a fundus image and information about a non-perfused area (NPA) in a region including the posterior pole of the fundus extracted from the fundus image; a display unit that displays the non-perfusion region superimposed on the fundus image; An image display device comprising:

[0122] <Third Modification>

[0123] In the above embodiment, the management server 140 executes the image processing program shown in Fig. 7 in advance, but the technology of the present disclosure is not limited to this. The image viewer 150 may transmit an image processing command to the management server 140, and the management server 140 may execute the image processing program shown in Fig. 6 in response to the command.

[0124] <Fourth Modification>

[0125] In the above embodiment, an example has been described in which a fundus image with an internal light irradiation angle of approximately 200 degrees is acquired by the ophthalmic apparatus 110. The technology of the present disclosure is not limited to this, and the technology of the present disclosure may also be applied to a fundus image captured by an ophthalmic apparatus with an internal irradiation angle of 100 degrees or less, or to a montage image in which multiple fundus images are combined.

[0126] <Fifth Modification>

[0127] In the above embodiment, fundus images are captured by the ophthalmologic apparatus 110 equipped with an SLO imaging unit, but the technology of the present disclosure may also be applied to images obtained by OCT angiography.

[0128] <Sixth Modification>

[0129] In the above embodiment, the management server 140 executes the image processing program. However, the technology of the present disclosure is not limited to this. For example, the ophthalmologic apparatus 110 or the image viewer 150 may execute the image processing program.

[0130] <Seventh Modification>

[0131] In the above embodiment, the ophthalmic system 100 including the ophthalmic apparatus 110, the laser treatment apparatus 120, the management server 140, and the image viewer 150 has been described as an example, but the technology of the present disclosure is not limited to this. For example, as a first example, the laser treatment apparatus 120 may be omitted, and the ophthalmic apparatus 110 may further have the functions of the laser treatment apparatus 120. Furthermore, as a second example, the ophthalmic apparatus 110 may further have the functions of at least one of the management server 140 and the image viewer 150. For example, if the ophthalmic apparatus 110 has the functions of the management server 140, the management server 140 can be omitted. In this case, the image processing program is executed by the ophthalmic apparatus 110 or the image viewer 150. Furthermore, if the ophthalmic apparatus 110 has the functions of the image viewer 150, the image viewer 150 can be omitted. As a third example, The management server 140 may be omitted, and the image viewer 150 may perform the functions of the management server 140 .

[0132] <Eighth Modification> In the above embodiment, the image processing control unit 206 detects the posterior pole NPA by executing the process shown in Fig. 9, but the technology of the present disclosure is not limited to this. For example, the posterior pole NPA may be detected by the process shown in Fig. 16.

[0133] Image processing of the posterior pole NPA will be described with reference to FIG. 16, in step S1304, a process for removing retinal blood vessels is performed on the acquired FA image. Then, a Gaussian filter is used to remove high frequency components from the image in which the retinal blood vessels have been processed, and a low frequency component image is created.

[0134] Next, in step S1306, the image processing control unit 206 corrects the brightness of the peripheral part of the fundus by removing the low-frequency component image from the FA image. Then, in step S1308, the image processing control unit 206 performs NPA detection processing. More specifically, from the fundus image obtained in S1306, in which brightness correction has been performed on the peripheral portion of the fundus, multiple pixels that are darker than a first darkness are extracted, and one or more areas in which the dark pixels that are darker than the first darkness are continuous and have an area equal to or greater than a predetermined area are detected as posterior pole NPAs. Then, in step S1310, the image processing control unit 206 extracts the contour of the detected posterior pole NPA, and the processing ends.

[0135] In this way, image processing for the posterior pole NPA is performed in the eighth modified example. Compared to the posterior pole NPA processing shown in Fig. 9, there is no need to perform extraction processing in steps S308 and S310, which enables high-speed processing and increases the number of fundus images processed per unit time.

[0136] <Ninth Variation> In the above embodiment, the image viewer 150 displays the screen 500 in which the contour image of the fundus region, the contour image of the posterior pole NPA, the contour image of the UWF-NPA, and the contour image of the erroneously detected posterior pole NPA are superimposed on the FA image, but the technology of the present disclosure is not limited to this. For example, an image in which the contour image of the fundus region, the contour image of the posterior pole NPA, the contour image of the UWF-NPA, and the contour image of the erroneously detected posterior pole NPA are superimposed on a color fundus image or a front image (en-face image) obtained from OCT data may be displayed. In this case, the FA image The image is aligned with another image such as a color fundus image, and each contour image is superimposed and displayed at the correct position on the color fundus image.

[0137] <Other variations>

[0138] The data processing described in the above embodiment is merely an example, and it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed, without departing from the spirit of the invention.

[0139] In addition, in the above embodiment, a case where data processing is realized by a software configuration using a computer is exemplified, but the technology of the present disclosure is not limited to this. For example, instead of a software configuration using a computer, an FPGA (Field Programmable Gate Array (FPGA)) may be used. Programmable Gate Array) or ASIC (Application Specific Integrated Circuit Data processing may be performed solely by a hardware configuration such as a specific integrated circuit (SCI), or a part of the data processing may be performed by a software configuration and the remaining part by a hardware configuration. You can do this. [Explanation of symbols]

[0140] 100 Ophthalmology Systems 110 Ophthalmological equipment 120 Laser Treatment Device 140 Management Server 150 Image Viewer 204 Display control unit 206 Image processing control unit 208 Processing section 262 CPU 254 Storage device

Claims

1. a fundus image acquisition unit for acquiring a fundus image of the subject's eye; an avascular region information acquiring unit that acquires information about an avascular region in the fundus of the subject's eye; an identifying unit that identifies an aperfusion region based on information about the avascular region acquired by the avascular region information acquiring unit, a distance to a blood vessel on the fundus for each pixel in the posterior pole region of the fundus image, and a pixel value for each pixel; An image processing device comprising:

2. The avascular region information acquisition unit obtaining a tomographic image including a lesion area or information about a laser treatment position in the subject's eye as information about the avascular area; The image processing device according to claim 1 .

3. The identification unit 3. The image processing device according to claim 1, wherein a region along the blood vessels on the fundus extracted from the fundus image and a region on the fundus image in which a feature value for pixels contained in the region is below a predetermined threshold is extracted as a candidate region for the non-perfusion region.

4. The identification unit 4. The image processing device according to claim 3, wherein when the candidate region is extracted from the fundus image using the predetermined threshold as a first threshold, an area equal to or less than a second threshold that is smaller than the first threshold based on an average value of pixel values ​​obtained from at least one of lightness, saturation, and hue of the extracted candidate region is extracted as the candidate region for the non-perfusion region.

5. The identification unit identifying the non-perfusion region by excluding false positive regions from the candidate region based on information about the avascular region; 5. The image processing device according to claim 3.

6. a peripheral specifying unit that performs a process different from the process for extracting the non-perfusion region on a peripheral region of the fundus image, the peripheral region being a periphery of the posterior pole region, to extract a peripheral non-perfusion region, The image processing device according to any one of claims 1 to 5.

7. The periphery specification unit extracting a region that is separated by a predetermined distance or more from the blood vessels on the fundus extracted from the fundus image as the peripheral non-perfusion region; The image processing device according to claim 6 .

8. The fundus image is a fundus image photographed by fluorescence photography or an OCT image acquired by an optical coherence tomography (OCT) camera. The image processing device according to any one of claims 1 to 7.

9. On the computer, A procedure for acquiring a fundus image of the subject's eye; a procedure for acquiring information about an avascular region in the fundus of the subject's eye; a step of identifying a non-perfusion region based on information about the avascular region, a distance to a blood vessel on the fundus at each pixel in the posterior pole region of the fundus image, and a pixel value at each pixel; A program that executes the following.

Images