Image processing method, image processing program, and image processing device

Abstract

SOLUTION: An image processing method includes: a step for acquiring an ocular fundus image of an eye to be examined; a step for extracting a candidate area, which is a candidate for a non-perfusion area, from a posterior pole area, which is a posterior pole part of the ocular fundus of the ocular fundus image; a step for acquiring a tomographic image of the eye to be examined; a step for extracting a predetermined area on the ocular fundus image that imaging light when the ocular fundus image is taken does not reach on the basis of the tomographic image; and a step for excluding the predetermined area from the candidate area, and identifying the non-perfusion area from the ocular fundus image.EFFECT: A piece of information is provided when a doctor performs diagnosis, determines and confirms a progress state and a treatment effect for diabetic retinopathy, diabetic macular edema, retina vein occlusion, etc. A piece of information is also provided when a doctor performs checking on early diagnosis.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The technology disclosed herein relates to an image processing method, an image processing program, and an image processing device. [Background technology]

[0002] Patent Document 1 discloses a technique for extracting an area where an abnormality occurs by analyzing a tomographic image of the fundus. It is desirable to be able to confirm an abnormality by analyzing the fundus image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2015 / 0366452 Summary of the Invention

[0004] The first aspect of the technology of the present disclosure is acquiring a fundus image of a subject's eye; A step of extracting a candidate region that is a non-perfusion region candidate from a posterior pole region that is a posterior pole part of the fundus from the fundus image; acquiring a tomographic image of the subject's eye; extracting a predetermined area on the fundus image that is not reached by imaging light when the fundus image is photographed, based on the tomographic image; identifying a non-perfusion region from the fundus image by excluding the predetermined region from the candidate region; The image processing method includes:

[0005] A second aspect of the technology of the present disclosure is 1 is an image processing program for causing a computer to execute the image processing method of the first aspect.

[0006] A third aspect of the technology of the present disclosure is a fundus image acquisition unit for acquiring a fundus image of a subject's eye; a tomographic image acquisition unit for acquiring a tomographic image of the subject's eye; a first extraction unit that extracts a candidate region that is a non-perfusion region candidate from a posterior pole region that is a posterior pole part of the fundus from the fundus image; a second extraction unit that extracts a predetermined area on the fundus image that is not reached by imaging light when the fundus image is photographed, based on the tomographic image; a specifying unit that specifies a non-perfusion region from the fundus image by excluding the predetermined region from the candidate region; The image processing device includes: [Brief description of the drawings]

[0007] [Figure 1] 1 is a block diagram of an ophthalmologic system according to an embodiment. [Diagram 2] 1 is a schematic diagram illustrating an overall configuration of an ophthalmologic apparatus according to an embodiment. [Diagram 3] FIG. 2 is a block diagram of an electrical system configuration of a management server according to an embodiment. [Figure 4] FIG. 2 is a block diagram of an electrical configuration of the image viewer according to the embodiment. [Diagram 5] FIG. 2 is a functional block diagram of a CPU of a management server according to the embodiment. [Figure 6] FIG. 2 is a functional block diagram of a CPU of an image viewer according to the embodiment. [Figure 7] 10 is a flowchart of an image processing program executed by the management server according to the embodiment. [Figure 8] FIG. 2 is a diagram showing a fundus region in a fundus image according to the embodiment. [Figure 9] 10 is a flowchart showing a process for detecting a nonperfused region of the posterior pole of the fundus according to an embodiment. [Figure 10] An explanatory diagram of the process of detecting non-perfused areas in the posterior pole of the fundus in an embodiment, where (A) shows primary candidates, (B) shows primary candidates to be excluded, and (C) shows the identified non-perfused areas in the posterior pole of the fundus. [Figure 11] 10 is a flowchart showing a flow of processing for detecting a nonperfusion region around the fundus according to an embodiment. [Figure 12] FIG. 4 is a diagram showing a display screen displayed on a display of an image viewer according to the embodiment. [Figure 13] FIG. 4 is a diagram showing a display screen displayed on a display of an image viewer according to the embodiment. [Figure 14] 13 is a flowchart showing a process flow according to a first modified example. [Figure 15] 13 is a flowchart showing a process flow according to a second modified example. [Figure 16] 13 is a flowchart showing a process flow according to an eighth modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments of the technology of the present disclosure will be described in detail with reference to the drawings.

[0009] The configuration of an ophthalmologic system 100 will be described with reference to Fig. 1. As shown in Fig. 1, the ophthalmologic system 100 includes an ophthalmologic apparatus 110, a laser treatment apparatus 120, a management server apparatus (hereinafter referred to as "management server") 140, and an image display apparatus (hereinafter referred to as "image viewer") 150.

[0010] The ophthalmologic device 110 acquires fundus images and tomographic images. The laser treatment device 120 is a device that supports the treatment of a lesion in the patient's eye 12. The laser treatment device 120 may be, for example, a treatment device such as a laser photocoagulation device that irradiates a laser beam to photocoagulate the irradiated area in order to suppress the progression of a disease in the lesion in the fundus of the patient. The laser treatment device 120 transmits information on the treatment performed on the eye 12 to the management server 140. For example, when a specific area of ​​the retina of the eye 12 is treated, the position of the specific area, the treatment time, and the treatment method are transmitted to the management server 140 as treatment information.

[0011] The management server 140 stores a plurality of fundus images obtained by photographing the fundus of a plurality of patients by the ophthalmologic apparatus 110 in association with the patient's ID. The management server 140 also detects a non-perfusion area (NPA) from a specified fundus image. The image viewer 150 displays the analysis result of the fundus image by the management server 140, for example, an image of an estimated non-perfusion area (NPA).

[0012] Here, the nonperfused area (NPA) is an area in the fundus where blood does not flow or where there is almost no blood flow due to blockage of the retinal capillary bed, etc. It is also an area where retinal ischemia occurs due to perfusion disorder.

[0013] The ophthalmologic apparatus 110 , the laser treatment apparatus 120 , the management server 140 , and the image viewer 150 are connected to each other via a network 160 .

[0014] As described above, in Fig. 1, the ophthalmic system 100 includes the laser treatment device 120, but the technology of the present disclosure is not limited to this. For example, the ophthalmic system 100 may replace the laser treatment device 120 with a measuring device such as a visual field measuring device that measures the visual field of a patient or an axial length measuring device that measures the axial length of the subject's eye 12. In addition, such a measuring device may be further added and connected to the network 130.

[0015] The management server 140 is an example of an "image processing device" of the technology of the present disclosure. The image viewer 150 is an example of an "image display device" of the technology of the present disclosure.

[0016] For the sake of convenience, the term "scanning laser ophthalmoscope" will be used hereinafter. Optical Coherence Tomography (OCT) is abbreviated as "OCT" and Optical Lung Ophthalmoscope (SLO) is abbreviated as "Optical Coherence Tomography".

[0017] Next, the configuration of the ophthalmic apparatus 110 will be described with reference to FIG.

[0018] When the ophthalmic apparatus 110 is placed on a horizontal plane, the horizontal direction is defined as the "X direction", the vertical direction to the horizontal plane is defined as the "Y direction", and the direction connecting the center of the pupil of the anterior part of the subject's eye 12 and the center of the eyeball is defined as the "Z direction". Therefore, the X direction, Y direction, and Z direction are perpendicular to each other.

[0019] The ophthalmologic apparatus 110 includes an imaging device 14 and a control device 16. The imaging device 14 is equipped with an SLO unit 18 and an OCT unit 20, and acquires a fundus image of the fundus of the subject's eye 12. Hereinafter, a two-dimensional fundus image acquired by the SLO unit 18 will be referred to as an SLO image. Also, a tomographic image or a front image (en-face image) of the retina created based on OCT data acquired by the OCT unit 20 will be referred to as an OCT image.

[0020] The control device 16 includes a computer having a central processing unit (CPU) 16A, a random access memory (RAM) 16B, a read-only memory (ROM) 16C, and an input / output (I / O) port 16D.

[0021] The control device 16 includes an input / display device 16E connected to the CPU 16A via an I / O port 16D. The input / display device 16E has a graphic user interface that displays an image of the subject's eye 12 and receives various instructions from a user. An example of the graphic user interface is a touch panel display.

[0022] The control device 16 also includes an image processing device 17 connected to the I / O port 16D. The image processing device 17 generates an image of the subject's eye 12 based on data obtained by the photographing device 14. The control device 16 is connected to a network 130 via a communication interface (not shown).

[0023] 2, the control device 16 of the ophthalmic apparatus 110 includes the input / display device 16E, but the technology of the present disclosure is not limited thereto. For example, the control device 16 of the ophthalmic apparatus 110 may not include the input / display device 16E, but may include a separate input / display device that is physically independent from the ophthalmic apparatus 110. In this case, the display device includes an image processing processor unit that operates under the control of the display control unit 204 of the CPU 16A of the control device 16. The image processing processor unit may display an SLO image or the like based on an image signal that is instructed to be output by the display control unit 204.

[0024] The imaging device 14 operates under the control of an imaging control unit 202 of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The imaging optical system 19 includes a first optical scanner 22, a second optical scanner 24, and a wide-angle optical system 30.

[0025] The first optical scanner 22 performs two-dimensional scanning in the X and Y directions with the light emitted from the SLO unit 18. The second optical scanner 24 performs two-dimensional scanning in the X and Y directions with the light emitted from the OCT unit 20. The optical element 24 may be any optical element capable of deflecting a light beam, such as a polygon mirror, a galvanometer mirror, or a combination thereof.

[0026] The wide-angle optical system 30 includes an objective optical system (not shown in FIG. 2) having a common optical system 28 , and a combining unit 26 that combines the light from the SLO unit 18 and the light from the OCT unit 20 .

[0027] The objective optical system of the common optical system 28 may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a catadioptric system combining concave mirrors and lenses. By using a wide-angle optical system using an elliptical mirror or a wide-angle lens, it becomes possible to photograph not only the central part of the fundus (the posterior pole of the fundus) but also the retina in the peripheral part of the fundus.

[0028] When a system including an elliptical mirror is used, the system using the elliptical mirror described in International Publication WO2016 / 103484 or International Publication WO2016 / 103489 may be used. The disclosures of International Publication WO2016 / 103484 and International Publication WO2016 / 103489 are each incorporated herein by reference in their entirety.

[0029] The wide-angle optical system 30 realizes observation of the fundus in a wide field of view (FOV) 12A. The FOV 12A indicates a range that can be photographed by the photographing device 14. The FOV 12A can be expressed as a viewing angle. In this embodiment, the viewing angle can be defined by an internal irradiation angle and an external irradiation angle. The external irradiation angle is an irradiation angle of a light beam irradiated from the ophthalmic device 110 to the subject's eye 12, the irradiation angle being defined based on the pupil 27. The internal irradiation angle is an irradiation angle of a light beam irradiated to the fundus F, the irradiation angle being defined based on the eyeball center O. The external irradiation angle and the internal irradiation angle are in a corresponding relationship. For example, when the external irradiation angle is 120 degrees, the internal irradiation angle corresponds to about 160 degrees. In this embodiment, the internal irradiation angle is 200 degrees.

[0030] Here, an SLO fundus image captured at an internal illumination angle of 160 degrees or more is referred to as a UWF-SLO fundus image. UWF is an abbreviation for Ultra Wide Field.

[0031] The SLO system is realized by a control device 16, an SLO unit 18, and an imaging optical system 19 shown in Fig. 2. The SLO system includes a wide-angle optical system 30, and therefore enables fundus imaging with a wide FOV 12A.

[0032] The SLO unit 18 includes a B (blue light) light source 40, a G (green light) light source 42, an R (red light) light source 44, and an IR (infrared (e.g., near-infrared) light source 46, as well as optical systems 48, 50, 52, 54, and 56 that reflect or transmit the light from the light sources 40, 42, 44, and 46 and guide them to one optical path. The optical systems 48, 50, and 56 are mirrors, and the optical systems 52 and 54 are beam splitters. The B light is reflected by the optical system 48, transmits through the optical system 50, and is reflected by the optical system 54, the G light is reflected by the optical systems 50 and 54, the R light is transmitted through the optical systems 52 and 54, and the IR light is reflected by the optical systems 52 and 56, and each is guided to one optical path.

[0033] The SLO unit 18 is configured to be able to switch between a light source that emits laser light of different wavelengths or a combination of light sources that emit light, such as a mode that emits R light and G light and a mode that emits infrared light. In the example shown in Fig. 2, three light sources are provided: a light source 42 of G light, a light source 44 of R light, and a light source 46 of IR light, but the technology of the present disclosure is not limited to this. For example, the SLO unit 18 may further include a light source of B light (blue light) and a light source of white light, and may emit light in various modes such as a mode that emits G light, R light, and B light, or a mode that emits only white light.

[0034] The light incident on the photographing optical system 19 from the SLO unit 18 is scanned in the X and Y directions by the first optical scanner 22. The scanning light is irradiated onto the fundus via the wide-angle optical system 30 and the pupil 27. The light reflected by the fundus is incident on the SLO unit 18 via the wide-angle optical system 30 and the first optical scanner 22.

[0035] The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits light other than B light from the posterior segment (fundus) of the subject's eye 12, and a beam splitter 58 that reflects G light and transmits light other than G light from the light that has transmitted through the beam splitter 64. The SLO unit 18 includes a beam splitter 60 that reflects R light and transmits light other than R light from the light that has transmitted through the beam splitter 58. The SLO unit 18 includes a beam splitter 62 that reflects IR light from the light that has transmitted through the beam splitter 60. The SLO unit 18 includes a B light detection element 70 that detects B light reflected by the beam splitter 64, a G light detection element 72 that detects G light reflected by the beam splitter 58, an R light detection element 74 that detects R light reflected by the beam splitter 60, and an IR light detection element 76 that detects IR light reflected by the beam splitter 62.

[0036] The light (reflected light reflected by the fundus) incident on the SLO unit 18 via the wide-angle optical system 30 and the first optical scanner 22 is reflected by the beam splitter 64 and received by the B light detection element 70 in the case of B light, and is reflected by the beam splitter 58 and received by the G light detection element 72 in the case of G light. The incident light passes through the beam splitter 58 in the case of R light, is reflected by the beam splitter 60, and is received by the R light detection element 74. The incident light passes through the beam splitters 58 and 60, is reflected by the beam splitter 62, and is received by the IR light detection element 76 in the case of IR light. The image processing device 17, which operates under the control of the CPU 16A, generates a UWF-SLO image using signals detected by the B light detection element 70, the G light detection element 72, the R light detection element 74, and the IR light detection element 76.

[0037] The UWF-SLO images include a UWF-SLO image (G-color fundus image) obtained by photographing the fundus in G color, and a UWF-SLO image (R-color fundus image) obtained by photographing the fundus in R color. The UWF-SLO images include a UWF-SLO image (B-color fundus image) obtained by photographing the fundus in B color, and a UWF-SLO image (IR fundus image) obtained by photographing the fundus in IR color.

[0038] The control device 16 also controls the light sources 40, 42, 44 to emit light simultaneously. The fundus of the subject's eye 12 is photographed simultaneously with the B light, the G light, and the R light, thereby obtaining a G-color fundus image, an R-color fundus image, and a B-color fundus image, each of which corresponds to each other. An RGB color fundus image is obtained from the G-color fundus image, the R-color fundus image, and the B-color fundus image. The control device 16 controls the light sources 42, 44 to emit light simultaneously, thereby obtaining a fundus of the subject's eye 12 simultaneously with the G light and the R light, thereby obtaining a G-color fundus image and an R-color fundus image, each of which corresponds to each other. An RG color fundus image is obtained from the G-color fundus image and the R-color fundus image.

[0039] The UWF-SLO images also include UWF-SLO fluorescent images that are photographed fluorescently using a contrast agent.

[0040] The image data of the B color fundus image, the G color fundus image, the R color fundus image, the IR fundus image, the RGB color fundus image, the RG color fundus image, and the UWF-SLO fluorescence image are sent from the ophthalmic device 110 to the management server 140 via a communication IF not shown.

[0041] The OCT system is realized by the control device 16, the OCT unit 20, and the photographing optical system 19 shown in FIG. 2. Since the OCT system includes the wide-angle optical system 30, the above-mentioned S Similar to LO fundus image capture, fundus imaging is possible with a wide FOV 12A. The OCT unit 20 includes a light source 20A, a sensor (detection element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

[0042] Light emitted from the light source 20A is branched by the first optical coupler 20C. One of the branched lights is collimated by the collimating lens 20E as measurement light, and then enters the photographing optical system 19. The measurement light is scanned in the X and Y directions by the second optical scanner 24. The scanning light passes through the wide-angle optical system 30 and the pupil 27 and is irradiated onto the fundus. The measurement light reflected by the fundus is entered into the OCT unit 20 via the wide-angle optical system 30 and the second optical scanner 24, and enters the second optical coupler 20F via the collimating lens 20E and the first optical coupler 20C.

[0043] The other light emitted from the light source 20A and branched by the first optical coupler 20C is incident on the reference optical system 20D as reference light, and passes through the reference optical system 20D to be incident on the second optical coupler 20F.

[0044] The light beams incident on the second optical coupler 20F, i.e., the measurement light reflected from the fundus and the reference light, are interfered with by the second optical coupler 20F to generate interference light. The interference light is received by the sensor 20B. The image processing device 17, which operates under the control of the image processing controller 206, generates OCT images such as tomographic images and en-face images based on the OCT data detected by the sensor 20B.

[0045] The OCT unit 20 can obtain OCT data of an OCT image which is a tomographic image of the subject's eye 12. As an example of an OCT image, a one-dimensional OCT image is an A-scan image obtained by performing a so-called A-scan by the ophthalmic apparatus 110, a two-dimensional OCT image is a B-scan image obtained by performing a so-called B-scan by the ophthalmic apparatus 110, and a three-dimensional OCT image is a C-scan image obtained by performing a so-called C-scan by the ophthalmic apparatus 110.

[0046] Here, an OCT fundus image captured at an internal illumination angle of 160 degrees or more is referred to as a UWF-OCT image.

[0047] The image data of the UWF-OCT image is sent from the ophthalmic apparatus 110 to the management server 140 via a communication IF (not shown), and is stored in the storage device 254 .

[0048] In this embodiment, the light source 20A is exemplified as a wavelength-swept type SS-OCT (Swept-Source OCT). The OCT system may be of various types, such as optical coherence tomography (OCT), time-domain optical coherence tomography (TD-OCT), or time-domain optical coherence tomography (TD-OCT).

[0049] Next, the configuration of the electrical system of the management server 140 will be described with reference to FIG. 3. As shown in FIG. 3, the management server 140 includes a computer main body 252. The computer main body 252 includes a CPU 262, a RAM 266, a ROM 264, and an input / output (I / O) port 268. The input / output (I / O) port 268 is connected to a storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I / F) 258. The storage device 254 is, for example, a non-volatile memory. The input / output (I / O) port 268 is connected to the network 130 via the communication interface (I / F) 258. Therefore, the management server 140 can communicate with the ophthalmic device 110, the laser treatment device 120, and the image viewer 150.

[0050] The management server 140 stores the data received from the ophthalmic apparatus 110 and the laser treatment apparatus 120 in the storage device 254 .

[0051] Next, the configuration of the electrical system of the image viewer 150 will be described with reference to FIG. 4. As shown in FIG. 4, the image viewer 150 includes a computer main body 152. The computer main body 152 includes a CPU 162, a RAM 166, a ROM 164, and an input / output (I / O) port 168. The input / output (I / O) port 168 is connected to a storage device 154, a display 156, a mouse 155M, a keyboard 155K, and a communication interface (I / F) 158. The storage device 154 is, for example, a non-volatile memory. The input / output (I / O) port 168 is connected to the network 130 via the communication interface (I / F) 158. Therefore, the image viewer 150 can communicate with the ophthalmic device 110 and the management server 140.

[0052] Next, various functions realized by the CPU 262 of the management server 140 executing the image processing program will be described with reference to Fig. 5. As shown in Fig. 5, the image processing program has a display control function, an image processing control function, and a processing function. By the CPU 262 executing the image processing program having these functions, the CPU 262 functions as a display control unit 204, an image processing control unit 206, and a processing unit 208 as shown in Fig. 5.

[0053] Next, various functions realized by the CPU 162 of the image viewer 150 executing the image processing program will be described with reference to Fig. 6. As shown in Fig. 6, the image processing program has a display control function, an image processing control function, and a processing function. By the CPU 162 executing the image processing program having these functions, the CPU 162 functions as the display control unit 104, the image processing control unit 106, and the processing unit 108 as shown in Fig. 6.

[0054] The image processing control unit 206 is an example of a "fundus image acquisition unit," a "first non-perfusion region extraction unit," and a "second non-perfusion region extraction unit" of the technology disclosed herein. The image processing control unit 106 is an example of the "acquisition unit" of the technology of the present disclosure, and the display 156 is an example of the "display unit" of the technology of the present disclosure.

[0055] Next, image processing by management server 140 will be described in detail with reference to Fig. 7. Image processing shown in the flowchart of Fig. 7 is realized by CPU 262 of management server 140 executing an image processing program. The image processing shown in Fig. 7 is an example of an image processing method of the technology of the present disclosure. Moreover, the display processing for displaying an image obtained by the image processing shown in Fig. 7 is an example of an image display method of the technology of the present disclosure.

[0056] It is assumed here that UWF-SLO image processing and UWF-OCT image processing are executed, and each data obtained by the execution of the processing (e.g., image data of the UWF-SLO image and image data of the UWF-OCT image) is stored in the memory device 254.

[0057] In this embodiment, as an example of a UWF-SLO image for detecting NPA, a UWF-SLO image obtained by fluorescein angiography (FA) is used. A case where a FA image (hereinafter referred to as an FA image) is used will be described. The FA image is suitable for detecting non-perfusion regions because the retinal capillaries are captured with high resolution. Image data of this FA image is sent from the ophthalmologic device 110 to the management server 140 via a communication IF (not shown) and stored in the storage device 254. In addition, fluorescent fundus imaging was performed using indocyanine green (ICG) as a contrast agent. UWF-SLO images taken using indocyanine green angio-graphy (IA) (hereinafter referred to as IA images) can also be used. In FA images and IA images, blood vessels in the fundus appear white.

[0058] 7, the image processing control unit 206 functioning in the management server 140 acquires an FA image from the storage device 254. The FA image is captured by the ophthalmic apparatus 110 and stored in the storage device 254.

[0059] In the next step S104, the image processing control unit 206 detects the fundus area of ​​the subject's eye 12 from the acquired FA image. In this step S104, the peripheral parts of the subject's eye 12, such as the patient's eyelashes and eyelids, and the area in which the elements constituting the ophthalmic device 110 are reflected are removed, and the area after removal is extracted as the fundus area, thereby detecting the fundus area of ​​the subject's eye 12. This process of extracting the fundus area can be realized by executing image processing that combines known processes such as binarization and morphology processing. FIG. 8 shows an image in which the fundus area has been extracted after the process of step 104 has been performed. The dashed lines in FIG. 8 are lines that typically indicate the edge of the fundus area, and there are no dashed lines in the image processed in step S104.

[0060] Next, the image processing control unit 206 detects a non-perfused area (NPA) from the detected fundus area.

[0061] In this embodiment, it is preferable to perform image processing according to the part of the fundus region of the subject eye 12 due to differences in the distribution of retinal blood vessels between the posterior pole of the fundus and its surrounding peripheral part. Therefore, in this embodiment, image processing is performed to classify and visualize the non-perfusion area (NPA) in the fundus region into a non-perfusion area in the posterior pole of the fundus (hereinafter referred to as posterior pole NPA) and a non-perfusion area in the periphery of the fundus (hereinafter referred to as UWF-NPA). The posterior pole NPA is searched for in the fundus region where retinal blood vessels are dense, and the UWF-NPA is searched for in the peripheral region of the fundus. Therefore, in this embodiment, processing optimal for detecting the posterior pole NPA (processing in FIG. 9 or FIG. 16 described later) and processing optimal for detecting the UWF-NPA (processing in FIG. 11 described later) are performed on the FA image.

[0062] Specifically, in step S106, the image processing control unit 206 detects posterior pole NPA in the fundus region of the subject's eye 12, and in step S108, performs image processing of posterior pole NPA to identify erroneously detected posterior pole NPA from among the detected posterior pole NPA. At the same time, in step S110, the image processing control unit 206 performs image processing of UWF-NPA to detect UWF-NPA in the fundus region of the subject's eye 12. Here, erroneously detected posterior pole NPA refers to a region that is detected as a posterior pole NPA but is unlikely to be a posterior pole NPA.

[0063] The posterior pole of the fundus of the subject eye 12 is an example of the "first region" of the technology disclosed herein, and the posterior pole NPA is an example of the "first non-perfusion region" of the technology disclosed herein. The fundus periphery, which is the periphery of the posterior pole of the fundus of the subject eye 12, is an example of the "second region" of the technology disclosed herein, and the UWF-NPA is an example of the "second non-perfusion region" of the technology disclosed herein. The erroneously detected posterior pole NPA, which is an area unlikely to be a posterior pole NPA, is an example of the "third non-perfusion region" of the technology disclosed herein.

[0064] Note that steps S106, S108, and S110 may be executed first, or simultaneously. After completing steps S106, S108, and S110, the image processing control unit 206 executes image processing to generate a display screen in step S112. As will be described in detail later, the generated display screen displays the contour of the posterior pole NPA so that the position of the posterior pole NPA can be easily recognized in the FA image, and the contour of the UWF-NPA so that the position of the UWF-NPA can be easily recognized, superimposed on the FA image. In step 114, the processing unit 208 transmits the generated display image of the display screen to the image viewer 150.

[0065] Next, image processing of the posterior pole NPA will be described with reference to FIGS. 9, in step S304, the image processing control unit 206 performs image enhancement processing for enhancing blood vessels on the acquired FA image. This is processing for highlighting blood vessels including capillaries, and is processing for estimating the posterior pole NPA with high accuracy.

[0066] The enhancement image processing can use various methods such as enhancement processing using an image histogram, such as histogram averaging or adaptive histogram equalization (CLAHE (Contrast Limited Adaptive Histogram Equalization)), contrast conversion processing based on gradation conversion, frequency enhancement processing of a specific frequency band such as unsharp mask, deconvolution processing such as Wiener filter, and morphology processing that enhances the shape of blood vessels, but histogram averaging or adaptive histogram equalization is preferable. Blood vessels are enhanced by the enhancement image processing.

[0067] Next, the image processing control unit 206 estimates multiple posterior pole NPAs from the FA image with enhanced blood vessels. Specifically, in step S306, the image processing control unit 206 selects a primary candidate for the posterior pole NPA. More specifically, multiple pixels with a first darkness or less are extracted from the FA image with enhanced blood vessels, and a single or multiple areas having an area of ​​a predetermined area or more where the dark pixels with the first darkness or less are continuous are selected as the primary candidate for the posterior pole NPA.

[0068] Here, a pixel that is darker than a first darkness refers to a pixel whose pixel value is equal to or less than a first predetermined value. Note that, as the pixel value, for example, a luminance value representing lightness may be used, but a value representing at least one of saturation and hue may be used instead of or together with the luminance value.

[0069] Next, the image processing control unit 206 executes the image processing in steps S308 and S310.

[0070] In step S308, the image processing control unit 206 selects only dark candidates from the single or multiple primary candidates for the posterior pole NPA based on the average value of pixel values ​​within the area of ​​each candidate. Specifically, the average value of pixel values ​​within each area of ​​the single or multiple primary candidates for the posterior pole NPA is calculated, and the single or multiple candidates for which the calculated average value is smaller than a second predetermined value are selected as dark areas. The second predetermined value is a value that is a predetermined value smaller than the first predetermined value. In other words, from the primary candidates for the first darkness, only candidates that are dark areas that are darker than a second darkness that is darker than the first darkness (candidates with a predetermined average pixel value or less) are extracted and set as the first secondary candidates.

[0071] In step S310, the image processing control unit 206 narrows down the multiple primary candidates for the posterior pole NPA to only the regions along the blood vessels. Specifically, the image processing control unit 206 first (1) extracts blood vessels. The blood vessels are extracted by a method such as morphological processing or binarization based on pixel values. The extracted region is called a blood vessel region. Next, the image processing control unit 206 (2) calculates the distance between the blood vessel region and the periphery of each region group of a single or multiple primary candidates or candidate groups for the posterior pole NPA by a method such as distance transformation, and selects a region where the distance is within a certain range. Here, being within a certain range means a first range that is greater than a first predetermined distance and smaller than a second predetermined distance that is greater than the first predetermined distance (i.e., along a running blood vessel).

[0072] In this way, in step S310, the image processing control unit 206 extracts, from the primary candidates, an area whose distance from the blood vessel is equal to or shorter than the first distance as a second secondary candidate. Note that the second secondary candidate may be an area within a certain range from the end of the blood vessel.

[0073] Either step S308 or step S310 may be executed first, or they may be executed simultaneously. After completing steps S308 and S310, the image processing control unit 206 executes image processing shown in step S312.

[0074] In step S312, the image processing control unit 206 performs an integration process to integrate the first secondary candidate and the second secondary candidate. Specifically, an area that is both the first secondary candidate (multiple dark areas) and the second secondary candidate (multiple areas along blood vessels) is extracted and specified as the posterior pole NPA.

[0075] Fig. 10 is a simplified representation of the processing results from step S306 to step S312, and is a schematic diagram showing an enlarged portion of the FA image. Fig. 10(A) shows four primary candidates 406, 408, 410, and 412 of the posterior pole NPA after the processing of step S306, together with a blood vessel 400. For convenience of explanation, the blood vessel 400 is shown in Fig. 10 with a black line. Fig. 10(B) shows primary candidates 406A, 408A, and 410A with solid lines as an example of the primary candidates estimated in the processing of steps S308 and S310. Fig. 10(B) also shows primary candidate 412 with a dotted line as an example of the primary candidate to be excluded. FIG. 10(C) shows putative candidates 406NPA, 408NPA, and 410NPA, which have been narrowed down from the primary candidates to secondary candidates common to the first and second secondary candidates, as identified posterior pole NPAs.

[0076] 9, the image processing control unit 206 extracts the contour of the posterior pole NPA in the FA image. The contour of the posterior pole NPA is an image to be superimposed on the FA image for display so that the position of the posterior pole NPA can be easily recognized on the FA image. In this manner, image processing for the posterior pole NPA is performed.

[0077] Next, the process of identifying erroneous detection of the posterior pole NPA in step S108 shown in FIG. 7 will be described.

[0078] The area detected as the posterior pole NPA as described above (step S106) may be an area that is not a posterior pole NPA (hereinafter, referred to as a non-NPA). Examples of non-NPA that are erroneously detected include photocoagulation spots, soft white spots, and spots caused by cataracts that prevent the imaging light from reaching the retina.

[0079] Therefore, in step S108 of Fig. 7, the image processing control unit 206 identifies whether the posterior pole NPA detected in step S106 is a posterior pole NPA that has been erroneously detected. Specifically, the image processing control unit 206 identifies whether the detected posterior pole NPA is a posterior pole NPA that has been erroneously detected by the following discrimination method.

[0080] In the first discrimination method, image processing is performed using image data of known photocoagulation spots with an image filter that detects photocoagulation spots, and the posterior pole NPA and photocoagulation spots are discriminated from each other. If the photocoagulation spot is discriminated, the posterior pole NPA discriminated as the photocoagulation spot is discriminated as a non-NPA. Note that the image filter used in the first discrimination method is not limited to the detection of photocoagulation spots, and it is possible to use an image filter that detects areas such as soft white spots and spots caused by the fact that the photographing light does not reach the retina due to cataracts.

[0081] The second discrimination method uses artificial intelligence that has been machine-learned on image data of known photocoagulation spots to discriminate between posterior pole NPA and photocoagulation spots, and if a photocoagulation spot is identified, the posterior pole NPA that has been identified as a photocoagulation spot is classified as a non-NPA. Note that the second discrimination method is not limited to machine-learning image data of photocoagulation spots, and may be machine-learned on image data showing areas such as soft white spots and spots caused by cataracts that do not allow the imaging light to reach the retina.

[0082] The third discrimination method involves obtaining data indicative of non-NPAs from another imaging device, and using the obtained data to distinguish non-NPAs from detected posterior pole NPAs. For example, in the case of soft exudates, the position of the soft exudates identified in the OCT B-scan image is identified. The posterior pole NPA at the identified position is excluded as a soft exudate. In addition, in the case of photocoagulation plaque, the position of laser irradiation by the laser treatment device 120 is identified. The posterior pole NPA at the identified laser irradiation position is excluded as a photocoagulation plaque. The data indicating non-NPA from other imaging devices used in the third discrimination method is an example of data for identifying non-NPA, and is not limited thereto, and may be any data for identifying non-NPA.

[0083] In this manner, by identifying whether a detected posterior pole NPA is an erroneously detected posterior pole NPA, it is possible to improve the detection accuracy of the posterior pole NPA. Furthermore, by eliminating erroneously detected posterior pole NPA, it becomes possible to extract only the posterior pole NPA, which is a non-perfused region that requires photocoagulation treatment.

[0084] Next, image processing of UWF-NPA will be described with reference to Fig. 11. UWF-NPA is an area located in the peripheral part of the fundus where no retinal blood vessels exist. 11, in step S400, the image processing control unit 206 performs image enhancement processing for enhancing blood vessel portions on the acquired FA image. This is processing for highlighting blood vessels including capillaries, and is processing for estimating UWF-NPA with high accuracy.

[0085] In step S402, the image processing control unit 206 performs blood vessel binarization processing to binarize the blood vessel enhanced image in which the blood vessel portions have been emphasized. In step S404, the image processing control unit 206 performs distance image creation processing to create a distance image using the blood vessel binarized image in which the blood vessel enhanced image in which the blood vessel portions have been emphasized has been binarized. The distance image is an image in which the luminance increases as the distance from the edge of the line segment (corresponding to the blood vessel portion) in the binarized image increases.

[0086] In step S406, the image processing control unit 206 performs a binarization process to binarize the distance image. In this binarization process, the distance image is binarized, and the area around the fundus, which is the periphery of the posterior pole of the fundus, is converted into a white area in the binarized distance image (a white area may remain in part of the posterior pole as well).

[0087] In step S408, the image processing control unit 206 performs a process of removing a region having a predetermined number of pixels or less. This process is a process of converting a region having a predetermined number of white pixels or less from a binarized distance image obtained by binarizing a distance image into a black region. Specifically, a plurality of white pixels are extracted from the binarized distance image, and a single or a plurality of regions having an area of ​​a predetermined area or less where white pixels are continuous are converted into a region having continuous black pixels. The binarized distance image includes a blood vessel portion in the posterior pole of the fundus of the white region and a region around the fundus that is the periphery of the posterior pole of the fundus. The number of pixels corresponding to the size, for example the width, of the blood vessel portion in the posterior pole of the fundus of the binarized distance image can be converted into a black region. As a result, the white region around the fundus remains in the binarized distance image.

[0088] In step S410, the image processing control unit 206 performs a binary process after removing white areas having a certain number of pixels or less. The contour of the UWF-NPA is extracted by extracting the contour of the white area remaining in the FA image. The contour of the UWF-NPA is an image to be superimposed on the FA image so that the position of the UWF-NPA can be easily recognized on the FA image. In this manner, image processing related to UWF-NPA is performed.

[0089] Next, a method for displaying the non-perfused area (NPA) will be described in more detail with reference to a screen 500 of the display 156 of the image viewer 150 shown in FIGS. 12 and 13 are examples of images relating to a fundus image of the subject's eye 12 displayed by executing the "image display method" of the technique of the present disclosure.

[0090] When the operator inputs the patient ID of the patient who will be observed by the fundus image, the image viewer 150 issues a command to the management server 140 to output the patient information. The management server 140 reads out the patient information corresponding to the patient ID. Then, the management server 140 reads out the FA image and performs the image processing described in FIG. 7. Then, the management server 140 stores the contour image of the fundus region corresponding to the obtained FA image, the contour image of the posterior pole NPA, the contour image of the UWF-NPA, and the contour image of the erroneously detected posterior pole NPA in the storage device 154 of the management server, and generates an image in which the contour image of the fundus region, the contour image of the posterior pole NPA, the contour image of the UWF-NPA, and the contour image of the erroneously detected posterior pole NPA are superimposed on the FA image, generates a display screen for the image viewer 150, and transmits the image data of the display screen to the image viewer 150. The display control unit 104 of the image viewer 150 controls the display of an image based on the image data of the display screen from the management server 140 on the display 156.

[0091] When an operator instructs the image viewer 150 to display a contour image (described later), the image viewer 150 generates a display screen according to the instruction and issues a command to the management server 140 to output image data of the display screen. The processing unit 108 of the image viewer 150 receives image data of the display screen corresponding to the instructed display mode of the contour image, and the display control unit 104 controls the display of the image on the display 156.

[0092] 12, the display contents of the non-perfused area (NPA) on the display 156 of the image viewer 150 are displayed on a screen 500. The screen 500 has a patient information display field 502, a fundus image display field 504, and an option instruction display field 506.

[0093] The patient information display field 502 has a patient ID display field 502A, a patient name display field 502B, an age display field 502C, a target eyeball display field 502D, a photographing date and time display field 502E, an axial length display field 502F, and a visual acuity display field 502G. The image viewer 150 acquires the patient ID, patient name, age, target eyeball, photographing date and time, axial length, and visual acuity stored in the management server 140. The image viewer 150 displays the acquired patient ID, patient name, age, target eyeball, photographing date and time, axial length, and visual acuity in the patient ID display field 502A, the patient name display field 502B, the target eyeball display field 502D, the photographing date and time display field 502E, the axial length display field 502F, and the visual acuity display field 502G, respectively.

[0094] The image viewer 150 displays the FA image of the patient in a fundus image display field 504 on the screen 500. In addition, the image viewer 150 displays a contour image of a nonperfused area (NPA) superimposed on the FA image to facilitate observation and diagnosis of the fundus of the subject's eye 12.

[0095] In the example shown in Fig. 12, a display form is shown in which a contour image 504A of the fundus region in the FA image, contour images 504B and 504C of the posterior pole NPA, and a contour image 504D of the UWF-NPA are displayed. In the example shown in Fig. 13, a display form is shown in which a contour image 504A of the fundus region in the FA image is not displayed, and contour images 504B and 504C of the posterior pole NPA, a contour image 504D of the UWF-NPA, and a posterior pole NPA 504E that has been erroneously detected are displayed. do.

[0096] An operator of the image viewer 150 (e.g., a doctor) may wish to change the display format of the image displayed in the fundus image display field 504. Therefore, the option instruction display field 506 has an instruction button for instructing selection of the display format of the image displayed in the fundus image display field 504 and a display field for displaying the instruction result.

[0097] In the example shown in Fig. 12, the option instruction display field 506 has an instruction button 506A for instructing in a pull-down format whether or not to display an outline image of the fundus region in the FA image and the display color, and a display field 506B for displaying the instruction result. The display form of the outline image of the fundus region displayed in the fundus image display field 504 is changed by an operator (e.g., a doctor) of the image viewer 150 selecting the instruction button 506A. For example, the operator operates an input means such as the mouse 155M to specify whether or not to display the outline image of the fundus region and the display color. When the display color and whether or not to display the outline image of the fundus region are specified, the image viewer 150 displays the outline image of the fundus region with the specified display color and whether or not to display it.

[0098] The option instruction display field 506 also has an instruction button 506C for instructing in a pull-down format whether or not to display the contour image of the posterior pole NPA and the display color, and a display field 506D for displaying the instruction result. The display form of the contour image of the posterior pole NPA displayed in the fundus image display field 504 is changed by the operator of the image viewer 150 selecting the instruction button 506C. For example, the operator operates an input means such as the mouse 155M to specify whether or not to display the contour image of the posterior pole NPA and the display color. When the display color and whether or not to display the contour image of the posterior pole NPA are specified, the image viewer 150 displays the contour image of the posterior pole NPA with the specified display color and whether or not to display it.

[0099] The option instruction display field 506 also has an instruction button 506E for instructing in a pull-down format whether or not to display the contour image of UWF-NPA and the display color, and a display field 506F for displaying the instruction result. The display form of the contour image of UWF-NPA displayed in the fundus image display field 504 is changed by the operator of the image viewer 150 selecting the instruction button 506E. For example, the operator operates an input means such as the mouse 155M to specify whether or not to display the contour image of UWF-NPA and the display color. When the display color and whether or not to display the contour image of UWF-NPA are specified, the image viewer 150 displays the contour image of UWF-NPA with the specified display color and whether or not to display it.

[0100] The option instruction display field 506 also has an instruction button 506G for instructing, in a pull-down format, whether or not to display and the display color of the contour image of the erroneously detected posterior pole NPA, which is an area that has been detected as a posterior pole NPA but is unlikely to be a posterior pole NPA, and a display field 506H for displaying the instruction result. The display form of the contour image of the erroneously detected posterior pole NPA displayed in the fundus image display field 504 is changed by the operator of the image viewer 150 selecting the instruction button 506G. For example, the operator operates an input means such as the mouse 155M to specify whether or not to display and the display color of the contour image of the erroneously detected posterior pole NPA. When the display of the contour image of the erroneously detected posterior pole NPA and the display color are specified, the image viewer 150 displays the contour image of the erroneously detected posterior pole NPA with the specified display of whether or not to display and the specified display color.

[0101] The image viewer 150 acquires the FA image, the contour image of the fundus region, the contour image of the posterior pole NPA, the contour image of the UWF-NPA, and the contour image of the erroneously detected posterior pole NPA from the management server 140. Then, the contour image according to the operator's instruction is superimposed on the FA image and displayed. Note that the FA image may include a nonperfusion area (NPA) or the like. The display image on which any contour image is superimposed may be generated by either the management server 140 or the image viewer 150 .

[0102] The display form of the contour image displayed in the fundus image display field 504 of the screen 500 by the image viewer 150 is not limited to the display form shown in Fig. 12 and Fig. 13. For example, the thickness of the line of the contour image and the type of line, such as a dotted line or a solid line, may be changed. This enables the operator to display the contour image in a different color, change the type of line of the contour image, or change both the color and the type of line of the contour image so that the operator can confirm the differences between the fundus region, the posterior pole NPA, the UWF-NPA, and the erroneously detected posterior pole NPA.

[0103] The posterior pole NPA, which is displayed in the fundus image display section 504 on the screen 500 by the image viewer 150 and identified by the contour image of the posterior pole NPA, is one of the pieces of information that a doctor uses to diagnose diabetic retinopathy, diabetic macular edema, retinal vein occlusion, etc., and to judge and confirm the progress and effectiveness of treatment. In addition, the UWF-NPA identified by the contour image of the UWF-NPA can be one of the pieces of information that doctors can use to confirm early diagnosis. For example, it can be useful for ophthalmologists to diagnose conditions that develop from the peripheral part of the fundus or where observation of the fundus is useful for confirming the diagnosis, such as "preproliferative diabetic retinopathy," "proliferative diabetic retinopathy," "branch retinal vein occlusion," "Coats' disease," and "non-infectious uveitis." Furthermore, detection of posterior pole NPA can be useful in determining the effectiveness of treatments such as photocoagulation and whether or not additional surgery is necessary. Furthermore, visualization of UWF-NPA can be useful for early diagnosis of diabetic retinopathy and other conditions, understanding the progression of the disease, and quantitatively understanding the effects of drug treatments such as anti-VEGF (Vascular Endothelial Growth Factor) drugs (anti-angiogenic drugs) and blood pressure control.

[0104] Next, various modified examples of the technique of the present disclosure will be described.

[0105] <First Modification> In the above embodiment, the image processing control unit 206 executes the image processing of the posterior pole NPA (steps S106 and S108) and the image processing of the UWF-NPA (step S110), but the technology of the present disclosure is not limited to this. For example, as shown in Fig. 14, the image processing control unit 206 may execute only the image processing of the UWF-NPA. Each process shown in Fig. 14 is similar to the above, so a detailed description will be omitted.

[0106] The first modified example includes the following technical content.

[0107] (1) An image processing method, comprising the steps of: acquiring a fundus image; Extracting an unperfused area (UWF-NPA) in an area including a periphery of the fundus area from the fundus image; An image processing method comprising:

[0108] (2) An image display method, comprising: acquiring information about a fundus image and an unperfused area (UWF-NPA) in an area including a periphery of the fundus area extracted from the fundus image; a step of superimposing and displaying the non-perfusion region on the fundus image; An image display method comprising:

[0109] (3) An image processing program comprising: An image processing program that causes a computer to execute the image processing method according to (1).

[0110] (4) An image display program, An image display program for causing a computer to execute the image display method according to (2).

[0111] (5) An image processing device, a fundus image acquisition unit for acquiring a fundus image; a non-perfusion area extraction unit that extracts a non-perfusion area (UWF-NPA) in an area including a periphery of the fundus area from the fundus image; An image processing device comprising:

[0112] (6) An image display device, an acquisition unit that acquires a fundus image and information regarding a non-perfused area (UWF-NPA) in an area including a periphery of the fundus area extracted from the fundus image; a display unit that displays the non-perfusion region superimposed on the fundus image; An image display device comprising:

[0113] <Second Modification> In the above embodiment, the image processing control unit 206 executes image processing of the posterior pole NPA and image processing of the UWF-NPA, and in the first modified example, the image processing control unit 206 executes only image processing of the UWF-NPA, but the technology of the present disclosure is not limited to this. For example, as shown in Fig. 15, the image processing control unit 206 may execute only image processing of the posterior pole NPA. Each process shown in Fig. 15 is the same as the above, so detailed description will be omitted.

[0114] The second modified example includes the following technical content.

[0115] (7) An image processing method, comprising the steps of: acquiring a fundus image; Extracting a non-perfused area (NPA) from the fundus image in an area including a center of the fundus area; An image processing method comprising:

[0116] (8) The image processing method according to (7), extracting a region that may be a false positive from the extracted non-perfusion region; excluding the region that may be a false positive from the extracted non-perfusion region; The image processing method further comprises:

[0117] (9) An image display method, comprising: obtaining a fundus image and information regarding a non-perfused area (NPA) in a region including a center of the fundus region extracted from the fundus image; a step of superimposing and displaying the non-perfusion region on the fundus image; An image display method comprising:

[0118] (10) An image processing program comprising: An image processing program for causing a computer to execute the image processing method according to (7) or (8).

[0119] (11) An image display program, An image display program for causing a computer to execute the image display method according to (9).

[0120] (12) An image processing device, a fundus image acquisition unit for acquiring a fundus image; a non-perfusion area extraction unit that extracts a non-perfusion area (NPA) in a region including a posterior pole of the fundus from the fundus image; An image processing device comprising:

[0121] (13) An image display device, An acquisition unit that acquires a fundus image and information regarding a nonperfused area (NPA) in a region including a posterior pole of the fundus region extracted from the fundus image; a display unit that displays the non-perfusion region superimposed on the fundus image; An image display device comprising:

[0122] <Third modified example>

[0123] In the above embodiment, the management server 140 executes the image processing program shown in Fig. 7 in advance, but the technology of the present disclosure is not limited to this. The image viewer 150 may transmit an image processing command to the management server 140, and the management server 140 may execute the image processing program of Fig. 6 in response to the command.

[0124] <Fourth modified example>

[0125] In the above embodiment, an example has been described in which a fundus image with an internal light irradiation angle of about 200 degrees is acquired by the ophthalmic apparatus 110. The technology of the present disclosure is not limited to this, and the technology of the present disclosure may be applied to a fundus image captured by an ophthalmic apparatus with an internal irradiation angle of 100 degrees or less, or a montage image in which multiple fundus images are combined.

[0126] <Fifth modified example>

[0127] In the above embodiment, a fundus image is captured by the ophthalmologic apparatus 110 equipped with an SLO imaging unit, but the technology of the present disclosure may also be applied to an image obtained by OCT angiography.

[0128] <Sixth Modification>

[0129] In the above embodiment, the management server 140 executes the image processing program. However, the technology of the present disclosure is not limited to this. For example, the ophthalmic apparatus 110 or the image viewer 150 may execute the image processing program.

[0130] <Seventh Variation>

[0131] In the above embodiment, the ophthalmic system 100 including the ophthalmic device 110, the laser treatment device 120, the management server 140, and the image viewer 150 has been described as an example, but the technology of the present disclosure is not limited thereto. For example, as a first example, the laser treatment device 120 may be omitted, and the ophthalmic device 110 may further have the functions of the laser treatment device 120. Also, as a second example, the ophthalmic device 110 may further have at least one of the functions of the management server 140 and the image viewer 150. For example, if the ophthalmic device 110 has the functions of the management server 140, the management server 140 can be omitted. In this case, the image processing program is executed by the ophthalmic device 110 or the image viewer 150. Also, if the ophthalmic device 110 has the functions of the image viewer 150, the image viewer 150 can be omitted. As a third example, The management server 140 may be omitted, and the image viewer 150 may perform the functions of the management server 140 .

[0132] <Eighth Modification> In the above embodiment, the image processing control unit 206 detects the posterior pole NPA by executing the process shown in Fig. 9, but the technology of the present disclosure is not limited to this. For example, the posterior pole NPA may be detected by the process shown in Fig. 16.

[0133] With reference to FIG. 16, image processing of the posterior pole NPA will be described. 16, in step S1304, a process for removing retinal blood vessels is performed on the acquired FA image. Then, a Gaussian filter is used to remove high frequency components from the image in which the retinal blood vessels have been processed, to generate a low frequency component image.

[0134] Next, in step S1306, the image processing control unit 206 corrects the luminance of the peripheral part of the fundus by removing the low-frequency component image from the FA image. Then, in step S1308, the image processing control unit 206 performs an NPA detection process. More specifically, from the fundus image obtained in S1306 in which the brightness of the fundus peripheral portion has been corrected, a plurality of pixels having a first darkness or less are extracted, and a single or a plurality of areas in which the dark pixels having the first darkness or less are continuous and have an area of ​​a predetermined area or more are detected as posterior pole NPA. Then, in step S1310, the image processing control unit 206 extracts the contour of the detected posterior pole NPA, and ends the process.

[0135] In this manner, image processing for the posterior pole NPA in the eighth modified example is performed. Compared with the posterior pole NPA processing shown in Fig. 9, there is no need to perform extraction processing in steps S308 and S310, which enables high-speed processing and increases the number of fundus images processed per unit time.

[0136] <Ninth Variation> In the above embodiment, the image viewer 150 displays the screen 500 in which the contour image of the fundus region, the contour image of the posterior pole NPA, the contour image of the UWF-NPA, and the contour image of the erroneously detected posterior pole NPA are superimposed on the FA image, but the technology of the present disclosure is not limited to this. For example, an image in which the contour image of the fundus region, the contour image of the posterior pole NPA, the contour image of the UWF-NPA, and the contour image of the erroneously detected posterior pole NPA are superimposed on a color fundus image or an en-face image obtained from OCT data may be displayed. In this case, the FA image The contour images are aligned with other images such as a color fundus image, and each contour image is superimposed and displayed in the correct position on the color fundus image.

[0137] <Other Modifications>

[0138] The data processing described in the above embodiment is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed without departing from the spirit of the invention.

[0139] In the above embodiment, the data processing is realized by a software configuration using a computer, but the technology of the present disclosure is not limited to this. For example, instead of a software configuration using a computer, a FPGA (Field Programmable Gate Array (FPGA) grammable gate array) or ASIC (Application Specific Integrated Circuit The data processing may be performed only by a hardware configuration such as a specific integrated circuit (SCI) or a part of the data processing may be performed by a software configuration and the remaining part by a hardware configuration. You can do so. [Explanation of symbols]

[0140] 100 Ophthalmology System 110 Ophthalmological equipment 120 Laser Treatment Device 140 Management Server 150 Image Viewer 204 Display control unit 206 Image processing control unit 208 Processing section 262 CPU 254 Storage device

Claims

1. a fundus image acquisition unit for acquiring a fundus image of a subject's eye; a first extraction unit that extracts a candidate region that is a non-perfusion region from a posterior pole region that is a posterior pole part of the fundus from the fundus image; a tomographic image acquisition unit for acquiring a tomographic image of the subject's eye; a second extraction unit that extracts a false detection region that may be a false detection from the candidate region based on a position of a lesion region in the tomographic image; a specifying unit that specifies a non-perfusion region from the fundus image by excluding the erroneous detection region from the candidate region; 2. An image processing device comprising:

2. a fundus image acquisition unit for acquiring a fundus image of a subject's eye; a first extraction unit that extracts a candidate region that is a non-perfusion region from a posterior pole region that is a posterior pole part of the fundus from the fundus image; a laser treatment position acquisition unit for acquiring a laser treatment position on the subject's eye; A second extraction unit that extracts a false detection region that may be a false detection from the candidate region based on the laser treatment position; a specifying unit that specifies a non-perfusion region from the fundus image by excluding the erroneous detection region from the candidate region; 2. An image processing device comprising:

3. a fundus image acquisition unit for acquiring a fundus image of a subject's eye; a first extraction unit that extracts a candidate region that is a non-perfusion region from a posterior pole region that is a posterior pole part of the fundus from the fundus image; a second extraction unit that extracts a false detection area that may be a false detection from the candidate area based on an image filter using a known fundus image having any of photocoagulation spots, soft exudates, and spots caused by a lack of light reaching the fundus during photography; a specifying unit that specifies a non-perfusion region from the fundus image of the subject's eye by excluding the erroneous detection region from the candidate region; 2. An image processing device comprising:

4. The first extraction unit is The image processing device according to claim 1 , further comprising: an area along the blood vessels on the fundus extracted from the fundus image and having a size equal to or smaller than a predetermined threshold, the area being extracted as the candidate area.

5. The first extraction unit is The image processing apparatus according to claim 4 , wherein an average of pixel values ​​calculated from at least one of brightness, saturation, and hue of the plurality of candidate regions extracted from the fundus image is set as the threshold value.

6. The image processing device according to claim 1 , further comprising a third extraction unit that performs a process different from the process for extracting the candidate region on a peripheral region of the fundus image, which is a peripheral portion of the posterior pole, to extract a peripheral non-perfusion region.

7. The third extraction unit is The image processing apparatus according to claim 6 , wherein a region separated by a predetermined distance or more from the blood vessels on the fundus extracted from the fundus image is extracted as the peripheral nonperfusion region.

8. The image processing device according to claim 1 , wherein the fundus image is a fundus image photographed with a fluorescent light or an OCT image obtained by an optical coherence tomography.

9. On the computer, A procedure for acquiring a fundus image of a subject's eye; A procedure for acquiring a tomographic image of the subject's eye; A step of extracting a candidate region that is a non-perfusion region candidate from a posterior pole region that is a posterior pole part of the fundus from the fundus image; extracting a false positive region that may be a false positive from the candidate region based on a position of a lesion region in the tomographic image; a step of excluding the false positive region from the candidate region and identifying a non-perfusion region from the fundus image; A program that executes the following.

10. On the computer, A procedure for acquiring a fundus image of a subject's eye; obtaining a laser treatment position of the subject's eye; A step of extracting a candidate region that is a non-perfusion region candidate from a posterior pole region that is a posterior pole part of the fundus from the fundus image; extracting a false detection region that may be a false detection from the candidate region based on the laser treatment position; a step of excluding the erroneous detection region from the candidate region and identifying a non-perfusion region from the fundus image; A program that executes the following.

11. On the computer, A procedure for acquiring a fundus image of a subject's eye; A step of extracting a candidate region that is a non-perfusion region candidate from a posterior pole region that is a posterior pole part of the fundus from the fundus image; extracting a false positive region from the candidate region based on an image filter using a fundus image having any of photocoagulation spots, soft exudates, and spots caused by a lack of light reaching the image during imaging; a step of excluding the erroneous detection region from the candidate region and identifying a non-perfusion region from the fundus image; A program that executes the following.

Images