Image processing method, image processing apparatus, and program

Abstract

Provided is an image processing method for making an ocular fundus image clear. The image processing method executed by a processor includes: the processor acquiring an ocular fundus image; the processor performing a first enhancement process on an image of at least a central region of the ocular fundus image and performing a second enhancement process different from the first enhancement process on an image of at least a peripheral region of the ocular fundus image around the central region; and the processor generating an enhanced image of the ocular fundus image based on a first image obtained by performing the first enhancement process and a second image obtained by performing the second enhancement process.

Description

TECHNICAL FIELD

[0001] The present application relates to an image processing method, an image processing apparatus, and a program. BACKGROUND

[0002] An image processing technique that clearly depicts a blood vessel region of a fundus image is disclosed in JP Patent No. 2008-229157. An image processing technique that achieves clearification of a fundus image is sought. SUMMARY

[0003] A first aspect of the technology disclosed herein provides an image processing method executed by a processor, the image processing method including: the processor acquiring a fundus image; the processor performing first enhancement processing on an image of at least a central region of the fundus image and performing second enhancement processing different from the first enhancement processing on an image of at least a peripheral region of the fundus image that is peripheral to the central region; and the processor generating an enhanced image of the fundus image based on a first image obtained by performing the first enhancement processing and a second image obtained by performing the second enhancement processing.

[0004] A second aspect of the technology disclosed herein provides an image processing method executed by a processor, the image processing method including: the processor acquiring a fundus image; the processor performing first clearification processing using a first parameter with respect to the fundus image and performing second clearification processing using a second parameter different from the first parameter with respect to the fundus image; and the processor generating a clearified fundus image based on an image obtained by performing the first clearification processing and the second clearification processing.

[0005] An image processing apparatus of a third aspect of the technology disclosed herein includes a processor and a memory connected to the processor, the processor configured to acquire a fundus image, perform first enhancement processing on an image of at least a central region of the fundus image and perform second enhancement processing different from the first enhancement processing on an image of at least a peripheral region of the fundus image that is peripheral to the central region, and generate an enhanced image of the fundus image based on a first image obtained by performing the first enhancement processing and a second image obtained by performing the second enhancement processing.

[0006] The image processing apparatus of the fourth aspect of the technology of the present disclosure includes a processor configured to acquire an eye fundus image, perform a first sharpening process using a first parameter with respect to the eye fundus image, perform a second sharpening process using a second parameter different from the first parameter with respect to the eye fundus image, and generate a sharpened eye fundus image based on images obtained by performing the first sharpening process and the second sharpening process.

[0007] The program of the fifth aspect of the technology of the present disclosure causes a computer to perform: acquiring an eye fundus image, performing a first enhancement process on an image of at least a central region of the eye fundus image and performing a second enhancement process different from the first enhancement process on an image of at least a peripheral region of the eye fundus image, and generating an enhanced image of the eye fundus image based on a first image obtained by performing the first enhancement process and a second image obtained by performing the second enhancement process.

[0008] The program of the sixth aspect of the technology of the present disclosure causes a computer to perform: acquiring an eye fundus image, performing a first sharpening process using a first parameter with respect to the eye fundus image, performing a second sharpening process using a second parameter different from the first parameter with respect to the eye fundus image, and generating a sharpened eye fundus image based on images obtained by performing the first sharpening process and the second sharpening process. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a block diagram of an ophthalmic system 100.

[0010] Figure 2 is a schematic configuration diagram showing the overall configuration of an ophthalmic apparatus 110.

[0011] Figure 3 is a block diagram of an electrical configuration of a server 140.

[0012] Figure 4 is a block diagram of the functions of a CPU 262 of the server 140.

[0013] Figure 5 is a flowchart of image processing based on the server 140.

[0014] Figure 6A is a flowchart of the sharpening process of step 504 of Figure 5

[0015] Figure 6B is a flowchart of the sharpened image analysis process of step 506 of Figure 5

[0016] Figure 7 ​​is a diagram showing a UWF fundus image G1 in an RGB color space.

[0017] Figure 8A is a diagram showing an image G11 of an L* component of an image in an L*a*b* color space converted from the UWF fundus image G1 in the RGB color space.

[0018] Figure 8B is a diagram showing a central region and a peripheral region in a fundus image.

[0019] Figure 9 is a diagram showing a first tile size Tc for CLAHE processing for an image of the central region of the image G11 of the L* component, and a second tile size Tp for CLAHE processing for an image of the peripheral region.

[0020] Figure 10 is a diagram showing a UWF fundus image G2 in an RGB color space after sharpening processing.

[0021] Figure 11 is a diagram showing an enhanced image in which a lesion portion is enhanced.

[0022] Figure 12 is a diagram showing a first fundus image display screen 1000A.

[0023] Figure 13 is a diagram showing a second fundus image display screen 1000B.

[0024] Figure 14 is a diagram showing a third fundus image display screen 1000C.

[0025] Figure 15 is a flowchart of a modification example of the sharpening processing of step 504 of Figure 5 DETAILED DESCRIPTION

[0026] Hereinafter, an embodiment of the present application will be explained in detail with reference to the drawings.

[0027] Reference will be made to Figure 1 The configuration of an ophthalmic system 100 will be explained. As shown in Figure 1 The ophthalmic system 100 is provided with an ophthalmic device 110, an ocular axial length measurer 120, a management server device (hereinafter referred to as "server") 140, and an image display device (hereinafter referred to as "viewer") 150. The ophthalmic device 110 acquires a fundus image. The ocular axial length measurer 120 measures an ocular axial length of a patient. The server 140 stores a fundus image obtained by photographing a fundus of a patient with the ophthalmic device 110 in correspondence with an ID of the patient. The viewer 150 displays medical information such as the fundus image acquired from the server 140.​

[0028] The ophthalmic apparatus 110, the axial length measuring apparatus 120, the server 140, and the viewer 150 are connected to each other via the network 130.

[0029] Next, the configuration of the ophthalmic apparatus 110 will be described with reference to Figure 2 The ophthalmic apparatus 110 is configured as follows.

[0030] For convenience of explanation, a scanning laser ophthalmoscope will be referred to as "SLO". In addition, optical coherence tomography will be referred to as "OCT".

[0031] Further, a horizontal direction in a case where the ophthalmic apparatus 110 is disposed on a horizontal plane will be referred to as an "X direction", a vertical direction with respect to the horizontal plane will be referred to as a "Y direction", and a direction linking the center of the pupil of the anterior segment of the eye of the eye under examination 12 and the center of the eyeball will be referred to as a "Z direction". Thus, the X direction, the Y direction, and the Z direction are perpendicular to each other.

[0032] The ophthalmic apparatus 110 includes a photographing apparatus 14 and a control apparatus 16. The photographing apparatus 14 has an SLO unit 18, an OCT unit 20, and a photographing optical system 19, and acquires a fundus image of a fundus of the eye under examination 12. Hereinafter, a two-dimensional fundus image acquired by the SLO unit 18 will be referred to as an SLO image. In addition, a tomographic image or an en-face image of the retina, or the like, created on the basis of OCT data acquired by the OCT unit 20 will be referred to as an OCT image.

[0033] The control apparatus 16 is provided with a computer having a CPU (Central Processing Unit) 16A, a RAM (Random Access Memory) 16B, a ROM (Read-Only memory) 16C, and an input / output (I / O) port 16D.

[0034] The control apparatus 16 is provided with an input / display apparatus 16E connected to the CPU 16A via the I / O port 16D. The input / display apparatus 16E has a graphical user interface that displays an image of the eye under examination 12 or receives various instructions from a user. As the graphical user interface, a touch panel, a display, or the like is exemplified.

[0035] Further, the control device 16 has an image processing device 16G connected to the I / O port 16D. The image processing device 16G generates an image of the examined eye 12 based on data obtained by the photographing device 14. The control device 16 has a communication interface (I / F) 16F connected to the I / O port 16D. The ophthalmic device 110 is connected to the axial length measurer 120, the server 140, and the viewer 150 via the communication interface (I / F) 16F and the network 130.

[0036] As described above, in Figure 2 which the control device 16 of the ophthalmic device 110 has the input / display device 16E, the technology of the present disclosure is not limited thereto. For example, the control device 16 of the ophthalmic device 110 does not have the input / display device 16E, and the ophthalmic device 110 can have a physically independent separate input / display device. In this case, the display device has an image processing processor unit that operates under the control of the CPU 16A of the control device 16. The image processing processor unit can also display an SLO image or the like based on an image signal outputted by the CPU 16A.

[0037] The photographing device 14 operates under the control of the CPU 16A of the control device 16. The photographing device 14 includes the SLO unit 18, a photographing optical system 19, and the OCT unit 20. The photographing optical system 19 includes a first optical scanner 22, a second optical scanner 24, and a wide-angle optical system 30.

[0038] The first optical scanner 22 performs two-dimensional scanning of light emitted from the SLO unit 18 in the X direction and the Y direction. The second optical scanner 24 performs two-dimensional scanning of light emitted from the OCT unit 20 in the X direction and the Y direction. The first optical scanner 22 and the second optical scanner 24 can be optical elements that can deflect a light beam, and for example, a polygon mirror or a galvanometer mirror or the like can be used. Further, a combination of these can also be used.

[0039] The wide-angle optical system 30 includes an object optical system (not illustrated in Figure 2 ) having a common optical system 28, and a combining section 26 that combines light from the SLO unit 18 and light from the OCT unit 20.

[0040] Further, the object optical system of the common optical system 28 can be a reflective optical system using a concave mirror or the like, or a refractive optical system using a wide-angle lens or the like, or a catadioptric optical system combining a concave mirror or a lens. By using a wide-angle optical system using an ellipsoidal mirror or a wide-angle lens or the like, not only the fundus center portion in which the optic papilla or the macula is present, but also the fundus peripheral portion in which the equator of the eyeball or the vortex vein is present can be imaged.

[0041] In the case of using a system including an ellipsoidal mirror, the configuration of a system using an ellipsoidal mirror recorded in International Publication WO2016 / 103484 or International Publication WO2016 / 103489 can also be used. The disclosure of International Publication WO2016 / 103484 and the respective disclosures of International Publication WO2016 / 103489 are incorporated by reference in their entirety in the present specification.

[0042] With the wide-angle optical system 30, observation based on a wide field of view (FOV: Field of View) 12A is realized at the fundus. The FOV 12A shows a range that can be captured with the photographing device 14. The FOV 12A can be expressed as an angle of view. The angle of view can be defined with an internal illumination angle and an external illumination angle in the present embodiment. The external illumination angle refers to an illumination angle defined with the pupil 27 as a reference with respect to an illumination angle of a light beam that is illuminated from the ophthalmic apparatus 110 to the examined eye 12. In addition, the internal illumination angle refers to an illumination angle defined with the eyeball center O as a reference with respect to an illumination angle of a light beam that is illuminated to the fundus. The external illumination angle and the internal illumination angle are in a corresponding relationship. For example, in the case where the external illumination angle is 120 degrees, the internal illumination angle approximately corresponds to 160 degrees. In the present embodiment, the internal illumination angle is set to 200 degrees.

[0043] 200 of the internal illumination angle is an example of a "defined value" of the technology of the present disclosure.

[0044] Here, an SLO fundus image that is captured with the internal illumination angle and with a captured angle of view of 160 degrees or more is referred to as a UWF-SLO fundus image. In addition, UWF refers to an abbreviation of UltraWide Field.

[0045] The SLO system is realized by Figure 2 The control device 16, the SLO unit 18, and the photographing optical system 19 shown are realized. The SLO system is provided with the wide-angle optical system 30, and thus, fundus photography based on a wide FOV 12A can be realized.

[0046] The SLO unit 18 has optical systems 48, 50, 52, 54, 56 that reflect or transmit the light of the light sources 40, 42, 44, 46 and guide the light to one optical path. The optical systems 48, 50, 56 are mirrors or transmissive mirrors, and the optical systems 52, 54 are beam splitters. The B light is reflected by the optical system 48, transmitted by the optical system 50, and reflected by the optical system 54, the G light is reflected by the optical systems 50, 54, the R light is transmitted by the optical systems 52, 54, and the IR light is reflected by the optical systems 56, 52, respectively, and guided to one optical path.

[0047] The SLO unit 18 is configured to be able to switch the combination of the light sources or light emitting sources that emit laser light of different wavelengths, such as a mode that emits G light, R light, and B light, a mode that emits infrared light, and the like. In the present embodiment, the SLO unit 18 is configured to be able to switch the combination of the light sources or light emitting sources that emit laser light of different wavelengths by switching the optical systems 48, 50, 52, 54, 56. Figure 2 In the illustrated example, there are four light sources, the B light (blue light) light source 40, the G light light source 42, the R light light source 44, and the IR light light source 46, but the technology of the present disclosure is not limited to this. For example, the SLO unit 18 can also have a white light light source, and emit light using various modes such as a mode that emits only white light.

[0048] The light incident on the photographing optical system 19 from the SLO unit 18 is scanned in the X direction and the Y direction by the first optical scanner 22. The scanned light is irradiated to the posterior eye portion of the eye to be examined 12 via the wide-angle optical system 30 and the pupil 27. The reflected light reflected by the fundus is incident on the SLO unit 18 via the wide-angle optical system 30 and the first optical scanner 22.

[0049] The SLO unit 18 has a beam splitter 64 that reflects B light among the light from the posterior eye portion (e.g., the fundus) of the eye to be examined 12 and transmits light other than the B light, and a beam splitter 58 that reflects G light among the light transmitted by the beam splitter 64 and transmits light other than the G light. The SLO unit 18 has a beam splitter 60 that reflects R light among the light transmitted by the beam splitter 58 and transmits light other than the R light. The SLO unit 18 has a beam splitter 62 that reflects IR light among the light transmitted by the beam splitter 60.

[0050] The SLO unit 18 has a plurality of light detection elements corresponding to the plurality of light sources. The SLO unit 18 has a B light detection element 70 that detects the B light reflected by the beam splitter 64, and a G light detection element 72 that detects the G light reflected by the beam splitter 58. The SLO unit 18 has an R light detection element 74 that detects the R light reflected by the beam splitter 60, and an IR light detection element 76 that detects the IR light reflected by the beam splitter 62.

[0051] In a case where the light (reflected light reflected by the fundus) incident on the SLO unit 18 via the wide-angle optical system 30 and the first optical scanner 22 is B light, it is reflected by the beam splitter 64 and received by the B light detection element 70, in a case where it is G light, it is transmitted through the beam splitter 64 and reflected by the beam splitter 58 and received by the G light detection element 72. In a case where the above-mentioned incident light is R light, it is transmitted through the beam splitters 64, 58, reflected by the beam splitter 60 and received by the R light detection element 74. In a case where the above-mentioned incident light is IR light, it is transmitted through the beam splitters 64, 58, 60, reflected by the beam splitter 62 and received by the IR light detection element 76. The image processing device 16G, which operates under the control of the CPU 16A, generates a UWF-SLO image using the signals detected by the B light detection element 70, the G light detection element 72, the R light detection element 74, and the IR light detection element 76.

[0052] The UWF-SLO image exists as a UWF-SLO image obtained by photographing the fundus with G color (G color fundus image) and a UWF-SLO image obtained by photographing the fundus with R color (R color fundus image). The UWF-SLO image exists as a UWF-SLO image obtained by photographing the fundus with B color (B color fundus image) and a UWF-SLO image obtained by photographing the fundus with IR (IR fundus image).

[0053] In addition, the control device 16 controls the light sources 40, 42, 44 in a simultaneous light emission manner. The fundus of the eye under examination 12 is simultaneously photographed with B light, G light, and R light, whereby G color fundus images, R color fundus images, and B color fundus images corresponding to each position are obtained. An RGB color fundus image is derived from the G color fundus images, the R color fundus images, and the B color fundus images. The control device 16 controls the light sources 42, 44 in a simultaneous light emission manner, and the fundus of the eye under examination 12 is simultaneously photographed with G light and R light, whereby G color fundus images and R color fundus images corresponding to each position are obtained. An RG color fundus image is derived from the G color fundus images and the R color fundus images.

[0054] As such, as the UWF-SLO image, specifically, there exist the B color fundus image, the G color fundus image, the R color fundus image, the IR fundus image, the RGB color fundus image, and the RG color fundus image. The respective image data of the UWF-SLO image and the information of the patient input via the input / display device 16E are transmitted from the ophthalmic device 110 to the server 140 via the communication interface (I / F) 16F. The respective image data of the UWF-SLO image and the information of the patient are stored in the storage device 254 in correspondence. Further, the information of the patient exists as, for example, a patient name ID, a name, an age, a visual acuity, a distinction between the right eye and the left eye, and the like. The operator inputs the information of the patient via the input / display device 16E.

[0055] The OCT system utilizes Figure 2 The control device 16, the OCT unit 20, and the photographing optical system 19 are implemented as illustrated. The OCT system is provided with a wide-angle optical system 30, and thus, as with the photographing of the SLO fundus image described above, fundus photographing based on the wide FOV 12A can be implemented. The OCT unit 20 includes a light source 20A, a sensor (detection element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimator lens 20E, and a second optical coupler 20F.

[0056] The light emitted from the light source 20A is branched by the first optical coupler 20C. The light on the one side after the branching is made into parallel light by the collimator lens 20E after being used as measurement light, and then is incident to the photographing optical system 19. The measurement light is scanned in the X direction and the Y direction by the second optical scanner 24. The scanned light is irradiated to the fundus via the wide-angle optical system 30 and the pupil 27. The measurement light reflected by the fundus is incident to the OCT unit 20 via the wide-angle optical system 30 and the second optical scanner 24, and is incident to the second optical coupler 20F via the collimator lens 20E and the first optical coupler 20C.

[0057] The other light branched by the first optical coupler 20C from the light source 20A is incident to the reference optical system 20D as reference light, and is incident to the second optical coupler 20F via the reference optical system 20D.

[0058] These lights incident to the second optical coupler 20F, that is, the measurement light reflected by the fundus and the reference light are interfered by the second optical coupler 20F to generate interference light. The interference light is received by the sensor 20B. An image processing device 16G that operates under the control of the CPU 16A generates an OCT image such as a tomographic image or an en-face image based on the OCT data detected by the sensor 20B.

[0059] Here, the OCT fundus image obtained by photographing at an internal illumination angle and with a photographing angle of view of 160 degrees or more is referred to as a UWF-OCT image.

[0060] The image data of the UWF-OCT image and the information of the patient are transmitted from the ophthalmic device 110 to the server 140 via a communication interface (I / F) 16F. The image data of the UWF-OCT image and the information of the patient are stored in the storage device 254 in correspondence.

[0061] Furthermore, in this embodiment, the light source 20A is SS-OCT (Swept-SourceOCT) of the wavelength scanning type, but it can also be an OCT system of various types such as SD-OCT (Spectral-Domain OCT) and TD-OCT (Time-Domain OCT).

[0062] Next, the axial length measuring device 120 will be described. The axial length measuring device 120 has two modes: a first mode and a second mode, which measure the axial length of the examined eye 12 in the axial direction. In the first mode, after directing light from a light source (not shown) to the examined eye 12, the device receives interference light from reflected light from the fundus and reflected light from the cornea, and measures the axial length based on the interference signal representing the received interference light. The second mode is a mode that uses ultrasound (not shown) to measure the axial length.

[0063] The axial length measuring device 120 sends the axial length measured using either the first mode or the second mode to the server 140. Alternatively, the axial length can be measured using both the first and second modes; in this case, the average of the axial lengths measured using both modes is sent to the server 140 as the axial length. The server 140 stores the patient's axial length corresponding to their patient name ID.

[0064] Next, refer to Figure 3 Describe the electrical configuration of server 140. For example... Figure 3 As shown, server 140 includes computer body 252. Computer body 252 has a CPU 262, RAM 266, ROM 264, and input / output (I / O) port 268 interconnected via bus 270. Input / output (I / O) port 268 is connected to storage device 254, display 256, mouse 255M, keyboard 255K, and communication interface (I / F) 258. Storage device 254 is constructed, for example, from non-volatile memory. Input / output (I / O) port 268 is connected to network 130 via communication interface (I / F) 258. Therefore, server 140 can communicate with ophthalmic device 110 and observer 150. Storage device 254 stores image processing program described later. Alternatively, the image processing program can be stored in ROM 264.

[0065] The image processing program is an example of a "program" according to the present disclosure. Storage device 254 and ROM 264 are examples of a "memory" or "computer-readable storage medium" according to the present disclosure. CPU 262 is an example of a "processor" according to the present disclosure.

[0066] The processing unit 208 of server 140 (see also)Figure 5 ) stores each data received from the ophthalmic apparatus 110 in the storage device 254. Specifically, the processing section 208 stores each image data of the UWF-SLO images and the image data of the UWF-OCT images and the information of the patient (the patient name ID and the like as described above) in the storage device 254 in correspondence. In addition, in a case where there is a lesion in the examined eye of the patient or a case where a surgery is performed on the lesion portion, the information of the lesion is input via the input / display device 16E of the ophthalmic apparatus 110 and transmitted to the server 140. The information of the lesion is stored in the storage device 254 in correspondence with the information of the patient. The information of the lesion includes the information of the position of the lesion portion, the name of the lesion, and in a case where a surgery is performed on the lesion portion, the name of the surgery, the date and time of the surgery, and the like.

[0067] The viewer 150 is provided with a computer having a CPU, a RAM, a ROM, and the like, and a display, and an image processing program is installed in the ROM, and based on an instruction of a user, the computer controls the display in a manner of displaying medical information such as a fundus image acquired from the server 140.

[0068] Next, various functions realized by the CPU 262 of the server 140 executing the image processing program will be described with reference to Figure 4 . The image processing program is provided with a display control function, an image processing function (a sharpening processing function, a fundus structure analysis function), and a processing function. The CPU 262 executes the image processing program having each function, and thereby the CPU 262 functions as the display control section 204, the image processing section 206 (the sharpening processing section 2060, the fundus structure analysis section 2062), and the processing section 208 as shown in Figure 4 .

[0069] Next, the image processing based on the server 140 will be described in detail using Figure 5 . The image processing shown in the flowchart of Figure 5 is realized by causing the CPU 262 of the server 140 to execute the image processing program. The image processing is started in a case where a patient name ID is input and an unillustrated start button displayed on the display 256 is operated.

[0070] In step 502, the processing section 208 acquires the UWF fundus image G1 stored in correspondence with the patient name ID from the storage device 254 as shown in Figure 7 . The UWF fundus image G1 is an RGB color fundus image in the UWF-SLO images, is a UWF fundus image in the RGB color space, and is a raw UWF fundus image on which no image processing is performed.

[0071] In step 504, the sharpening processing section 2060 performs a sharpening process, the details of which will be described later. In step 506, the fundus structure analysis section 2062 analyzes the sharpened image, the details of which will be described later.

[0072] In step 508, the processing section 208 generates image identification flags, and sets the image identification flags with respect to the original UWF fundus image (UWF fundus image G1 (refer to Figure 7 )) and the UWF fundus image after the sharpening process. Specifically, the processing section 208 establishes a correlation flag = 0 with respect to the original UWF fundus image. The processing section 208 establishes a correlation flag = 1 with respect to the UWF fundus image after the sharpening process (sharpened image).

[0073] In step 510, the processing section 208 saves (stores) the original UWF fundus image in the storage device 254 in correspondence with the flag = 0, and saves (stores) the sharpened image in the storage device 254 in correspondence with the flag = 1.

[0074] In step 512, the processing section 208 saves (stores) data of the analysis result obtained by the analysis performed by the fundus structure analysis section 2062 with respect to the UWF fundus image after the sharpening process in the storage device 254.

[0075] In step 514, the processing section 208 outputs (transmits) the original UWF fundus image to the viewer 150 in a state in which the correlation flag = 0 is established, and outputs (transmits) the UWF fundus image after the sharpening process to the viewer 150 in a state in which the correlation flag = 1 is established, to the viewer 150, and also outputs (transmits) the analysis data to the viewer 150 via the communication interface 258.

[0076] Next, the process of step 504 will be described with reference to Figure 6A to perform the sharpening process of step 504.

[0077] In step 602, the sharpening processing section 2060 converts the UWF fundus image G1 (refer to Figure 7 ) in the RGB color space into an image in the L*a*b* color space. Thus, an image G11 (refer to Figure 8A ) of the L* component, an image of the a* component, and an image of the b* component are obtained.

[0078] The L*a*b* color space is an example of the "color space having three components of a lightness component representing luminance, and a first color component and a second color component which are different two chroma components" of the technology of the present disclosure. Since the image G11 (refer to Figure 8AThe image of the a* component and the image of the b* component are each an image of the fundus, so the color space fundus image having three components of luminance, two different chrominances is acquired by the process of step 602.

[0079] That is, the Lab space fundus image is converted from the RGB fundus image in the three primary color space (RGB) to the color complementary space (L*a*b*), and the sharpening process of step 504 is performed.

[0080] The color complementary space (L*a*b*) is also called the CIELAB color space or "CIE 1976 L*a*b* color space".

[0081] In addition, it is also possible to convert to the CIELUV color space (or "CIE 1976 L*u*v* color space") as another color space.

[0082] In order for the user to more effectively recognize the sharpening effect, it is preferable to perform the sharpening process in the color complementary space (L*a*b*) in which the difference in the recognition of color is converted to a quantitative difference, because the user (the observer who views the image (for example, an ophthalmologist)) can more effectively recognize the sharpening effect. That is, it is more preferable to cognitively separate the independent luminance component and the chrominance component in the color complementary space (L*a*b*) and perform the sharpening process on each component independently, compared to performing the process with respect to each color component of R, G, B in the RGB color space. Therefore, it is possible to generate an image in which the difference in recognition is effectively perceived using the sharpening process in the color complementary space (L*a*b*). Therefore, the sharpening process of the technology of the present disclosure is preferably performed after being converted to the color complementary space (L*a*b*).

[0083] In step 604, the sharpening processing section 2060 extracts an image of a region of a circle of a prescribed radius with the center of the image of the L*a*b* color space as the center from each image of the L*a*b* color space as an image of a central region.

[0084] Here, the central region refers to a region including the optic disc and the macula in the fundus image, and is the posterior pole of the fundus. The peripheral region refers to a region outside the central region, and is the equator and the peripheral region of the fundus. Specifically, as shown in FIG. 6, the region surrounded by the dotted circle including the optic disc DD and the macula MM is the central region CR, and the outside of the dotted circle is the peripheral region. Figure 8B

[0085] Further, the central position of the image of the L*a*b* color space is the position at which the fundus of the eye 12 under examination intersects the optical axis.

[0086] ​In step 606, the sharpening processing unit 2060 sets the first parameter as a parameter for the CLAHE processing performed relative to the image of the central region.

[0087] Here, CLAHE (Contrast Limited Adaptive Histogram Equalization) refers to the process of dividing an image into multiple regions, performing histogram smoothing on each region, and then performing interpolation processes such as bilinear interpolation at the boundaries of each region to adjust the image's contrast.

[0088] The CLAHE processing parameter includes the tile size. CLAHE processing is performed locally on an image (an image in L*a*b* color space). Specifically, the image is divided into multiple quadrilateral regions, and CLAHE processing is performed on each region. Each region is called a tile, and the size of the tile is called the tile size.

[0089] In this embodiment, in step 606, as Figure 9 As shown, the sharpening processing unit 2060 sets a first tile size Tc as the tile size. This will be explained in detail later, but a second tile size Tp is set relative to the image of the peripheral region located around the central region. Relative to the first tile size Tc used for CLAHE processing in the central region, the second tile size Tp used for CLAHE processing in the peripheral region has a tile side length (the tile is a square) that is 1.4 times that of Tc, and its area is set to twice that of Tc. That is, the size of the first tile size Tc is smaller than the second tile size Tp. This is because, in order to capture the spherical fundus, the peripheral region is distorted relative to the central region.

[0090] Thus, the original UWF fundus image G1 (obtained by shooting at an internal illumination angle and a shooting angle of more than 160 degrees) is shown. Figure 7 In the image, the peripheral region shows greater distortion compared to the central region. In other words, the actual fundus area corresponding to the same specified area in the original UWF fundus image G1 is larger in the peripheral region than in the central region.

[0091] Therefore, in this embodiment, the size of the second block Tp is larger than the size of the first block Tc.

[0092] Specifically, the actual fundus area corresponding to the central region of the original UWF fundus image G1 is set as Sc, and the actual fundus area corresponding to the peripheral region of the original UWF fundus image G1 is set as Sp. When Sp = n2Sc, it is set as Tp = n2Tc.

[0093] Therefore, the degree of enhancement of the CLAHE processing based on the components of L*a*b* of the image of the peripheral region is set to be larger than the degree of enhancement of the CLAHE processing based on the components of L*a*b* of the image of the central region.

[0094] In step 608, the sharpening processing section 2060 performs the CLAHE processing with the 1st parameter with respect to the components of L*a*b* of the image of the central region.

[0095] In step 610, the sharpening processing section 2060 extracts the image of the region other than the central region from the image of the L*a*b* color space as the image of the peripheral region.

[0096] In step 612, the sharpening processing section 2060 sets the 2nd parameter as the parameter of the CLAHE processing performed with respect to the image of the peripheral region. In the present embodiment, the sharpening processing section 2060 sets the 2nd tile size Tp larger than the 1st tile size Tc as the tile size.

[0097] In step 614, the sharpening processing section 2060 performs the CLAHE processing with the 2nd parameter with respect to the image of the peripheral region of the components of L*a*b* of the image of the peripheral region.

[0098] In step 616, the sharpening processing section 2060 synthesizes the components of the respective images of the central region and the peripheral region. In step 618, the sharpening processing section 2060 synthesizes the respective images of the central region and the peripheral region whose components are synthesized. Thus, the sharpened UWF fundus image of the L*a*b* color space is obtained.

[0099] In step 620, as shown in Figure 10 , the sharpening processing section 2060 performs the image G2 conversion of the RGB color space with respect to the sharpened UWF fundus image of the L*a*b* color space.

[0100] In the image G2( Figure 10 ) of the RGB color space, the blood vessels are sharpened, and the blood vessels that are difficult to see in the original UWF fundus image G1( Figure 7 ) become visible. In addition, the lesion portions such as hemorrhage, white spots, and retinal detachment are also sharpened.

[0101] Next, the analysis processing of the sharpened image of step 506 is described with reference to Figure 6B .

[0102] In step 652, the fundus structure analysis section 2062 determines whether information on the lesion is registered in correspondence with the patient name ID that is the current object.

[0103] In the case where information on the lesion is not registered in correspondence with the patient name ID that is the current object, the analysis processing is ended.

[0104] In the case where information on the lesion is registered in correspondence with the patient name ID that is the current object, the fundus structure analysis section 2062 generates an enhanced image that enhances the lesion portion, and stores it in the storage device 254 as analysis data.

[0105] As described above, the analysis data also is output (transmitted) to the viewer 150 in step 508, and therefore, in the viewer 150, an image that enhances the lesion portion shown in FIG. 8 is displayed. Thus, the lesion portion is visualized. Figure 5 Figure 11 As described above, the analysis data also is output (transmitted) to the viewer 150 in step 508, and therefore, in the viewer 150, an image that enhances the lesion portion shown in FIG. 8 is displayed. Thus, the lesion portion is visualized.

[0106] For example, first, in the case where there is a lesion portion in the vitreous body and surgery is performed on the lesion portion, as shown in FIG. 7, the fundus structure analysis section 2062 generates an enhanced image that changes the color of the retinal blood vessels of the surgery site 702, or superimposes the retinal blood vessels before the surgery. Figure 11

[0107] The viewer 150 displays the enhanced image according to the instruction of the ophthalmologist at the time of diagnosing the examined eye. The viewer 150 changes the color to display the retinal blood vessels of the surgery site 702, or causes the retinal blood vessels of the surgery site 702 to overlap the retinal blood vessels before the surgery to display. Thus, the misalignment of the retinal blood vessels of the surgery site 702 and the retinal blood vessels before the surgery can be confirmed. The viewer 150 can also display only the retinal blood vessels after the surgery of the surgery site 702, or only the retinal blood vessels before the surgery of the surgery site 702, or alternately display these.

[0108] Second, in the case where there is a lesion of retinal detachment, the fundus structure analysis section 2062 creates an enhanced image that superimposes a red frame on the site of retinal detachment 708.

[0109] The viewer 150 displays the enhanced image according to the instruction of the ophthalmologist at the time of diagnosing the examined eye. In this case, the viewer 150 can also flicker display the site of retinal detachment 708, or reverse flicker display, or display the position and size.

[0110] Third, in the case where there is a lesion of punctate hemorrhage, the fundus structure analysis section 2062 changes the color of the site of punctate hemorrhage 706, or instead of changing the color, or on this basis, creates an enhanced image that superimposes the number of sites of punctate hemorrhage. ​​

[0111] The observer 150 displays the enhanced image according to the instruction of the ophthalmologist at the time of diagnosing the eye under examination. In this case, the observer 150 can flicker display the site of the pinpoint hemorrhage 706, or reverse flicker display the site of the pinpoint hemorrhage 706, or display the count value of the number. The observer 150 can display the convex lens button M in the case of clicking the vicinity of the site of the pinpoint hemorrhage 706, and when the convex lens button is pressed, the site of the pinpoint hemorrhage 706 is displayed in magnification.

[0112] Fourth, in the case of a lesion in which a white spot is present, the fundus structure analysis section 2062 changes the color of the site of the white spot 704A, 704B, or instead of changing the color, or on this basis, creates an enhanced image in which the number of white spots is superimposed.

[0113] The observer 150 displays the enhanced image according to the instruction of the ophthalmologist at the time of diagnosing the eye under examination. In this case, the observer 150 can flicker display the site of the pinpoint hemorrhage 706, or reverse flicker display the site of the pinpoint hemorrhage 706, or display the count value of the number. The observer 150 can display the convex lens button M in the case of clicking the vicinity of the site of the pinpoint hemorrhage 706, and when the convex lens button is pressed, the site of the pinpoint hemorrhage 706 is displayed in magnification.

[0114] The ophthalmologist inputs the patient name ID to the observer 150 at the time of diagnosing the eye under examination of the patient. The observer 150 instructs in such a manner that the image data and the like of the eye under examination corresponding to the patient name ID are transmitted to the server 140. The server 140 transmits the patient name, the patient's age, the patient's visual acuity, information of whether it is the left eye or the right eye, the axial length, the photographing date, and the image data corresponding to the patient name ID to the observer 150 together with the patient name ID.

[0115] The observer 150 that receives the patient name ID, the patient name, the patient's age, the patient's visual acuity, information of whether it is the left eye or the right eye, the axial length, the photographing date, and the image data displays Figure 12 The 1st fundus image display screen 1000A shown is displayed on the display.

[0116] As shown in Figure 12 The 1st fundus image display screen 1000A has a patient information display field 1002, and a 1st fundus image information display field 1004A.

[0117] The patient information display field 1002 has respective display fields 1012 to 1022 for displaying the patient name ID, the patient name, the patient's age, the patient's visual acuity, information of whether it is the left eye or the right eye, the axial length, and a screen switching button 1024. The display fields 1012 to 1022 display the received patient name ID, the patient name, the patient's age, the patient's visual acuity, information of whether it is the left eye or the right eye, and the axial length.

[0118] The first fundus image information display field 1004A has a photographing date display field 1030, an original UWF fundus image display field 1032A, a sharpened UWF fundus image display field 1032B, and an information display field 1034, and selection buttons 1036A to 1036D.

[0119] The photographing date display field 1030 displays the photographing date (YYY / MM / DD). The information display field 1034 displays the user's (ophthalmologist's) comments or records at the time of diagnosis as text.

[0120] The original UWF fundus image display field 1032A displays the original UWF fundus image G1( Figure 7 ). The sharpened UWF fundus image display field 1032B displays the sharpened UWF fundus image G2( Figure 10 ).

[0121] The mode 1 selected by the selection button 1036A is a chroma fundus camera style correction mode. The chroma fundus camera style correction mode is a mode in which the chroma of the sharpened UWF fundus image G2 displayed in the sharpened UWF fundus image display field 1032B is changed to the chroma of an image captured by a fundus camera.

[0122] The mode 2 selected by the selection button 1036B is a mist removal processing mode. The mist removal processing mode is a mode in which mist (for example, fog, etc.) is removed from the sharpened UWF fundus image G2 (mist removal processing). The mist removal processing is disclosed in the following papers and patent documents.

[0123] (Papers)

[0124] He, Kaiming. "Single Image Haze Removal Using Dark Channel Prior." Thesis, The Chinese University of Hong Kong, 2011.

[0125] (Patent Documents)

[0126] JP Patent No. 6225255

[0127] The mode 3 selected by the selection button 1036C is a pachy-choroid confirmation mode. The pachy-choroid confirmation mode is a mode in which the proportion of the red component is enhanced compared to the proportions of the components of other colors (green, blue) in the sharpened UWF fundus image G2 displayed in the sharpened UWF fundus image display field 1032B. For example, in a case where the selection button 1036C of the mode 3 is operated on an image in which the proportions of the red, green, and blue components are equal in the sharpened UWF fundus image G2, the proportion of the red component is set to 8 / 10, the proportions of the components of other colors (green, blue) are each set to 1 / 10, and the like. When the proportion of the red component is enhanced compared to the proportions of the components of other colors (green, blue) in the sharpened UWF fundus image G2, red light reaches the choroid by passing through the retina, and thus the portion of the blood vessels of the choroid is more enhanced than the blood vessels of the retina. Thus, it is possible to confirm the state of pachy-choroid.

[0128] As such, the object of the image processing performed in order to confirm the state of pachy-choroid can not be the sharpened UWF fundus image G2, but can be the UWF fundus image G1 of the RGB color space (refer to FIG. 1) as follows. Figure 7 Specifically, first, the information of the red (R) component of the UWF fundus image G1 of the R light that reaches the choroid by passing through the retina includes the information of the blood vessels of each of the retina and the choroid. The information of the green (G) component of the UWF fundus image G1 of the G light that reaches only the retina does not include the information of the blood vessels of the choroid, but includes the information of the blood vessels of the retina. The blood vessels of the choroid are extracted from the information of the red (R) component and the information of the green (G) component. Thus, the processing section 208 of the server 140 extracts the blood vessels of the choroid from the UWF fundus image G1 of the RGB color space (refer to FIG. 1) separately from the image processing of the server 140, and stores the blood vessels of the choroid in the storage device 254. The viewer 150 receives the information of the blood vessels of the choroid from the server 140 in a case where the selection button 1036C of the mode 3 is operated, and superimposes the blood vessels of the choroid on the sharpened UWF fundus image G2 displayed in the sharpened UWF fundus image display field 1032B. Figure 5 Figure 7

[0129] ​​The mode 4 selected by the selection button 1036D is a vitreous surgery result confirmation mode. The vitreous surgery result confirmation mode is a mode in which the proportion of the green color component is enhanced compared to the proportions of the components of other colors (red, blue) in the sharpened UWF fundus image G2 displayed in the sharpened UWF fundus image display field 1032B. For example, in a case where the selection button 1036D of the mode 4 is operated on an image in which the components of the red, green, and blue colors are equal in the sharpened UWF fundus image G2, the proportion of the green color component is set to 8 / 10, and the proportions of the components of the other colors (red, blue) are each set to 1 / 10, or the like. When the proportion of the green color component is enhanced compared to the proportions of the components of other colors (red, blue) in the sharpened UWF fundus image G2, the light of the green color does not pass through the retina and does not reach the choroid, and thus only the blood vessels of the retina are enhanced. Thus, it is possible to confirm the state after the surgery of the blood vessels of the retina that are the targets of the vitreous surgery.

[0130] In a case where the selection buttons 1036A to 1036D are operated, the viewer 150 performs the above-described processing corresponding to the mode on the sharpened UWF fundus image G2 displayed in the sharpened UWF fundus image display field 1032B. The viewer 150 displays the sharpened UWF fundus image on which the above-described processing corresponding to the mode is performed in the UWF fundus image display field 1032B.

[0131] The above-described processing corresponding to the mode is not limited to being performed by the viewer 150, and for example, the viewer 150 can first instruct the processing corresponding to the mode to the server 140, and the server 140 can perform the processing. The server 140 transmits the sharpened UWF fundus image after the processing to the viewer 150, and the viewer 150 displays the sharpened UWF fundus image after the processing in the UWF fundus image display field 1032B.

[0132] The processing corresponding to the mode can also be performed by another image processing apparatus that is connected to the network 130 in addition to the viewer 150 and the server 140.

[0133] Further, the sharpened UWF fundus image on which the above-described processing corresponding to the mode is performed and the sharpened UWF fundus image G2 can also be displayed together.

[0134] In a case where the screen switching button 1024 of Figure 12 is operated, the viewer 150 displays the second fundus image display screen 1000B shown in Figure 13 in the display.

[0135] The first fundus image display screen 1000A and the second fundus image display screen 1000B are substantially the same, and therefore the same reference numerals are assigned to the same parts and the description thereof is omitted, and only the different parts are described.

[0136] The second fundus image display screen 1000B has an original UWF fundus image display field 1032A and a sharpened UWF fundus image display field 1032B instead of the Figure 12 original UWF fundus image display field 1032A and the sharpened UWF fundus image display field 1032B, and has an original UWF fundus image partial display field 1032C and a sharpened UWF fundus image partial display field 1032D.

[0137] The size of the image display field in which the original UWF fundus image partial display field 1032C and the sharpened UWF fundus image partial display field 1032D are combined is the same as that of the UWF fundus image display field 1032A or the UWF fundus image display field 1032B. Figure 12

[0138] A slide bar 1032E is provided at the boundary between the original UWF fundus image partial display field 1032C and the sharpened UWF fundus image partial display field 1032D.

[0139] The slide bar 1032E is movable to the original UWF fundus image partial display field 1032C side (left direction) or the sharpened UWF fundus image partial display field 1032D side (right direction) as indicated by an arrow 1032F. Figure 13 Figure 13

[0140] When the slide bar 1032E is moved to the left direction, the range of the UWF fundus image partial display field 1032C becomes narrow and the range of the UWF fundus image partial display field 1032D becomes wide. When the slide bar 1032E is moved to the right direction, the range of the UWF fundus image partial display field 1032C becomes wide and the range of the UWF fundus image partial display field 1032D becomes narrow. Figure 13 Figure 13

[0141] When the screen switching button 1024 is operated, the viewer 150 displays the third fundus image display screen 1000C shown in Fig. 10C on the display. Figure 13 Figure 14 The second fundus image display screen 1000B and the third fundus image display screen 1000C are substantially the same, and therefore the same reference numerals are assigned to the same parts and the description thereof is omitted, and only the different parts are described.

[0142]

[0143] ​​​​​​​The 3rd fundus image display screen 1000C is replaced with Figure 13 the original UWF fundus image portion display field 1032C, the sharpening-processed UWF fundus image portion display field 1032D, and the both display field 1032G are displayed.

[0144] The sharpening-processed UWF fundus image G2 is displayed in the portion 1032G2 outside the frame of the both display field 1032G. The portion of the original UWF fundus image corresponding to the portion divided by the frame is displayed in the portion 1032G1 divided by the frame.

[0145] In terms of the frame itself, for example, if a portion other than the corner of the frame is dragged, the both display field 1032G can be moved, and when the corner of the frame is dragged and moved, the size of the frame can be enlarged and reduced.

[0146] As such, the sharpening-processed UWF fundus image G2 is displayed in the portion 1032G2 outside the frame, and the portion of the original UWF fundus image corresponding to the portion divided by the frame is displayed in the portion 1032G1 divided by the frame. Therefore, the ophthalmologist can confirm the sharpening-processed UWF fundus image G2 as a whole while confirming the content before the image processing for a portion of the image.

[0147] Further, for example, in a case where a portion of the both display field 1032G is clicked or in a case where the reverse button is set and the reverse button is operated, the portion of the original UWF fundus image is displayed in the portion 1032G2 outside the frame, and the sharpening-processed UWF fundus image G2 corresponding to the portion divided by the frame is displayed in the portion 1032G1 divided by the frame.

[0148] As described above, in the technology of the present disclosure, the sharpening processing unit 2060 performs the CLAHE processing with respect to each component of L*a*b* of the image of the central region and the peripheral region with the parameter corresponding to each region. Specifically, the sharpening processing unit 2060 performs the CLAHE processing with respect to each component of L*a*b* of the image of the central region with the 1st tile size Tc, and performs the CLAHE processing with respect to each component of L*a*b* of the image of the peripheral region with the 2nd tile size Tp.

[0149] As described above, in the original UWF fundus image G1( Figure 7 ) obtained by photographing at the internal illumination angle and with the photographing angle of view of 160 degrees or more, the distortion of the image of the peripheral region is larger than that of the central region. In other words, the peripheral region of the area of the actual fundus corresponding to the same prescribed area in the original UWF fundus image G1 is larger than the central region.

[0150] Thus, in the present embodiment, the size of the second tile size Tp is increased compared to the first tile size Tc.

[0151] Specifically, the area of the actual fundus corresponding to the portion of the central region of the original UWF fundus image G1 is set as Sc, the area of the actual fundus corresponding to the portion of the peripheral region of the original UWF fundus image G1 is set as Sp, and when Sp = n2Sc, then Tp = n2Tc.

[0152] Thus, the degree of enhancement of the CLAHE processing based on each component of L*a*b* of the image for the peripheral region is set to be greater than the degree of enhancement of the CLAHE processing based on each component of L*a*b* of the image for the central region.

[0153] As such, in the present embodiment, the CLAHE processing is performed with parameters corresponding to the central region and the peripheral region, and thus, compared to the case where the CLAHE processing is uniformly performed with certain parameters with respect to each region, the thickness of the enhanced blood vessels in each region can be made uniform, and thus, the contrast of the image in each region can be unified, and the image can be clarified.

[0154] Further, in the technology of the present disclosure, the clarification processing section 2060 can also perform the clarification processing with respect to the UWF fundus image of the RGB color space. However, in the present embodiment, the clarification processing section 2060 converts the UWF fundus image of the RGB color space into an image of the L*a*b* color space, and performs the clarification processing on each image of the L*a*b* color space.

[0155] In the clarification processing of the UWF fundus image of the RGB color space, the clarification processing is performed only with respect to the luminance (brightness).

[0156] However, in the clarification processing of the image of the L*a*b* color space, the clarification processing section 2060 performs the clarification processing not only with respect to the value of L* (luminance), but also with respect to the hue and chroma of red or green as a*, and the hue and chroma of yellow or blue as b*. Thus, the luminance, the hue, and the chroma are enhanced. Thus, the UWF fundus image can be made clearer.

[0157] Next, a modification example of the clarification processing of step 504 of the process of Figure 15 will be described with reference to Figure 5 .

[0158] In the clarification processing of the above-described embodiment (with reference to Figure 6AIn the case of the CLAHE processing in the flowchart of FIG. 6, the sharpening processing section 2060 extracts an image of the central region and an image of the peripheral region from the fundus image, and performs CLAHE processing that increases the degree of enhancement of the image of the peripheral region compared to the image of the central region, with respect to the extracted image of the peripheral region and the extracted image of the central region.

[0159] The technology of the present disclosure is not limited to this. For example, the sharpening processing section 2060 performs CLAHE processing that differs in degree of enhancement with respect to the image G11 of the L* component (refer to FIG. 6A), Figure 8A the image of the a* component, and the image of the b* component, respectively, generates the 1st image and the 2nd image with respect to each component, and synthesizes the 1st image and the 2nd image at a prescribed mixing ratio. Specifically, the sharpening processing section 2060 synthesizes the 1st image and the 2nd image in a manner in which the proportion of the 2nd image becomes higher than the proportion of the 1st image with respect to each component as the position of the pixel goes from the center to the periphery. More specifically, as described below.

[0160] In step 602, the sharpening processing section 2060 converts the UWF fundus image G1 of the RGB color space (refer to FIG. 6A) Figure 7 into an image of the L*a*b* color space. Thereby, the image G11 of the L* component (refer to FIG. 6B), Figure 8A the image of the a* component, and the image of the b* component are obtained.

[0161] In step 607, the sharpening processing section 2060 sets the 1st parameter for the image of the central region as a parameter of the CLAHE processing. Further, the 1st parameter is the same as the 1st parameter of step 606 of Figure 6A

[0162] In step 609, the sharpening processing section 2060 performs CLAHE processing with respect to each component of the image as a whole of the L*a*b* color space with the 1st parameter, and thereby generates the 1st processing-terminated image with respect to each component.

[0163] In step 613, the sharpening processing section 2060 sets the 2nd parameter for the image of the peripheral region as a parameter of the CLAHE processing. Further, the 2nd parameter is the same as the 2nd parameter of step 612 of Figure 6A

[0164] In step 615, the sharpening processing section 2060 performs CLAHE processing with respect to each component of the image as a whole of the L*a*b* color space with the 2nd parameter, and thereby generates the 2nd processing-terminated image with respect to each component.

[0165] ​​In step 617, the sharpening processing section 2060 synthesizes the first processing end image and the second processing end image of each component of the image of the L*a*b* color space in a manner such that the proportion of the second processing end image becomes higher than the proportion of the first processing end image as the position of the pixel goes from the center to the periphery.

[0166] For example, in the central position, the first processing end image : the second processing end image = 1 : 0, and in the most peripheral position, the first processing end image : the second processing end image = 0 : 1. As the position goes from the center to the most periphery, the proportion of the first processing end image becomes smaller than 1 and the proportion of the second processing end image becomes larger than 0.

[0167] In step 619, the sharpening processing section 2060 converts each component of the image of the L*a*b* color space after the first processing end image and the second processing end image are synthesized into an image of the RGB color space.

[0168] As explained above, in Figure 15 In the modification example shown, the first image and the second image are synthesized in a prescribed mixing ratio for each component, and the sharpening processing section 2060 synthesizes the first image and the second image for each component in a manner such that the proportion of the second image becomes higher than the proportion of the first image as the position of the pixel goes from the center to the periphery.

[0169] The technology of the present disclosure is not limited to this, and the sharpening processing section 2060 can also synthesize the first image and the second image for each component in a manner such that the proportion of the first image becomes more than the proportion of the second image in the central region and the proportion of the second image becomes higher than the proportion of the first image in the peripheral region.

[0170] In each of the examples explained above, the tile size was used as a parameter of the CLAHE processing, but the technology of the present disclosure is not limited to this, and a stretch coefficient can also be used.

[0171] Here, the stretch coefficient is a coefficient that limits the degree of enhancement of the contrast with respect to the brightness of the image.

[0172] The stretch coefficient is set to a value corresponding to each of the central region and the peripheral region. Specifically, in terms of the stretch coefficient, the first stretch coefficient is set for the central region, and the second stretch coefficient is set for the peripheral region. The second stretch coefficient is larger than the first stretch coefficient. Since the influence of the aberration of the peripheral region is large compared to the central region, in terms of the UWF fundus image G1, the sharpness looks worse as the peripheral region is approached compared to the central region. Therefore, since the contrast enhancement is increased more as the peripheral region is approached, the second stretch coefficient is set to be larger than the first stretch coefficient.

[0173] In each of the above-described examples, the CLAHE processing is used as the sharpening processing, but the technology of the present disclosure is not limited thereto, and for example, non-sharpening filter processing (frequency processing) can also be used.

[0174] In addition, a contrast enhancement processing other than the CLAHE processing, such as deconvolution processing, histogram equalization processing, haze removal processing, chroma correction processing, noise reduction processing, or a combination of these, can also be used.

[0175] In the non-sharpening filter processing, a sharpening parameter is used.

[0176] The sharpening parameter is a coefficient that specifies the degree of enhancement of the contrast of the lightness and darkness of the image.

[0177] The non-sharpening filter processing refers to processing that enhances the high-frequency components of the image. The original image is intentionally smoothed (blurred) to create a difference image from the original image, and thus a sharpening component of the original image is created. The sharpening of the original image is performed by adding the difference image to the original image. A constant that specifies the proportion at which the difference image is added to the original image is referred to as a non-sharpening parameter.

[0178] In the case where the non-sharpening filter processing is used, in step 606 of the processing of Figure 6A the sharpening processing unit 2060 sets a first sharpening parameter as the sharpening parameter for the non-sharpening filter processing of the image of the central region. In step 608, the sharpening processing unit 2060 performs the non-sharpening filter processing with respect to each component of L*a*b* of the image of the central region at the first sharpening parameter.

[0179] In step 612, the sharpening processing unit 2060 sets a second sharpening parameter as the sharpening parameter for the non-sharpening filter processing of the image of the peripheral region. In step 608, the sharpening processing unit 2060 performs the non-sharpening filter processing with respect to each component of L*a*b* of the image of the peripheral region at the second sharpening parameter. The second sharpening parameter is larger than the first sharpening parameter. This is because the influence of the aberration of the peripheral region is greater than that of the central region, and the influence of the aberration of the peripheral region is reduced more than that of the central region, so the second sharpening parameter is increased more than the first sharpening parameter.

[0180] Furthermore, the sharpening processing unit 2060 can perform both the CLAHE processing and the non-sharpening filter processing as the sharpening processing. In this case, the parameters of at least one of the CLAHE processing and the non-sharpening filter processing can be set to values corresponding to each of the central region and the peripheral region.

[0181] For example, the parameters of both the CLAHE processing and the non-sharpening filter processing can be set to values corresponding to each of the central region and the peripheral region, or the parameters of the CLAHE processing can be set constant in the central region and the peripheral region, and the non-sharpening filter processing can be set to values corresponding to each of the central region and the peripheral region. Specifically, for example, the tile size of the CLAHE processing can be set constant in the central region and the peripheral region (an intermediate size between the tile size Tc and the tile size Tp), the non-sharpening filter processing can be set to the first sharpening parameter for the central region, and the second sharpening parameter for the peripheral region. The peripheral region is more likely to be blurred (the image is blurred due to the influence of aberration of the optical system that captures the fundus, aberration caused by a large angle of incidence of light rays that strike the pupil of the eyeball, and the like) than the central region, and therefore, the second sharpening parameter is preferably larger than the first sharpening parameter, and more sharpening components are added.

[0182] Furthermore, a contrast enhancement processing other than the CLAHE processing can be performed instead of or simultaneously with at least one of the CLAHE processing and the non-sharpening filter processing. In the contrast enhancement processing other than the CLAHE processing, the degree of enhancement is increased more in the peripheral region than in the central region according to each of the central region and the peripheral region.

[0183] In the example described above, the regions whose parameters are changed by the sharpening processing are the central region and the peripheral region, but the technology of the present disclosure is not limited to this, and can be a plurality of regions of three or more regions. Specifically, the sharpening processing unit 2060 performs the sharpening processing using parameters that increase as distance from the center increases for a plurality of regions of three or more regions.

[0184] In the example described above, the UWF fundus image G1 is transmitted from the ophthalmic apparatus 110 having the SLO system to the server 140, but the technology of the present disclosure is not limited to this. For example, the fundus camera can be connected to the network 130, and a fundus image having a smaller angle of view than the ophthalmic apparatus 110 having the SLO system can be transmitted from the fundus camera to the server 140. In this case, the ophthalmic apparatus 110 transmits the UWF fundus image G1 to the server 140 in correspondence with a flag indicating a UWF fundus image, and the fundus camera transmits a fundus image to the server 140 in correspondence with a flag indicating a fundus image other than a UWF fundus image. The server 140 determines whether the image of the object to be processed corresponding to the patient name ID is a UWF fundus image on the basis of the flag indicating a UWF fundus image and the flag indicating a fundus image other than a UWF fundus image. The server 140 performs the image processing of the UWF fundus image G1 in a case where it is determined that the image of the object to be processed is a UWF fundus image. Figure 5

[0185] ​Further, it is also possible to configure such that the ophthalmic apparatus 110 transmits the UWF fundus image G1 to the server 140 in correspondence with the flag indicating the UWF fundus image, and the fundus camera transmits the fundus image in correspondence with the flag indicating the UWF fundus image. Conversely, it is also possible to configure such that the ophthalmic apparatus 110 transmits the UWF fundus image G1 to the server 140 in correspondence with the flag indicating the UWF fundus image, and the fundus camera transmits the fundus image in correspondence with the flag indicating the UWF fundus image.

[0186] In the above-described examples, Figure 5 The image processing is executed by the server 140, but the technology of the present disclosure is not limited thereto. For example, it is also possible to execute by the ophthalmic apparatus 110 or the viewer 150, or by another image processing apparatus connected further to the network 130.

[0187] In the above-described examples, the case where the image processing is implemented by a software configuration using a computer is exemplified, but the technology of the present disclosure is not limited thereto. For example, it is also possible to execute the image processing by a hardware configuration using only an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) or the like instead of a software configuration using a computer. It is also possible to execute part of the image processing by a software configuration and the other processing by a hardware configuration.

[0188] As such, the technology of the present disclosure includes the following 1st technology and 2nd technology because it includes the case where the image processing is implemented by a software configuration using a computer and the case where the image processing is implemented by a configuration other than a software configuration using a computer.

[0189] The image processing apparatus includes:

[0190] an acquisition section that acquires a fundus image;

[0191] an execution section that executes a 1st enhancement process on an image of at least a central region of the fundus image and executes a 2nd enhancement process different from the 1st enhancement process on an image of at least a peripheral region of the fundus image around the central region; and

[0192] a generation section that generates an enhanced image of the fundus image based on a 1st image obtained by executing the 1st enhancement process and a 2nd image obtained by executing the 2nd enhancement process.

[0193] Further, the clarification processing section 2060 of the above-described embodiment is an example of the "acquisition section", "execution section", and "generation section" of the above-described 1st technology.

[0194] The following is a second technology suggested from the above disclosure described in the above manner.

[0195] The image processing method includes:

[0196] The acquisition section acquires a fundus image;

[0197] The execution section executes a first enhancement process on an image of at least a central region of the fundus image, and executes a second enhancement process different from the first enhancement process on an image of at least a peripheral region of the fundus image that is peripheral to the central region; and

[0198] The generation section generates an enhanced image of the fundus image based on a first image obtained by executing the first enhancement process, and a second image obtained by executing the second enhancement process.

[0199] The following is a third technology suggested from the above disclosure.

[0200] A computer program product for image processing, which has a computer readable storage medium, per se, that is not a transitory signal,

[0201] The program is stored in the computer readable storage medium,

[0202] The program causes a computer to execute:

[0203] acquire a fundus image;

[0204] execute a first enhancement process on an image of at least a central region of the fundus image, and execute a second enhancement process different from the first enhancement process on an image of at least a peripheral region of the fundus image that is peripheral to the central region; and

[0205] generate an enhanced image of the fundus image based on a first image obtained by executing the first enhancement process, and a second image obtained by executing the second enhancement process.

[0206] The image processing described above is at least one example. Therefore, needless steps can be deleted, new steps can be added, or the processing order can be changed, of course, without departing from the gist.

[0207] All documents, patent applications, and technical standards cited in this specification are hereby incorporated by reference to the extent that each document, patent application, and technical standard is specifically and individually indicated to be incorporated by reference in its entirety.

Claims

1. An image processing method executed by a processor, the image processing method being characterized by comprising: The processor acquires fundus images; The processor performs a first enhancement process on the central region of the fundus image, including at least the optic nerve head and macula, and performs a second enhancement process, different from the first enhancement process, on the peripheral region of the fundus image, including at least the central region. as well as The processor generates an enhanced image of the fundus image based on the first image obtained by performing the first enhancement process and the second image obtained by performing the second enhancement process.

2. The image processing method according to claim 1, characterized in that, The processor extracts images of the central region and the peripheral region from the fundus image, performs a first enhancement process on the extracted central region image and a second enhancement process on the extracted peripheral region image.

3. The image processing method according to claim 1, characterized in that, The processor performs the first enhancement process and the second enhancement process on the fundus image. The processor generates the enhanced image by combining the first image and the second image at a predetermined mixing ratio.

4. The image processing method according to claim 3, characterized in that, The processor generates the enhanced image by combining the first image and the second image in such a way that the scale of the second image becomes higher than that of the first image as the pixel position moves from the center to the periphery.

5. The image processing method according to claim 3, characterized in that, The processor generates the enhanced image by combining the first image and the second image in such a way that the scale of the first image becomes higher than that of the second image in the central region and the scale of the second image becomes higher than that of the first image in the peripheral region.

6. The image processing method according to any one of claims 1 to 5, characterized in that, The fundus image is a fundus image with three components: a lightness component representing brightness, and a first color component and a second color component representing two different chromaticities.

7. The image processing method according to any one of claims 1 to 4, characterized in that, The degree of enhancement specified by the parameters of the second enhancement process is greater than the degree of enhancement specified by the parameters used for the first enhancement process.

8. The image processing method according to any one of claims 1 to 5, characterized in that, The first enhancement process and the second enhancement process are processes that enhance the contrast of the image.

9. The image processing method according to any one of claims 1 to 5, characterized in that, The fundus image is either a first fundus image with an internal illumination angle above a predetermined value or a second fundus image with an internal illumination angle below the predetermined value. The processor performs the first enhancement process and the second enhancement process only when the fundus image is the first fundus image.

10. The image processing method according to any one of claims 1 to 5, characterized in that, The fundus image is an image converted from a fundus image in the RGB color space to a fundus image in the complementary color space of the L*a*b* color space.

11. An image processing method executed by a processor, the image processing method being characterized by comprising: The processor acquires fundus images; The processor performs a first sharpening process using a first parameter relative to the central region of the fundus image, including the optic nerve head and macula, and performs a second sharpening process using a second parameter different from the first parameter relative to the peripheral region surrounding the central region of the fundus image. as well as The processor generates a sharpened fundus image based on the image obtained after performing the first sharpening process and the second sharpening process.

12. An image processing method executed by a processor, characterized in that, The processor acquires fundus images. A first sharpened image is generated by the processor performing a first sharpening process using a first parameter on the entire region of the fundus image, and a second sharpened image is generated by performing a second sharpening process using a second parameter on the entire region of the fundus image. The processor generates a central region of a sharpened fundus image, including the optic nerve head and macula, by combining the first sharpened image and the second sharpened image at a first mixing ratio, and generates a peripheral region surrounding the central region of the sharpened fundus image by combining the first sharpened image and the second sharpened image at a second mixing ratio different from the first mixing ratio.

13. The image processing method according to claim 11 or 12, characterized in that, The first parameter and the second parameter are the dimensions of the regions where the first and second sharpening processes are performed. The first and second sharpening processes are performed for each of the regions.

14. The image processing method according to claim 11 or 12, characterized in that, The first and second sharpening processes are CLAHE processes.

15. The image processing method according to claim 11 or 12, characterized in that, The clarified fundus image is a fundus image in complementary color space.

16. The image processing method according to claim 15, characterized in that, The complementary color space is L*a*b* complementary color space.

17. The image processing method according to claim 11 or 12, characterized in that, It also has: Perform RGB space conversion on the clarified fundus image; and Output the cleared fundus image after the RGB space conversion.

18. An image processing apparatus, characterized in that, It includes a processor and a memory connected to the processor. The processor is configured such that, Acquire fundus images, A first enhancement process is performed on the central region of the fundus image, including at least the optic nerve head and macula, and a second enhancement process, different from the first enhancement process, is performed on the peripheral region of the fundus image, including at least the area surrounding the central region. An enhanced image of the fundus image is generated based on the first image obtained by performing the first enhancement process and the second image obtained by performing the second enhancement process.

19. An image processing apparatus, characterized in that, It includes a processor and a memory connected to the processor. The processor includes: Acquire fundus images; A first sharpening process using a first parameter is performed relative to the central region of the fundus image, including the optic nerve head and macula, and a second sharpening process using a second parameter different from the first parameter is performed relative to the peripheral region surrounding the central region of the fundus image; and A sharpened fundus image is generated based on the image obtained after the first and second sharpening processes have been performed.

20. A program product containing a program, characterized in that, The program causes the computer to execute: Acquire fundus images, A first enhancement process is performed on the central region of the fundus image, including at least the optic nerve head and macula, and a second enhancement process, different from the first enhancement process, is performed on the peripheral region of the fundus image, including at least the area surrounding the central region. An enhanced image of the fundus image is generated based on the first image obtained by performing the first enhancement process and the second image obtained by performing the second enhancement process.

21. A program product containing a program, characterized in that, The program causes the computer to execute: Acquire fundus images, A first sharpening process using a first parameter is performed relative to the central region of the fundus image, including the optic nerve head and macula, and a second sharpening process using a second parameter different from the first parameter is performed relative to the peripheral region surrounding the central region of the fundus image. A sharpened fundus image is generated based on the image obtained after the first and second sharpening processes have been performed.

Images