Information output device, fundus image acquisition device, information output method, and information output program

Abstract

To examine a disease state based on feature amounts related to the position, the size or the range on fundus of a structure existing on the fundus of subject eye.SOLUTION: A fundus image formed by imaging a fundus of subject eye is acquired, and a first feature amount and a second feature amount are extracted respectively as feature amounts of the subject eye from the acquired fundus image, and information on a disease risk of the subject eye is outputted from the extracted first feature amount and second feature amount.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information output device, an ophthalmic fundus imaging device, an information output method, and an information output program.

Background Art

[0002] There is a demand for diagnosing a pathological condition based on a feature amount related to the position, size, or range of a structure existing in the fundus of an eye to be examined on the fundus using a pre-trained model.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] An information output device according to a first aspect of the technology of the present disclosure includes an acquisition unit that acquires a fundus image in which the fundus of an eye to be examined is photographed, an extraction unit that extracts a first feature amount and a second feature amount as feature amounts of the eye to be examined from the fundus image, and an output unit that outputs information regarding the disease risk of the eye to be examined from the first feature amount and the second feature amount.

[0005] An information output method according to a second aspect of the technology of the present disclosure includes a step of acquiring a fundus image in which the fundus of an eye to be examined is photographed, a step of extracting a first feature amount and a second feature amount as feature amounts of the eye to be examined from the fundus image, and a step of outputting information regarding the disease risk of the eye to be examined from the first feature amount and the second feature amount.

[0006] An information output program according to a third aspect of the technology of the present disclosure causes a computer to function as an acquisition unit that acquires a fundus image in which the fundus of an eye to be examined is photographed, an extraction unit that extracts a first feature amount and a second feature amount as feature amounts of the eye to be examined from the fundus image, and an output unit that outputs information regarding the disease risk of the eye to be examined from the first feature amount and the second feature amount. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram of the ophthalmology system 100. [Figure 2] This is a schematic diagram showing the overall configuration of the ophthalmic device 110. [Figure 3] This is a block diagram of the electrical system configuration of management server 140. [Figure 4] This is a block diagram of the functions of CPU 262 on management server 140. [Figure 5] This is a block diagram of the functions of the image processing unit 206 on the CPU 262 of the management server 140. [Figure 6] This flowchart shows an example of the process for building a predictive model in this embodiment. [Figure 7] This is a schematic diagram of choroidal vascular imaging (CLA). [Figure 8] This is a schematic diagram showing a binarized image 300 generated from choroidal vascular images (CLA). [Figure 9] This is an explanatory diagram showing the regions in fundus images that are used to assess disease risk. [Figure 10] This flowchart shows an example of processing using a disease risk prediction model after training. [Figure 11] This is a schematic diagram showing the display image 500 shown on the display 256 of the management server 140. [Figure 12] This is a schematic diagram showing the time-dependent changes in the fundus image of display image 500, which is displayed on the display 256 of the management server 140. [Modes for carrying out the invention]

[0008] This embodiment will be described in detail below with reference to the drawings.

[0009] Referring to Figure 1, the configuration of the ophthalmology system 100 will be explained. As shown in Figure 1, the ophthalmology system 100 comprises an ophthalmology device 110, a management server device (hereinafter referred to as the "management server") 140, and a display device (hereinafter referred to as the "viewer") 150. The ophthalmology device 110 acquires fundus images. The management server 140 stores multiple fundus images and axial lengths obtained by the ophthalmology device 110 capturing the funduses of multiple patients, corresponding to the patient's ID. The viewer 150 displays the fundus images and analysis results acquired by the management server 140.

[0010] The viewer 150 includes a display 156 that displays fundus images and analysis results acquired by the management server 140, a mouse 155M and a keyboard 155K for operation.

[0011] The ophthalmic device 110, the management server 140, and the viewer 150 are interconnected via the network 130. The viewer 150 is a client in the client-server system and is connected via the network. Multiple viewer 150s may be connected to the network 130. In addition, to ensure system redundancy, multiple management servers 140 may be connected via the network 130.

[0012] If the ophthalmic device 110 is equipped with image processing capabilities and the viewer 150 with image viewing capabilities, then the ophthalmic device 110 can acquire fundus images, process them, and view them in a standalone state. Furthermore, if the management server 140 is equipped with the viewer 150 with image viewing capabilities, then the configuration of the ophthalmic device 110 and the management server 140 can acquire fundus images, process them, and view them.

[0013] Note that a diagnostic support device that performs image analysis using other ophthalmic devices (examination devices such as visual field measurement and intraocular pressure measurement) and AI (Artificial Intelligence) may be connected to the ophthalmic device 110, the management server 140, and the viewer 150 via the network 130. In the present disclosure, each component (device, etc.) may exist alone or in two or more as long as there is no contradiction.

[0014] Next, referring to FIG. 2, the configuration of the ophthalmic device 110 will be described. For convenience of explanation, a scanning laser ophthalmoscope is referred to as "SLO". Also, an optical coherence tomography is referred to as "OCT".

[0015] Note that when the ophthalmic device 110 is installed on a horizontal plane, the horizontal direction is the "X direction", the vertical direction with respect to the horizontal plane is the "Y direction", and the direction connecting the center of the pupil of the anterior eye part of the subject eye 12 and the center of the eyeball is the "Z direction". Therefore, the X direction, the Y direction, and the Z direction are perpendicular to each other.

[0016] The ophthalmic device 110 includes a photographing device 14 and a control device 16. The photographing device 14 includes an SLO unit 18 and an OCT unit 20, and acquires a fundus image of the fundus of the subject eye 12. Hereinafter, the two-dimensional fundus image acquired by the SLO unit 18 is referred to as an SLO image. Also, a tomographic image or an en-face image of the retina created based on the OCT data acquired by the OCT unit 20 is referred to as an OCT image.

[0017] [[ID=1​​​​The control device 16 includes an input / output display device 16E connected to the CPU 16A via the I / O port 16D. The input / output display device 16E has a graphic user interface for displaying an image of the eye under examination 12 and receiving various instructions from the user. Examples of the graphic user interface include a touch panel display.

[0019] Further, the control device 16 includes an image processing device 17 connected to the I / O port 16D. The image processing device 17 generates an image of the eye under examination 12 based on the data obtained by the imaging device 14. Note that the control device 16 is connected to the network 130 via a communication interface (I / F) 16F.

[0020] As described above, in FIG. 2, the control device 16 of the ophthalmic device 110 includes the input / output display device 16E, but the technology of the present disclosure is not limited thereto. For example, the control device 16 of the ophthalmic device 110 may not include the input / output display device 16E and may include a separate input / output display device physically independent of the ophthalmic device 110. In this case, the display device includes an image processing processor unit that operates under the control of the CPU 16A of the control device 16. The image processing processor unit may display an SLO image or the like based on the image signal output-instructed by the CPU 16A.

[0021] The imaging device 14 operates under the control of the CPU 16A of the control device 16. The imaging device 14 includes an SLO unit 18, an imaging optical system 19, and an OCT unit 20. The imaging optical system 19 includes a scanner 22 and a wide-angle optical system 30.

[0022] The scanner 22 two-dimensionally scans the light emitted from the SLO unit 18 in the X and Y directions and two-dimensionally scans the light emitted from the OCT unit 20 in the X and Y directions. The scanner 22 may be an optical element capable of deflecting a light beam. For example, a polygon mirror, a galvanometer mirror, or the like can be used. Further, a combination thereof may also be used.

[0023] The wide-angle optical system 30 includes an objective optical system (not shown in Figure 2) provided on the side of the eye under examination 12, and a combining unit (not shown in Figure 2) that combines light from the SLO unit 18 and light from the OCT unit 20.

[0024] The objective optical system may be a reflective optical system using a concave mirror such as an elliptical mirror, a refractive optical system using a wide-angle lens, or a reflective-refractive optical system combining a concave mirror and lenses. By using a wide-angle optical system using an elliptical mirror or a wide-angle lens, it becomes possible to photograph not only the central part of the fundus but also the peripheral part of the fundus.

[0025] When using a system that includes an elliptical mirror, a configuration using an elliptical mirror as described in International Publication WO2016 / 103484 or International Publication WO2016 / 103489 is also acceptable. Each of the disclosures in International Publication WO2016 / 103484 and International Publication WO2016 / 103489 is incorporated herein by reference in their entirety.

[0026] The wide-angle optical system 30 enables observation of the fundus with a wide field of view (FOV) 12A. The FOV 12A indicates the range that can be captured by the imaging device 14. The FOV 12A can be expressed as the field of view angle. In this embodiment, the field of view angle can be defined by the internal illumination angle and the external illumination angle. The external illumination angle is the illumination angle of the light beam irradiated from the ophthalmic device 110 onto the eye under examination 12, defined with respect to the pupil 27. The internal illumination angle is the illumination angle of the light beam irradiated onto the fundus, defined with respect to the center O of the eyeball. The external illumination angle and the internal illumination angle are in a corresponding relationship. For example, if the external illumination angle is 120 degrees, the internal illumination angle corresponds to approximately 160 degrees. In this embodiment, the internal illumination angle is set to 200 degrees.

[0027] Here, SLO fundus images obtained by imaging with an internal illumination angle of 160 degrees or more are referred to as UWF-SLO fundus images. UWF stands for UltraWide Field.

[0028] The SLO system is implemented by the control device 16, SLO unit 18, and imaging optical system 19 shown in Figure 2. Because the SLO system includes a wide-angle optical system 30, it enables fundus imaging with a wide FOV 12A.

[0029] The SLO unit 18 includes a light source 40 for blue light (B), a light source 42 for green light (G), a light source 44 for red light (R), and a light source 46 for infrared light (IR, e.g., near-infrared light), and optical systems 48, 50, 52, 54, and 56 that reflect or transmit the light from the light sources 40, 42, 44, and 46 into a single optical path. Optical systems 48, 50, and 56 are mirrors, and optical systems 52 and 54 are beam splitters. The B light is reflected by optical system 48, transmitted through optical system 50, and reflected by optical system 54; the G light is reflected by optical systems 50 and 54; the R light is transmitted through optical systems 52 and 54; and the IR light is reflected by optical systems 52 and 56, and each is guided into a single optical path.

[0030] The SLO unit 18 is configured to allow switching between combinations of light sources that emit or emit laser light of different wavelengths, such as modes that emit G light, R light, and B light, and modes that emit infrared light. In the example shown in Figure 2, there are four light sources: a B light (blue light) light source 40, a G light light source 42, an R light light source 44, and an IR light light source 46, but the technology of this disclosure is not limited to this. For example, the SLO unit 18 may further include a white light light source and emit light in various modes, such as a mode that emits only white light.

[0031] Light incident from the SLO unit 18 into the imaging optical system 19 is scanned in the X and Y directions by the scanner 22. The scanned light passes through the wide-angle optical system 30 and the pupil 27 and illuminates the posterior segment (fundus) of the eye under examination 12. The reflected light reflected by the fundus passes through the wide-angle optical system 30 and the scanner 22 and is incident back into the SLO unit 18.

[0032] The SLO unit 18 includes a beam splitter 64 that reflects B light and transmits other light from the posterior segment (fundus) of the eye under examination 12, and a beam splitter 58 that reflects G light and transmits other light from the light transmitted through the beam splitter 64. The SLO unit 18 also includes a beam splitter 60 that reflects R light and transmits other light from the light transmitted through the beam splitter 58. The SLO unit 18 also includes a beam splitter 62 that reflects IR light from the light transmitted through the beam splitter 60. The SLO unit 18 includes a B light detection element 70 that detects B light reflected by the beam splitter 64, a G light detection element 72 that detects G light reflected by the beam splitter 58, an R light detection element 74 that detects R light reflected by the beam splitter 60, and an IR light detection element 76 that detects IR light reflected by the beam splitter 62.

[0033] Light incident on the SLO unit 18 via the wide-angle optical system 30 and scanner 22 (reflected light reflected by the fundus) is, in the case of B light, reflected by beam splitter 64 and received by B light detection element 70, and in the case of G light, transmitted through beam splitter 64, reflected by beam splitter 58 and received by G light detection element 72. In the case of R light, the incident light is transmitted through beam splitters 64 and 58, reflected by beam splitter 60 and received by R light detection element 74. In the case of IR light, the incident light is transmitted through beam splitters 64, 58 and 60, reflected by beam splitter 62 and received by IR light detection element 76. The image processing device 17, operating under the control of CPU 16A, generates a UWF-SLO image using the signals detected by the B light detection element 70, G light detection element 72, R light detection element 74, and IR light detection element 76. Examples of B light detection elements 70, G light detection elements 72, R light detection elements 74, and IR light detection elements 76 include PD (photodiode) and APD (avalanche photodiode).

[0034] In the SLO unit 18, the light reflected (scattered) from the fundus of the eye, which is the target object, passes through the scanner 22 and reaches the photodetector, so it always returns to the same position, i.e., the position where the B photodetector 70, G photodetector 72, R photodetector 74, and IR photodetector 76 are located. Therefore, it is not necessary to configure the photodetector in a planar (2-dimensional) manner like an area sensor, and a point-shaped (0-dimensional) detector such as a PD or APD is optimal in this embodiment. However, it is also possible to use a line sensor (1-dimensional) or an area sensor (2-dimensional), not limited to PDs or APDs.

[0035] UWF-SLO images include those obtained by imaging the fundus with green light (green fundus images) and those obtained by imaging the fundus with red light (red fundus images). UWF-SLO images also include those obtained by imaging the fundus with blue light (blue fundus images) and those obtained by imaging the fundus with infrared light (IR fundus images).

[0036] Furthermore, the control device 16 controls the light sources 40, 42, and 44 to emit light simultaneously. By simultaneously photographing the fundus of the eye under examination 12 with B light, G light, and R light, G-color fundus images, R-color fundus images, and B-color fundus images are obtained, with each position corresponding to the others. An RGB color fundus image is obtained from the G-color fundus image, R-color fundus image, and B-color fundus image. The control device 16 controls the light sources 42 and 44 to emit light simultaneously, and by simultaneously photographing the fundus of the eye under examination 12 with G light and R light, G-color fundus images and R-color fundus images are obtained, with each position corresponding to the others. By mixing the G-color fundus image and R-color fundus image at a predetermined mixing ratio, an RG color fundus image is obtained.

[0037] UWF-SLO images also include UWF-SLO images (video) captured by ICG fluorescence imaging. When indocyanine green (ICG) is injected into the blood vessels, it reaches the fundus of the eye, first reaching the retina, then the choroid, and passes through the choroid. The UWF-SLO image (video) is a moving image from the moment indocyanine green (ICG) is injected into the blood vessels and reaches the retina, until after it has passed through the choroid.

[0038] Image data for B-color fundus images, G-color fundus images, R-color fundus images, IR fundus images, RGB-color fundus images, RG-color fundus images, and UWF-SLO images are sent from the ophthalmic device 110 to the management server 140 via the communication interface 16F.

[0039] The OCT system is implemented by the control device 16, OCT unit 20, and imaging optical system 19 shown in Figure 2. Because the OCT system is equipped with a wide-angle optical system 30, in addition to acquiring the SLO fundus image described above, it enables fundus imaging with a wide FOV 12A. On the UWF-SLO image, the user specifies the position where the OCT image will be acquired, and the OCT image is acquired by scanning (capturing) the specified position.

[0040] The OCT unit 20 includes a light source 20A, a sensor (detection element) 20B, a first optical coupler 20C, a reference optical system 20D, a collimating lens 20E, and a second optical coupler 20F.

[0041] Light emitted from the light source 20A is split by the first optical coupler 20C. One of the split beams of light is made into parallel light by the collimating lens 20E and then incident on the imaging optical system 19 as measurement light. The measurement light is scanned in the X and Y directions by the scanner 22. The scanning light is irradiated onto the fundus of the eye via the wide-angle optical system 30 and the pupil 27. The measurement light reflected by the fundus of the eye is incident on the OCT unit 20 via the wide-angle optical system 30 and the scanner 22, and then incident on the second optical coupler 20F via the collimating lens 20E and the first optical coupler 20C.

[0042] The other beam of light emitted from the light source 20A and branched by the first optical coupler 20C is incident on the reference optical system 20D as reference light, and then, via the reference optical system 20D, is incident on the second optical coupler 20F.

[0043] These lights incident on the second optical coupler 20F, i.e., the measurement light reflected from the fundus and the reference light, interfere with each other at the second optical coupler 20F to generate interference light. The interference light is received by the sensor 20B. The image processing device 17, operating under the control of the image processing unit 206, generates OCT images such as tomographic images and en-face images based on the OCT data detected by the sensor 20B.

[0044] The OCT image data and positional information related to the OCT acquisition location (information indicating the area and location scanned during OCT acquisition, such as pixel position and coordinate data on the UWF-SLO image, or the scanner drive signal) are sent from the ophthalmic device 110 to the management server 140 via the communication I / F 16F and stored in the storage device 254.

[0045] In this embodiment, the light source 20A is exemplified as a wavelength-swept type SS-OCT (Swept-Source OCT), but various other types of OCT systems such as SD-OCT (Spectral-Domain OCT) and TD-OCT (Time-Domain OCT) may also be used.

[0046] Next, the electrical system configuration of the management server 140 will be described with reference to Figure 3. As shown in Figure 3, the management server 140 includes a computer unit 252. The computer unit 252 has a CPU 262, RAM 266, ROM 264, and input / output (I / O) ports 268. A storage device 254, a display 256, a mouse 255M, a keyboard 255K, and a communication interface (I / F) 258 are connected to the I / O ports 268. The storage device 254 is composed of non-volatile memory such as an HDD (hard disk drive) or an SSD (solid state drive). The input / output (I / O) ports 268 are connected to the network 130 via the communication I / F 258. Therefore, the management server 140 can communicate with the ophthalmic device 110 and the viewer 150. The storage device 254 stores an image processing program, which will be described later. The image processing program may also be stored in the ROM 264. The CPU 262 corresponds to the "processor" in the technology of this disclosure. Furthermore, ROM264 and RAM266 correspond to the "memory" of the technology of this disclosure.

[0047] The management server 140 stores each piece of data received from the ophthalmic device 110 in the storage device 254, and also performs various image processing and data processing using the CPU 262.

[0048] Next, referring to Figure 4, various functions realized by the CPU 262 of the management server 140 executing the image processing program will be described. The image processing program includes display control functions, image processing functions, and processing functions. When the CPU 262 executes the image processing program having these functions, the CPU 262 functions as a display control unit 204, an image processing unit 206, and a processing unit 208, as shown in Figure 4.

[0049] Next, with reference to Figure 5, the various functions of the image processing unit 206 will be described. The image processing unit 206 functions as an image feature acquisition unit 2060 that performs image processing such as generating images with enhanced choroidal blood vessels from fundus images, identifying fundus structures such as vortex veins from fundus images, and acquiring feature quantities indicating the position, size, or range of said structures on the fundus, and a machine learning analysis unit 2062 that uses machine learning to diagnose the presence or absence of disease, the effectiveness of treatment for said disease, or the progression of the disease state from the feature quantities of structures on the fundus. The image feature acquisition unit 2060 and the machine learning analysis unit 2062 correspond to the "extraction unit" of the technology of this disclosure.

[0050] Figure 6 is a flowchart showing an example of the process for building a predictive model in this embodiment. The process shown in Figure 6 is executed on the management server 140.

[0051] In step 100, the image processing unit 206 acquires a UWF-SLO image from the storage device 254 as an input image.

[0052] In step 102, the image processing unit 206 performs preprocessing to detect vortex veins from the acquired UWF-SLO images (R-color fundus image and G-color fundus image). Step 102 also performs denoising to remove noise from the UWF-SLO images. A median filter or the like is applied for noise reduction.

[0053] The image processing unit 206 extracts retinal blood vessels from the G-color fundus image by applying a black hat filter to the G-color fundus image after noise reduction.

[0054] The image processing unit 206 removes retinal blood vessels from the R-color fundus image by using positional information of retinal blood vessels extracted from the G-color fundus image to fill in the retinal blood vessel structures with the same value as the surrounding pixels through inpainting. This process removes the retinal blood vessels from the R-color fundus image, generating an image in which only choroidal blood vessels are visible.

[0055] Furthermore, the image processing unit 206 obtains an image from which low-frequency components have been removed from the R-color fundus image after inpainting. To remove low-frequency components, well-known image processing methods such as frequency filtering and spatial filtering are applied.

[0056] Furthermore, the image processing unit 206 enhances the choroidal vessels in the red-color fundus image by applying, for example, contrast-limited adaptive histogram equalization to the red-color fundus image data in which the retinal vessels have been removed and the choroidal vessels remain. As a result, the choroidal vessel image CLA shown in Figure 7 is created. The created choroidal vessel image CLA is stored in the storage device 254, completing the preprocessing in step 102, and the procedure proceeds to step 104.

[0057] In the above example, a choroidal vascular image (CLA) is generated from a red-colored fundus image and a green-colored fundus image. However, the image processing unit 206 may also generate a choroidal vascular image (CLA) from a green-colored fundus image and an IR-colored fundus image. Furthermore, the image processing unit 206 may generate a choroidal vascular image (CLA) from a blue-colored fundus image and either a red-colored or IR-colored fundus image.

[0058] In step 104, the image processing unit 206 detects vortex veins by generating a binarized image from the choroidal vascular image CLA in which vortex veins, which are structures of the fundus, are detected. The binarized image is generated by binarizing the choroidal vascular image CLA.

[0059] Vortex veins include a linear portion composed of thin blood vessels and an ampulla where the blood vessels constituting the linear portion converge to form a thicker blood vessel. In this embodiment, by combining a first binarized fundus image suitable for detecting the ampulla and a second binarized fundus image suitable for detecting the linear portion, a fundus image in which the linear and ampulla portions of the vortex veins are clearly depicted can be obtained.

[0060] Then, the image obtained from the first binarization process and the image obtained from the second binarization process are combined (combined) to obtain a combined image as shown in Figure 8. The image combination is a logical OR operation. In this embodiment, pixels that are bright areas in either the image obtained from the first binarization process or the image obtained from the second binarization process are processed to become pixels that represent bright areas indicating choroidal blood vessels.

[0061] Figure 8 is a schematic diagram showing the binarized image 300 generated in step 104. In the binarized image 300, vortex veins 302V1, 302V2, 302V3, and 302V4 (hereinafter abbreviated as "each vortex vein" as needed) are displayed in white against a black background. Therefore, in the binarized image 300, sets of pixels with a brightness value above a threshold can be detected as each vortex vein.

[0062] In step 106, the image processing unit 206 calculates multiple feature quantities for each vortex vein detected in the binarized image 300. The feature quantities calculated in step 106 include the number of vortex veins, the distance from a predetermined position in the fundus to each vortex vein, the angle of each vortex vein from a predetermined position in the fundus, the diameter of the surrounding blood vessels of each vortex vein, and the thickness of the blood vessel wall, and also include the axial length of the eye. It is preferable that the position and angle values ​​are calculated on a spherical surface using spherical trigonometry.

[0063] In Figure 8, coordinates are set with the macula or ONH (optic nerve head) as the origin 304. The image processing unit 206 detects the optic nerve head by detecting the brightest point in the UWF-SLO image. The image processing unit 206 also detects the macula by detecting the darkest point in the UWF-SLO image.

[0064] The image processing unit 206 then calculates the following features on the spherical surface related to the eye 12: the distance 306 between the origin 304 and the center of the ampulla of each vortex vein, the angle θ between the line connecting the origin 304 and the center of the ampulla of each vortex vein and the horizontal axis of the Cartesian coordinate system, and the thickness of the linear vessels of each vortex vein, which are the surrounding blood vessels. The thickness of the linear vessels of each vortex vein is calculated, for example, by counting the number of pixels at the intersection of the analysis circles 302C1, 302C2, 302C3, and 302C4 with each vortex vein, and assuming that the cross-section of the linear vessel of each vortex vein is circular, the thickness of the linear vessel of each vortex vein is calculated from the counted number of pixels. The thickness of the blood vessel wall is calculated using OCT images, which are tomographic images, in combination.

[0065] Alternatively, as a characteristic feature of each vortex vein, the area of ​​each region 302A1, 302A2, 302A3, and 302A4, enclosed by lines connecting the tips of the linear portions of each vortex vein, may be adopted.

[0066] Step 108 involves obtaining disease risk information for the 12 eyes examined. The disease risk information obtained includes, for example, age-related macular degeneration, pachychoroid diseases such as polypoidal choroidal vasculopathy (PCV), and diabetic macular degeneration.

[0067] In step 110, machine learning is performed using training data that associates fundus images with multiple features calculated in step 106 and disease risk information. There are no particular restrictions on the machine learning method, but it is preferable to select from a group of methods consisting of Naive Bayes, Random Forest, Support Vector Machines, and AdaBoost. The features to be selected may be the measured quantities themselves, or dimensionality reduction may be performed using methods such as PCA to select the main features and perform calculations. It is preferable to adjust the feature values ​​by normalizing them. In step 110, the correlation between each feature and the disease risk information is derived.

[0068] In this embodiment, the risk of disease is determined for each feature. For the risk of a feature, the importance value of the feature as an important predictor calculated by the Relieff method or the like may be used. For example, features whose important predictor is above a predetermined threshold are designated as the first feature, and features below that threshold are designated as the second feature. In addition to judgment based on an image of the entire fundus, the fundus image may be divided into several regions, and features may be extracted for each region.

[0069] For the purpose of defining features by dividing them, features may be defined by dividing them. For example, the position, size, and range of the inferior nasal vortex vein may be defined as the first feature, and the position, size, and range of the superior temporal vortex vein may be defined as the second feature.

[0070] Figure 9 is an explanatory diagram showing the regions in a fundus image that are subject to assessment of disease risk. In Figure 9, the fundus is divided into the posterior pole 310, the middle margin 312, and the peripheral region 314, centered on the origin 304, which is the macula or ONH (optic nerve head). For example, the posterior pole 310 is in the range of 50 degrees when converted to an internal illumination angle from the center of the eyeball towards the origin 304, the middle margin 312 is in the range of 50 to 100 degrees, and the peripheral region 314 is in the range of 100 to 220 degrees. The feature quantity extracted from the image of the entire fundus may be used as the first feature quantity, and the feature quantities extracted from the middle margin 312 and the peripheral region 314, or from the peripheral region 314, may be used as the second feature quantity. The first feature quantity may be extracted from the peripheral region 314 where vortex veins are located, and the feature quantities of blood vessels connected to the vortex veins may be extracted from the posterior pole 310 or the middle margin 312 as the second feature quantity.

[0071] In step 112, the image processing unit 206 constructs a disease risk prediction model through learning from training data and then terminates the process.

[0072] Figure 10 is a flowchart showing an example of processing using a disease risk prediction model after learning. The steps from step 200 to step 206 are the same as the steps from step 100 to step 106 in Figure 6, so a detailed explanation is omitted.

[0073] Then, in step 208, the image processing unit 206 calculates the disease risk and / or the disease risk for each feature using multiple features based on the constructed predictive model, and then terminates the process.

[0074] Figure 11 is a schematic diagram showing the display image 500 displayed on the display 256 of the management server 140. In addition to the display 256 of the management server 140, the display image 500 may also be displayed on the display 156 of the viewer 150 or on the input / display device 16E of the ophthalmic device 110.

[0075] As shown in Figure 11, the display image 500 has an information display area 502 and an image display area 504. The information display area 502 has a patient ID display area 512, a patient name display area 514, an age display area 516, a visual acuity display area 518, a right eye / left eye display area 520, and an axial length display area 522.

[0076] The image display area 504 can display a binarized fundus image 540 in which each vortex vein is made visible, a disease risk 550 derived from the fundus image 540, and disease-specific risk by important predictor parameter 552, which shows the degree of influence of each parameter (important predictor) in a specific disease. In addition, the image display area 504 displays the year, month, and day 540T on which the fundus image 540 was acquired, and the treatment history 554 is displayed chronologically in the lower part of the image display area 504.

[0077] The disease risk 550 and the disease-specific important predictor parameter risk 552, displayed in the image display area 504, are calculated using the process shown in Figure 10 and displayed as percentages. The image display area 504 may also display feature quantities such as the number of vortex veins, the positional information of the vortex veins relative to a predetermined location in the fundus, the diameter of the blood vessels constituting the vortex veins, the thickness of the vortex vein walls, and the axial length of the eye.

[0078] Figure 12 is a schematic diagram showing the time course of the fundus image of the display image 500 shown on the display 256 of the management server 140. The display image 500 may be displayed on the display 156 of the viewer 150, or on the input / display device 16E of the ophthalmic device 110, in addition to the display 256 of the management server 140, as in the case of Figure 11.

[0079] As shown in Figure 12, the image display area 504 displays fundus image 540 from date 540T, fundus image 542 from date 542T, and fundus image 546 from date 544T. In chronological order, the fundus images were acquired in the order of date 540T, date 542T, and date 544T.

[0080] In the fundus images 540, 542, and 544 of the patient shown in Figure 12, the vortex veins 302V1 and 302V2, respectively, show an enlarged trend, suggesting a specific disease. In Figure 12, the time-series change in the risk of a specific disease is displayed as the disease-specific risk value over time 556. The disease-specific risk value over time 556 is calculated using the process shown in Figure 10 and is displayed as a percentage.

[0081] Furthermore, the image display area 504 can display an urgency level 558 indicating the need for immediate action in response to changes in disease risk, etc.

[0082] As described above, in this embodiment, disease risk is determined by a fundus image and feature quantities including information such as the location of vortex veins.

[0083] Processing the entire fundus image with a CNN model can lead to difficulties such as judging disease risk without a rigorous examination of disease location. However, by determining disease risk using the fundus image and features that include information such as the location of vortex veins, it becomes possible to output a detailed disease risk that includes disease location.

[0084] Furthermore, by processing only a portion of the fundus image, the computational load on the management server 140 is reduced, and disease risk can be output quickly.

[0085] The image processing in each embodiment described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0086] The embodiments described above assume image processing using a computer-based software configuration, but the technology of this disclosure is not limited thereto. For example, instead of a computer-based software configuration, image processing may be performed solely by a hardware configuration such as an FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit). Alternatively, some of the image processing may be performed by a software configuration, and the remaining processing by a hardware configuration.

[0087] Thus, the technology disclosed herein includes cases where image processing is implemented using computer-based software and cases where it is not, and therefore includes the following technologies.

[0088] (First technology) The system comprises memory and a processor connected to the memory, The aforementioned processor, The steps include: obtaining a fundus image taken of the fundus of the eye being examined, The steps include extracting a first feature and a second feature from the fundus image as feature quantities of the eye under examination, The steps include outputting information regarding the disease risk of the eye being examined from the aforementioned first feature and the aforementioned second feature, An information output device that performs the following actions.

[0089] (Second technology) Image processing performed by a processor, The steps include: obtaining a fundus image taken of the fundus of the eye being examined, The steps include extracting a first feature and a second feature from the fundus image as feature quantities of the eye under examination, The steps include outputting information regarding the disease risk of the eye being examined from the aforementioned first feature and the aforementioned second feature, A method for outputting information that includes this information.

[0090] (Third technology) A computer program product for image processing, The aforementioned computer program product includes a computer-readable storage medium that is not itself a temporary signal, The aforementioned computer-readable storage medium stores a program. The aforementioned program is installed on the computer. The steps include: obtaining a fundus image taken of the fundus of the eye being examined, The steps include extracting a first feature and a second feature from the fundus image as feature quantities of the eye under examination, The steps include outputting information regarding the disease risk of the eye being examined from the aforementioned first feature and the aforementioned second feature, A computer program product that executes a command.

[0091] All documents, patent applications, and technical standards described herein are incorporated by reference in the same manner as when each individual document, patent application, and technical standard is specifically and individually incorporated by reference. [Explanation of symbols]

[0092] 12. Eye to be examined 14. Imaging device 19. Imaging optical system 100 Ophthalmic Systems 110 Ophthalmological equipment 130 Networks 140 Management Server 150 Viewers 204 Display Control Unit 206 Image Processing Unit 208 Processing Unit 252 Computer main unit 254 Storage device 256 displays 262 CPU 264 ROM 266 RAM 268 ports 300 binarized images 302A1, 302A2, 302A3, 302A4 area Analysis circles 302C1, 302C2, 302C3, and 302C4 302V1, 302V2, 302V3, 302V4 Vortex Veins 304 Origin 306 distance 310 Posterior pole 312 Middle edge 314 Peripheral area 500 displayed images 502 Information display area 504 Image display area 2060 Image Feature Acquisition Unit 2062 Machine Learning Analysis Department θ angle

Claims

1. An acquisition unit that acquires fundus images of the eye under examination, The extraction unit extracts a first feature from the entire area of ​​the fundus image and a second feature from the peripheral area of ​​the macula of the fundus image, as the feature quantities of the eye under examination. An output unit that outputs information regarding the disease risk of the eye being examined, based on the first feature and the second feature, An information output device equipped with the following features.

2. The information output device according to claim 1, wherein the extraction unit extracts the second feature quantity from a region other than the peripheral region, which is the post pole portion.

3. The information output device according to claim 1 or 2, wherein the extraction unit extracts the first feature quantity and the second feature quantity, which include any of the following: the number of vortex veins extracted from the fundus image, positional information of the vortex veins relative to a predetermined position in the fundus, the diameter of the blood vessels constituting the vortex veins, the thickness of the blood vessel walls of the vortex veins, and the axial length of the eye.

4. An information output device according to any one of claims 1 to 3, comprising a display unit that displays the first feature quantity, the second feature quantity, and the information.

5. The information output device according to claim 4, wherein the display unit displays the fundus image.

6. The information output device according to any one of claims 1 to 5, wherein the extraction unit extracts the first feature quantity and the second feature quantity from the SLO image or OCT image of the eye to be examined.

7. The extraction unit is executed using one of the following algorithms: Naive Bayes, Random Forest, and Support Vector Machine, or by combining one of these algorithms with AdaBoost, and a learning model is constructed by learning the first feature and the second feature based on the fundus image, any one of claims 1 to 6. The information output device described in the section.

8. An information output device according to any one of claims 1 to 7, A fundus image acquisition device comprising an imaging unit for capturing the fundus image.

9. The fundus image acquisition device according to claim 8, wherein the imaging unit captures an SLO image or an OCT image of the eye to be examined.

10. A computer, An information output program that functions as an acquisition unit that acquires a fundus image of the fundus of the eye under examination, an extraction unit that extracts a first feature from the entire image region of the fundus image and a second feature from the peripheral region of the macula of the fundus image as feature quantities of the eye under examination, and an output unit that outputs information regarding the disease risk of the eye under examination from the first feature quantity and the second feature quantity.

Images