Method and apparatus for acquiring fundus information

Abstract

To provide an eyeground information acquisition method which does not need the correct positioning of an imaging unit to a subject eye with a simple optical system constitution and enables observation and imaging to be performed even in a case where a distance from the subject eye is far.SOLUTION: An eyeground information acquisition apparatus comprises an infinite distance correction optical system which forms an image of light passing through an objective lens as a portion of an eyeground image of a subject eye by an imaging unit arranged so as to be opposed to the objective lens by using a portion defined by a range of at least a pupil of the subject eye of a cornea and a crystalline lens of the subject eye as the objective lens opposed to the eyeground of the subject eye. An eyeground information acquisition method comprises an eyeground image acquisition step of acquiring an image of the subject eye including the eyeground image reflected in the range of the pupil of the subject eye in the imaging unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0006]

[0001] The present invention relates to a method for obtaining fundus information and a fundus information acquisition device.

Background Art

[0002] In fundus observation, the eyeball has a cornea and a lens, which are convex lenses with a high refractive index. Therefore, the light emerging from the fundus is strongly refracted by the cornea and the lens and forms an image at a very short distance from the eyeball. Thus, even when an examiner observes by peering into the subject's eye, it is impossible to directly observe the fundus of the subject eye.

[0003] Therefore, in fundus observation, it is generally common to use either a direct ophthalmoscope optical system or an indirect ophthalmoscope optical system.

[0004] In the direct ophthalmoscope optical system, a concave lens cancels out the refraction of the cornea and the lens to focus on the fundus and observe the fundus. At this time, since the examiner needs to bring his own eye close to the subject eye, the examiner can only observe an angle of about 10 degrees inside the eyeball at the tip of the pupil.

[0005] In the indirect ophthalmoscope optical system, the examiner observes the fundus of the subject eye by generating an intermediate image of an inverted image of the fundus by using a convex lens. The range that the examiner can observe is an angle of about 50 degrees inside the eyeball at the tip of the pupil. At this time, when the projection light is the same as the visual axis of the subject eye, the fundus cannot be seen due to corneal reflection. Also, when the projection light is tilted by several degrees from the visual axis of the subject eye, the fundus part to be observed and the fundus part irradiated with the projection light are different, so it becomes impossible to observe. Therefore, it is common for the examiner to slightly tilt the projection light and the visual axis of the subject eye by 1 to 2 degrees, or to project light using a ring-shaped light or the like to observe the fundus of the subject eye while avoiding corneal reflection.

[0006] Next, as a method for photographing the fundus, the so-called table-mounted fundus camera is generally used. This fundus camera is a type of inverted ophthalmoscope optical system, and is an evolution of the conventional inverted ophthalmoscope optical system. Furthermore, the Scanning Laser Ophthalmoscope (SLO) is also a type of system that further develops the inverted ophthalmoscope optical system, and because it uses a laser for fundus observation, using a strong convex lens tends to cause reflection, so it has specifications that make complex improvements to the conventional inverted ophthalmoscope observation method, such as using a concave mirror instead of some convex mirrors.

[0007] In recent years, particularly in developed countries, retinal diseases leading to blindness, such as age-related macular degeneration and diabetic retinopathy, have been increasing due to aging populations and the rise in diabetes. As a simple and early detection method for these diseases, the manufacture and sale of so-called mobile fundus cameras is attracting attention. Furthermore, such mobile fundus cameras are also in high demand in developing countries where the penetration rate of conventional fundus cameras is low.

[0008] Based on the above, in recent years, mobile fundus cameras that can be used in combination with smartphones and the like have been manufactured and sold. For example, Patent Document 1 describes a close-up imaging device that is detachably attached to a mobile terminal, comprising a light source, an imaging camera lens, and predetermined components, as a solution to the problem of easily enabling both anterior segment and fundus imaging by attaching it to a mobile terminal such as a smartphone. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] International Publication No. 2019 / 146792 [Overview of the project] [Problems that the invention aims to solve]

[0010] However, such mobile fundus cameras need to capture images at a very close distance to the eye being examined. Fundus cameras that utilize LED lights from smartphones, in particular, require the light emitted from the light source to illuminate and capture the fundus of the eye being examined. This can cause the pupil of the eye to constrict significantly due to the glare, making observation difficult for the examiner.

[0011] Therefore, methods such as using pupil-dilating drugs for observation are being considered. However, this observation method is extremely dangerous if the anterior segment examination is insufficient and narrow-angle angiotensin (where the distance between the cornea and lens of the eye is short, and when the pupil is dilated, the iris moves to the periphery and comes into contact with the cornea, blocking the drainage of intraocular fluid and causing a rapid increase in intraocular pressure, leading to blindness) is overlooked. For this reason, its implementation in developing countries is particularly risky. Moreover, the examiner must project the observation light onto the pupil of the eye, aligning all x, y, and z coordinates to within an error of 1 mm, which presents the problem of difficulty in achieving accurate positioning.

[0012] In addition, the objective lens needs to be positioned a few centimeters from the eyeball at a distance of approximately 20D (where D is the reciprocal of the focal length (m)), making it difficult to take images from a distance of 10cm or more. While components are manufactured worldwide to fix the illumination and imaging units in the appropriate position relative to the subject's eye, the need for precise alignment between the subject's eye and the smartphone remains unchanged. Furthermore, it can be said that it is completely impossible to take images naturally in a way that the subject does not become aware of their fundus being photographed during daily life.

[0013] This invention has been made in view of the above-mentioned problems, and aims to provide a method for acquiring fundus information that does not require precise positioning of the imaging unit relative to the eye under test, and that allows observation and imaging even when the distance from the eye under test is sufficiently large, using a simple optical system configuration. [Means for solving the problem]

[0014] The inventors of this invention conducted diligent research to solve the above problems. As a result, they discovered that the above problems can be solved by a fundus information acquisition method equipped with an infinity correction optical system, and thus completed the present invention.

[0015] In other words, a fundus information acquisition method according to one aspect of the present invention includes an infinity-correcting optical system that uses at least a portion of the cornea and lens of the eye defined by the range of the pupil of the eye as an objective lens facing the fundus of the eye, and images the light that has passed through the objective lens as part of a fundus image of the eye by an imaging unit positioned opposite the objective lens, and includes a fundus image acquisition step in which the imaging unit acquires an image of the eye, including a fundus image that is captured within the range of the pupil of the eye.

[0016] Another embodiment of the present invention relates to a fundus information acquisition method which includes an infinity correction optical system that uses at least a portion of the cornea and lens of the same test eye, defined by the range of the pupil of the test eye, as an objective lens facing the fundus of the test eye, and images the light that has passed through the objective lens as part of a fundus image of the test eye by an imaging unit positioned opposite the objective lens, and acquires multiple images of the test eye, including a fundus image that is captured within the range of the pupil of the test eye, while changing the relative position between the test eye and the imaging unit; a position information identification step which identifies the position of the fundus image in the fundus based on information obtained from the fundus image; and a fundus image synthesis step which generates a composite fundus image for the test eye by arranging multiple fundus images based on the position in the fundus identified for the fundus image.

[0017] In the above-described method for acquiring fundus information, the position information determination step may involve determining the position of the fundus image in the fundus based on characteristic features that appear in the fundus image originating from the components of the eye under test.

[0018] In the above-described method for acquiring fundus information, the feature portions that appear in the fundus image are the first reflection from the portion including the corneal apex of the subject eye and the second reflection from the anterior capsule of the lens of the subject eye. In the position information identification step, the position of the fundus image in the fundus may be identified based on the positional relationship between the bright spot of the first reflection and the bright spot of the second reflection.

[0019] In the above fundus information acquisition method, in the fundus image synthesis step, if the multiple fundus images acquired in the continuous fundus image acquisition step do not correspond to all regions of the fundus of the subject eye, a mathematical model trained by machine learning or deep learning may be used to identify the eyeball position corresponding to the region not acquired in the continuous fundus image acquisition step, and the continuous fundus image acquisition step may be performed at that eyeball position to acquire fundus images corresponding to the region not acquired in the continuous fundus image acquisition step.

[0020] In the above fundus information acquisition method, in the fundus image synthesis step, if the multiple fundus images acquired in the continuous fundus image acquisition step do not cover all regions of the fundus of the subject eye, a mathematical model trained by machine learning or deep learning may be used to supplement the fundus images corresponding to the regions not acquired in the continuous fundus image acquisition step using a standard image of the entire fundus of the subject eye.

[0021] In the above-described method for acquiring fundus information, multiple fundus images acquired in the sequential fundus image acquisition process may be used as training data for the mathematical model.

[0022] In the above fundus information acquisition method, the fundus image synthesis step may involve performing focus correction using one or more super-resolution techniques on each of the multiple fundus images acquired in the continuous fundus image acquisition step, and / or on the synthesized fundus image for the eye under test.

[0023] In the above method for acquiring fundus information, an abnormal site estimation information acquisition step of inputting a synthesized fundus image into a mathematical model trained by machine learning or deep learning using image data related to retinal diseases as training data to obtain information regarding the presence or absence of an abnormal site and information regarding the position of the abnormal site may be further provided.

[0024] In the above method for acquiring fundus information, a display step of displaying the synthesized fundus image on a display unit of the imaging unit may be further provided.

[0025] In the above method for acquiring fundus information, the imaging unit may be a camera of a mobile terminal.

[0026] In the above method for acquiring fundus information, the imaging unit may be a camera of a stationary device.

[0027] Further, a fundus information acquisition device according to an aspect of the present invention uses at least a site defined by the range of the pupil of the subject eye among the cornea and the lens of the subject eye as an objective lens facing the fundus of the subject eye, and an imaging unit disposed opposite to the objective lens forms an image of the light passing through the objective lens as a part of the fundus image of the subject eye. Based on an infinity-corrected optical system, an image of the subject eye including a fundus image captured within the range of the pupil of the subject eye is acquired by the imaging unit.

Advantages of the Invention

[0028] According to the present invention, with a simple optical system configuration, accurate alignment of the imaging unit with respect to the subject eye is not required, and observation and imaging can be performed even when the distance from the subject eye is sufficiently large.

Brief Description of the Drawings

[0029] [Figure 1] It is a schematic diagram of an infinity-corrected optical system. [Figure 2] It is a diagram showing an example of an image of a subject eye including a fundus image acquired in the fundus image acquisition step. [Figure 3]This is an example of a diagram illustrating changes in the scenery reflected in the subject's eye, changes in the limbus of the cornea of ​​the subject's eye, movement of the pupillary center within the limbus, or changes in the limbus of the pupil according to one embodiment of the present invention. [Figure 4] This is an example of a diagram showing multiple fundus images arranged together. [Figure 5] This is a schematic diagram of a synthetic fundus image according to one embodiment of the present invention. [Figure 6] This is a schematic diagram illustrating an example of using a mobile device as a fundus information acquisition device 4. [Figure 7] This is a schematic diagram showing an example of using a stationary device as a fundus information acquisition device 4. [Figure 8] This is a flowchart illustrating the implementation of a fundus information acquisition device according to one embodiment of the present invention. [Figure 9] This is the hardware configuration of a fundus information acquisition device according to one embodiment of the present invention. [Figure 10] This is a functional block configuration of a fundus information acquisition device according to one embodiment of the present invention. [Modes for carrying out the invention]

[0030] Embodiments of the present invention will be described with reference to the attached drawings. The following embodiments are provided to facilitate understanding of the present invention and are not intended to limit its interpretation. Furthermore, the present invention can be modified in various ways without departing from its essence. Those skilled in the art can also adopt embodiments in which each of the elements described below is replaced with equivalent components, and such embodiments are also included within the scope of the present invention.

[0031] 1. Fundus information acquisition device 4 Figure 1 shows an example of a fundus information acquisition device 4 that acquires an image of a test eye based on an infinity correction optical system 10. The fundus information acquisition device 4 of this embodiment uses at least a portion of the cornea 23 and lens 22 of the same test eye, defined by the range of the pupil 24 of the test eye 20, as an objective lens facing the fundus 21 of the test eye 20. Based on an infinity correction optical system 10 that images the reflected light 31 that has passed through the objective lens as part of the fundus image of the test eye 20 in an imaging unit 40 positioned opposite the objective lens, the imaging unit 40 acquires an image of the test eye 20, including the fundus image that is captured within the range of the pupil 24 of the test eye 20.

[0032] <Hardware Configuration> Figure 9 shows an example of the hardware configuration of the fundus information acquisition device 4. The fundus information acquisition device 4 includes a processor 11 such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), a storage device 12 such as memory, an HDD (Hard Disk Drive) and / or an SSD (Solid State Drive), a communication interface 13 for wired or wireless communication, an input device 14 for receiving input operations, and an output device 15 for outputting information. The input device 14 is, for example, a keyboard, a touch panel, a mouse and / or a microphone. The output device 15 is, for example, a display, a touch panel and / or a speaker.

[0033] In this embodiment, the imaging unit 40 may be configured to be included in the input device 14, or it may be configured to be included in an external device of the fundus information acquisition device 10, and fundus images acquired by the imaging unit 40 may be input from the input device 14 via a communication line or the like.

[0034] <Functional Block Configuration> Figure 10 shows an example of the functional block configuration of the fundus information acquisition device 4. The fundus information acquisition device 4 includes a storage unit 110 and a control unit 120. The storage unit 110 can be implemented using a storage device 12 provided by the fundus information acquisition device 4. The control unit 120 can be implemented by the processor 11 of the fundus information acquisition device 4 executing a program stored in the storage device 12. This program can be stored in a storage medium. The storage medium on which the program is stored may be a non-transitory computer-readable medium. The non-transitory storage medium is not particularly limited, but may be, for example, a USB memory stick or a CD-ROM.

[0035] Figure 8 is a flowchart showing each step in the implementation of the fundus information acquisition device 4 according to one embodiment of the present invention, and each step is executed by the control unit 120 of the fundus information acquisition device 4. Each step will be described in detail below with reference to Figure 8.

[0036] The fundus information acquisition device 4 includes an infinity correction step S101 and a fundus image acquisition step S102, and by performing these steps, it acquires fundus information of the subject eye 20. The fundus information acquisition device 4 may also include a position information identification step S103, a fundus image synthesis step S104, and a display step S105. Furthermore, the fundus information acquisition device 4 may include other steps as needed.

[0037] 1.1. Infinity correction process S101 As shown in Figure 8, the infinity correction step S101 is a step for configuring the infinity correction optical system 10. Here, the infinity correction optical system is configured so that light passing through the objective lens forms a parallel beam of light. In this embodiment, as shown in Figure 1, in the infinity correction optical system 10, the reflected light 31 that leaves the fundus 21 of the subject eye 20, which is the object to be observed, does not form an image in the objective lens, which is the crystalline lens 22 and the cornea 23, but enters the imaging lens 41 of the imaging unit 40 as a parallel beam of light 32 at infinity, and the imaging lens 41 forms an image on the image sensor 42. The fundus information acquisition device 4 acquires an image of the subject eye 20, including a fundus image that is captured within the range of the pupil of the subject eye, in the imaging unit 40.

[0038] Conventional techniques using inverted ophthalmoscope optics for fundus observation and fundus photography require precise positioning of the imaging unit relative to the eye under examination. Similarly, conventional fundus photography using straight ophthalmoscope optics also requires precise positioning of the imaging unit relative to the eye under examination, and because it is necessary to take images at a very close distance to the eye, there were risks associated with illuminating the eye at close range.

[0039] On the other hand, in this embodiment, by configuring the infinity correction optical system 10, it is not necessary to adjust the position of the imaging lens, etc., to match the position of the intermediate image formed via the objective lens, as in conventional inverted ophthalmoscope optical systems. Therefore, the distance between the objective lens and the imaging lens, etc., can be freely adjusted, and fundus observation can be performed easily. In addition, since the test eye 20 can be kept at a sufficient distance from the imaging unit 40, it can be used easily and safely.

[0040] In this embodiment, the means for inducing the subject to look at infinity are not particularly limited, but for example, the fundus information acquisition device 4 may be equipped with a fixation target projection means. In this configuration, the fundus information acquisition device 4 has a fixation lamp that emits visible light, and by turning on the fixation lamp, a fixation target can be presented to the subject, and the subject's eye can be guided to look in the direction of infinity.

[0041] In this embodiment, "infinity" does not necessarily mean that the light emitted from the fundus 21 becomes a parallel beam of light in the strict sense after passing through the lens 22 and cornea 23. It goes without saying that it is preferable for the light that leaves the fundus 21 and passes through the lens 22 and cornea 23 to form a parallel beam of light when the subject looks precisely at infinity, but it is sufficient if it approximates a parallel beam of light to the extent that the subject eye 20 can be observed with a certain degree of accuracy that satisfies the minimum practical requirements. As a certain degree of accuracy that satisfies the minimum practical requirements, it is sufficient to be able to obtain information about the color of the fundus 21 of the subject eye 20.

[0042] If the fundus information obtained in this embodiment includes at least information about the color of the fundus 21, it can be used to diagnose glaucoma and optic nerve atrophy. Furthermore, if it includes information about the shape of the fundus 21, it can be used to diagnose further retinal diseases and the like.

[0043] Furthermore, in the imaging unit 40 of the fundus information acquisition device 4, which acquires an image of the test eye 20 based on the infinity correction optical system 10, two or more imaging lenses constituting an imaging lens 41 as a system may be used as needed. When two or more imaging lenses are used, the lens that forms an image on the image sensor 42 is called the first imaging lens, and the other lens is called the second imaging lens. The second imaging lens may form an intermediate image.

[0044] 1.1.1. Objective lens In this embodiment, at least a portion 51 of the cornea 23 and lens 22 of the subject eye 20, defined by the range of the pupil 24 of the subject eye 20, is used as the objective lens facing the fundus 21 of the subject eye 20. The objective lens only needs to be configured such that the reflected light 31 emitted from the fundus 21 becomes a parallel beam of light 32 when the subject looks at infinity. If the subject is wearing glasses or the like, the objective lens means an optical system formed integrally with at least a portion of the cornea 23 and lens 22 of the subject eye 20, defined by the range of the pupil 24 of the subject eye 20, and the glasses.

[0045] 1.1.1.1. Pupil 24 The pupil is the opening surrounded by the iris of the eye, and its diameter changes in response to the amount of light. Changes in pupil diameter contribute to adjusting the amount of light projected onto the retina. Therefore, when strong light is shone on the subject eye 20, pupil constriction occurs, and the pupil diameter decreases, resulting in a tendency for the area in which the fundus 21 can be observed to narrow. For this reason, in this embodiment, it is preferable to acquire fundus images in a state where the ambient light is as dark as possible.

[0046] 1.1.1.2. Crystalline lens 22 The lens is a transparent, convex-lens-shaped substance located at the front of the eyeball that refracts light from the outside to form an image on the retina. The lens 22 automatically adjusts so that it becomes thicker when a person is looking at something close up and thinner when looking at something far away, allowing the eye to focus on both near and far objects.

[0047] In this embodiment, the thickness of the lens 22 is adjusted so that it focuses on infinity by directing the subject to look at infinity.

[0048] 1.1.1.3.Cornea 23 The cornea is a transparent membrane that covers the front surface of the eyeball and, together with the lens, functions as a convex lens.

[0049] 1.1.2. Imaging unit 40 In this embodiment, the imaging unit 40 includes an imaging lens 41 and an image sensor 42 for forming an image of a parallel light beam 32 that has passed through the crystalline lens 22, which is the objective lens, and the cornea 23. The imaging lens 41 can have any desired refractive index.

[0050] The image sensor 42 is sensitive to at least visible light and infrared light. The image sensor 42 is not particularly limited, but it is preferable that it has a viewing function that processes several adjacent photoelectric conversion elements together as one pixel. As a result, the image sensor 42 can exhibit sufficient sensitivity to obtain a fundus image even with reflected light 31 from the fundus 21 of the test eye 20, where the amount of light is low.

[0051] Figure 6 is a schematic diagram showing an example of using a mobile terminal as a fundus information acquisition device 4. For example, the camera of the mobile terminal is used as the imaging unit 40. The mobile terminal is not particularly limited, but examples include information processing devices such as smartphones, tablet computers, notebook computers, or workstation computers. By using the camera of the mobile terminal as the imaging unit 40, a simple optical system using the mobile terminal can be configured as shown in Figure 6, and fundus information can be easily acquired. Here, a terminal refers to a device that is connected to a line or network and can act as the main entity for communicating with other devices.

[0052] Figure 7 is a schematic diagram showing an example of using a stationary device as a fundus information acquisition device 4. For example, the camera of the stationary device is used as the imaging unit 40. When a stationary device is used as a fundus information acquisition device, as shown in Figure 7, the relative position between the subject eye 20 and the imaging unit 40 can be easily changed by the subject moving while facing the stationary device. Furthermore, the fundus image acquisition process S102 in this embodiment can be performed continuously for a certain period of time without any operation by the subject.

[0053] In this case, when a stationary device is used as the imaging unit 40, the distance between the subject eye 20 and the imaging unit 40 tends to be larger compared to when a mobile terminal is used as the imaging unit 40, and consequently, the resolution of the obtained fundus image tends to decrease. Therefore, it is preferable to further include a second imaging lens that forms an intermediate image of the fundus image of the subject eye 20 in order to acquire fundus information while increasing the resolution. In this embodiment, the second imaging lens refers to the lens that forms the intermediate image and does not refer to the lens that forms an image on the image sensor 42.

[0054] Furthermore, the stationary device may be a smart mirror. By using a smart mirror, subjects can be photographed naturally and without conscious effort in their daily lives. Here, a smart mirror refers to a mirror-type terminal that has internet connectivity and can reflect the subject's own image, and a camera capable of photographing the subject's eyes.

[0055] The imaging unit 40 may optionally include a first imaging lens, a light source provided around the first imaging lens, and a polarizing plate provided in front of the first imaging lens. By providing a light source, which is a light irradiation means, the accuracy of fundus information can be improved based on the reflected light from the subject eye 20. Furthermore, by providing a polarizing plate, surface reflections of light from objects other than the subject eye 20 can be removed, and in particular, the colors of the resulting fundus image can be made more vivid.

[0056] The light surrounding the first imaging lens may be a ring-shaped light positioned on the outer circumference of the first imaging lens. By using such a light, the light emitted from the light can be more uniformly irradiated onto the subject eye 20, and a high-precision fundus image can be obtained.

[0057] When a mobile terminal or a camera of a stationary device is used as the imaging unit 40, the light provided around the first imaging lens may be a light built into the mobile terminal or stationary device. This makes it easier to acquire fundus image information.

[0058] 1.2. Fundus image acquisition step S102 As shown in Figure 8, the fundus image acquisition step S102 is a step of acquiring a fundus image. Conventional known means can be used as means for acquiring the fundus image. The image acquisition means is not particularly limited, but for example, it can be acquired by taking a picture with a camera or other photographic means. In the image acquisition means, it is preferable to include a light irradiation means so that an image of the subject eye is acquired based on the reflected light from the subject eye. Such a light irradiation means can improve the accuracy of information, especially regarding the color of the fundus.

[0059] Furthermore, the fundus image acquisition process S102 includes, in one embodiment, a continuous fundus image acquisition process. The continuous fundus image acquisition process is a process for continuously acquiring multiple fundus images while changing the relative position between the subject eye 20 and the imaging unit 40. This will be explained in detail below.

[0060] 1.2.1. Means for changing the relative position between the subject eye 20 and the imaging unit 40 A conventionally known method can be used to change the relative position between the subject eye 20 and the imaging unit 40. This method is not particularly limited, but for example, it may be changed by having the subject change the direction in which they look at infinity without changing the position of the imaging unit 40, or by changing the position of the imaging unit 40 without changing the position of the subject.

[0061] The means for changing the direction in which the subject views infinity are not particularly limited, but one example is to use a fixation target projection means that presents multiple fixation targets. The fixation target projection means has a fixation lamp that emits visible light, and by turning on the fixation lamp, multiple fixation targets can be presented to the subject, and the subject eye 20 can be guided to look in multiple directions.

[0062] The number of images acquired is not particularly limited, as they are used in the fundus image synthesis process S104, but for example, 30 or more is preferred, 100 or more is more preferred, and 300 or more is even more preferred.

[0063] 1.3. Location information identification step S103 As shown in Figure 8, the position information identification step S103 is a step of identifying the position of the fundus image in the fundus 21 based on information obtained from the eye image of the subject. Preferably, this step is performed by identifying the position of the fundus image in the fundus 21 based on characteristic parts that appear in the eye image of the subject, originating from the components of the eye 20 of the subject. This makes it possible to improve the accuracy of the fundus information that can be obtained.

[0064] Figure 2 shows an example of a subject eye image including a fundus image acquired by a fundus image acquisition means. Figure 4 shows an example of a diagram in which multiple fundus images are arranged. First, as one embodiment of the process, the characteristic portion is defined as a first reflection from the portion including the corneal apex of the subject eye 20 and a second reflection from the anterior capsule of the lens of the subject eye 20, and the position of the fundus image in the fundus 21 is determined based on the positional relationship between the bright spot 52 of the first reflection and the bright spot 53 of the second reflection. Specifically, the distance between the center point 61 of the subject eye 20 and the fundus image is determined based on the distance between 52 and 53, and further, the orientation of the fundus image from the center point 61 of the subject eye 20 is determined by arranging the fundus image so that a straight line passing through 52 and 53 passes through the center point 61 of the eyeball. This method of determination reduces the number of computer calculations required in the generation of a composite fundus image, making it possible to acquire fundus information quickly and easily.

[0065] Figure 3 is an example of a diagram illustrating changes in the scenery reflected in the pupil of the subject eye, such as the movement of the scenery reflected in the pupil of the subject eye, changes in the limbus of the cornea of ​​the subject eye, movement of the pupillary center within the limbus, or changes in the limbus of the pupil. In particular, in an infinity correction optical system 10 where the incident light is insufficient, such as when only ambient light is used, possible embodiments of this process include: detecting a change in the scenery 54 reflected in the subject eye 20 that occurs in response to the rotational movement 59 of the subject eye, thereby determining the distance between the center point 61 of the subject eye 20 and the fundus image; detecting a change in the limbus 55 that occurs in response to the rotational movement 59 of the eye, such as the limbus 55 changing from a circular to a substantially elliptical shape, thereby determining the distance between the center point 61 of the subject eye 20 and the fundus image; detecting a movement 58 of the pupillary center within the limbus that occurs in response to the rotational movement 59 of the eye, thereby determining the distance between the center point 61 of the subject eye 20 and the fundus image; or detecting a change in the pupillary limbus 57 that occurs in response to the rotational movement 59, thereby determining the distance between the center point 61 of the subject eye 20 and the fundus image. It is even more preferable to combine these determination methods.

[0066] Furthermore, as another embodiment of the process, conventionally known means for determining the position of the fundus image in the fundus 21 can be used, and are not particularly limited, but for example, an embodiment can be made that includes a feature point detection means for detecting feature points in an image and a means for simultaneously storing the feature quantities and coordinate data of the detected feature points. Here, since the acquired fundus images are fundus images of the same subject eye 20, among the multiple fundus images there are images that capture common fundus regions and feature points, so the position information of other fundus images can be determined based on one fundus image. More specifically, for example, the intersections of blood vessels displayed in each fundus image can be extracted as feature points, correlation matching of image data around the intersections of blood vessels can be performed between two fundus images to find common pairs of intersections, and the position information can be determined by superimposing the two fundus images based on the position of the found pairs of intersections.

[0067] Here, "among multiple fundus images, there are images that capture common fundus regions and feature points" does not mean that all fundus images of the same subject eye 20 capture common fundus regions and feature points. Rather, it means that among the multiple fundus images acquired, there are images that capture common fundus regions and feature points, even if they are not necessarily the same fundus regions. 1.4. Fundus image synthesis step S104

[0068] As shown in Figure 8, the fundus image synthesis step S104 is a step in which multiple fundus images are arranged based on the positions identified in the position information identification step S103 to generate a composite fundus image for the subject eye 20.

[0069] In this embodiment, since an infinity correction optical system 10 is used, the range of the fundus 21 captured in the fundus image is narrower compared to an inverted ophthalmoscope optical system. However, by including the fundus image synthesis step S104, it is possible to generate a composite fundus image for observing the fundus 21 over a wide area.

[0070] Furthermore, by including the fundus image synthesis process S104, precise positioning of the imaging unit 40 relative to the subject eye 20 is not required, and more detailed observation and imaging becomes possible even when the distance between the imaging unit 40 and the subject eye 20 is relatively large.

[0071] Figure 5 shows an example of a composite fundus image. The composite fundus image is not particularly limited, but for example, it can be generated by arranging fundus images 72 based on the position information identified in the position information identification step S103, and then connecting them together by superimposing common fundus regions, including blood vessels 71, etc., captured in each fundus image 72.

[0072] In this case, if the multiple fundus images 72 acquired in the fundus image acquisition step S102 do not correspond to all regions of the fundus 21 of the subject eye 20, as in the first embodiment, a mathematical model trained by machine learning or deep learning can be used to identify an eye position where fundus images 72 corresponding to the regions not acquired in the fundus image acquisition step S102 can be captured, and by performing the fundus image acquisition step S102 at that eye position, fundus images 72 corresponding to the regions not acquired in the fundus image acquisition step S102 can be supplemented.

[0073] Furthermore, as a second embodiment described above, a mathematical model trained by machine learning or deep learning can be used to complement the fundus image 72 corresponding to the unacquired region in the fundus image acquisition step S102 by using a standard image of the entire fundus of the subject eye 20. A standard image refers to a fundus image acquired by a conventionally known image acquisition method. Conventionally known standard image acquisition methods are not particularly limited, but examples include fundus photography using a wide-angle scanning ophthalmoscope.

[0074] Here, as training data for the mathematical models in the first and second embodiments described above, multiple fundus images 72 acquired in the fundus image acquisition step S102 can be used.

[0075] Furthermore, as a third embodiment described above, as a method for supplementing fundus images 72 corresponding to areas not acquired in the fundus image acquisition step S102, multiple fundus images 72 acquired in the fundus image acquisition step S102 can be used as training data for the mathematical model to generate and supplement fundus images 72 corresponding to areas not acquired in the fundus image acquisition step S102. Although this increases the number of calculations performed by the computer, it eliminates the need to prepare standard images for the subject's fundus 21, thus enabling convenient fundus observation.

[0076] Here, a machine learning model (hereinafter referred to as a machine learning model) is an example of a mathematical model. A machine learning model includes a model that has a predetermined model structure and parameters that change through the learning process, and whose recognition accuracy is improved by optimizing its processing parameters based on experience obtained from training data. In other words, a machine learning model is a model that learns the optimal processing parameters through the learning process. The algorithm of a machine learning model can be, for example, a support vector machine, logistic regression, random forest, neural network, etc., and is not particularly limited in type, but from the viewpoint of obtaining a synthetic fundus image even when there is little training data, it is preferable to use a neural network. The machine learning model that performs this learning may have already undergone some learning using the training data, or it may be one that has not yet been learned.

[0077] The fundus image synthesis step S104 further preferably performs focus correction using one or more super-resolution techniques on each of the multiple fundus images 72 acquired in the fundus image acquisition step S102, and / or on the synthesized fundus image for the subject eye 20. This improves the resolution of the fundus image. Super-resolution techniques are techniques that convert existing low-resolution (LR) images into high-resolution (HR) images using a software algorithm method through image processing; in simpler terms, they are techniques that increase resolution by shifting pixels.

[0078] 1.5. Display process S105 As shown in Figure 8, the display step S105 is a step in which a composite fundus image is displayed. In this embodiment, it is preferable that the fundus information acquisition device 4 has a display step S105. This makes it easier to check the fundus information of the subject.

[0079] Furthermore, by including a display step S105, in the first embodiment, when the multiple fundus images 72 acquired in the fundus image acquisition step S102 do not correspond to all areas of the fundus 21 of the subject eye 20, a mathematical model trained by machine learning or deep learning can be used to identify an eye position where fundus images 72 corresponding to areas not acquired in the fundus image acquisition step S102 can be captured, and when the fundus image acquisition step S102 is performed at that eye position, it is possible to more easily guide the subject to look in a desired direction.

[0080] 1.6. Other processes In this embodiment, the fundus information acquisition device 4 may include other steps as needed, in addition to the steps described above. Other steps are not particularly limited, but for example, an abnormal area estimation information acquisition step may be included. In this embodiment, the device may further include an abnormal area estimation information acquisition step, in which the synthesized fundus image is input into a mathematical model trained by machine learning or deep learning using image data related to retinal diseases as training data, thereby acquiring information regarding the estimation of the presence or absence of abnormal areas and information regarding the estimated location of abnormal areas. This can be used to easily and quickly detect retinal diseases and the like. [Industrial applicability]

[0081] The present invention makes it possible to acquire still images and videos for use in ophthalmic examinations using, for example, a smartphone or other mobile device, although this is not particularly limited. Furthermore, the fundus information acquisition method according to the present invention can be constructed at a very low cost without using special parts, compared to fundus observation methods using slit-lamp microscopes or handheld slit-lamp microscopes, and is therefore expected to be widely adopted around the world.

[0082] Furthermore, it can be used clinically. Specifically, in ophthalmologists' examinations, the eye of the patient is observed and diagnosed using a table-mounted fundus camera. However, this is difficult and requires skilled techniques when examining children or bedridden elderly people. With this invention, however, regardless of the subject's posture, fundus images can be captured very easily using a mobile device or the like simply by having them look at infinity. Subsequently, a composite fundus image can be generated using a commonly used composite image acquisition method and used for observation and diagnosis. Therefore, it is expected to be used in remote medical care in rural areas and in aid to developing countries.

[0083] Furthermore, the present invention can also be used for observing and photographing the fundus of animals. In particular, it is possible to capture still images and videos of the eyes of companion animals such as pets, and of large animals kept in zoos where aligning the imaging unit with the eye of the subject is especially difficult, and to obtain ophthalmic findings of the above animals.

[0084] Furthermore, by analyzing the data as big data using AI, it is expected that the accuracy of ophthalmologists' diagnoses can be improved. Ultimately, it can be used as a self-diagnosis tool for subjects, making it possible to further develop ophthalmological practice itself. [Explanation of Symbols]

[0085] 4…Fundus information acquisition device, 10…Infinity correction optical system, 20…Subject eye, 21…Fundus, 22…Lens, 23…Cornea, 24…Pupil, 25…Retina, 26…Optic nerve head, 31…Light, 32…Parallel light beam, 40…Imaging unit, 41…Imaging lens, 42…Image sensor, 51…Area defined by the pupil's range, 52…Bright spot of the first reflection, 53…Bright spot of the second reflection, 54…Scene reflected in the subject eye, 55…Limus, 56…Changes in the limbus, 57…Limus of the pupil, 58…Movement of the pupil's center within the limbus, 59…Rotational movement of the eyeball, 61…Center point of the eyeball, 71…Blood vessels, 72…Fundus image, 11…Processor, 12…Storage device, 13…Communication interface, 14…Input device, 15…Output device, 110…Memory unit, 120…Control unit

Claims

1. A step of configuring an infinity-correcting optical system in which, when a subject looks at infinity, light emanating from the fundus of the subject's eye and passing through the objective lens enters an imaging unit positioned opposite the objective lens as a parallel beam or in a state approximating a parallel beam, and forms an image as part of the fundus image of the subject's eye, wherein the objective lens uses at least a portion of the cornea and lens of the subject's eye that is defined by the range of the pupil of the subject's eye; The imaging unit performs a fundus image acquisition step, in which multiple images of the subject eye are acquired, including a fundus image captured within the range of the pupil of the subject eye, while changing the relative position between the subject eye and the imaging unit. A position information identification step, which identifies the position of the fundus image in the fundus based on the information obtained from the subject's eye image, A fundus image synthesis step, which generates a composite fundus image for the subject eye by arranging multiple fundus images based on the position in the fundus identified for the fundus image, A method for acquiring fundus information, comprising the following features.

2. In the position information identification step, the position of the fundus image in the fundus is identified based on the characteristic portion that originates from the component of the eye being examined and appears in the image of the eye being examined. The method for obtaining fundus information according to claim 1.

3. The characteristic features appearing in the image of the eye are a first reflection from the portion including the corneal apex of the eye, and a second reflection from the anterior capsule of the lens of the eye. In the position information identification step, the position of the fundus image in the fundus is identified based on the positional relationship between the bright spot of the first reflection and the bright spot of the second reflection. The method for obtaining fundus information according to claim 2.

4. The characteristic features that appeared in the image of the eye being tested are the scenery reflected in the eye. In the position information identification step, the position of the fundus image in the fundus is identified based on the change in the scenery reflected in the subject's eye. The method for obtaining fundus information according to claim 2.

5. The characteristic portion that appeared in the image of the eye under investigation is the limbus of the eye under investigation, the center of the pupil within the limbus, or the pupillary limbus. In the position information identification step, the position of the fundus image in the fundus is identified based on the limbus of the eye being examined, the movement of the pupillary center within the limbus, or changes in the limbus. The method for obtaining fundus information according to claim 2.

6. In the fundus image synthesis step, when the multiple fundus images acquired in the fundus image acquisition step do not cover all regions of the fundus of the subject eye, a mathematical model trained by machine learning or deep learning is used to identify the eyeball position corresponding to the region not acquired in the fundus image acquisition step, and the fundus image acquisition step is performed at that eyeball position to supplement the fundus images corresponding to the region not acquired in the fundus image acquisition step. A method for acquiring fundus information according to any one of claims 1 to 5.

7. In the fundus image synthesis step, if the multiple fundus images acquired in the fundus image acquisition step do not cover all regions of the fundus of the subject eye, a mathematical model trained by machine learning or deep learning is used to supplement the fundus images corresponding to the regions not acquired in the fundus image acquisition step using a standard image of the entire fundus of the subject eye. A method for acquiring fundus information according to any one of claims 1 to 6.

8. As training data for the mathematical model, multiple fundus images acquired in the fundus image acquisition process are used. The method for obtaining fundus information according to claim 6 or 7.

9. In the fundus image synthesis step, focus correction is performed on each of the multiple fundus images acquired in the fundus image acquisition step, and / or on the synthesized fundus image for the eye under test, using one or more super-resolution techniques. A method for acquiring fundus information according to any one of claims 1 to 8.

10. The system further comprises an abnormality site estimation information acquisition step, in which the synthesized fundus image is input into a mathematical model trained by machine learning or deep learning using image data related to retinal diseases as training data, thereby acquiring information regarding the estimation of the presence or absence of abnormal areas and information regarding the estimation of the location of abnormal areas. A method for acquiring fundus information according to any one of claims 1 to 9.

11. The system further comprises a display step of displaying the aforementioned composite fundus image. A method for acquiring fundus information according to any one of claims 1 to 10.

12. The imaging unit is the camera of a mobile device. A method for acquiring fundus information according to any one of claims 1 to 11.

13. The imaging unit is a camera of a stationary device. A method for acquiring fundus information according to any one of claims 1 to 11.

14. A fundus information acquisition device equipped with an imaging unit, An infinity-correcting optical system in which, when a subject looks at infinity, light emanating from the fundus of the subject's eye and passing through the objective lens enters the imaging unit positioned opposite the objective lens as a parallel beam or in a state approximating a parallel beam, and forms an image as part of the fundus image of the subject's eye, wherein the objective lens uses at least a portion of the cornea and lens of the subject's eye that is defined by the range of the pupil of the subject's eye, and means for acquiring multiple images of the subject's eye, including a fundus image that is captured within the range of the pupil of the subject's eye, while changing the relative position between the subject's eye and the imaging unit. A means for determining the position of the fundus image in the fundus based on the information obtained from the subject eye image, A means for generating a composite fundus image for the eye under test by arranging multiple fundus images based on the position in the fundus identified for the fundus image, A fundus information acquisition device equipped with the following features.

Images