Fundus information acquisition method and fundus information acquisition apparatus

Abstract

Provided is an eye fundus information acquisition method in which, by a simple optical system configuration, accurate alignment of a photographing section with respect to a subject eye is not required, and observation and photographing can be performed even at a long distance from the subject eye. The eye fundus information acquisition method includes an infinity-corrected optical system that uses at least a portion of a cornea and a lens of the same subject eye, which is defined by a range of a pupil of the subject eye, as an objective lens that opposes an eye fundus of the subject eye, and causes light that has passed through the objective lens to be imaged as a portion of an eye fundus image of the subject eye by a photographing section that is disposed opposite the objective lens.

Description

[0001] Cross-reference of related applications

[0002] This application is based on Japanese Patent Application No. 2021-178241, filed on October 29, 2021, and its contents are incorporated herein by reference. Technical Field

[0003] This disclosure relates to a method and device for acquiring fundus information. Background Technology

[0004] In fundus examination, because the eyeball has a cornea and a lens, both of which are convex lenses with high refractive index, light emanating from the fundus is significantly refracted through the cornea and lens, forming an image at a very close distance to the eyeball. Therefore, even if the examiner observes the subject's eye by peering into it, they cannot directly observe the fundus of the eye being examined. Thus, fundus examination typically uses either the direct ophthalmoscopy optical system or the indirect ophthalmoscopy optical system.

[0005] In a direct ophthalmoscopy optical system, concave lenses are used to counteract the refraction of the cornea and lens, allowing for focused observation of the fundus. Because the examiner needs to bring their own eye close to the eye being examined, they can only observe approximately 10 degrees of the eyeball behind the pupil.

[0006] In an indirect ophthalmoscopy optical system, the examiner observes the fundus of the tested eye by generating an intermediate image of the inverted fundus obtained using a convex lens. The examiner can observe an angle of approximately 50 degrees within the eyeball behind the pupil. At this angle, if the projected light is aligned with the visual axis of the tested eye, the fundus cannot be seen due to corneal reflection. Furthermore, if the projected light is tilted several degrees away from the visual axis of the tested eye, the observed portion of the fundus differs from the portion illuminated by the projected light, making observation impossible. Therefore, examiners typically avoid corneal reflection and observe the fundus by tilting the projected light only 1-2 degrees away from the visual axis of the tested eye or by using a ring-shaped lamp or other projection methods.

[0007] Secondly, as a method of fundus imaging, a stage-mounted fundus camera is typically used. This fundus camera is a type of indirect ophthalmoscope optical system, developed from existing indirect ophthalmoscope optical systems. Additionally, the Scanning Laser Ophthalmoscope (SLO) is another system developed through further advancements in indirect ophthalmoscope optical systems. Since lasers are used in fundus observation, using high-power convex lenses can easily cause reflections. Therefore, methods such as replacing some convex lenses with concave mirrors have been developed, adding complex modifications to existing types of indirect ophthalmoscope-based observation methods.

[0008] In recent years, especially in developed countries, retinal diseases that can lead to blindness, such as age-related macular degeneration and diabetic retinopathy, have increased with the aging population and the rise in diabetes. As a simple method for early detection of these diseases, the manufacture and sale of mobile fundus cameras have attracted considerable attention. Furthermore, such mobile fundus cameras are in high demand even in developing countries where the penetration rate of conventional fundus cameras is low.

[0009] Based on the above, in recent years, there has been a focus on manufacturing and selling mobile fundus cameras that can be used in conjunction with smartphones and the like. For example, Patent Document 1 describes a technical problem that allows for easy imaging of both the anterior eye and the fundus by being mounted on a mobile terminal such as a smartphone, and provides a close-up imaging device that includes a light source, a camera lens, and predetermined components as a solution. This close-up imaging device can be detachably mounted on the mobile terminal.

[0010] Existing technical documents

[0011] Patent documents

[0012] Patent Document 1: International Publication No. 2019 / 146792 Summary of the Invention

[0013] However, such mobile fundus cameras require taking pictures at a very close distance to the eye being tested. In particular, fundus cameras that utilize LED lights from smartphones or similar devices present a problem: because the fundus of the eye being tested is illuminated by the light from its own lamp while observation and imaging are being performed, the examiner's observation becomes difficult due to the pupil of the eye being tested constricting sharply due to the light.

[0014] Therefore, methods for observation, such as using mydriatic drugs, are being researched. However, this method is extremely dangerous in cases where the anterior eye examination is insufficient and narrow-angle eyes are overlooked (where the distance between the cornea and lens of the tested eye is short; if mydriasis is performed, the iris moving to the periphery will come into contact with the cornea, blocking the outlet of intraocular fluid, causing a sudden increase in intraocular pressure and leading to blindness). Therefore, its implementation in developing countries carries a particularly high risk. Furthermore, the examiner needs to align the observation light relative to the pupil of the tested eye, setting the position coordinates to the xyz axes and ensuring the entire observation light is within an error range of 1 mm, which presents a challenge in achieving accurate positional alignment.

[0015] Furthermore, the objective lens needs to be placed approximately 20D (D refers to the reciprocal of the focal length (m)) a few centimeters away from the eyeball, making it difficult to take pictures at distances greater than 10cm. Regarding methods for placing the objective lens a few centimeters away from the eyeball, parts have been developed worldwide that pre-fix the illumination and imaging parts to a suitable position relative to the eye being tested. However, the need for precise alignment between the tested eye and a smartphone, etc., remains unchanged. Moreover, it is practically impossible to take pictures naturally in everyday life without realizing that the fundus of the tested eye is being photographed.

[0016] This disclosure is made in view of the above-mentioned problems, and its purpose is to provide a method for acquiring fundus information. This method, through the structure of a simple optical system, does not require the precise alignment of the imaging unit with respect to the eye being tested, and can perform observation and imaging even at a sufficiently far distance from the eye being tested.

[0017] The inventors conducted in-depth research to solve the aforementioned technical problems. As a result, they discovered that a method for acquiring fundus information using an infinity-corrected optical system could solve these problems, thus completing this invention.

[0018] That is, the fundus information acquisition method disclosed herein includes an infinity-corrected optical system. The infinity-corrected optical system uses a portion of the cornea and lens of the eye under test, defined at least by the pupil of the eye under test, as an objective lens facing the fundus of the eye under test. The light passing through the objective lens is imaged as part of the fundus image of the eye under test by an imaging unit arranged opposite to the objective lens. The fundus information acquisition method includes a fundus image acquisition step, in which an image of the eye under test, including the fundus image reflected within the pupil of the eye under test, is acquired in the imaging unit.

[0019] Another aspect of this disclosure relates to a fundus information acquisition method comprising an infinity-corrected optical system. This infinity-corrected optical system uses a portion of the cornea and lens of the same tested eye, defined at least by the pupil of the tested eye, as an objective lens opposing the fundus of the tested eye. An imaging unit positioned opposite the objective lens images the light passing through it as part of a fundus image of the tested eye. The fundus information acquisition method comprises: a continuous fundus image acquisition step, acquiring multiple images of the tested eye, including fundus images projected onto the pupil of the tested eye, while changing the relative position of the tested eye and the imaging unit; a position information determination step, determining the position of the fundus images in the fundus based on information obtained from the fundus images; and a fundus image synthesis step, configuring multiple fundus images and generating a synthesized fundus image of the tested eye based on the determined position of the fundus images in the fundus.

[0020] In the above-mentioned method for acquiring fundus information, the position of the fundus image in the fundus can also be determined in the position information determination step based on the feature parts that appear in the fundus image from the structural parts of the tested eye.

[0021] In the above-mentioned method for acquiring fundus information, the feature portion displayed in the fundus image may be a first reflection from a portion including the corneal apex of the tested eye and a second reflection from the anterior capsule of the lens of the tested eye. In the position information determination step, the position of the fundus image in the fundus is determined based on the positional relationship between the bright spots of the first reflection and the bright spots of the second reflection.

[0022] In the above-mentioned fundus information acquisition method, in the fundus image synthesis process, when multiple fundus images acquired in the continuous fundus image acquisition process do not correspond to the entire area of ​​the fundus of the tested eye, a mathematical model trained by machine learning or deep learning is used to determine the eyeball position corresponding to the area not acquired in the continuous fundus image acquisition process, and the continuous fundus image acquisition process is performed at the eyeball position, thereby acquiring the fundus image corresponding to the area not acquired in the continuous fundus image acquisition process.

[0023] In the above-mentioned fundus information acquisition method, it is also possible that, in the fundus image synthesis process, when multiple fundus images acquired in the continuous fundus image acquisition process do not correspond to the entire area of ​​the fundus of the tested eye, a mathematical model trained by machine learning or deep learning is used to supplement the fundus images corresponding to the areas not acquired in the continuous fundus image acquisition process with a standard image of the entire fundus of the tested eye.

[0024] In the above-mentioned method for acquiring fundus information, multiple fundus images acquired in the continuous fundus image acquisition process can also be used as training data for a mathematical model.

[0025] In the above-mentioned method for acquiring fundus information, it is also possible to use one or more super-resolution techniques in the fundus image synthesis process to perform focus correction on each of the multiple fundus images acquired in the continuous fundus image acquisition process and / or the synthesized fundus image about the tested eye.

[0026] In the above-mentioned fundus information acquisition method, the fundus information acquisition method may also include an abnormal site estimation information acquisition step. In the abnormal site estimation information acquisition step, by inputting the synthesized fundus image into a mathematical model trained by machine learning or deep learning using image data related to retinal diseases as training data, information related to the estimation of the presence or absence of abnormal sites and information related to the estimation of the location of abnormal sites are obtained.

[0027] In the above-described method for acquiring fundus information, the method may also include a display step of displaying the synthesized fundus image on the display unit of the imaging unit.

[0028] In the above-mentioned methods for acquiring fundus information, the imaging unit can also be a camera of a mobile terminal.

[0029] In the above-mentioned methods for acquiring fundus information, the imaging part can also be a camera with a fixed device.

[0030] In addition, one aspect of the present disclosure relates to a fundus information acquisition device based on an infinity-corrected optical system that uses a portion of the cornea and lens of the eye being tested, defined at least by the pupil of the eye being tested, as an objective lens opposing the fundus of the eye being tested, and uses an imaging unit configured opposite to the objective lens to image the light passing through the objective lens as part of the fundus image of the eye being tested. The imaging unit acquires an image of the eye being tested, including the fundus image reflected within the pupil of the eye being tested.

[0031] The effects of the invention

[0032] According to this disclosure, due to the simple structure of the optical system, it is not necessary to accurately align the imaging unit with the eye being tested, and observation and imaging can be performed even when the distance from the eye being tested is sufficiently far. Attached Figure Description

[0033] Figure 1 A simplified diagram of an optical system for infinity correction.

[0034] Figure 2 This figure illustrates an example of a test eye image that includes fundus images acquired during the fundus image acquisition process.

[0035] Figure 3This is an example of a diagram illustrating changes in the scenery mapped onto the eye being tested, changes in the limbus of the eye being tested, movement of the pupil center within the limbus, or changes in the pupillary margin, according to one embodiment.

[0036] Figure 4 This is an example of a diagram with multiple fundus images.

[0037] Figure 5 This is a simplified diagram of a synthetic fundus image according to one embodiment.

[0038] Figure 6 A simplified diagram illustrating an example of using a mobile terminal as a fundus information acquisition device 4.

[0039] Figure 7 A simplified diagram illustrating an example of using a fixed device as a fundus information acquisition device 4.

[0040] Figure 8 This is a flowchart illustrating the implementation of a fundus information acquisition device according to one embodiment.

[0041] Figure 9 The hardware structure of a fundus information acquisition device according to one embodiment is shown.

[0042] Figure 10 This is a functional block structure of a fundus information acquisition device according to one embodiment. Detailed Implementation

[0043] The summary of this disclosure will be described with reference to the accompanying drawings. It should be noted that the following embodiments are provided to facilitate understanding of this disclosure and are not intended to limit its interpretation. Furthermore, various modifications can be made to this disclosure without departing from its spirit. Moreover, those skilled in the art can employ embodiments in which the elements described below are replaced with equivalent elements, and such embodiments are also included within the scope of this disclosure.

[0044] 1. Fundus information acquisition device 4

[0045] Figure 1This figure illustrates an example of a fundus information acquisition device 4 that acquires an image of a tested eye based on an infinity-corrected optical system 10. The fundus information acquisition device 4 of this embodiment is based on an infinity-corrected optical system 10 that uses a portion of the cornea 23 and lens 22 of the same tested eye, defined by the range of the pupil 24 of the tested eye, as an objective lens opposing the fundus 21 of the tested eye 20. An imaging unit 40, positioned opposite the objective lens, images the reflected light 31 passing through the objective lens as part of the fundus image of the tested eye 20. The imaging unit 40 acquires an image of the tested eye 20 including the fundus image projected within the range of the pupil 24 of the tested eye 20.

[0046] <Hardware Structure>

[0047] Figure 9 This figure illustrates an example of the hardware structure of the fundus information acquisition device 4. The fundus information acquisition device 4 includes a processor 11 such as a CPU (Central Processing Unit) and GPU (Graphical Processing Unit), a storage device 12 such as a memory, HDD (Hard Disk Drive) and / or SSD (Solid State Drive), a communication interface 13 for wired or wireless communication, an input device 14 for accepting input operations, and an output device 15 for outputting information. The input device 14 may be, for example, a keyboard, touch panel, mouse, and / or microphone. The output device 15 may be, for example, a display, touch panel, and / or speaker.

[0048] In this embodiment, the imaging unit 40 may be configured to be included in the input device 14, or it may be configured to be included in an external device of the fundus information acquisition device 10, and the fundus image acquired by the imaging unit 40 may be input from the input device 14 via a communication line or the like.

[0049] <Functional Block Structure>

[0050] Figure 10This figure illustrates an example of the functional block structure of the fundus information acquisition device 4. The fundus information acquisition device 4 includes a storage unit 110 and a control unit 120. The storage unit 110 can be implemented using the storage device 12 included in the fundus information acquisition device 4. Furthermore, the control unit 120 can be implemented by executing a program stored in the storage device 12 through the processor 11 of the fundus information acquisition device 4. This program can also be stored on a storage medium. The storage medium storing the program can also be a non-transitory computer-readable medium. There are no particular limitations on the non-transitory storage medium; for example, it could be a USB flash drive or a CD-ROM.

[0051] Figure 8 To illustrate the flowcharts of the various steps involved in implementing the fundus information acquisition device 4 according to an embodiment of this disclosure, each step is executed by the control unit 120 of the fundus information acquisition device 4. The following uses... Figure 8 Each process is described in detail.

[0052] The fundus information acquisition device 4 includes an infinity correction step S101 and a fundus image acquisition step S102, and acquires fundus information of the tested eye 20 by performing these steps. Additionally, the fundus information acquisition device 4 may also include a position information determination step S103, a fundus image synthesis step S104, and a display step S105. Furthermore, the fundus information acquisition device 4 may include other steps as needed.

[0053] 1.1. Infinity calibration procedure S101

[0054] like Figure 8 As shown, the infinity correction process S101 is a process for constructing the infinity correction optical system 10. Here, the infinity correction optical system refers to a system constructed in such a way that the light passing through the objective lens forms a parallel beam. In this embodiment, as... Figure 1 As shown, in the infinity correction optical system 10, the reflected light 31 emitted from the fundus 21 of the tested eye 20, which is the object being observed, does not form an image in the lens 22 and cornea 23, which serve as the objective lens. Instead, it enters the imaging lens 41 of the imaging unit 40 as a parallel beam 32 at infinity, and forms an image in the imaging element 42 through the imaging lens 41. The fundus information acquisition device 4 acquires an image of the tested eye 20 in the imaging unit 40, including the fundus image within the range of the pupil of the tested eye.

[0055] In fundus observation and imaging using indirect ophthalmoscopy optical systems, which are currently used as existing technology, accurate alignment of the imaging unit relative to the eye being examined is required. Similarly, fundus imaging using direct ophthalmoscopy optical systems, which are currently used as existing technology, requires not only accurate alignment of the imaging unit relative to the eye being examined, but also involves imaging at a very close distance to the eye, which poses risks due to the exposure to light at close range.

[0056] On the other hand, in this embodiment, by constructing an infinity-corrected optical system 10, since it is not necessary to adjust the position of the imaging lens, etc., by aligning it with the position of the intermediate image formed by the objective lens as in conventional indirect ophthalmoscopy optical systems, the distance between the objective lens and the imaging lens, etc., can be freely adjusted, enabling convenient fundus observation, etc. Furthermore, since the eye being tested 20 can be ensured to be at a sufficiently large distance from the imaging unit 40, it can be used conveniently and safely.

[0057] In this embodiment, there are no particular limitations on the unit used to prompt the subject to look at infinity. For example, the fundus information acquisition device 4 can be equipped with a fixation target projection unit. Thus, the fundus information acquisition device 4 can have a fixation lamp that emits visible light, and by illuminating the fixation lamp, a fixation target is presented to the subject, guiding the subject's eye to look infinity.

[0058] Here, regarding infinity in this embodiment, it is not strictly necessary for the light emitted from the fundus 21 to become a parallel beam after passing through the lens 22 and cornea 23. Ideally, the light emitted from the fundus 21 and passing through the lens 22 and cornea 23 should form a parallel beam when the subject is strictly looking at infinity. However, it is sufficient if the light observed in the tested eye 20 approximates a parallel beam to a fixed level of accuracy that meets minimum practical requirements. This fixed level of accuracy, which meets minimum practical requirements, is sufficient to obtain information about the color of the fundus 21 of the tested eye 20.

[0059] If the fundus information obtained in this embodiment includes at least information about the color of the fundus 21, it can be used in the diagnosis of glaucoma and optic nerve atrophy. Furthermore, if it includes information about the shape of the fundus 21, it can be used in further diagnosis of retinal diseases, etc.

[0060] Furthermore, in the imaging unit 40 of the fundus information acquisition device 4 that acquires images of the tested eye 20 based on the infinity correction optical system 10, two or more imaging lenses that configure the imaging lens 41 as a system may be used as needed. When using two or more imaging lenses, the lens that images in the imaging element 42 is called the first imaging lens, and the other lens is called the second imaging lens. The second imaging lens may also form an intermediate image.

[0061] 1.1.1. Objective lens

[0062] In this embodiment, a portion 51 of the cornea 23 and lens 22 of the eye being tested, defined at least by the pupil 24 of the eye being tested, is used as an objective lens opposing the fundus 21 of the eye being tested. This objective lens is configured such that reflected light 31 emitted from the fundus 21 becomes a parallel beam 32 when the subject is looking at infinity. When the subject uses eyeglasses or similar devices, the term "objective lens" refers to an optical system that integrates the portion of the cornea 23 and lens 22 of the eye being tested, defined at least by the pupil 24 of the eye being tested, with eyeglasses.

[0063] 1.1.1.1. Pupil 24

[0064] The pupil is an opening surrounded by the iris of the eye, and its diameter changes according to the amount of light. This change in pupil diameter helps adjust the amount of light projected onto the retina. Therefore, if the eye 20 being tested is illuminated with strong light, the pupil 24 will constrict, and the resulting reduction in pupil diameter narrows the area from which the fundus 21 can be observed. Therefore, in this embodiment, regarding ambient light, it is preferable to acquire fundus images in a state where it is as dark as possible.

[0065] 1.1.1.2. Lens 22

[0066] The lens is a transparent, convex lens-like substance located at the front of the eyeball that refracts light from the outside, thus forming an image on the retina. The lens 22 automatically adjusts by thickening when viewing near objects and thinning when viewing distant objects, thereby enabling focusing at both near and far distances.

[0067] In this embodiment, the thickness of the lens 22 is adjusted by prompting the subject to look at infinity, thereby focusing at infinity.

[0068] 1.1.1.3. Cornea 23

[0069] The cornea is a transparent membrane covering the front surface of the eyeball, and together with the lens, it functions as a convex lens.

[0070] 1.1.2. Filming Department 40

[0071] In this embodiment, the imaging unit 40 includes an imaging lens 41 for imaging a parallel light beam 32 that passes through the lens 22 (which serves as the objective lens) and the cornea 23, as well as an imaging element 42. The imaging lens 41 can be a lens with a desired refractive index.

[0072] The imaging element 42 is sensitive to at least visible and infrared light. The imaging element 42 is not particularly limited, but preferably has a viewing function that integrates several adjacent photoelectric conversion elements into a single pixel for processing. Thus, the imaging element 42 can maintain sensitivity in acquiring fundus images even in low-light conditions such as reflected light 31 from the fundus 21 of the tested eye 20.

[0073] Figure 6 This is a simplified diagram illustrating an example of using a mobile terminal as a fundus information acquisition device 4. For example, the camera of the mobile terminal is used as the imaging unit 40. The mobile terminal is not particularly limited; examples include information processing devices such as smartphones, tablet computers, personal computers, or workstation computers. By using the camera of the mobile terminal as the imaging unit 40, such as… Figure 6 As shown, a simple optical system can be constructed using a mobile terminal, enabling easy acquisition of fundus information. Here, "terminal" refers to a machine that is connected to a line or network and can function as a communication device with other machines.

[0074] Figure 7 This is a simplified diagram illustrating an example of using a fixed device as a fundus information acquisition device 4. For example, the camera of the fixed device is used as the imaging unit 40. In the case where the fixed device is used as a fundus information acquisition device, such as... Figure 7 As shown, by having the subject move while facing the fixed device, the relative position of the tested eye 20 and the imaging unit 40 can be easily changed. Furthermore, the fundus image acquisition process S102 of this embodiment can be performed continuously for a fixed time without the subject performing any operation.

[0075] Here, when a fixed device is used as the imaging unit 40, compared to when a mobile terminal is used as the imaging unit 40, the distance between the tested eye 20 and the imaging unit 40 tends to increase, and the resolution of the obtained fundus image tends to decrease accordingly. Therefore, it is preferable to further improve the resolution and acquire fundus information by also having a second imaging lens that forms an intermediate image of the fundus image of the tested eye 20. It should be noted that in this embodiment, the second imaging lens refers to the lens that forms the intermediate image, rather than the lens that forms an image in the imaging element 42.

[0076] Alternatively, this fixed device can also be a smart mirror. Using a smart mirror, it is possible to take photos naturally without the subject being aware of it in their daily life. Here, a smart mirror refers to a mirror-like terminal that has network connectivity, reflects the subject's own image, and has a camera capable of capturing the subject's eyes.

[0077] The imaging unit 40 may also include, as needed, a first imaging lens, a lamp disposed around the first imaging lens, and a polarizing plate disposed at the front end of the first imaging lens. By including a lamp as a light illumination unit, the accuracy of fundus information can be improved based on the light returned from the tested eye 20. In addition, by providing a polarizing plate, surface reflections of light from objects other than the tested eye 20 can be eliminated, and in particular, the colors of the obtained fundus image can be made more vivid.

[0078] The lamp positioned around the first imaging lens can also be a ring-shaped lamp arranged around the outer periphery of the first imaging lens. By using such a lamp, the light emitted from the lamp can be more uniformly irradiated onto the eye being tested 20, thereby obtaining a high-precision fundus image.

[0079] When a camera from a mobile terminal or a fixed device is used as the imaging unit 40, the light surrounding the first imaging lens can also be a light built into the mobile terminal or the fixed device. With this configuration, it is easier to acquire fundus image information.

[0080] 1.2. Fundus image acquisition process S102

[0081] like Figure 8 As shown, the fundus image acquisition step S102 is a step of acquiring a fundus image. As the unit for acquiring the fundus image, existing and known units can be used. The image acquisition unit is not particularly limited; for example, it can acquire the image by taking a picture using a camera or other imaging unit. In this image acquisition unit, it is preferable to have a light illumination unit that acquires an image of the tested eye based on light reflected from the tested eye. Such a light illumination unit can particularly improve the accuracy of information regarding the color of the fundus.

[0082] Furthermore, regarding the fundus image acquisition process S102, one embodiment includes a continuous fundus image acquisition process. The continuous fundus image acquisition process is a process for continuously acquiring multiple fundus images while changing the relative position of the tested eye 20 and the imaging unit 40. This will be described in detail below.

[0083] 1.2.1. Unit that changes the relative position of the tested eye 20 and the imaging unit 40

[0084] Regarding the unit for changing the relative position of the tested eye 20 and the imaging unit 40, existing known methods can be used. There are no particular limitations to this method; for example, the relative position can be changed by altering the direction in which the subject looks at infinity without changing the position of the imaging unit 40. Alternatively, the relative position can be changed by altering the position of the imaging unit 40 without changing the position of the subject.

[0085] There are no particular limitations on the unit used to enable the test subject to view indefinitely. For example, a fixation target projection unit that displays multiple fixation targets can be used. This fixation target projection unit has a fixation lamp that emits visible light, which can present multiple fixation targets to the test subject by illuminating the fixation lamp and guide the test subject's eye 20 to view in multiple directions.

[0086] Regarding the number of images acquired, since it is used in the fundus image synthesis process S104, there is no particular limitation. For example, it is preferably 30 or more, more preferably 100 or more, and even more preferably 300 or more.

[0087] 1.3. Location information determination process S103

[0088] like Figure 8 As shown, the location information determination step S103 is a step of determining the location of the fundus image in the fundus 21 based on information obtained from the image of the tested eye. Preferably, this step is performed in a manner that the location of the fundus image in the fundus 21 is determined based on the feature portions of the image of the tested eye that are manifested from the structural portions of the tested eye 20. This can improve the accuracy of the fundus information that can be obtained.

[0089] Figure 2 This diagram illustrates an example of a test eye image including a fundus image acquired by a fundus image acquisition unit. Additionally, Figure 4 Here is an example of a diagram with multiple fundus images. First, as one embodiment of this process, the reflection from the portion including the corneal apex of the tested eye 20 is designated as the first reflection, and the reflection from the anterior capsule of the tested eye 20 is designated as the second reflection. The position of the fundus image in the fundus 21 is determined based on the positional relationship between the bright spot 52 of the first reflection and the bright spot 53 of the second reflection. Specifically, the distance between the center point 61 of the tested eye 20 and the fundus image is determined based on the distance between 52 and 53. Furthermore, the fundus images are configured such that a straight line passing through 52 and 53 passes through the center point 61 of the eyeball, thereby determining the orientation of the fundus image away from the center point 61 of the tested eye 20. This method reduces the amount of computational processing required by the computer in generating the synthesized fundus image, enabling rapid and convenient acquisition of fundus information.

[0090] Figure 3 An example of a diagram illustrating changes in the view reflected in the pupil of the tested eye, such as movement of the view reflected in the pupil, changes in the limbus of the tested eye, movement of the pupil center within the limbus, or changes in the pupillary margin. In particular, in the case of using only ambient light, in the infinity correction optical system 10 where the incident light is insufficient, as an implementation of this process, the following methods can be used: determining the distance between the center point 61 of the tested eye 20 and the fundus image by detecting changes in the scenery 54 reflected in the tested eye 20 caused by the rotational movement 59 of the eyeball corresponding to the tested eye; determining the distance between the center point 61 of the tested eye 20 and the fundus image by detecting changes 56 in the limbus caused by the rotational movement 59 of the eyeball, such as the corneal limbus 55 deforming from a circle to a roughly elliptical shape in response to the rotational movement 59 of the eyeball; determining the distance between the center point 61 of the tested eye 20 and the fundus image by detecting the movement 58 of the pupil center in the limbus corresponding to the rotational movement 59 of the eyeball; or determining the distance between the center point 61 of the tested eye 20 and the fundus image by detecting changes in the pupillary rim 57 caused by the rotational movement 59 of the eyeball, and more preferably, combining these determination methods.

[0091] Alternatively, as another implementation of this process, existing known units for determining the position of the fundus image in the fundus 21 can be used, but there are no particular limitations. For example, an implementation that includes a feature point detection unit for detecting feature points in the image, and a unit that simultaneously stores the feature values ​​and coordinate data of the detected feature points, can be used. Here, since the acquired fundus image is a fundus image of the same tested eye 20, since there are images of common fundus regions and feature points among multiple fundus images, the position information of other fundus images can be determined based on one fundus image. More specifically, for example, the intersections of blood vessels displayed in each fundus image are extracted as feature points, the image data around the intersections of blood vessels are correlated and matched between two fundus images to find common intersection pairs, and the two fundus images are superimposed based on the position of the found intersection pairs, thereby determining the position information.

[0092] Here, "images containing common fundus regions and feature points exist among multiple fundus images" does not mean that common fundus regions and feature points were captured in all fundus images of the same tested eye 20, but rather that although they may not be the same fundus region, there are images among the acquired fundus images that reflect common fundus regions and feature points.

[0093] 1.4. Fundus image synthesis process S104

[0094] As Figure 8As shown, the fundus image synthesis process S104 is a process of arranging multiple fundus images at the positions determined by the position information determination process S103 and generating a synthetic fundus image of the tested eye 20.

[0095] In this embodiment, since the infinity correction optical system 10 is used, the area of ​​the fundus 21 reflected in the fundus image is narrower compared with the indirect ophthalmoscopy optical system. However, by providing the fundus image synthesis process S104, a synthetic fundus image for observing the fundus 21 over a wide range can be generated.

[0096] Furthermore, by incorporating the fundus image synthesis process S104, precise alignment of the imaging unit 40 with respect to the tested eye 20 is not required. Even when the distance between the imaging unit 40 and the tested eye 20 is relatively far, more detailed observation and imaging can be performed.

[0097] Figure 5 This is an example of synthesizing a fundus image. There are no particular limitations on the synthesized fundus image. For example, after the fundus image 72 is configured based on the position information determined in the position information determination step S103, it can be generated by stitching together common fundus regions, including blood vessels 71 and the like, that are mapped into each fundus image 72 in an overlapping manner.

[0098] In the case where the multiple fundus images 72 acquired in the fundus image acquisition process S102 do not correspond to the entire area of ​​the fundus 21 of the tested eye 20, as a first embodiment, a mathematical model trained by machine learning or deep learning is used to determine the eyeball position where fundus images 72 corresponding to the areas not acquired in the fundus image acquisition process S102 can be captured, and the fundus image acquisition process S102 is performed at the eyeball position, thereby supplementing the fundus images 72 corresponding to the areas not acquired in the fundus image acquisition process S102.

[0099] Furthermore, as a second embodiment described above, a mathematical model trained through machine learning or deep learning is used to supplement the fundus images 72 corresponding to the regions not acquired in the fundus image acquisition step S102 using a standard image of the entire fundus of the tested eye 20. The standard image refers to a fundus image acquired using a conventionally known image acquisition method. There are no particular limitations on conventionally known standard image acquisition methods; examples include fundus imaging based on a wide-angle scanning ophthalmoscopy.

[0100] Here, multiple fundus images 72 acquired in the fundus image acquisition process S102 can be used as training data for the mathematical models in the first and second embodiments described above.

[0101] Furthermore, as a third embodiment described above, as a method for supplementing fundus images 72 corresponding to regions not acquired in the fundus image acquisition step S102, by using multiple fundus images 72 acquired in the fundus image acquisition step S102 as training data for the mathematical model, fundus images 72 corresponding to regions not acquired in the fundus image acquisition step S102 can be generated and supplemented. Therefore, although the amount of computation by the computer increases, fundus observation can be performed easily because it is not necessary to prepare the standard image of the subject's fundus 21.

[0102] Here, the learning model trained through machine learning (hereinafter referred to as the machine learning model) is an example of a mathematical model. A machine learning model includes a model with a predetermined model structure and parameters that change according to the learning process, and improves recognition accuracy by optimizing its processing parameters based on experience gained from training data. That is, a machine learning model is a model that learns optimal processing parameters through learning processes. The algorithm for a machine learning model can use, for example, support vector machines, logistic regression, random forests, neural networks, etc., with no particular limitation on the type. However, from the viewpoint of obtaining synthetic fundus images even with limited training data, neural networks are preferred. The machine learning model performing this learning includes models that have already undergone some kind of learning using training data, as well as models before learning.

[0103] In the fundus image synthesis step S104, preferably, one or more super-resolution techniques are used to further correct the focus of each of the multiple fundus images 72 acquired in the fundus image acquisition step S102 and / or the synthesized fundus image with respect to the tested eye 20. This improves the resolution of the fundus image. It should be noted that super-resolution technology is a technique that uses image processing methods and software algorithms to convert existing low-resolution (LR) images into high-resolution (HR) images; simply put, it is a technique that improves resolution through pixel shifting.

[0104] 1.5. Display process S105

[0105] As Figure 8 As shown, the display step S105 is the step of displaying the synthesized fundus image. Preferably, the fundus information acquisition device 4 in this embodiment has a display step S105. This makes it easier to confirm the fundus information of the subject.

[0106] Furthermore, by including the display process S105, as a first embodiment in the case where the multiple fundus images 72 acquired in the fundus image acquisition process S102 do not correspond to the entire area of ​​the fundus 21 of the tested eye 20, a mathematical model trained by machine learning or deep learning is used to determine the eye position where fundus images 72 corresponding to the areas not acquired in the fundus image acquisition process S102 can be captured. When the fundus image acquisition process S102 is performed at the eye position, it is easier to guide the subject to look in the desired direction.

[0107] 1.6. Other processes

[0108] In the fundus information acquisition device 4 of this embodiment, in addition to the above-described steps, other steps may be included as needed. There are no particular limitations on these other steps; for example, a step for acquiring estimation information of abnormal areas can be included. In this embodiment, a step for acquiring estimation information of abnormal areas may also be included. In this step, the synthesized fundus image is input into a mathematical model trained using machine learning or deep learning with image data related to retinal diseases as training data to acquire information related to the estimation of the presence or absence of abnormal areas and information related to the estimated location of abnormal areas. This allows for the early and convenient detection of retinal diseases, etc.

[0109] Industrial application

[0110] According to this disclosure, without particular limitation, for example, still images and videos for ophthalmic examinations can be acquired using mobile terminals such as smartphones. Furthermore, the fundus information acquisition method disclosed herein can be widely adopted worldwide because it can be constructed at a very low cost without the use of special parts, compared to fundus observation methods using slit-lamp microscopes or handheld slit-lamp microscopes.

[0111] Furthermore, it can also be used clinically. Specifically, while ophthalmologists use tabletop fundus cameras to observe / diagnose the tested eye during examinations, this presents challenges and requires skilled techniques when examining children or bedridden elderly individuals. However, in this invention, regardless of the subject's posture, as long as they are encouraged to look at infinity, fundus images can be easily captured using a mobile terminal or similar device. These images are then used to generate synthetic fundus images via a commonly used synthetic image acquisition unit, enabling observation and diagnosis. Therefore, its application in rural telemedicine and support for developing countries is expected.

[0112] Furthermore, this disclosure can also be used for fundus observation and photography of animals. In particular, it can capture still images and videos of the eyes of pets and other animals kept for entertainment, as well as large animals in zoos where it is particularly difficult to align the camera lens with the eye being tested, thereby obtaining ophthalmic diagnostic results for these animals.

[0113] Furthermore, the analysis of big data through AI is expected to improve the diagnostic accuracy of ophthalmologists. Ultimately, it can be used as a self-diagnostic tool for those being tested, thereby further advancing ophthalmological diagnosis and treatment itself.

[0114] Explanation of reference numerals in the attached figures

[0115] 4: Fundus information acquisition device; 10: Infinity correction optical system; 20: Test eye; 21: Fundus; 22: Lens; 23: Cornea; 24: Pupil; 25: Retina; 26: Optic nerve head; 31: Light; 32: Parallel beam; 40: Imaging unit; 41: Imaging lens; 42: Imaging element; 51: Area defined by the pupil; 52: Bright spot of the first reflection; 53: Bright spot of the second reflection; 54: Scenery reflected in the test eye; 55: Limb of the cornea; 56: Changes in the limbus; 57: Pupil limbus; 58: Movement of the pupil center within the limbus; 59: Rotation of the eyeball; 61: Center point of the eyeball; 71: Blood vessel; 72: Fundus image; 11: Processor; 12: Storage device; 13: Communication IF; 14: Input device; 15: Output device; 110: Storage unit; 120: Control unit.

Claims

1. A method for acquiring fundus information, wherein, The fundus information acquisition method includes an infinity-corrected optical system. This system uses a portion of the cornea and lens of the same tested eye, defined at least by the pupil of that eye, as an objective lens opposing the fundus of the tested eye. An imaging unit positioned opposite this objective lens images the light passing through it as a portion of the fundus image of the tested eye. The method for acquiring fundus information includes: The continuous acquisition process of fundus images involves changing the relative position of the eye under test and the imaging unit while acquiring multiple images of the eye under test, including fundus images within the range reflected in the pupil of the eye under test. The location information determination process determines the location of the fundus image in the fundus based on information obtained from the tested eye image; as well as The fundus image synthesis process involves configuring multiple fundus images and generating a synthesized fundus image of the tested eye based on the position of the fundus image determined in the fundus.

2. The method for acquiring fundus information according to claim 1, wherein, In the position information determination process, the position of the fundus image in the fundus is determined based on the feature portion that appears in the image of the eye subject to be tested, which originates from the structural portion of the eye subject to be tested.

3. The method for acquiring fundus information according to claim 2, wherein, The characteristic portions that appear in the image of the tested eye are a first reflection from a portion including the corneal apex of the tested eye, and a second reflection from the anterior capsule of the lens of the tested eye. In the position information determination process, the position of the fundus image in the fundus is determined based on the positional relationship between the first reflected bright spot and the second reflected bright spot.

4. The method for acquiring fundus information according to claim 2, wherein, The features displayed in the image of the tested eye are the scenery mapped into the tested eye. In the location information determination process, the position of the fundus image in the fundus is determined based on the changes in the scenery mapped in the eye being tested.

5. The method for acquiring fundus information according to claim 2, wherein, The characteristic features that appear in the image of the tested eye are the limbus of the tested eye, the center of the pupil within the limbus, or the pupillary margin. In the position information determination process, the position of the fundus image in the fundus is determined based on the limbus of the tested eye, the movement of the pupil center within the limbus, or the change of the pupil limbus.

6. The method for acquiring fundus information according to claim 1, wherein, In the fundus image synthesis process, when multiple fundus images acquired in the continuous fundus image acquisition process do not correspond to the entire area of ​​the fundus of the tested eye, a mathematical model trained by machine learning or deep learning is used to determine the eyeball position corresponding to the area not acquired in the continuous fundus image acquisition process, and the continuous fundus image acquisition process is performed at the eyeball position, thereby supplementing the fundus images corresponding to the area not acquired in the continuous fundus image acquisition process.

7. The method for acquiring fundus information according to claim 1, wherein, In the fundus image synthesis process, when multiple fundus images acquired in the continuous fundus image acquisition process do not correspond to the entire area of ​​the fundus of the tested eye, a mathematical model trained by machine learning or deep learning is used to supplement the fundus images corresponding to the areas not acquired in the continuous fundus image acquisition process with a standard image of the entire fundus of the tested eye.

8. The method for acquiring fundus information according to claim 6 or 7, wherein, Multiple fundus images acquired during the continuous fundus image acquisition process are used as training data for the mathematical model.

9. The method for acquiring fundus information according to claim 1, wherein, In the fundus image synthesis process, one or more super-resolution techniques are used to perform focus correction on each of the multiple fundus images acquired in the continuous fundus image acquisition process and / or on the synthesized fundus image of the tested eye.

10. The method for acquiring fundus information according to claim 1, wherein, The fundus information acquisition method further includes an abnormality estimation information acquisition step. In the abnormality estimation information acquisition step, the synthetic fundus image is input into a mathematical model trained by machine learning or deep learning using image data related to retinal diseases as training data to obtain information related to the estimation of the presence or absence of abnormalities and information related to the estimation of the location of abnormalities.

11. The method for acquiring fundus information according to claim 1, wherein, The fundus information acquisition method also includes a display step for displaying the synthesized fundus image.

12. The method for acquiring fundus information according to claim 1, wherein, The shooting unit is the camera of the mobile terminal.

13. The method for acquiring fundus information according to claim 1, wherein, The shooting unit is a fixed camera.

14. A device for acquiring fundus information, wherein, The fundus information acquisition device is based on an infinity-corrected optical system that uses an objective lens, defined at least by the pupil of the cornea and lens of the tested eye, as an objective lens opposing the fundus of the tested eye, and uses an imaging unit disposed opposite to the objective lens to image the light passing through the objective lens as part of the fundus image of the tested eye. The fundus information acquisition device includes: A unit that acquires multiple images of the tested eye, including fundus images within the range of the pupil of the tested eye, while changing the relative position of the tested eye and the imaging unit. A unit for determining the location of the fundus image in the fundus based on information obtained from the tested eye image; as well as A unit that configures multiple fundus images and generates a synthetic fundus image of the tested eye based on the position determined by the fundus images in the fundus.

Images