Eyeball observation assistance method, eyeball observation assistance device, and eyeball observation assistance program

Abstract

Provided is a technology for assisting with efficient observation for ascertaining the structure of an eyeball.　This eyeball observation assistance method comprises: a first step for acquiring depth information of an eyeball structure in an eye to be examined; a second step for generating position information on a first part and a second part of the eye to be examined, on the basis of the depth information and a model for identifying parts of the eyeball structure; a third step for generating distance information on the distance between the first part and the second part on the basis of the position information; a fourth step for generating an eyeball tomographic image including the position information; and a fifth step for outputting the distance information and the eyeball tomographic image.

Description

Eyeball observation support method, eyeball observation support device, and eyeball observation support program CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Patent Application No. 2024-015537 filed on February 5, 2024, the contents of which are incorporated herein by reference.

[0002] The present disclosure relates to an eyeball observation support method.

[0003] During vitreous surgery and intravitreal injection, the access site to the vitreous is important. Around the vitreous are important structures such as the cornea, which transmits light into the eye, the retina, which is involved in visual function, and the ciliary rugae, which are rich in blood vessels. Damage to these structures during surgery or injection can lead to disorders such as traumatic cataracts and floaters, rhegmatogenous retinal detachment, and even blindness. Meanwhile, the ciliary body is known to contain a region called the pars plana, which allows safe access to the vitreous. Therefore, the ciliary body contains both the pars plana, which poses a significant risk if damaged, and the pars plana, which allows safe access to the vitreous.

[0004] Furthermore, Non-Patent Document 1 discloses that the length of the ciliary body is affected by the axial length, which is a finding that contributes to identifying the position of the ciliary body pars plana.

[0005] Lincke JB, Keller S, Amaral J, Zinkernagel MS, Schuerch K. Ciliary body length revisited by anterior segment optical coherence tomography: implications for safe access to the pars plana for intravitreal injections. Graefes Arch Clin Exp Ophthalmol. 2021;259(6):1435-1441. doi:10.1007 / s00417-020-04967-3

[0006] However, since the size of the eyeball and the length of the eye parts vary from individual to individual, it is desirable to be able to measure directly for each individual.

[0007] Furthermore, because there was no good method for estimating the location of the pars plana, vitreous injections were generally performed 3.5 mm from the limbus, lens-sparing vitreous surgery 3 mm, and non-lens-sparing vitreous surgery 4 mm from the limbus. However, these distances vary depending on the eye, making direct measurement desirable. In particular, in long-axis eyes, surgical instruments are often insufficient in length to reach the macular region, which is crucial for visual function. While using longer instruments can solve this problem, it increases instrument flexibility, making surgery more difficult. Therefore, observing the most peripheral retina from behind the iris and puncturing the very edge of the retina are crucial for safe vitreous treatment.

[0008] Even the widest fundus camera models can only capture 200 degrees of light, capturing only 80% of the retina. Common methods for observing the most peripheral retina from behind the iris include observing the pars plana ciliary body by applying intensely painful pressure to the eyeball, using an ultrasound biomicroscope with poor resolution, or using anterior segment optical coherence tomography (ASOCT), which has poor penetration depth. ASOCT can obtain tomographic images of the anterior segment by irradiating the subject's eye with near-infrared light and analyzing the reflection from intraocular structures.

[0009] However, the method of compressing the eyeball is extremely painful, the method using an ultrasound biomicroscope does not provide sufficient resolution to identify the ciliary body planum, and the method using ASOCT does not provide sufficient depth information to identify the ciliary body planum.

[0010] In addition, from the back of the iris to the most peripheral part of the retina, there may be findings such as iris cysts, ciliary body cysts, and retinal cysts that interfere with visual function or surgery, and while it is desirable to suture the intraocular lens into the ciliary sulcus, this is difficult to observe, so it is generally sewn blindly 2 mm from the limbus. Therefore, it is desirable to accurately measure these findings as well.

[0011] Therefore, an object of the present disclosure is to provide a technology that supports efficient observation for understanding the structure of the eyeball.

[0012] An eyeball observation support method according to one aspect of the present disclosure includes a first step of acquiring depth information of an eyeball structure in a test eye, a second step of generating position information regarding a first portion and a second portion in the test eye based on the depth information and a model that identifies portions of the eyeball structure, a third step of generating distance information regarding the distance between the first portion and the second portion based on the position information, a fourth step of generating a tomographic image of the eyeball including the position information, and a fifth step of outputting the distance information and the tomographic image of the eyeball.

[0013] According to the present disclosure, it is possible to provide a technology that supports efficient observation for understanding the structure of the eyeball.

[0014] 1 is a diagram for explaining an eyeball structure. FIG. 1 is a diagram for explaining an eyeball structure. FIG. 1 is a diagram for explaining an example of a configuration of an eyeball observation support system. FIG. 2 is a diagram for explaining an example of using a mobile terminal or a fixed device as a detection device. FIG. 3 is a diagram for explaining an example of using a mobile terminal or a fixed device as an eyeball observation support device. FIG. 4 is a diagram for explaining an example of a functional configuration of an eyeball observation support system. FIG. 5 is a diagram for explaining an example of a hardware configuration of a detection device and an eyeball observation support device. FIG. 6 is a diagram for explaining an example of a functional configuration of an eyeball observation support device. FIG. 7 is a diagram for explaining an example of depth information. FIG. 8 is a diagram for explaining an example of a distance between parts of an eyeball structure. FIG. 9 is a diagram for explaining an example of a distance between parts of an eyeball structure. FIG. 10 is a flowchart for explaining an example of a processing procedure for eyeball observation support by an eyeball observation support device. FIG. 11 is a diagram for explaining a state in which depth information of an eyeball structure of a subject's eye is detected using a distance measuring sensor. FIG. 12 is a diagram for explaining a state in which depth information is detected using distance measuring sensors located at various relative positions as seen from the subject's eye. FIG. 13 is a diagram for explaining an example of a screen display. FIG. 14 is a diagram for explaining an example of a screen display. FIG. 15 is a diagram for explaining an example of a screen display.

[0015] An embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described with reference to the accompanying drawings, but the present invention is not limited to this and various modifications are possible without departing from the spirit of the present invention. In each drawing, parts with the same reference numerals have the same or similar configurations. The dimensional ratios of the drawings are not limited to those shown in the drawings.

[0016] 1. Eyeball Structure FIG. 1 is a diagram explaining the structure of the human eyeball. The structure of the eyeball is divided into an outer wall and internal organs. The outer wall of the eyeball consists of three layers: from the outside, the fibrous membrane, the vascular membrane, and the endophthalm. The fibrous membrane includes the cornea 20 in the front and the sclera 23 in the back. The vascular membrane includes the choroid 24, the ciliary body 21, and the iris 28. The endophthalm includes the retina 25. The internal organs of the eyeball include the lens 29, aqueous humor, and vitreous body 27. Aqueous humor includes the anterior aqueous humor, which fills the area inside the eyeball surrounded by the cornea and iris (anterior chamber 70), and the posterior aqueous humor, which fills the area behind the anterior chamber, separated by the iris (posterior chamber 80).

[0017] In the present disclosure, the eyeball is described as a human eyeball, but the scope of application of the present disclosure is not necessarily limited to human eyeballs and may be the eyeball of an animal, including a pet animal, a model animal, etc. Furthermore, in the present disclosure, the subject eye is not limited to an eyeball affected by a disease, but may be an eyeball to be observed, an eyeball of a subject affected by a disease, or an eyeball of a subject who may be affected by a disease. The subject eye may be an eyeball with a specific axial length, such as a long axial eye (including myopic or hyperopic eyeballs), or an eyeball with an unspecified axial length.

[0018] FIG. 2 is a diagram for explaining the structure of the human eyeball from the peripheral part of the iris to the most peripheral part of the retina.

[0019] The area present at the periphery of the cornea 20 is the bulbar conjunctiva 22. The cornea, bulbar conjunctiva 22, and sclera 23 are sometimes referred to as the iris and white of the eye, respectively. The boundary between the cornea 20 and the sclera 23 is the limbus 60.

[0020] The area inside the eyeball surrounded by the cornea 20 and the iris 28 is the anterior chamber 70. Behind the anterior chamber 70, separated by the iris 28, is the posterior chamber 80. The depression located on the periphery of the posterior chamber 80 is the ciliary sulcus 50.

[0021] The ciliary body 21 is composed of a ciliary rugae wall 21a, which is rich in blood vessels, and a ciliary body plana 21b. It is known that damage to the blood vessels in the ciliary rugae wall 21a can cause intraocular bleeding, leading to the risk of floaters and blindness. The ciliary body plana 21b is physiologically more stable than the ciliary rugae wall 21a and is known as a region that allows safe access to the vitreous body 27. The posterior end 40 of the ciliary body is the most peripheral portion of the uveal tract of the retina.

[0022] Vitreous surgery is required to treat vitreous diseases, such as retinal detachment, diabetic retinopathy, macular hole, epiretinal membrane, and branch retinal vein occlusion, which are diseases that cause abnormalities in the vitreous. Vitreous surgery involves observing the structure of the eyeball from the peripheral iris to the most peripheral part of the retina, drilling a hole in the ciliary body plana using a specialized instrument such as a cannula, and then irradiating the vitreous with a laser or replacing gas or liquid. When inserting an intraocular lens, it is desirable to observe the structure of the eyeball and suture the lens into the ciliary sulcus 50. However, because it is difficult to observe the area around the ciliary sulcus from outside the body, the position of the ciliary sulcus 50 is determined based on its distance from the corneal limbus 60, and the intraocular lens is then blindly sutured into the eye.

[0023] 3 is a diagram showing an example of the configuration of an eyeball observation support system 1 according to this embodiment. The eyeball observation support system 1 includes a detection device 2 and an eyeball observation support device 3. The detection device 2 includes a distance measurement sensor 4. The detection device 2 and the eyeball observation support device 3 are connected to each other so as to be able to communicate with each other via a wired or wireless communication network such as the Internet, an intranet, a wireless LAN, or mobile communication.

[0024] The detection device 2 is a device used by a user such as a patient (hereinafter sometimes referred to as a "subject"), an ophthalmologist, a nurse, or a technician to detect the structure of the eyeball.

[0025] The detection device 2 can be any device as long as it has a means for detecting an ocular structure and a means for exchanging data with the eyeball observation support device 3. In the present disclosure, the means for detecting an ocular structure is a means for collecting depth information of the ocular structure. The detection device 2 can be, for example, a mobile terminal device such as a mobile phone, a smartphone, a tablet device, a wearable device, or a laptop PC (Personal Computer). For example, a stationary device such as a smart mirror can also be used. In the present disclosure, a smart mirror is a mirror-type terminal that has an internet connection and is equipped with a mirror that can reflect the subject's image and a distance measurement sensor. Sensing devices such as a camera or an image sensor can also be used. Furthermore, examination devices conventionally used in ophthalmic examinations, such as an OCT device, an ultrasound imaging device, an X-ray CT device, or an MRI device, can also be used. Using a mobile terminal or a stationary device enables observation of the most peripheral part of the retina from the back surface of the iris using a simpler optical system configuration. Furthermore, when a fixed device is used, the subject moves while facing the fixed device, which makes it possible to easily change the relative position between the subject's eye and the distance measuring sensor 4. Furthermore, the step of acquiring depth information of the ocular structure of the subject's eye can be performed for a certain period of time without any operation by the subject.

[0026] 4A is a diagram showing an example in which a mobile terminal 501 is used as the detection device 2. FIG. 4B is a diagram showing an example in which a stationary device 502 is used as the detection device 2.

[0027] In the present disclosure, a user is a person who uses the detection device 2 and the eyeball observation support device 3. The user is not limited to one person, and may be multiple people. For example, a subject may use the detection device 2 and the eyeball observation support device 3. For example, a subject may use the detection device 2, and an ophthalmologist, technician, or other person different from the subject may use the eyeball observation support device 3.

[0028] In the present disclosure, the means for detecting the ocular structure of the detection device 2 includes a distance measurement unit. The distance measurement unit has a function of measuring various distances based on the positional relationship between the detection device 2 and the subject's eye. The distance measurement unit can be realized, for example, by an input device included in the detection device 2. The distance measurement unit of the detection device 2 includes, for example, a distance measurement sensor 4.

[0029] In the present disclosure, the eyeball observation support system 1 is described as including the detection device 2. However, the detection device 2 is not necessarily required, and the functions of the detection device 2 may be performed by the eyeball observation support device 3. When the eyeball observation support device 3 is configured to perform the functions of the detection device 2, the eyeball observation support device 3 may be any device that, in addition to the functions performed by the eyeball observation support device 3, further includes a means for detecting an eyeball structure, which is the function performed by the detection device 2. In such a case, the eyeball observation support device 3 may be, for example, any of the various mobile terminals, stationary devices, sensing devices, inspection devices, etc., exemplified as devices that can be used for the detection device 2. For example, FIG. 5A illustrates an example in which a mobile terminal 501 is used as the eyeball observation support device 3. FIG. 5B illustrates an example in which a stationary device 502 is used as the eyeball observation support device 3.

[0030] Furthermore, in this disclosure, the detection device 2 is described as being equipped with a ranging sensor 4, but the detection device 2 does not necessarily have to be equipped with a ranging sensor 4, and the ranging sensor 4 may be connected to the detection device 2 by wire or wirelessly.

[0031] Examples of the distance measurement sensor 4 include a distance measurement sensor using an optical system such as LiDAR (Light Detection and Ranging), a distance measurement sensor using radio waves such as RADAR (Radio Detection and Ranging), and a distance measurement sensor using ultrasonic waves. Among these, optical distance measurement sensors are preferred from the viewpoint of easily acquiring depth information. Examples of LiDAR include a ToF (Time of Flight) type and an FMCW (Frequency Modulated Continuous Wave) type.

[0032] It is preferable to use an FMCW type LiDAR for the ranging sensor 4. Since FMCW type LiDAR emits a laser beam that continuously shifts between different frequencies, the detector of the ranging sensor 4 collects reflected light and measures the reflection time, allowing it to distinguish a specific frequency pattern from other light sources. This allows for high-speed and accurate operation under all illumination conditions. Therefore, even when the distance between the subject's eye and the ranging sensor 4 is large and light from surrounding light sources and its reflected light easily enters the detector, more accurate depth information tends to be obtained. Furthermore, as in the present disclosure, even when the reflection intensity of reflected light originating from the ocular structure is very weak, more accurate depth information tends to be obtained.

[0033] In the case of a distance measuring sensor 4 using an optical system, it is equipped with a light source, a beam splitter, and a detector, and measurement light is incident on the test eye from the light source, and the reflected light from each eye structure such as the conjunctiva, sclera, and ciliary body is detected by the detector to measure the distance between the distance measuring sensor 4 and each eye structure of the test eye.

[0034] The eyeball observation support device 3 is an information processing device that has the function of supporting the observation of the eyeball structure. Any device can be used as the eyeball observation support device 3 as long as it has means for sending and receiving data with the detection device 2. The eyeball observation support device 3 is, for example, a general-purpose computer such as a workstation computer. It may be configured as a single computer, or may be configured as multiple computers distributed over a communication network N.

[0035] 6 is a diagram showing an example of the configuration of the eyeball observation support system 1. The detection device 2 of the eyeball observation support system 1 exchanges data via a communication IF 11 of the eyeball observation support device 3. For example, the detection device 2 outputs information relating to the distance between the distance measurement sensor 4 and each ocular structure of the subject's eye, measured by the distance measurement sensor 4, and inputs the information to the eyeball observation support device 3 via the communication IF 11 of the eyeball observation support device 3.

[0036] 7 is a diagram showing an example of the hardware configuration of the detection device 2 and the eyeball observation support device 3. The detection device 2 and the eyeball observation support device 3 include a communication IF (Interface) 11 for wireless or wired communication, a storage device 12 such as a memory (e.g., RAM (Random Access Memory) or ROM (Read Only Memory)), a hard disk drive (HDD) and / or a solid state drive (SSD), an input device 13 for accepting input operations, an output device 14 for outputting information, and a control device 15 such as a CPU (Central Processing Unit) or GPU (Graphical Processing Unit). The input device 13 is, for example, a keyboard, a touch panel, a mouse, and / or a microphone. The output device 14 is, for example, a display, a touch panel, and / or a speaker, etc. The communication IF 11, the storage device 12, the input device 13, the output device 14, and the control device 15 are connected by one or more communication buses 10.

[0037] 8 is a diagram showing an example of the functional configuration of the eyeball observation support device 3. The eyeball observation support device 3 includes a storage unit 201, an acquisition unit 202, a model construction unit 203, a first position information generation unit 204, a distance information generation unit 205, a second position information generation unit 206, an eyeball tomogram generation unit 207, and an output control unit 208. The storage unit 201 can be realized using the storage device 12 included in the eyeball observation support device 3. The acquisition unit 202, the model construction unit 203, the first position information generation unit 204, the distance information generation unit 205, the second position information generation unit 206, the eyeball tomogram generation unit 207, and the output control unit 208 can be realized by the control device 15 included in the eyeball observation support device 3 executing a program stored in the storage device 12. The program can be stored in a storage medium. The storage medium storing the program may be a non-transitory computer-readable medium. The non-transitory storage medium is not particularly limited, and may be, for example, a Universal Serial Bus (USB) memory or a Compact Disc Read-Only Memory (CD-ROM).

[0038] In the present disclosure, the eyeball observation support device 3 is described as including the model construction unit 203 and the second position information generation unit 206, but it is not necessary for the eyeball observation support device 3 to have these functions. For example, the function performed by the model construction unit 203 may be performed by an information processing device different from the eyeball observation support device 3, and the eyeball observation support device 3 may use a model constructed in the different information processing device.

[0039] <Storage Unit> The storage unit 201 stores various data necessary for the eyeball observation support device 3 to execute eyeball observation support.

[0040] <Acquisition Unit> The acquisition unit 202 has a function of acquiring depth information of the ocular structure of the subject's eye.

[0041] In the present disclosure, the term "ocular structure" refers to a structure that constitutes the eyeball, such as the cornea 20, ciliary body 21, bulbar conjunctiva 22, sclera 23, choroid 24, retina 25, vitreous body 27, iris 28, lens 29, and zonule 30 shown in FIG.

[0042] In the present disclosure, depth information of an ocular structure is information relating to the distance from a sensor located at a predetermined relative position as viewed from the subject's eye to the ocular structure. By combining multiple pieces of depth information, it is possible to understand the three-dimensional structure of the subject's eye.

[0043] 9 is a diagram illustrating an example of depth information. Information relating to P1, P2, P3, P4, P5, and P6 is an example of depth information relating to the cornea 20. Information relating to Q1, Q2, Q3, and Q4 is an example of depth information relating to the iris 28. Information relating to R1, R2, R3, R4, R5, and R6 is an example of depth information relating to the retina 25.

[0044] In the present disclosure, "obtain" means to accept input of information (including depth information) obtained by detecting the eyeball structure.

[0045] The depth information of the ocular structures in the subject's eye may be detected using the distance measurement sensor 4. When, for example, a distance measurement sensor 4 using an optical system is used to detect the depth information of the ocular structures in the subject's eye, the depth information of the ocular structures can be obtained from the characteristics of each reflected light of the measurement light emitted from the distance measurement sensor 4 located at a predetermined relative position with respect to the subject's eye and reflected by each ocular structure.

[0046] Acquiring the depth information of the ocular structure may include receiving an input of an image or video including the depth information of the ocular structure of the subject's eye. Examples of the image including the depth information of the ocular structure include a two-dimensional tomographic image and a three-dimensional tomographic image.

[0047] The acquiring unit 202 may receive information about the subject input by a user. The information about the subject input by the user may include, for example, examination data including information about the subject's visual acuity level, age, sex, nationality, medical history, medical treatment history, disease history, symptoms, contact lens use history, axial length, etc.

[0048] <Model Construction Unit> The model construction unit 203 has a function of constructing a region identification model that identifies regions of the eyeball structure based on the information acquired by the acquisition unit 202 and information about the subject input by the user.

[0049] A first region, a second region, and / or a third region of the subject's eye can be identified by inputting the depth information of the ocular structure of the subject's eye acquired by the acquisition unit 202 into the model constructed by the model construction unit 203. In one aspect, a different predetermined region can also be identified from information on the position of the identified predetermined region and information on the measured distance.

[0050] The region identification model is a mathematical model (hereinafter also referred to as "model"). The region identification model may be a learning model trained by machine learning or deep learning. The learning model has a predetermined model structure and parameters that vary through learning processing, and the processing parameters are optimized based on experience gained from training data, thereby improving the identification accuracy. The learning model is a model that learns optimal processing parameters through learning processing. The learning model algorithm can be, for example, a support vector machine, logistic regression, random forest, neural network, etc. Neural networks are preferable because they can be used even when there is a small amount of training data.

[0051] The model construction unit 203 may construct a part identification model by referring to the information stored in the storage unit 201 .

[0052] The model constructed by the model construction unit 203 may be constructed so that images and videos including depth information of the ocular structure of the subject's eye can be input. This allows the user to input images and videos including depth information of the ocular structure, which have been collected using an examination device conventionally used in ophthalmic examinations, into the part identification model.

[0053] The model construction unit 203 periodically performs learning based on the information acquired by the acquisition unit 202 and information input by the user, and updates the model as appropriate.

[0054] The model construction unit 203 may perform learning using the results of identification and distance measurement by particularly skilled photographers and measurers as training data. This makes it possible to easily inform the surgeon of a site on the sclera that can be safely punctured under general imaging conditions, even in ASOCT, where the penetration depth is generally insufficient. The model construction unit 203 may further have a function of assisting measurements in cases where the image quality is extremely poor by inputting the distance information generated by the distance information generation unit 205 (described later) into a mathematical model trained by machine learning or deep learning using data related to ocular diseases such as race, age, sex, height, and axial length as training data.

[0055] <First Position Information Generating Unit> The first position information generating unit 204 has a function of generating first position information relating to two or more parts of the subject's eye based on the information acquired by the acquiring unit 202 and the part identification model.

[0056] In the present disclosure, the position information is information that specifies the three-dimensional position of a structure in the entire structure of the subject's eye. In the present disclosure, the position information generated by the first position information generating unit 204 is referred to as first position information.

[0057] For example, the first position information generating unit 204 searches for a predetermined site in the subject's eye based on the site identification model and identifies a position estimated to be the predetermined site. At this time, the position estimated to be the predetermined site is not limited to one, but may be multiple. Furthermore, the estimated positions may have different reliability levels.

[0058] The first position information generated by the first position information generating unit 204 may include information regarding the reliability of each position estimated to be a predetermined part.

[0059] The first position information generating unit 204 may further generate position information regarding a third region of the subject's eye. This allows the position information regarding the third region to be taken into account in addition to the first and second regions in the processing performed by the distance information generating unit, second position information generating unit, and ocular tomographic image generating unit (described later). This is expected to improve the accuracy of distance measurement, identification of the fourth region, and generation of ocular tomographic images. It is preferable that at least one of the first, second, and third regions is difficult to identify from an image of the eye captured from the front. For example, the first, second, and third regions may be selected from the ciliary sulcus, the posterior end of the ciliary body, and the limbus, respectively. For example, the first region may be the ciliary sulcus, the second region may be the posterior end of the ciliary body, and the third region may be the limbus. For example, the first region may be the limbus, the second region may be the posterior end of the ciliary body, and the third region may be the ciliary sulcus. For example, the first location may be the limbus, the second location may be the ciliary sulcus, and the third location may be the posterior tip of the ciliary body.

[0060] <Distance information generation unit> The distance information generation unit 205 has the function of measuring various distances related to the ocular structure of the test eye based on the depth information of the test eye acquired by the acquisition unit 202 and the position information corresponding to two or more specified parts of the test eye identified by the first position information generation unit 204.

[0061] Various distances related to the ocular structure of the subject's eye include, for example, the distance between a first portion and a second portion, the distance between the first portion and a third portion, and the distance between the second portion and a third portion. In the present disclosure, the distance between portions is, for example, the linear distance from one portion to another portion.

[0062] The distance between the sites is measured by measuring the distance between a predetermined site on the test eye and an estimated position. Here, if the estimated site on the test eye is one or more positions each having a different reliability, the distance between the positions with the highest reliability is measured.

[0063] 10 is a diagram showing an example of the distance between parts of the eyeball structure. In the example of Fig. 10, the distance d1 between the posterior end of the ciliary body 40 and the ciliary sulcus 50 is shown.

[0064] 11 is a diagram showing an example of the distance between parts of the eyeball structure, in which the distance d2 between the posterior end of the ciliary body 40 and the limbus 60 is shown.

[0065] Furthermore, the distance information generating unit 205 may generate distance information regarding a fourth region of the essential eye based on position information generated by the second position information generating unit 206. For example, when the position of the ciliary body in the subject's eye is identified by the second position information generating unit 206, the distance information generating unit 205 may measure the total length or diameter of the ciliary body.

[0066] <Second Position Information Generator> The second position information generator 206 has a function of generating second position information regarding a fourth region of the subject's eye based on the first position information corresponding to two or more predetermined regions of the subject's eye generated by the first position information generator 204 and various distances generated by the distance information generator 205. The fourth region is preferably a region that is difficult to identify from an image captured from the front of the eyeball. The fourth region is, for example, the ciliary body, the ciliary body pars plana, or a cyst. The fourth region may also be a recommended puncture region.

[0067] In the present disclosure, the location information generated by the second location information generating unit 206 is referred to as second location information.

[0068] For example, the second position information generating unit 206 identifies a position that is a predetermined distance or more and a predetermined distance or less from the first position in the direction toward the second position as the position of a fourth position in the test eye, based on first position information corresponding to the first and second positions in the test eye generated by the first position information generating unit 204 and distance information corresponding to the distance between the first and second positions generated by the distance information generating unit 205. Then, the second position information generating unit 206 generates second position information corresponding to the identified fourth position. For example, when the first position is the limbus and the second position is the posterior end of the ciliary body, the second position information generating unit 206 identifies a position that is 2 mm or more and 5 mm or less from the first position in the direction toward the second position as the fourth position in the test eye.

[0069] Furthermore, for example, the second position information generating unit 206 identifies a position corresponding to a predetermined percentage of the distance between the first or second portion from the first portion or the second portion as the position of a fourth portion of the test eye, based on the first position information corresponding to the first and second portions of the test eye generated by the first position information generating unit 204 and the distance information corresponding to the distance between the first and second portions generated by the distance information generating unit 205. Then, it generates second position information corresponding to the identified fourth portion. For example, when the first portion is the limbus, the second portion is the posterior end of the ciliary body, and the distance between the first and second portions is 7 mm, the second position information generating unit 206 identifies a portion corresponding to 2.8 mm (40% of 7 mm) from the posterior end of the ciliary body as the fourth portion of the test eye.

[0070] The second position information generating unit 206 may generate second position information regarding a fourth region of the subject's eye using the region identification model. For example, the second position information generating unit 206 identifies a position estimated to be the fourth region of the subject's eye based on the first position information corresponding to two or more predetermined regions of the subject's eye generated by the first position information generating unit 204, the various distances generated by the distance information generating unit 205, and the region identification model.

[0071] The number of locations estimated as the fourth part by the second location information generating unit 206 is not limited to one, and may be multiple. Furthermore, the estimated locations may have different degrees of reliability.

[0072] The second position information generated by the second position information generating unit 206 may include information regarding the reliability of each position estimated to be the fourth part.

[0073] The second position information generating unit 206 may refer to information regarding the axial length of the subject. The axial length of the eyeball varies among individuals and also varies depending on the subject's visual acuity level, etc. The length of the ciliary body is affected by the axial length. Therefore, by referring to information regarding the axial length of the subject, the accuracy of identifying the location of the ocular structure of the subject's eye is improved. The axial length of the subject may be calculated based on depth information of the ocular structure acquired by the acquiring unit 202, or may be included in information about the subject input by the user.

[0074] <Eyeball tomogram generating unit> The eyeball tomogram generating unit 207 has a function of generating an eyeball tomogram including the first position information generated by the first position information generating unit 204. For example, the eyeball tomogram generating unit 207 generates an eyeball tomogram including display of search points corresponding to positions estimated to be the first part and the second part, generated by the first position information generating unit 204.

[0075] The eye tomographic image generating unit 207 may generate an eye tomographic image including the first position information generated by the first position information generating unit 204 and the second position information generated by the second position information generating unit 206. For example, the eye tomographic image generating unit 207 generates an eye tomographic image including a display of search points corresponding to positions estimated to be the first part and the second part, generated by the first position information generating unit 204, and a display of a search point corresponding to a position estimated to be the fourth part, generated by the second position information generating unit 206.

[0076] The tomographic image of the eye may be a two-dimensional tomographic image or a three-dimensional tomographic image.

[0077] If the search points corresponding to one or more locations have different reliability levels, the display of each search point may be adjusted according to the reliability of the search point. For example, the color shading of the display of each search point may be used to visualize the strength of the reliability. For example, the search points may be displayed in a heat map format.

[0078] The ocular tomographic image generating unit 207 may generate an ocular tomographic image including the blood vessels and blood flow of the subject's eye. For example, it may generate an ocular tomographic image including the blood vessels and blood flow of the ciliary body. This makes it easier to understand the structure of the ciliary body.

[0079] The eyeball tomographic image generating unit 207 may generate an eyeball tomographic image that displays the recommended puncture site in the subject's eye as a heat map or the puncture point itself.

[0080] <Output Control Unit> The output control unit 208 has a function of controlling the output of the distance information generated by the distance information generation unit 205 and the ocular tomographic image generated by the ocular tomographic image generation unit 207. For example, the output control unit 208 controls the output of the ocular tomographic image of the subject's eye and the distance between a first portion and a second portion of the subject's eye so that the user can visually recognize the distance. For example, the output control unit 208 may control the output of the ocular tomographic image of the subject's eye so that the distance between the first portion and the second portion is displayed numerically, or ... together with a scale bar.

[0081] The output control unit 208 may perform control so that the depth information acquired by the acquisition unit 202 is displayed.

[0082] 5. Operation FIG. 12 is a flowchart showing an example of a processing procedure for supporting eyeball observation by the eyeball observation support system 1. The eyeball observation support system 1 acquires depth information of the eyeball structure of the subject's eye (step S301). The eyeball observation support system 1 identifies a first region and a second region of the subject's eye based on the depth information and a region-specific model of the eyeball structure (step S302). The eyeball observation support system 1 measures the distance between the first region and the second region of the subject's eye identified in step S302 (step S303). The eyeball observation support system 1 generates a tomographic image of the subject's eye based on the depth information acquired in step S301 and information on the positions of the first and second regions of the subject's eye identified in step S302 (step S304). The eyeball observation support system 1 controls the output of the distance information measured in step S303 and the tomographic image generated in step S304 (step S305).

[0083] Each step shown in FIG. 12 will be described below.

[0084] (Step S301: Acquiring Depth Information of Ocular Structures) The eyeball observation support system 1 acquires depth information of ocular structures of the subject's eye (step S301). For example, a user uses the distance measuring sensor 4 of the detection device 2 to direct measurement light from a light source to the subject's eye, the detection device 2 detects the characteristics of the reflected light reflected by each ocular structure of the subject's eye, and the eyeball observation support device 3 acquires depth information regarding the distance from the distance measuring sensor 4 to the ocular structure of the subject's eye based on the characteristics of the reflected light.

[0085] The distance between the distance measuring sensor 4 and the subject's eye is, for example, 3 cm or more, 10 cm or more, 20 cm or more, or 30 cm or more. The lower limit of the distance between the distance measuring sensor 4 and the subject's eye is, for example, 1 mm, 3 mm, 5 mm, or 1 cm. The upper limit of the distance between the distance measuring sensor 4 and the subject's eye is, for example, 10 m, 5 m, 3 m, 1 m, or 50 cm.

[0086] The number of pieces of depth information to be acquired is, for example, 30 or more, 100 or more, or 300 or more. Since the depth information is used in the eye tomographic image generating process described later, it is preferable that the number of pieces of depth information is large. Here, the number of pieces of depth information is counted as one for each relative position between the subject's eye E and the distance measuring sensor 4.

[0087] Fig. 13(1) is a diagram showing how depth information of the ocular structure of the subject's eye E is detected using the distance measurement sensor 4. The depth information acquired in step S301 is depth information detected by facing the distance measurement sensor 4 to the subject's eye E, as shown in Fig. 13(1). Axis a (la) indicates the optical axis of the measurement light (measurement laser beam). Axis b (lb) indicates the gaze direction of the subject's eye E.

[0088] 14(2) and 14(3) are diagrams illustrating how depth information is detected using a distance measurement sensor 4 at various relative positions relative to the subject's eye E. L denotes a light source that emits measurement light from the distance measurement sensor 4. A denotes the angle A formed by the intersection of axis a (la) indicating the optical axis of the measurement light and axis b (lb) indicating the gaze direction of the subject's eye E. In the process of acquiring depth information of the ocular structure, an index may be used to guide the angle of the eye. The angle A may be, for example, 45 degrees or more, 50 degrees or more, 60 degrees or more, 70 degrees or less, 60 degrees or less, or 50 degrees or less. Among these, an angle between 45 degrees and 60 degrees is preferable. Depth information detected when angle A is between 45 degrees and 60 degrees tends to be useful because it easily captures information around the safe access area of ​​the vitreous body. Therefore, depth information detected when angle A is between 45 degrees and 60 degrees enables efficient understanding of the ocular structure.

[0089] In the example of Figure 14 (2), depth information is acquired with the subject's eye E tilted at an arbitrary angle with respect to the light source L, as shown by a10. As a means for tilting the subject's eye E, for example, a fixation target projection means may be used, which presents a fixed index (fixation target) at multiple positions for the subject's eye E to gaze at, thereby changing the gaze direction of the subject's eye E. The fixation target projection means has a fixation lamp that emits visible light, and by turning on the fixation lamp, presents the subject with a fixation target at multiple positions, thereby guiding the subject's eye E to gaze in an arbitrary direction. By using the fixation target projection means, it is possible to easily detect depth information even when a stationary device or a non-movable examination device is used.

[0090] In the example of Figure 14 (3), depth information is acquired with the light source L tilted arbitrarily with respect to the subject's eye E, as indicated by a11. As a means for tilting the light source L, for example, a detection device 2 equipped with a movable light source L or a detection device 2 equipped with multiple light sources L may be used. As a means for tilting the light source L, a detection device 2 equipped with audio guidance means that guides the subject to move the detection device 2 to an arbitrary position by audio guidance or the like may also be used. By using a means for tilting the light source L, it is possible to detect depth information without the subject having to take any action.

[0091] In the step of acquiring depth information of the eyeball structure, multiple pieces of depth information detected while changing the relative position between the distance measuring sensor 4 and the subject's eye may be acquired. This is expected to improve the accuracy of the position information generated in step S302. In addition, in one aspect, it is expected to improve the accuracy of site identification by the site identification model.

[0092] In the step of acquiring depth information of the eyeball structure, resolution processing may be performed using adaptive optics. For example, a distance measuring sensor 4 that applies adaptive optics may be used. Here, adaptive optics is a technology that removes wavefront aberration to increase resolution.

[0093] (Step S302: Identifying Regions) The eyeball observation support system 1 identifies two or more regions of the subject's eye based on the depth information of the subject's eye and the region identification model of the eyeball structure (step S302). For example, the eyeball observation support device 3 inputs the depth information of the subject's eye into the region identification model to identify two or more regions of the subject's eye.

[0094] (Step S303: Measuring Distances) The eyeball observation support system 1 measures various distances related to the ocular structure of the subject's eye based on the positions of the first and second portions of the subject's eye identified in step S302 (step S303). For example, the eyeball observation support device 3 measures the distances between two or more predetermined portions of the actual eyeball based on the depth information of the subject's eye acquired in step S301 and information related to the positions of the two or more predetermined portions of the identified subject's eye.

[0095] (Step S304: Generating a tomographic image of the eye) The eye observation support system 1 generates a tomographic image of the eye of the subject based on the depth information acquired in step S301 and the information on the positions of the first and second parts of the subject eye identified in step S302 (step S304).

[0096] (Step S305: Output Control) The eyeball observation support system 1 controls to output information on the distance measured in step S303 and the tomographic image of the eyeball generated in step S304 (step S305). Figures 15 to 17 are diagrams for explaining examples of screen displays output by the output control step.

[0097] 15 , a generated tomographic image of the eye d21, a display d22 of a plurality of search points indicating the position estimated to be the first portion of the eye, a display d23 of a plurality of search points indicating the position estimated to be the second portion of the eye, a display d24 of a line connecting the first portion and the second portion, and information d25 of the distance between the first portion and the second portion are displayed. As shown in FIG. 15 , by displaying the display d24 of the line connecting the first portion and the second portion and information of the distance between the first portion and the second portion together with the tomographic image of the eye, the user can efficiently grasp the positions of the first portion and the second portion of the eye.

[0098] 16 , a generated tomographic image of the eye d31, a display d32 of a plurality of search points indicating positions estimated to be the first region in the test eye, a display d33 of a plurality of search points indicating positions estimated to be the second region in the test eye, and a scale bar d34 are displayed. As shown in FIG. 16 , by displaying the scale bar together with the tomographic image of the eye, the user can efficiently grasp the distance between the first region and the second region in the test eye and the positions of the regions between the first region and the second region.

[0099] 17 shows an example of a screen display in which an unprocessed ocular tomogram d11 of the subject's eye before the analysis process has been uploaded, a button d12 for downloading the unprocessed ocular tomogram, a button d13 for uploading a new unprocessed ocular tomogram, a generated ocular tomogram d14 of the subject's eye generated by the analysis process, information d15 about the distance between predetermined sites in the subject's eye, and a button d16 for downloading the generated ocular tomogram. As shown in FIG. 17 , by displaying the unprocessed ocular tomogram together with the generated ocular tomogram, the user can compare the ocular tomograms before and after processing by the eye observation support device 3 of the present disclosure and can confirm whether the generated ocular tomogram is reliable.

[0100] (Other Steps) The eyeball observation support method using the eyeball observation support system 1 may include other steps in addition to the steps described above. For example, it may include a step of identifying a different part of the subject's eye from information about two or more predetermined positions on the subject's eye identified in step S302 and information about the distance measured in step S303. Such a step can be performed by the second position information generating unit 206 of the eyeball observation support device 3. By including such a step, the user can easily understand the structure of the eyeball even in situations where it is difficult to refer to the knowledge of an expert.

[0101] 6. Program A program according to one aspect of the present invention causes one or more computers to execute the following steps: a first step of acquiring depth information of an ocular structure in an eye to be examined; a second step of generating position information regarding a first portion and a second portion in the eye to be examined based on the depth information and a portion-specific model of the ocular structure; a third step of generating distance information regarding the distance between the first portion and the second portion based on the position information; a fourth step of generating a tomographic image of the eye based on the position information; and a fifth step of outputting the distance information and the tomographic image of the eye.

[0102] The present invention is not limited to the above-described embodiment and can be embodied in various other forms without departing from the spirit and scope of the present invention. Therefore, the above-described embodiment is merely illustrative in all respects and should not be construed as limiting. For example, the above-described information processing steps can be arbitrarily changed in order or executed in parallel as long as no inconsistency occurs in the processing content. For example, while the above-described embodiment illustrates an example of acquiring distance information between specific parts, the present invention is not limited thereto and can also be applied to acquiring information about the area or volume of specific parts.

[0103] For example, without limitation, during examinations by general ophthalmologists, vitreous surgical instruments are inserted from a standard position, which can lead to intraocular bleeding and rhegmatogenous retinal detachment in eyes with atypical ciliary body sizes. This can make macular manipulation particularly difficult in highly myopic eyes. However, the present invention makes it possible to determine ciliary body length, which was difficult to measure using conventional techniques, and use it for observation and diagnosis. Furthermore, it is possible to provide a method that can acquire various anterior segment distance information using a simple machine learning technique, even if the image quality of ASOCT images is low. Therefore, it is expected that this technology will be widely adopted in vitreous treatment and vitreous observation settings around the world. Furthermore, it is expected that the present invention will enable ciliary body length, which was difficult to measure using conventional techniques, to be included as information on a patient's profile. Having information on ciliary body length and the position of the ciliary body pars plana as information on a patient's profile in advance can be useful during surgery and diagnosis.

[0104] Furthermore, the present invention can be similarly used for observing and photographing the eyes of animals. The structure of the anterior segment of the eye varies greatly among species. Small animals, in particular, lack the pars plana ciliary body, unlike humans. Furthermore, their spherical lenses can significantly compress the vitreous, making access to the vitreous difficult. It is possible to generate tomographic images of the eyes of these animals, allowing the acquisition of ophthalmic findings.

[0105] 1...eyeball observation support system, 2...detection device, 2a...mobile terminal type detection device, 2b...fixed type detection device, 3...eyeball observation support device, 4...distance measurement sensor, 10...communication bus, 11...communication IF, 12...storage device, 13...input device, 14...output device, 15...control device, 20...cornea, 21...ciliary body, 21a...ciliary rugae wall portion, 21b...ciliary body plana portion, 22...bulbar conjunctiva, 23...sclera, 24...choroid, 25...retina, 26...optic nerve, 27...vitreous body, 28...iris, 29...lens, 30...zonules of Zinn, 40...ciliary Posterior end of body, 50...ciliary sulcus, 60...corneal limbus, 201...storage unit, 202...acquisition unit, 203...model construction unit, 204...first position information generation unit, 205...distance information generation unit, 206...second position information generation unit, 207...ocular tomographic image generation unit, 208...output control unit, 208a...display control unit, E...examined eye, axis a...optical axis of measurement light, axis b...gaze direction of examinee eye, A...angle generated by the intersection of la indicating the optical axis of the measurement light and lb indicating the gaze direction of examinee eye E, L...light source, a10...movement of examinee eye, a11...movement of light source

Claims

1. An eyeball observation support method comprising: a first step of acquiring depth information of the eyeball structure in the subject's eye; a second step of generating position information regarding a first portion and a second portion of the subject's eye based on the depth information and a model that identifies the portion of the eyeball structure; a third step of generating distance information regarding the distance between the first portion and the second portion based on the position information; a fourth step of generating a tomographic image of the eyeball including the position information; and a fifth step of outputting the distance information and the tomographic image of the eyeball.

2. The eyeball observation support method according to claim 1, wherein the depth information is acquired using a distance measuring sensor.

3. The eyeball observation support method according to claim 1, wherein the first portion is the limbus of the subject's eye, and the second portion is the posterior end of the ciliary body of the subject's eye.

4. The eyeball observation support method according to claim 1, wherein the depth information is acquired when the angle between the gaze direction of the eyeball of the subject's eye and the optical axis of the measurement light is between approximately 45 degrees and approximately 60 degrees.

5. The eyeball observation support method according to claim 1, wherein the model is a mathematical model trained by machine learning or deep learning.

6. An eyeball observation support method as described in claim 1, wherein the positional information regarding the first portion includes information regarding one or more first search points that are estimated by the model to be the first portion in the subject's eye, and the positional information regarding the second portion includes information regarding one or more second search points that are estimated by the model to be the second portion in the subject's eye.

7. An eyeball observation support method as described in claim 6, wherein the position information regarding the first portion further includes information regarding the reliability corresponding to each of the first search points, and the position information regarding the second portion further includes information regarding the reliability corresponding to each of the second search points.

8. The eyeball observation support method according to claim 1, wherein the second step further generates position information relating to a third portion of the subject's eye based on the depth information and the model.

9. The eyeball observation support method according to claim 2, wherein the distance measuring sensor is a distance measurement unit of a mobile terminal.

10. The eyeball observation support method according to claim 2, wherein the distance measuring sensor is a distance measuring unit of a stationary device.

11. An eyeball observation support device comprising: an acquisition unit that acquires depth information of the eyeball structure in the subject's eye; a position information generation unit that generates position information regarding a first portion and a second portion in the subject's eye based on the depth information and a model that identifies the portion of the eyeball structure; a distance information generation unit that generates distance information regarding the distance between the first portion and the second portion based on the position information; an eyeball tomographic image generation unit that generates an eyeball tomographic image including the position information; and an output control unit that controls the output of the distance information and the eyeball tomographic image.

12. A program that causes one or more computers to execute the following steps: a first step of acquiring depth information of the ocular structure in the subject's eye; a second step of generating position information regarding a first portion and a second portion in the subject's eye based on the depth information and a model that identifies the portions of the ocular structure; a third step of generating distance information regarding the distance between the first portion and the second portion based on the position information; a fourth step of generating a tomographic image of the eye including the position information; and a fifth step of outputting the distance information and the tomographic image of the eye.

Images