Ophthalmic observation apparatus, control method thereof, program, and recording medium

Abstract

An ophthalmic observation apparatus (1) of an exemplary embodiment includes a dynamic image generation section (30, 40), an analysis section (211), a target position decision section (212), and a display control section (200). The dynamic image generation section captures an eye under examination to generate a dynamic image. The analysis section analyzes a still image included in the dynamic image generated by the dynamic image generation section to detect an image of a predetermined site of the eye under examination. The target position decision section decides a target position of a predetermined treatment based on at least the image of the predetermined site detected by the analysis section. The display control section causes a display device (3) to display the dynamic image generated by the dynamic image generation section and target position information indicating the target position decided by the target position decision section.

Description

Technical Field

[0001] (Mutual references between related applications)

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 106,087, filed October 27, 2020, entitled “APPARATUS AND METHOD FOROPHTHALMIC OBSERVATION”, which is incorporated herein by reference in its entirety.

[0003] This disclosure relates to an ophthalmic observation device, its control method, procedure, and recording medium. Background Technology

[0004] An ophthalmic observation device is used to observe a patient's eye (the eye being examined). Ophthalmic observation is performed in various situations, such as examinations, surgeries, and treatments, to monitor the condition of the eye being examined.

[0005] Traditional ophthalmic observation devices provide the user with a magnified image obtained from an objective lens and a zoom optical system via an eyepiece lens. However, in recent years, ophthalmic observation devices with the following structure have emerged: they use a camera element to capture a magnified image obtained from an objective lens and a zoom optical system, and display the resulting image (called a digital ophthalmic observation device). Types of digital ophthalmic observation devices include surgical microscopes, slit-lamp microscopes, and fundus cameras. Furthermore, various ophthalmic examination devices, such as refractometers, corneal astigmatism meters, tonometers, corneal endothelial microscopes, wavefront aberrometers, and microperimeters, also incorporate functions as digital ophthalmic observation devices.

[0006] Furthermore, in recent years, ophthalmic observation devices have included those utilizing optical scanning (scanning ophthalmic observation devices). Such ophthalmic devices include scanning laser ophthalmoscopes (SLO) and optical coherence tomography (OCT) devices.

[0007] Typically, ophthalmic observation devices provide users (e.g., physicians and other healthcare professionals) with dynamic images of the eye being examined. Digital ophthalmic observation devices are typically configured to perform dynamic image capture using infrared and / or visible light as illumination and real-time dynamic image display of the resulting images. On the other hand, scanning ophthalmic observation devices are typically configured to perform data collection based on repeated light scans, real-time image reconstruction based on the sequentially collected datasets, and real-time dynamic image display of the sequentially reconstructed images. These real-time dynamic images are referred to as observation images or live images.

[0008] Ophthalmic observation devices capable of providing real-time dynamic images are also used in surgery. There are many types of ophthalmic surgery, but many involve inserting instruments into the eye. Incisions are made in the eye tissues during instrument insertion. For example, in cataract surgery using an intraocular lens (IOL) to replace the lens, corneal incision (cornosclerotomy) and anterior capsulotomy (CCC) are typically performed. Similarly, in refractive surgery to retain an intraocular contact lens (ICL; also known as a replica IOL), corneal incision is usually performed. Furthermore, in minimally invasive glaucoma surgery (MIGS), incision of the fibrous struts and insertion of a support into them are performed. Additionally, in vitrectomy using light guides (illumination) and various other instruments, conjunctival and scleral incisions are typically performed.

[0009] Such incisions in the eye tissue need to be made in appropriate locations. However, there are few areas on the eye that can be marked, making it difficult to mark the eye itself, and the eyeball moves during surgery, making it difficult to determine the appropriate target location for the incision. The same applies to procedures other than incisions.

[0010] Patent Document 1: Japanese Patent Application Publication No. 2019-162336 Summary of the Invention

[0011] One of the purposes of this disclosure is to provide a new technique for making ophthalmic surgery easier.

[0012] Several exemplary embodiments are ophthalmic observation devices for observing an eye under examination, which may include: a dynamic image generation unit that captures the eye under examination to generate a dynamic image; an analysis unit that analyzes still images included in the dynamic image to detect an image of a predetermined region of the eye under examination; a target position determination unit that determines a target position for a predetermined treatment based at least on the image of the predetermined region detected by the analysis unit; and a display control unit that causes a display device to display the dynamic image and target position information indicating the target position.

[0013] In several exemplary ophthalmic observation devices, the target location determination unit may also determine the target location based at least on the image of the predetermined region detected by the analysis unit and the measurement data of the examined eye acquired in advance.

[0014] In several exemplary ophthalmic observation devices, it is also possible to further include a measurement unit that acquires measurement data of the examined eye, and the target location determination unit determines the target location based at least on the image of the predetermined region detected by the analysis unit and the measurement data acquired by the measurement unit.

[0015] In several exemplary ophthalmic observation devices, the target location determination unit may include an evaluation unit that evaluates the impact of the predetermined treatment on the examined eye based on the measurement data obtained by the measurement unit, and the target location determination unit determines the target location based at least on the image of the predetermined site detected by the analysis unit and the result obtained by the evaluation unit.

[0016] In several exemplary ophthalmic observation devices, the target position determination unit may also determine the target position as the position at a predetermined distance from the image of the predetermined region detected by the analysis unit.

[0017] In several exemplary ophthalmic observation devices, when a first treatment and a second treatment are applied to the examined eye, the analysis unit may also be configured to perform the following detections: a first detection, analyzing a first still image generated by the dynamic image generation unit before the first treatment to detect an image of a first portion of the examined eye; and a second detection, analyzing a second still image generated by the dynamic image generation unit before the second treatment to detect an image of a second portion of the examined eye. Furthermore, the target position determination unit may also be configured to perform the following decisions: a first decision, determining a first target position as the target position for the first treatment based at least on the image of the first portion; and a second decision, determining a second target position as the target position for the second treatment based at least on the image of the second portion. Additionally, the display control unit may also be configured to perform the following controls: a first display control, displaying the dynamic image and first target position information indicating the first target position for the first treatment; and a second display control, displaying the dynamic image and second target position information indicating the second target position for the second treatment.

[0018] In several exemplary ophthalmic observation devices, a treatment transfer detection unit is also included for detecting the transfer from the first treatment to the second treatment.

[0019] Several exemplary embodiments are methods for controlling an ophthalmic observation device, the ophthalmic observation device including a processor and generating a dynamic image of an eye being examined, or the processor being configured to: analyze a still image included in the dynamic image and detect an image of a predetermined region of the eye being examined; determine a target position for a predetermined treatment based at least on the detected image of the predetermined region; and cause a display device to display the dynamic image and target position information indicating the target position.

[0020] Several instantiations are a program that causes a computer to execute any one of the instantiations.

[0021] Several exemplary modes are computer-readable, non-transitory recording media that record the program of any one of the exemplary modes.

[0022] According to the illustrative approach, it is possible to make ophthalmic surgery easier. Attached Figure Description

[0023] Figure 1 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device (ophthalmic surgical microscope) according to an exemplary embodiment.

[0024] Figure 2 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0025] Figure 3 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0026] Figure 4 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0027] Figure 5 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0028] Figure 6 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0029] Figure 7 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0030] Figure 8A This is a flowchart illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0031] Figure 8B This is a flowchart illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0032] Figure 9A This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0033] Figure 9B This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0034] Figure 9C This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0035] Figure 9DThis is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0036] Figure 9E This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0037] Figure 10A This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0038] Figure 10B This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0039] Figure 10C This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0040] Figure 11 This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment. Detailed Implementation

[0041] Referring to the accompanying drawings, several exemplary embodiments of the ophthalmic observation device, the method and procedure for controlling it, and the recording medium are described in detail. Furthermore, matters described in the documents cited in this specification, any known techniques, and exemplary embodiments can be combined.

[0042] The exemplary ophthalmic observation devices are used in medical procedures such as surgery, examination, and treatment to monitor the condition of the examined eye. The exemplary ophthalmic observation devices described below are surgical microscope systems, but are not limited to surgical microscope systems. For example, an ophthalmic observation device can be any of the following: a slit-lamp microscope, fundus camera, refractometer, corneal astigmatism meter, tonometer, corneal endothelial microscope, wavefront aberrometer, microperimeter, SLO, and OCT device, and can also be a system including any and more of these. More commonly, an ophthalmic observation device can be any ophthalmic device with observation capabilities.

[0043] The observation site using ophthalmic observation devices can be any part of the examined eye, including any part of the anterior and / or posterior eye. Examples of anterior eye observation sites include the cornea, iris, anterior chamber, anterior chamber angle, lens, ciliary body, and zonules of Qin. Examples of posterior eye observation sites include the retina, choroid, sclera, and vitreous body. The observation site is not limited to ocular tissues; it can also be any part of the eyelids, meibomian glands, or orbit used in ophthalmology (and / or other specialties) for observation.

[0044] Ophthalmic observation devices are used, for example, in surgeries where artificial objects are inserted into the eye. These artificial objects can be any device that remains inside the eye, such as an intraocular lens, an intraocular contact lens, or a MIGS device (stent). Additionally, artificial objects can also be any medical device, such as light guides, surgical instruments, examination instruments, or treatment instruments.

[0045] At least a portion of the functionality of the elements disclosed in this specification is implemented using a circuitry or processing circuitry. The circuitry or processing circuitry includes any of the following: a common processor, a special-purpose processor, an integrated circuit, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a programmable logic device (e.g., a SPLD (Simple Programmable Logic Device), a CPLD (Complex Programmable Logic Device), an FPGA (Field Programmable Gate Array)), conventional circuitry, and any combination thereof. A processor is considered to be a processing circuitry or circuitry including transistors and / or other circuitry. In this disclosure, circuitry, units, components, or similar terms are hardware that performs at least a portion of the disclosed functionality or hardware programmed to perform at least a portion of the disclosed functionality. The hardware may be the hardware disclosed in this specification or known hardware programmed and / or configured to perform at least a portion of the functions described herein. In the case where the hardware is a processor considered as a type of circuit structure, the circuit structure, unit, component, or similar term is a combination of hardware and software used to constitute the hardware and / or processor.

[0046] <Ophthalmic Observation Device>

[0047] Figure 1 The structure of an ophthalmic observation device is shown as an example.

[0048] The ophthalmic observation device 1 (surgical microscope system) of the embodiment includes an operating device 2, a display device 3, and a surgical microscope 10. In several embodiments, the surgical microscope 10 may include at least one of the operating device 2 and the display device 3. Additionally, in several embodiments, the display device 3 may not be included in the ophthalmic observation device 1. That is, the display device 3 may be a peripheral device of the ophthalmic observation device 1.

[0049] <Operating Device 2>

[0050] Operating device 2 includes operating equipment and / or input devices. For example, operating device 2 may also include buttons, switches, mice, keyboards, trackballs, operation panels, dials, etc. Typically, operating device 2 includes a foot switch, similar to that of a general ophthalmic surgical microscope. Alternatively, it may be configured to be operated using voice recognition, gaze input, etc.

[0051] <Display Device 3>

[0052] Display device 3 displays images of the examined eye acquired by surgical microscope 10. Display device 3 includes display devices such as flat panel displays. Alternatively, display device 3 may also include various display devices such as touch panels. A typical display device 3 includes a large-screen display. Display device 3 may include one or more display devices. In cases where display device 3 includes two or more display devices, for example, one may be a larger-screen display device and another a smaller-screen display device. Furthermore, a structure can be adopted where multiple display areas are set on a single display device to display multiple pieces of information.

[0053] The operating device 2 and the display device 3 do not need to be separate devices. For example, a device that integrates operating and display functions, such as a touch panel, can be used as the display device 3. In this case, the operating device 2 includes the touch panel and a computer program. Operations on the operating device 2 are input to a processor (not shown) as electrical signals. Alternatively, operations and information input can be performed using a graphical user interface (GUI) displayed on the display device 3 and the operating device 2. In several ways, the functions of the operating device 2 and the display device 3 can also be implemented using a touch screen.

[0054] <Surgical Microscope 10>

[0055] The surgical microscope 10 is used to observe the eye (examined eye) of a patient in a supine position. The surgical microscope 10 captures images of the examined eye to generate digital image data. In particular, the surgical microscope 10 generates a dynamic image of the examined eye. The dynamic image (image) generated by the surgical microscope 10 is transmitted to and displayed on the display device 3 via wired and / or wireless signal lines. The user (surgical operator) can perform surgery while observing the examined eye through the displayed image. In addition to such image observation, several types of surgical microscopes 10 also allow observation via conventional eyepiece lenses.

[0056] In several embodiments, the surgical microscope 10 includes a communication device for transmitting and receiving electrical signals between itself and the operating device 2. The operating device 2 receives user operations and generates corresponding electrical signals (operation signals). The operation signals are transmitted to the surgical microscope 10 via wired and / or wireless signal lines. The surgical microscope 10 performs processing corresponding to the received operation signals.

[0057] <Optical System of Surgical Microscope 10>

[0058] Hereinafter, for ease of explanation, the optical system structure of the surgical microscope 10 will be described as follows: the direction of the optical axis of the objective lens will be defined as the z-direction (e.g., the vertical direction during surgery, the up-down direction), a predetermined direction orthogonal to the z-direction will be defined as the x-direction (e.g., the horizontal direction during surgery, the left-right direction for the surgeon and the patient), and a direction orthogonal to both the z-direction and the x-direction will be defined as the y-direction (e.g., the horizontal direction during surgery, the front-back direction for the surgeon, the body axis direction for the patient).

[0059] Furthermore, the following description primarily focuses on the case where the observation optical system has a pair of left and right optical systems (optical systems that allow binocular observation). However, other types of observation optical systems can have a monocular observation optical system, and those skilled in the art will recognize that the structure described below can be applied to such systems.

[0060] Figure 2 An example of the structure of the optical system of the surgical microscope 10 is shown. Figure 2 This diagram shows a schematic top view of the optical system viewed from above and a schematic side view of the optical system viewed from the side. For the sake of simplicity, the illumination optical system 30, which is placed above the objective lens 20, is omitted from the top view.

[0061] The surgical microscope 10 includes an objective lens 20, a dichroic mirror DM1, an illumination optics system 30, and an observation optics system 40. The observation optics system 40 includes a zoom extender 50 and a camera 60. In several configurations, either the illumination optics system 30 or the observation optics system 40 includes the dichroic mirror DM1.

[0062] Objective lens 20 is positioned facing the eye being examined. Objective lens 20 is positioned such that its optical axis is along the z-direction. Objective lens 20 may also include two or more lenses.

[0063] The dichroic mirror DM1 couples the optical path of the illumination optical system 30 with the optical path of the observation optical system 40. The dichroic mirror DM1 is placed between the illumination optical system 30 and the objective lens 20. The dichroic mirror DM1 transmits the illumination light from the illumination optical system 30 and guides it through the objective lens 20 to the eye being examined, and reflects and guides the reflected light from the eye being examined through the objective lens 20 to the camera 60 of the observation optical system 40.

[0064] The dichroic mirror DM1 coaxially couples the optical paths of the illumination optical system 30 and the observation optical system 40. That is, the optical axes of the illumination optical system 30 and the observation optical system 40 intersect on the dichroic mirror DM1. When the illumination optical system 30 includes a left-eye illumination optical system (31L) and a right-eye illumination optical system (31R), and the observation optical system 40 includes a left-eye observation optical system 40L and a right-eye observation optical system 40R, the dichroic mirror DM1 coaxially couples the optical paths of the left-eye illumination optical system (first illumination optical system 31L) with those of the left-eye observation optical system 40L, and coaxially couples the optical paths of the right-eye illumination optical system (first illumination optical system 31R) with those of the right-eye observation optical system 40R.

[0065] The illumination optical system 30 is an optical system used to illuminate the eye being examined with the aid of the objective lens 20. The illumination optical system 30 can be configured to illuminate the eye being examined using any one of two or more illumination lights with different color temperatures. Under the control of the control unit (200) described later, the illumination optical system 30 projects illumination light of a specified color temperature onto the eye being examined.

[0066] The illumination optical system 30 includes first illumination optical systems 31L and 31R and a second illumination optical system 32.

[0067] The optical axes OL and OR of the first illumination optical system 31L and 31R are respectively positioned approximately coaxially with the optical axis of the objective lens 20. This enables coaxial illumination and allows the acquisition of a transillumination image utilizing diffuse reflection from the fundus. In this method, the transillumination image of the examined eye can be observed with both eyes.

[0068] The second illumination optical system 32 is positioned such that its optical axis OS is off-center from the optical axis of the objective lens 20. The first illumination optical systems 31L and 31R, as well as the second illumination optical system 32, are positioned such that the offset of the optical axis OS relative to the optical axis of the objective lens 20 is greater than the offset of the optical axes OL and OR relative to the optical axis of the objective lens 20. This enables so-called "angled illumination (oblique illumination)," preventing the intrusion of ghosting caused by corneal reflections, etc., while allowing binocular observation of the examined eye. Furthermore, it allows for detailed observation of the areas and tissues of the examined eye.

[0069] The first illumination optical system 31L includes a light source 31LA and a condenser lens 31LB. The light source 31LA outputs illumination light of a wavelength in the visible region, for example, with a color temperature of 3000K (Kelvin). The illumination light output from the light source 31LA passes through the condenser lens 31LB, is transmitted through the dichroic mirror DM1, and is incident on the eye being examined through the objective lens 20.

[0070] The first illumination optical system 31R includes a light source 31RA and a condenser lens 31RB. The light source 31RA also outputs illumination light of a wavelength in the visible region, for example, with a color temperature of 3000K. The illumination light output from the light source 31RA passes through the condenser lens 31RB, is transmitted through the dichroic mirror DM1, and is incident on the eye being examined through the objective lens 20.

[0071] The second illumination optical system 32 includes a light source 32A and a condenser lens 32B. The light source 32A outputs illumination light with a wavelength in the visible range, for example, a color temperature of 4000K to 6000K. The illumination light output from the light source 32A passes through the condenser lens 32B and, without passing through the dichroic mirror DM1, passes through the objective lens 20 and enters the eye being examined.

[0072] That is, the color temperature of the illumination light from the first illumination optical system 31L and 31R is lower than the color temperature of the illumination light from the second illumination optical system 32. With such a structure, the eye under examination can be observed using the first illumination optical system 31L and 31R with a warm color, and the structure and manner of the eye under examination can be observed in detail.

[0073] In several ways, the optical axes OL and OR can be moved relatively relative to the optical axis of the objective lens 20. This relative movement is in the direction intersecting the optical axis of the objective lens 20, and is represented by a displacement vector in at least one of the x and y components that is not zero. In several ways, the optical axes OL and OR can be moved independently. On the other hand, in several ways, the optical axes OL and OR can be moved as a unit. For example, the surgical microscope 10 includes a moving mechanism (31d) that moves the first illumination optical systems 31L and 31R independently or as a unit, by which the first illumination optical systems 31L and 31R are moved independently or as a unit in a direction intersecting the optical axis of the objective lens 20. This allows adjustment of the visual effect on the examined eye. In several ways, the moving mechanism is operated under the control of a control unit (200) described later.

[0074] In several ways, the optical axis OS can be moved relative to the optical axis of the objective lens 20. This relative movement is in the direction intersecting the optical axis of the objective lens 20, and is expressed as a displacement vector in at least one of the x and y components that is not zero. For example, the surgical microscope 10 includes a moving mechanism (32d) that moves the second illumination optical system 32 in a direction intersecting the optical axis of the objective lens 20. This allows for adjustment of the visual effect of the concavity and convexity of the examined eye area and tissue. In several ways, the moving mechanism is operated under the control of a control unit (200), described later.

[0075] As described above, in this configuration, the illumination optical system 30 is placed directly above the objective lens 20 (at the position of the transmission direction of the dichroic mirror DM1), and the observation optical system 40 is placed at the position of the reflection direction of the dichroic mirror DM1. For example, the observation optical system 40 may be positioned such that the angle formed by the optical axis of the observation optical system 40 and the plane (xy plane) orthogonal to the optical axis of the objective lens 20 is ±20 degrees or less.

[0076] According to the structure of this method, the observation optical system 40, whose optical path length is longer than that of the illumination optical system 30, is typically placed approximately parallel to the xy plane. Therefore, unlike conventional surgical microscopes where the observation optical system is placed vertically in front of the surgeon's eyes, it does not obstruct the surgeon's field of vision. Consequently, the surgeon can easily observe the image displayed on the front-facing display device 3. In other words, the visibility of displayed information (images, reflections, and other reference information of the examined eye) during surgery is improved. Furthermore, since no housing is placed in front of the surgeon's eyes, it does not create a feeling of pressure on the surgeon, thus reducing the surgeon's workload.

[0077] The observation optical system 40 is an optical system for observing an image formed based on the reflected light from the illumination light incident from the examined eye via the objective lens 20. In this configuration, the observation optical system 40 provides the image to the imaging element of the camera 60.

[0078] As described above, the observation optical system 40 includes a left-eye observation optical system 40L and a right-eye observation optical system 40R. The structure of the left-eye observation optical system 40L is the same as that of the right-eye observation optical system 40R. In several ways, the optical placement of the left-eye observation optical system 40L and the right-eye observation optical system 40R can also be changed independently of each other.

[0079] The zoom expander 50 is also referred to as a beam expander, variable beam expander, etc. The zoom expander 50 includes a left-eye zoom expander 50L and a right-eye zoom expander 50R. The structure of the left-eye zoom expander 50L is the same as that of the right-eye zoom expander 50R. In several ways, the optical placement of the left-eye zoom expander 50L and the right-eye zoom expander 50R can also be changed independently.

[0080] The zoom extender 50L for the left eye includes multiple zoom lenses 51L, 52L, and 53L. At least one of the multiple zoom lenses 51L, 52L, and 53L is movable in the optical axis direction via a zoom mechanism (not shown).

[0081] Similarly, the zoom extender 50R for the right eye includes multiple zoom lenses 51R, 52R and 53R, at least one of which is movable in the optical axis direction via a zoom mechanism (not shown).

[0082] The zoom mechanism can be configured to move each zoom lens of the left-eye zoom extender 50L and each zoom lens of the right-eye zoom extender 50R independently or as a unit towards the optical axis. This changes the magnification when photographing the eye being examined. In several configurations, the zoom mechanism is operated under the control of the control unit (200), described later.

[0083] The video camera 60 is a device that captures an image formed by the observation optical system 40 and generates digital image data; typically, it is a digital camera (digital video camera). The video camera 60 includes a left-eye video camera 60L and a right-eye video camera 60R. The structure of the left-eye video camera 60L is the same as that of the right-eye video camera 60R. In several ways, the optical placement of the left-eye video camera 60L and the right-eye video camera 60R can be changed independently of each other.

[0084] The left-eye camera 60L includes an imaging lens 61L and an image sensor 62L. The imaging lens 61L forms an image on the imaging surface of the image sensor 62L based on the reflected light from the zoom extender 50L. The image sensor 62L is a region sensor, typically a charge-coupled device (CCD) image sensor or a complementary metal-oxide-semiconductor (CMOS) image sensor. The image sensor 62L operates under the control of a control unit (200) described later.

[0085] The right-eye camera 60R includes an imaging lens 61R and an image sensor 62R. The imaging lens 61R forms an image on the imaging surface of the image sensor 62R based on the reflected light from the right-eye zoom extender 50R. The image sensor 62R is a region sensor, typically a CCD image sensor or a CMOS image sensor. The image sensor 62R operates under the control of a control unit (200) described later.

[0086] <Processing System>

[0087] This describes the processing system of the ophthalmic observation device 1. Figure 3 and Figure 4 An example of the basic structure of a processing system is shown, and Figure 5 , Figure 6 as well as Figure 7 Several examples of specific structures are shown. Any two or more of the various structural examples described below can be combined, at least partially. Furthermore, the structure of the processing system is not limited to these examples.

[0088] The control unit 200 controls each part of the ophthalmic observation device 1. The control unit 200 includes a main control unit 201 and a storage unit 202. The main control unit 201 includes a processor that controls each part of the ophthalmic observation device 1. For example, in order to realize the functions of this method, the processor can read and execute programs stored in the storage unit 202 or other storage devices, and can utilize (reference, process, calculate, etc.) data and information stored in the storage unit 202 or other storage devices.

[0089] The main control unit 201 is capable of controlling the light sources 31LA, 31RA and 32A of the illumination optical system 30, the imaging elements 62L and 62R of the observation optical system 40, the movement mechanisms 31d and 32d, the zoom mechanisms 50Ld and 50Rd, the operation device 2, and the display device 3.

[0090] The control of light source 31LA includes turning the light source on and off, adjusting the light intensity, and adjusting the aperture. The control of light source 31RA includes turning the light source on and off, adjusting the light intensity, and adjusting the aperture. The main control unit 201 can exclusively control light sources 31LA and 31RA. The control of light source 32A includes turning the light source on and off, adjusting the light intensity, and adjusting the aperture.

[0091] When the lighting optical system 30 includes a light source capable of changing color temperature, the main control unit 201 can change the color temperature of the output lighting light by controlling the light source.

[0092] The control of image sensor 62L includes exposure adjustment, gain adjustment, and imaging rate adjustment. The control of image sensor 62R includes exposure adjustment, gain adjustment, and imaging rate adjustment. Furthermore, the main control unit 201 can control image sensors 62L and 62R so that their imaging timing is synchronized or the difference between their imaging timings is within a predetermined time. Moreover, the main control unit 201 can control the readout of digital data obtained from image sensors 62L and 62R.

[0093] The moving mechanism 31d moves the light sources 31LA and 31RA independently or integrally in a direction intersecting the optical axis of the objective lens 20. The main control unit 201 can move the optical axes OL and OR independently or integrally relative to the optical axis of the objective lens 20 by controlling the moving mechanism 31d.

[0094] The moving mechanism 32d moves the light source 32A in a direction intersecting the optical axis of the objective lens 20. The main control unit 201 can move the optical axis OS relative to the optical axis of the objective lens 20 by controlling the moving mechanism 32d.

[0095] The moving mechanism 70 moves the surgical microscope 10. For example, the moving mechanism 70 is configured to integrate at least a portion of the illumination optics 30 with the observation optics 40. This allows for changing the relative positions of at least a portion of the illumination optics 30 and the observation optics 40 relative to the examined eye while maintaining the relative positional relationship between them. In several configurations, the moving mechanism 70 is configured to move the first illumination optics 31L and 31R integrally with the observation optics 40. This allows for changing the relative positions of the first illumination optics 31L and 31R and the observation optics 40 relative to the examined eye while maintaining coaxial illumination. In several configurations, the moving mechanism 70 is configured to move the second illumination optics 32 integrally with the observation optics 40. This allows for changing the relative positions of the second illumination optics 32 and the observation optics 40 relative to the examined eye while maintaining the oblique illumination angle. In several configurations, the moving mechanism 70 is designed to move the first illumination optical systems 31L and 31R, the second illumination optical system 32, and the observation optical system 40 as a single unit. This allows for changing the relative positions of the illumination optical system 30 and the observation optical system 40 with respect to the eye being examined, while maintaining both coaxial illumination and oblique illumination angles. The moving mechanism 70 operates under the control of the control unit 200.

[0096] In several ways, the main control unit 201 can control at least two of the moving mechanisms 31d, 32d and 70 in conjunction.

[0097] The zoom mechanism 50Ld moves at least one of the multiple zoom lenses 51L to 53L of the left-eye zoom extender 50L in the optical axis direction. The main control unit 201 can change the magnification of the left-eye observation optical system 40L by controlling the zoom mechanism 50Ld.

[0098] Similarly, the zoom mechanism 50Rd moves at least one of the multiple zoom lenses 51R to 53R of the right-eye zoom extender 50R in the optical axis direction. The main control unit 201 can change the magnification of the right-eye observation optical system 40R by controlling the zoom mechanism 50Rd.

[0099] Control of the operating device 2 includes operation permission control, operation prohibition control, transmission control and / or reception control of operation signals from the operating device 2, etc. The main control unit 201 receives the operation signals generated by the operating device 2 and executes the control corresponding to the received signals.

[0100] Control of the display device 3 includes information display control, etc. The main control unit 201, acting as a display control unit, enables the display device 3 to display various types of information. For example, the main control unit 201 can enable the display device 3 to display images based on digital image data generated by the imaging elements 62L and 62R. Typically, the display device 3 can display moving images (images) based on digital image data (image signals) generated by the imaging elements 62L and 62R, and can also display still images (frames) included in the moving images. Furthermore, the main control unit 201 can enable the display device 3 to display images (moving images, still images, etc.) obtained by processing the digital image data generated by the imaging elements 62L and 62R. Additionally, the main control unit 201 can enable the display device 3 to display any information generated by the ophthalmic observation device 1, and any information acquired by the ophthalmic observation device 1 from external sources.

[0101] The main control unit 201, acting as a display control unit, creates a left-eye image based on digital image data generated by the imaging element 62L and a right-eye image based on digital image data generated by the imaging element 62R. The display device 3 can then display the created left-eye and right-eye images in a stereoscopic manner. For example, the main control unit 201 can create a pair of parallax images based on the left-eye and right-eye images, and the display device 3 can then display this pair of parallax images. Users (such as surgical operators) can use known stereoscopic observation methods to recognize the pair of parallax images as stereoscopic images. Any stereoscopic observation method applicable to this method can be used, such as a naked-eye stereoscopic observation method, a stereoscopic observation method using assistive devices (polarized glasses, etc.), a stereoscopic observation method utilizing image processing (image synthesis, rendering, etc.) of the left-eye and right-eye images, a stereoscopic observation method that displays a pair of parallax images simultaneously, a stereoscopic observation method that switches between displaying a pair of parallax images, or a stereoscopic observation method that combines two or more of these methods.

[0102] The data processing unit 210 performs various data processing operations. Several examples of the processing that the data processing unit 210 can perform are described below. The data processing unit 210 (its components) includes a processor that operates according to predetermined software (programs), and is implemented through hardware and software cooperation.

[0103] Several examples of the processing that the data processing unit 210 can perform, together with the associated elements, are described. Figure 4 An example of the basic structure of the data processing unit 210 is shown. In this example, the data processing unit 210 includes an analysis unit 211 and a target position determination unit 212.

[0104] The surgical microscope 10 captures images of the examined eye and generates dynamic images (live images). The control unit 200 captures frames (still images) of the dynamic images and inputs them to the analysis unit 211. For example, the control unit 200 captures frames from the dynamic images in response to manual or automatic triggering.

[0105] The analysis unit 211 analyzes the frames captured from the moving image and detects the image of a predetermined part of the eye being examined (i.e., the image region corresponding to the predetermined part of the eye being examined). The image region detected by the analysis unit 211 based on the frame may be, for example, an image region determined to be the image of that part, or an image region obtained by processing the image of that part (and its vicinity) (e.g., an approximate shape such as an approximate ellipse or an approximate circle).

[0106] The predetermined site of the examined eye detected by the analysis unit 211 can be any site. When the object of observation in the ophthalmic observation device 1 is the anterior eye, the analysis unit 211 may be configured to detect, for example, any one of the following: pupil (whole or characteristic sites such as pupil edge, pupil center, pupil centroid, etc.), cornea (whole or characteristic sites such as corneal rim, corneal edge, corneal center, corneal apex, etc.), iris (whole or characteristic sites such as inner iris edge, outer iris edge, iris pattern, etc.), anterior chamber (whole or characteristic sites such as anterior boundary, posterior boundary, etc.), anterior chamber angle (whole or peripheral area, etc.), lens (whole or characteristic sites such as lens capsule, anterior capsule, posterior capsule, lens nucleus, etc.), ciliary body, zona simuli, blood vessels, and lesion sites. When the object of observation in the ophthalmic observation device 1 is the posterior eye, the analysis unit 211 can be configured to detect, for example, any one of the following: optic nerve head, macula, blood vessels, retina (whole, surface, or one or more sub-tissues), choroid (whole, anterior, posterior, or one or more sub-tissues), sclera (whole, anterior, posterior, or one or more sub-tissues), vitreous body (whole, opacity, floaters, detached tissue, etc.), and lesion sites. When the object of observation in the ophthalmic observation device 1 is not the eyeball, the analysis unit 211 can be configured to detect any location (tissue) such as the eyelid, meibomian glands, or eye socket. The location detected by the analysis unit 211 can be determined according to the illumination method, surgical site, surgical method, etc.

[0107] The analysis unit 211 can detect the image of a predetermined region of the eye being examined from a still image using an arbitrary region extraction method. For example, when detecting the image of a region with brightness characteristics, the analysis unit 211 can be configured to use brightness thresholding processing such as binarization to detect the image of the predetermined region of the eye being examined from the still image. When detecting the image of a region with shape characteristics, the analysis unit 211 can be configured to use shape analysis processing such as pattern matching to detect the image of the predetermined region of the eye being examined from the still image. When detecting the image of a region with hue characteristics, the analysis unit 211 can be configured to use color analysis processing such as feature color extraction to detect the image of the predetermined region of the eye being examined from the still image. In addition, the analysis unit 211 can be configured to detect the image of the predetermined region of the eye being examined by applying the segmentation used to determine the image of the predetermined region of the eye being examined to the still image. Generally, segmentation is a process of determining local regions in an image. Segmentation can include any known image processing technique, such as segmentation using image processing such as edge detection and / or segmentation using machine learning (e.g., deep learning).

[0108] The image of the predetermined region of the examined eye (indicating its position, extent, etc.) detected by the analysis unit 211 is input to the target position determination unit 212. The target position determination unit 212 determines the target position for the predetermined treatment based at least on the image of the predetermined region detected by the analysis unit 211.

[0109] The planned procedure can be any medical action, including procedures during surgery or treatment. In several cases, the planned procedure can also be any one or more of the following during cataract surgery: incision (keratotomy, keratosclerotomy), ophthalmic viscoelastic injection, cleft phacoemulsification (CCC), lens emulsification aspiration, IOL insertion, ophthalmic viscoelastic removal, and incision closure. In several cases, the planned procedure can be any procedure during refractive surgery, any procedure during minimally invasive glaucoma surgery (MIGS), any procedure during vitrectomy, or any procedure during other ophthalmic surgeries.

[0110] Several examples of target location determination processing are explained. In the first example, a predicted value for the examined eye is calculated. The measurement data obtained from the examined eye using the predicted value is stored in the storage unit 202 of the ophthalmic observation device 1 or in a storage device accessible by the ophthalmic observation device 1. This can be applied in the first example. Figure 5 The structure shown is an example. The data processing unit 210A is... Figure 4 The data processing unit 210 is an example, and the target position determination unit 212A is an example of the target position determination unit 212.

[0111] In cases where the intended procedure is ocular incision (corneal incision, corneoscleral incision, conjunctival incision, scleral incision, etc.), the predicted quantity may be, for example, any one of aberration measurement of the examined eye using a wavefront sensor, corneal shape measurement of the examined eye using a corneal topography device, or corneal shape measurement of the examined eye using an anterior ocular OCT device. Wavefront sensors and corneal topography devices are described, for example, in International Publication No. 2003 / 022138. Anterior ocular OCT is described, for example, in International Publication No. 2017 / 154348.

[0112] Figure 5 The measurement data 203 shown is an example of data obtained from the examined eye through a predicted quantity. Measurement data 203 may include, for example, aberration data (corneal aberration data) of the examined eye obtained using a wavefront sensor, corneal shape data of the examined eye obtained using a corneal topography instrument, corneal shape data of the examined eye obtained using an anterior ocular OCT device, etc.

[0113] The target location determination unit 212A in the first example is configured to determine the target location of the predetermined treatment based at least on the image and measurement data 203 of the predetermined part of the examined eye detected by the analysis unit 211. When the predetermined treatment is an ocular incision (corneal incision, corneal-scleral incision, etc.), the target location determination unit 212A can determine the location that minimizes corneal astigmatism obtained by the incision.

[0114] For example, the target location determination unit 212A can determine the location where the incision will have a smaller impact on corneal astigmatism based on measurement data 203 (corneal shape data, corneal aberration data, etc.). Typically, the steep meridian position of the cornea is incised to correct the corneal astigmatism that occurred before the incision caused by the incision (steep meridian incision). In this case, the measurement data 203 may also include data indicating the steep meridian position of the cornea of ​​the examined eye (data indicating the angular direction of the steep meridian). More commonly, the measurement data 203 may also include data indicating an approximate target position for determining the specific target position in the examined eye.

[0115] For example, the target position determination unit 212A determines the position corresponding to the steep meridian direction in the corneal helix of the examined eye (or a closed curve at a predetermined distance from the corneal helix to the outside, etc.) based on the steep meridian data included in the measurement data 203 or the steep meridian data obtained from the measurement data 203 and the image of the corneal helix detected by the analysis unit 211, and can set this position as the target cutting position of the examined eye.

[0116] More commonly, the target location determination unit 212A determines the position corresponding to the approximate target location in the predetermined part of the eye being examined (or the part that meets the predetermined conditions for the predetermined part) based on the approximate target location included in the measurement data 203 or the approximate target location calculated based on the measurement data 203 and the image of the predetermined part detected by the analysis unit 211, and can set the position as the target applicable position for the predetermined treatment of the eye being examined.

[0117] The number of target locations can be arbitrary. In cases where the intended procedure is an ophthalmotomy, for example, the location of the main incision (main wound, main incision site) and one or more lateral incision sites (lateral wounds, lateral incision sites) can be determined. The number and location of target locations can be set based on various conditions such as the type of intended procedure, the type and severity of the disease, the type of instruments used, and the doctor's preferences.

[0118] In the second example of target location determination, measurements of the examined eye are performed using ophthalmic observation device 1 (or a system including this device). That is, in the second example, the predictive measurement required in the first example is not necessary. Therefore, the second example can be applied even without prepared measurement data.

[0119] Furthermore, the second example can arbitrarily refer to the measurement data obtained through the predicted quantity. For example, several methods can be configured to derive a third measurement data based on both the first measurement data obtained through the predicted quantity and the second measurement data obtained from the ophthalmic observation device 1, and then refer to this third measurement data for target location determination processing. The process of deriving the third measurement data based on the first and second measurement data can include any statistical processing. Alternatively, several methods can be configured to select one of the first measurement data obtained through the predicted quantity and the second measurement data obtained from the ophthalmic observation device 1, and refer to the selected measurement data for target location determination processing.

[0120] The measurement data obtained from the examined eye by the ophthalmic observation device 1 is stored in the storage unit 202 of the ophthalmic observation device 1 or in a storage device accessible by the ophthalmic observation device 1. In the second example, it can be applied... Figure 6 The structure shown is an example. The data processing unit 210B is... Figure 4 The data processing unit 210 is an example, and the target position determination unit 212B is an example of the target position determination unit 212.

[0121] Figure 6 The ophthalmic observation device 1 shown also includes a measurement unit 220. The measurement unit 220 has arbitrary ophthalmic measurement functions for acquiring measurement data of the examined eye. Similar to the ophthalmic measurement device for predictive measurements also applicable to the first example, in the case where the predetermined treatment is ophthalmotomy, the measurement unit 220 may include any one of a wavefront sensor, a corneal topography instrument, and an anterior ocular OCT device.

[0122] Figure 6 The measurement data 203 shown is an example of data obtained by the measurement unit 220 from the eye being examined. For example, it may include any one of the aberration data (corneal aberration data) of the eye being examined obtained using a wavefront sensor, the corneal shape data of the eye being examined obtained using a corneal topography instrument, and the corneal shape data of the eye being examined obtained using an anterior ocular OCT device.

[0123] The target location determination unit 212B in the second example is configured to determine the target location for predetermined treatment based at least on the image of the predetermined part of the examined eye detected by the analysis unit 211 and the measurement data 203 obtained by the measurement unit 220.

[0124] Similar to the target location determination unit 212A in the first example, when the predetermined treatment is ocular incision, the target location determination unit 212B can determine the location that minimizes corneal astigmatism caused by the incision. The attributes of the target location and the content of the target location determination process in the second example can be the same as in the first example.

[0125] As with cases involving corneal astigmatism obtained through ocular incision, when determining the target location by considering the impact of the planned treatment on the examined eye, the following approach may be applicable. Figure 7 The structure shown is an example. The data processing unit 210C is... Figure 6 The data processing unit 210B is an example, and the target position determination unit 212C is an example of the target position determination unit 212B.

[0126] The target location determination unit 212C includes an evaluation unit 213. The evaluation unit 213 is configured to evaluate the impact of a predetermined treatment on the examined eye based on measurement data acquired by the measurement unit 220. For example, the evaluation unit 213 evaluates the impact of ocular incisions (corneal incision, corneoscleral incision, etc.) on corneal astigmatism in the examined eye. In several ways, the evaluation process may include, for example, determining the steep meridian direction based on corneal shape data and corneal aberration data, determining the refractive power along the steep meridian direction, determining the flat meridian direction, determining the refractive power along the flat meridian direction, and determining the distribution of refractive power, etc.

[0127] Furthermore, the evaluation process using several methods can simulate the corneal state after eye incision based on the corneal state before eye incision (corneal shape data, corneal aberration data, steep meridian direction, refractive power in the steep meridian direction, flat meridian direction, refractive power in the flat meridian direction, distribution of refractive power, etc.). The simulation of several examples can be rule-based processing based on a predetermined simulation program. Alternatively, the simulation of several examples can be performed using a machine learning system (neural network, inference model) created through supervised learning based on training data, where the training data includes the corneal state before eye incision, attributes of eye incision (incision location, incision shape, incision size, etc.), and the corneal state after eye incision. The simulations that can be performed by the evaluation unit 213 are not limited to this; for example, simulations using other methods such as unsupervised learning can be included, as well as simulations that at least locally combine two or more methods.

[0128] Figure 7 The target position determination unit 212C is configured to determine the target position for the predetermined treatment of the examined eye based at least on the image of the predetermined region detected by the analysis unit 211 and the result obtained by the evaluation unit 213. For example, as the target position for the predetermined treatment of the examined eye, the target position determination unit 212C can determine the position that is presumed to be the position where corneal astigmatism generated by eyeball incision is minimized.

[0129] In the third case of target location determination, the target location for the examined eye is determined without referring to the measurement data of the examined eye as in the first and second cases. This third case can be applied... Figure 4 (or Figures 5 to 7The structural example shown is from any one of the examples. In the third example, the target position determination unit 212 is configured to determine the position at which the image of a predetermined part of the eye under examination, detected by the analysis unit 211, is separated from the image by a predetermined distance as the target position for a predetermined treatment of the eye under examination. The predetermined distance is defined, for example, as an actual distance (millimeters, micrometers, etc.) or an image distance (number of pixels, etc.).

[0130] For example, in anterior capsulotomy (CCC) during cataract surgery, the analysis unit 211 can determine the characteristics of the anterior portion of the examined eye (image of the corneal rim, its size, image of the pupillary margin, its size, etc.). The target position determination unit 212 can determine a pattern of predetermined shape and size based on the characteristics of the anterior portion determined by the analysis unit 211. This pattern is, for example, a circle with a diameter of 6 mm. This circle can be, for example, a circle with a radius of 3 mm based on the center of an approximate circle (approximate ellipse) of the corneal rim, a circle with a radius of 3 mm based on the center of an approximate circle (approximate ellipse) of the pupillary margin, an approximate circle of the corneal rim enlarged (reduced) to an isotropic circle with a diameter of 6 mm, or an approximate circle of the corneal limbus enlarged (reduced) to an isotropic circle with a diameter of 6 mm. Furthermore, the shape and size of the pattern can be arbitrarily changed by the user.

[0131] Furthermore, during retinal vitrectomy, the analysis unit 211 can detect the image of the corneal helix of the examined eye. The target position determination unit 212 can set the target position for inserting surgical instruments to a position spaced outward from the detected image of the corneal helix by a predetermined distance (e.g., 3-4 mm). In addition, the shape and size of the graphic can be arbitrarily changed by the user.

[0132] This section explains an example of the target location determination process (including distance calculation process) applicable to the third example (or other examples). Hereinafter, we will explain the case where the predetermined location of the examined eye detected by the analysis unit 211 is the corneal rim (outer edge of the iris), but the same process can be performed even when other locations (e.g., the pupillary rim) are used as the target.

[0133] As a preparatory step, an anterior segment of the eye being examined is photographed before surgery. The distance between the eye being examined and the imaging device (running distance WD) is recorded during the photographing process. The corneal concentric circle is detected from the image obtained through the photograph, and the size (e.g., diameter) of the detected corneal concentric circle is measured and recorded. Furthermore, the method for determining the size of the corneal concentric circle beforehand can be arbitrary. For example, the size can be measured based on an anterior segment OCT image (e.g., a cross-sectional image or a three-dimensional image), or the size of the corneal concentric circle can be measured directly without using an image.

[0134] In the above preparation, during the operation, the ophthalmic observation device 1 is used to photograph the eye being examined and obtain observation images (surgical microscope 10), the running distance during the photographing is recorded (control unit 200), the corneal helix is ​​detected from the observation images obtained by the photographing (analysis unit 211), the size of the detected corneal helix is ​​measured (data processing unit 210), and the measured value of the obtained size is recorded (control unit 200).

[0135] Furthermore, the target location determination unit 212 calculates the relationship between the scale of the preoperative image (e.g., the actual distance per pixel) and the scale of the intraoperative observation image (e.g., the scale ratio of the two images) based on the relationship between the preoperative image's travel distance and the intraoperative image's travel distance (e.g., the ratio of the two image scales). The target location determination unit 212 then determines a location separated from the corneal helix detected in the intraoperative image by a predetermined distance (e.g., 3-4 mm) by the calculated relationship between the preoperative image scale and the intraoperative image scale. In this example, the control unit 200 (main control unit 201) causes the display device 3 to display the observation image (live image) of the examined eye, and can display target location information indicating the determined location (target location) on the observation image. The target location information can be displayed in any manner, such as a circular, elliptical, or arc-shaped marker.

[0136] Several variations are illustrated below. During preoperative imaging, the image can be taken with the ruler aligned with the eye being examined (anterior eye), and the actual distance can be determined based on the ruler projected onto the resulting image. Similarly, during surgical imaging, the image can be taken with the ruler aligned with the eye being examined (anterior eye), and the actual distance can be determined based on the ruler projected onto the resulting image. The distance from the corneal helix to the target location is 3–4 mm in the examples described, but it can be arbitrary. For example, the distance from the corneal helix to the target location can be set according to the surgeon's preference, corneal shape (e.g., corneal curvature or corneal curvature distribution), etc.

[0137] In application Figures 4 to 7 In the case of any of the structural examples or when other structural examples are applied, the control unit 200 receives a real-time image of the eye being examined from the surgical microscope 10 and displays it on the display device 3, while also displaying target position information on the display device 3 indicating the target position determined by the analysis unit 211 and the target position determination unit 212.

[0138] According to the ophthalmic observation device 1 with such a structure, the target location of the predetermined treatment applicable to the examined eye can be provided together with the live image, so that the user can grasp the location where the predetermined treatment is to be applied in real time. As a result, it is possible to make surgery (or treatment, etc.) easier, shorten the time of surgery (or treatment, etc.), reduce and prevent treatment errors, and reduce the burden on patients.

[0139] The ophthalmic observation device 1 can update the target position information displayed in real time along with the live images generated by the surgical microscope 10. Therefore, the analysis unit 211, in parallel with the dynamic images generated by the surgical microscope 10, sequentially detects images of predetermined areas of the examined eye based on frames (still images) sequentially generated by the surgical microscope 10. In parallel with dynamic image generation and image detection, the target position determination unit 212 sequentially determines the target position for the predetermined treatment based at least on the images of the predetermined areas sequentially detected by the analysis unit 211. In parallel with dynamic image generation, image detection, and target position determination, the control unit 200 performs the display of the live images (sequential frame updates) and the updating of target position information based on the target positions sequentially determined by the target position determination unit 212 (sequential switching). According to this structure, the user can monitor changes in the target position for the predetermined treatment in real time while observing the live images of the examined eye. Furthermore, changes in the target position are caused by activities of the examined eye, etc.

[0140] When two or more procedures are performed sequentially or in parallel, the ophthalmic observation device 1 can perform different procedures for each procedure. For example, in cataract surgery, it can perform procedures such as wound creation (corneal incision, corneoscleral incision), ophthalmic viscoelastic injection, anterior capsule incision (CCC), lens emulsification aspiration, IOL insertion, ophthalmic viscoelastic removal, and wound closure.

[0141] Typically, when the first to Nth treatments are applied to the eye being examined, the analysis unit 211 analyzes the nth still image generated by the surgical microscope 10 before the nth treatment and is able to detect the image of the nth part of the eye being examined.

[0142] Here, N is an integer greater than or equal to 2, and n is any integer greater than or equal to 1 and less than N. For any two integers n1 and n2 (n1 ≠ n2) greater than or equal to 1 and less than N, the n1th disposal and the n2th disposal can be the same disposal or different disposal, and the n1th part and the n2th part can be the same part or different parts.

[0143] Furthermore, the target position determination unit 212 can determine the nth target position as the target position for the nth processing, based at least on the image of the nth part detected from the nth still image.

[0144] In addition, the control unit 200 can display the live image acquired by the surgical microscope 10 and the nth target position information indicating the nth target position, so as to perform the nth treatment (nth display control).

[0145] When performing two or more procedures sequentially, the ophthalmic observation device 1 can also be configured to automatically detect the transfer of procedures. For example, the data processing unit 210 can be configured to detect the transfer from the first procedure to the second procedure (procedure transfer detection unit). Alternatively, the ophthalmic observation device 1 can be configured to detect the transfer of procedures based on user instructions (operation input, voice input, etc.).

[0146] In cataract surgery, procedures such as incision, injection of ophthalmic viscoelastic agent, anterior capsule incision, lens emulsification and aspiration, IOL insertion, removal of ophthalmic viscoelastic agent, and incision closure are performed sequentially. Most surgeries and treatments follow a predetermined process. Therefore, a specific action mode corresponding to a particular surgery (or treatment) can be pre-programmed with the processing content of each step of the procedure (such as detection, target location determination, and target location information display).

[0147] By referring to such preset information, the ophthalmic observation device 1 can automatically detect the situation of being transferred to a new process (new treatment) and can execute the action mode that is pre-associated with the new process.

[0148] Automatic detection of transfer can be achieved, for example, by analyzing live images and detecting the state of specific parts of the eye being examined, and / or by analyzing live images and detecting images of instruments or intraocular retention devices.

[0149] For example, automated detection of transfer to the anterior capsulotomy (CCC) during cataract surgery can be achieved by combining the detection of the already formed corneal incision with the detection of the unresolved anterior capsulotomy. Alternatively, automated detection of transfer to the anterior capsulotomy (CCC) can also be achieved by combining the detection of the already formed corneal incision with the detection of the image of the CCC instrument.

[0150] Furthermore, the procedure to be automatically detected is not limited to anterior capsulotomy (CCC), but can be any procedure within cataract surgery (other procedures), any procedure within any other surgery, or any procedure within any medical act (treatment, diagnosis, etc.). The specific method for automatically detecting procedures other than anterior capsulotomy (CCC) can be performed using the same approach as described with respect to anterior capsulotomy (CCC).

[0151] <Actions and Usage>

[0152] Explain the operation and usage of ophthalmic observation device 1. Figure 8A and 8B An example of the operation and use of the ophthalmic observation device 1 is shown. This example illustrates its application to the corneal incision and anterior capsule incision (CCC) stages of cataract surgery. However, the essentially same operation and use can be performed even in other stages of cataract surgery or in other medical procedures (surgery, treatment, diagnosis, etc.).

[0153] (S1: Start generating and displaying live images)

[0154] First, the user performs a pre-defined operation using the operating device 2, thereby initiating the generation and display of a live image of the examined eye (anterior eye) by the ophthalmic observation device 1. Specifically, the surgical microscope 10 illuminates the examined eye via the illumination optics system 30 while generating digital image data (image) of the examined eye using the imaging elements 62L and 62R. The generated image (live image 301) is then displayed in real time on the display device 3 (see reference). Figure 9A In other words, the dynamic image acquired by the surgical microscope 10 is displayed as a live image (observation image) on the display device 3. The user can perform surgery while observing this live image.

[0155] (S2: Select cataract surgery mode)

[0156] The ophthalmic observation device 1, for example, corresponds to the user's instructions, causing the display device 3 to display a screen for selecting an operation mode. The operation mode options may include, for example, cataract surgery mode, vitrectomy mode, etc. The user uses the operating device 2 to select a predetermined operation mode. In this example, the cataract surgery mode is selected.

[0157] (S3: Transfer to the corneal incision stage)

[0158] Cataract surgery was initiated, and the treatment phase shifted to corneal incision.

[0159] (S4: Detect the image of the predetermined region based on the live image)

[0160] Next, the analysis unit 211 detects an image of a predetermined region of the eye being examined based on the live image (frames) generated at the beginning of step S1. The region to be detected for corneal incision may be, for example, the pupillary rim or the corneal rim.

[0161] In several methods, for detecting images of predetermined regions, the analysis unit 211 first applies binarization, edge detection, and segmentation to frames of the real-world image 301. Figure 9B In the example shown, the image 302 of the pupil edge is detected based on the live image 301.

[0162] (S5: Determine the target location for corneal incision)

[0163] Next, the target location determination unit 212 determines the corneal incision target location in the live image based at least on the image of the predetermined region (pupil margin) detected in step S4. The target location determination unit 212 performs, for example, any of the aforementioned processing examples.

[0164] exist Figure 9C In the example shown, the target position determination unit 212 first determines a portion 303 located at a predetermined distance outward from the image 302 of the pupillary margin detected in step S4. Next, the target position determination unit 212 calculates the steep meridian 304 of the cornea of ​​the examined eye (see reference 203) based on previously acquired measurement data 203 (corneal shape data, corneal aberration data, etc.). Figure 9D The target location determination unit 212 determines the corneal incision target location as the position where a portion 303, separated by a predetermined distance from the image 302 at the pupillary margin, intersects with the steep meridian 304. Figure 9D In the example shown, two corneal incision target locations, 305a and 305b, are set.

[0165] (S6: Display the target location information for corneal incision on the live image)

[0166] Next, the control unit 200 displays target location information, which indicates the corneal incision target location determined in step S5, on the live image of the examined eye.

[0167] exist Figure 9E In the example shown, the live image 301 displays the respective representations of... Figure 9D The example shown includes two target location information 306a and 306b for the two determined corneal incision target locations 305a and 305b. Target location information 306a and 306b represent the areas to be incised during the corneal incision, and are respectively arc-shaped marker images. The method of the arc-shaped marker images (length, radius of curvature, thickness, etc.) can be preset, and can also be adjusted arbitrarily by the user. This allows the user to easily determine the location of the cornea to be incised. Furthermore, in this example, it is easier to achieve a steep meridian incision aimed at reducing postoperative corneal astigmatism.

[0168] (S7: Transfer to anterior capsule incision?)

[0169] Steps S4 to S6 are repeated at least until the corneal incision is completed. Typically, steps S4 to S6 are repeated until ophthalmic viscoelastic injection begins or until anterior capsule incision (CCC) begins. Steps S4 to S6 are applied sequentially to frames acquired as live image 301. For example, steps S4 to S6 can be applied to all frames acquired as live image 301 (time-series image) or to frames selected from all frames acquired as live image 301. Frame selection can be, for example, a rejection process performed at predetermined intervals. Through such repeated processing, even if the corneal incision target position changes due to movement of the examined eye, the user can easily and in real-time monitor the location of the cornea to be incised.

[0170] The corneal incision target location information (e.g., an arc-shaped marker image) can be displayed at any stage after the corneal incision. For example, the corneal incision target location information can be displayed in response to the user's actions. Therefore, the user can confirm the target location during the corneal incision at the desired time. For example, because the side incision port is small, it may be missed during the procedure; however, in this example, the position of the side incision port can be confirmed at any time. Furthermore, when the area indicated by the corneal incision target location information is actually incised, the control unit 200 can store the coordinate information of that area in the storage unit 202, etc. Similarly, when a different area than the area indicated by the corneal incision target location information is actually incised, the control unit 200 can store the coordinate information of the actually incised area in the storage unit 202, etc. In this case, the coordinate information of the area indicated by the corneal incision target location information can also be stored in the storage unit 202, etc.

[0171] (S8: Detect the image of the predetermined region from the live image)

[0172] When the procedure proceeds to the anterior capsulotomy (CCC) stage, the analysis unit 211 detects an image of a predetermined region of the eye being examined based on the live image (frames) generated in step S1. The region targeted for detection during anterior capsulotomy may be, for example, the pupillary rim or the corneal rim.

[0173] In several methods, for detecting images of predetermined regions, the analysis unit 211 first applies binarization, edge detection, and segmentation to frames of the real-world image 301. Figure 10A In the example shown, similar to the corneal incision stage, the image 311 of the pupillary margin is detected based on the live image 301.

[0174] (S9: Determines the target location for anterior capsule incision)

[0175] Next, the target position determination unit 212 determines the target position for anterior capsule incision in the live image, at least based on the image of the predetermined region (pupil margin) detected in step S8. The target position determination unit 212 performs, for example, any of the aforementioned processing examples.

[0176] exist Figure 10B In the example shown, the target location determination unit 212 determines a location separated from the image 311 of the pupillary margin detected in step S8 by a predetermined distance inward, and sets the determined location as the anterior capsule incision target location 312.

[0177] (S10: Display the target location information for anterior capsule incision on the live image)

[0178] Next, the control unit 200 displays target location information, indicating the corneal incision target location determined in step S9, on the live image of the examined eye. Figure 10C In the example shown, the live image 301 displays the representation of... Figure 10B The example shown includes target location information 313 for the determined anterior capsule incision target location 312. Target location information 313 is a circular marker image (or possibly an arc-shaped marker image), along which the anterior capsule is incised. The circular marker image can be preset and can also be adjusted arbitrarily by the user. This allows the user to easily determine the location of the anterior capsule to be incised.

[0179] (S11: Transfer to the next step?)

[0180] Steps S8 to S10 are repeated at least until the anterior capsule incision is completed. Typically, steps S8 to S10 are repeated until lens emulsification aspiration begins. Steps S8 to S10 are applied sequentially to frames acquired as live image 301. For example, steps S8 to S10 can be applied to all frames acquired as live image 301 (time-series image) or to frames selected from all frames acquired as live image 301. Frame selection can be, for example, a rejection process performed at predetermined intervals. Through such repeated processing, even if the corneal incision target position changes due to movement of the examined eye, the user can easily and in real-time monitor the location of the cornea to be incised.

[0181] Even at any stage after anterior capsule incision, the target location information of the anterior capsule incision (e.g., a circular marker image) can be displayed. For example, the target location information of the anterior capsule incision can be displayed in response to user operations. Thus, the user can confirm the target location during the anterior capsule incision at the desired time. If the area indicated by the target location information of the anterior capsule incision is actually incised, the control unit 200 can store the coordinate information of that area in the storage unit 202, etc. Similarly, if a different area than the area indicated by the target location information of the anterior capsule incision is actually incised, the control unit 200 can store the coordinate information of the actually incised area in the storage unit 202, etc. In this case, the coordinate information of the area indicated by the target location information of the anterior capsule incision can also be stored in the storage unit 202, etc.

[0182] (S12: Take other measures)

[0183] If the anterior capsule incision is completed, the patient will then undergo subsequent procedures (lens emulsification and aspiration, IOL insertion, removal of ophthalmic viscoelastic agent, wound closure, etc.). This completes the cataract surgery (end).

[0184] To briefly explain another method. In retinal vitrectomy, for example, the analysis unit 211 applies image processing (binarization, etc.) to the observed image and detects the image of the corneal helix. The target position determination unit 212 determines the target position as a part separated from the image of the corneal helix by a predetermined distance (e.g., 3 to 4 millimeters) from the outside. The control unit 200 displays (e.g., a circular mark) at the target position.

[0185] Figure 11 An example of information displayed for performing vitrectomy is shown. In this example, a circular marker image 403, representing a portion spaced outward from the image 402 of the corneal helix at a predetermined distance (e.g., 3-4 mm), is displayed along with a live image 401 acquired by a surgical microscope 10. The user can perform a scleral incision (conjunctival incision) and create a port during vitrectomy while referring to the marker image 403. The port provides a path for inserting light guides (illumination) and various instruments into the eye. Figure 11 The device provides three ports 404a, 404b, and 404c formed on the marker image 403. The ophthalmic observation device 1 is able to record the coordinate information of each formed port, and can, for example, display the marker image indicating the position of the port on the live image in response to a user's request.

[0186] As described above, ophthalmic surgery is typically performed by inserting instruments into the eye. In this disclosure, for example, the optimal treatment sites (e.g., instrument insertion sites, incision sites, perforation sites, etc.) can be displayed on a live image. For example, by displaying markers (e.g., images) indicating the target treatment location during the surgical procedure, marking the target treatment location within the user's field of vision, the surgery can be successfully completed.

[0187] Furthermore, in this disclosure, by sequentially processing multiple frames acquired as live images, it is possible to track the location of the target being processed. Additionally, after the actual processing is performed, the processed area can be displayed on the live image.

[0188] This disclosure describes its application in cataract surgery (during corneal incision, anterior capsule incision, etc.) and retinal vitreous surgery; however, those skilled in the art will recognize that the application of this disclosure is not limited thereto.

[0189] Furthermore, in several methods, during corneal incision in cataract surgery, a pattern (e.g., an arc-shaped line) can be displayed as the incision target based on features of the anterior eye (corneal rim, pupil, their dimensions, etc.) and measurement data of the examined eye. The size and shape of this pattern are variable. Additionally, before or during surgery, corneal data can be acquired using wavefront sensors, corneal topography, and anterior eye OCT to identify areas with minimal impact on corneal astigmatism. Based on the live image, characteristic locations (e.g., pupillary margin, pupillary center) can be determined, thereby enabling marking for corneal incision.

[0190] In several methods, during anterior capsule incision in cataract surgery, a pattern (e.g., a circle with a diameter of 6 mm) determined based on the characteristics of the anterior eye (corneal orbicularis oculi, pupil, their sizes, etc.) can be displayed as the incision target. The size and shape of this pattern are variable. Furthermore, before or during surgery, predetermined data (e.g., corneal orbicularis oculi diameter, pupil diameter) is acquired to determine the size of the displayed marker image (pattern). Feature areas (e.g., corneal orbicularis oculi, pupillary margin) are detected from the live image, and a marker image of the determined size is displayed, thereby enabling marking for anterior capsule incision.

[0191] In several methods, during retinal vitrectomy, a graphic (e.g., a circular image) serving as an instrument insertion target (perforation target, port formation target) can be displayed at a predetermined distance (e.g., 3–4 mm) lateral to the corneal rim. The size and shape of this graphic are variable.

[0192] Even in surgeries, treatments, examinations, etc., other than these, target location information indicating the location where the predetermined treatment should be applied can be displayed along with live images.

[0193] As a result, procedures such as surgery, treatment, and examination can be completed more easily, in less time, with fewer errors and less burden on patients.

[0194] <Methods for controlling ophthalmic observation devices>

[0195] An exemplary embodiment (such as the ophthalmic observation device 1 described above) provides a method for controlling an ophthalmic observation device. Any aspect related to the ophthalmic observation device 1 of the described embodiment can be combined with the exemplary method described below.

[0196] An ophthalmic observation device controlled by an illustrative method includes a processor (e.g., a control unit 200 and a data processing unit 210) that generates a dynamic image of the eye being examined (surgical microscope 10). The illustrative method first analyzes still images included in the dynamic image using the processor and detects an image of a predetermined region of the eye being examined. Furthermore, the illustrative method determines a target location for a predetermined treatment based at least on the image of the predetermined region detected from the still image. Additionally, the illustrative method causes a display device to display the dynamic image and displays target location information indicating the determined target location.

[0197] The method described in this exemplary manner can achieve the same function and effect as the ophthalmic observation device 1 of the embodiment. Furthermore, by combining the aspects related to the ophthalmic observation device 1 of the embodiment with the method of the exemplary manner, the resulting method can achieve the function and effect corresponding to the combined aspects.

[0198] <program>

[0199] An exemplary embodiment provides a program for causing a computer to execute the method of the exemplary mode. Matters related to the ophthalmic observation device 1 of the described embodiment can be combined with such a program.

[0200] According to such a procedure, the same function and effect as the ophthalmic observation device 1 of the described embodiment can be achieved. Furthermore, by combining matters related to the ophthalmic observation device 1 of the described embodiment with the procedure, the resulting procedure can achieve the function and effect corresponding to the combined matters.

[0201] <Recording Medium>

[0202] An exemplary embodiment provides a computer-readable, non-transitory recording medium on which the program described herein is recorded. Matters relating to the ophthalmic observation device 1 of the embodiment can be combined with such a recording medium. The non-transitory recording medium can be of any type, including, for example, a magnetic disk, an optical disk, a optical disc, a semiconductor memory, etc.

[0203] Based on such a recording medium, it can perform the same function and effect as the ophthalmic observation device 1 of the described embodiment. Furthermore, by combining matters related to the ophthalmic observation device 1 of the described embodiment with the recording medium, the resulting recording medium can perform the function and effect corresponding to the combined matters.

[0204] This disclosure only illustrates the embodiments, and any modifications, omissions, additions, substitutions, etc., can be implemented within the scope of this disclosure and its equivalents.

[0205] (Explanation of reference numerals in the attached diagram)

[0206] 1: Ophthalmic observation device; 2: Operating device; 3: Display device; 10: Surgical microscope; 30: Illumination optical system; 40: Observation optical system; 200: Control unit; 203: Measurement data; 210, 210A, 210B, 210C: Data processing unit; 211: Analysis unit; 212, 212A, 212B, 212C: Target position determination unit; 213: Evaluation unit; 220: Measurement unit.

Claims

1. An ophthalmic observation device for observing an examined eye, wherein, The ophthalmic observation device includes: The dynamic image generation unit captures the examined eye and generates a dynamic image; The analysis unit analyzes the still images included in the dynamic image to detect the image of a predetermined part of the eye being examined; The target location determination unit determines the target location for predetermined processing based at least on the image of the predetermined region detected by the analysis unit; and The display control unit causes the display device to display the dynamic image and target location information indicating the target location, and also causes the target location information to be displayed on the dynamic image. The target location is the corneal incision target location.

2. The ophthalmic observation device according to claim 1, wherein, The target location determination unit determines the target location based at least on the image of the predetermined region detected by the analysis unit and the measurement data of the examined eye that has been acquired in advance.

3. The ophthalmic observation device according to claim 1, wherein, The ophthalmic observation device also includes: The measurement unit acquires measurement data of the eye being examined. The target location determination unit determines the target location based at least on the image of the predetermined region detected by the analysis unit and the measurement data obtained by the measurement unit.

4. The ophthalmic observation device according to claim 3, wherein, The target location determination unit includes an evaluation unit that evaluates the impact of the predetermined treatment on the examined eye based on the measurement data obtained by the measurement unit. The target location determination unit determines the target location based at least on the image of the predetermined region detected by the analysis unit and the result obtained by the evaluation unit.

5. The ophthalmic observation device according to any one of claims 1 to 4, wherein, The target position determination unit determines the target position as the position at a predetermined distance from the image of the predetermined part detected by the analysis unit.

6. The ophthalmic observation device according to any one of claims 1 to 4, wherein, When the first and second treatments are applied to the examined eye, The analysis unit performs a first detection and a second detection. The first detection analyzes a first still image generated by the dynamic image generation unit before the first processing to detect an image of a first portion of the examined eye. The second detection analyzes a second still image generated by the dynamic image generation unit before the second processing to detect an image of a second portion of the examined eye. The target location determination unit performs a first decision and a second decision. The first decision determines a first target location as the target location for the first action based at least on the image of the first region, and the second decision determines a second target location as the target location for the second action based at least on the image of the second region. The display control unit performs first display control and second display control. The first display control causes the display of the dynamic image and first target position information indicating the first target position in order to perform the first action. The second display control causes the display of the dynamic image and second target position information indicating the second target position in order to perform the second action.

7. The ophthalmic observation device according to claim 6, wherein, The ophthalmic observation device also includes: The disposal transfer detection unit is used to detect the transfer from the first disposal to the second disposal.

8. A method for controlling an ophthalmic observation device, the ophthalmic observation device including a processor, and generating a dynamic image of the eye being examined, wherein, The processor performs the following steps: The image of a predetermined region of the examined eye is detected by analyzing the still images included in the dynamic image. Based at least on the image of the detected predetermined region, the target location for the predetermined treatment is determined. The display device displays the dynamic image and target location information indicating the target location, and the target location information is displayed on the dynamic image. The target location is the corneal incision target location.

9. A computer-readable, non-transitory recording medium containing a program that causes a computer to execute the method of claim 8.

Images