Ophthalmic observation apparatus, ophthalmic image processing apparatus, ophthalmic image processing method, program, and recording medium

Abstract

An ophthalmic observation apparatus (1) of an exemplary embodiment is used for observing an eye to be examined. The ophthalmic observation apparatus (1) includes a dynamic image generation section (surgical microscope 10), an image processing section (data processing section 210, image processing section 211), and a display control section (main control section 201). The dynamic image generation section photographs the eye to be examined to generate a first dynamic image. The image processing section applies first image processing using a plurality of different values of a predetermined image parameter to still images included in the first dynamic image generated by the dynamic image generation section, respectively, to produce a plurality of processed images. The display control section causes a first display device (display device 3) to display the plurality of processed images produced by the image processing section.

Description

Technical Field

[0001] This disclosure relates to ophthalmic observation devices, ophthalmic image processing devices, ophthalmic image processing methods, programs, and recording media. Background Technology

[0002] An ophthalmic observation device is a device used to observe a patient's eye (the eye being examined). It is used in various situations such as examinations, surgeries, and treatments to monitor the condition of the eye being examined.

[0003] Conventional ophthalmic observation devices provide users with a magnified image obtained through an objective lens and a zoom optical system via an eyepiece lens. However, in recent years, ophthalmic observation devices with the following structure have emerged: they capture a magnified image obtained through an objective lens and a zoom optical system using an imaging element, and display the captured image (a first-type ophthalmic observation device). Such ophthalmic observation devices include slit-lamp microscopes, surgical microscopes, fundus cameras, etc. Furthermore, various ophthalmic examination devices, such as refractometers, corneal astigmatometers, tonometers, corneal endothelial microscopes, wavefront aberrometers, and microperimeters, also incorporate the function of an ophthalmic observation device as a first-type device.

[0004] Furthermore, in recent years, ophthalmic observation devices have also utilized optical scanning (a second type of ophthalmic observation device). Such ophthalmic devices include scanning laser ophthalmoscopy (SLO) and optical coherence tomography (OCT) devices.

[0005] Typically, ophthalmic observation devices provide dynamic images of the examined eye to a user (e.g., a doctor or other healthcare professional). Typically, a first-type ophthalmic observation device is configured to perform dynamic image capture using infrared and / or visible light as illumination and real-time dynamic image display of the resulting image. On the other hand, a second-type ophthalmic observation device is typically configured to perform data collection based on repeated light scans, real-time image reconstruction based on the successively collected dataset, and real-time dynamic image display of the successively reconstructed image. The real-time dynamic images provided in this way are referred to as observation images.

[0006] To provide good viewing images, image quality adjustments are necessary. The desired image quality varies from user to user, and also depends on the examination, type of surgery, and stage. For example, some doctors prefer a reddish image, while others prefer a greenish one. Furthermore, cataract surgery, one of the most common ophthalmic surgeries, involves alignment, incision, ophthalmic viscoelastic injection, anterior capsulotomy (CCC), lens emulsification and drainage, lens cortical drainage, intraocular lens (IOL) insertion, IOL centering, ophthalmic viscoelastic removal, and incision closure. However, the desired image quality sometimes varies depending on the stage. Additionally, sometimes the image quality of the area of ​​interest needs to be selectively improved. Moreover, the desired image quality can also differ depending on the condition of the eye being examined.

[0007] Thus, the desired image quality varies greatly. However, in traditional ophthalmic observation devices, image quality adjustments were manually performed each time, which was tedious and inconvenient, and a major cause of prolonged examination and surgical times. On the other hand, while automatic image quality adjustments have been considered, manual adjustments ultimately remain necessary given the diversity of desired image quality.

[0008] Patent Document 1: Japanese Patent Application Publication No. 2003-310556

[0009] Patent Document 2: Japanese Patent Application Publication No. 2009-118955 Summary of the Invention

[0010] One of the objectives of this invention is to provide a new technique for facilitating ophthalmic observation.

[0011] Several exemplary embodiments are ophthalmic observation devices for observing an eye under examination, comprising: a dynamic image generation unit that captures the eye under examination to generate a first dynamic image; an image processing unit that applies first image processing, using multiple different values ​​of predetermined image parameters, to each still image included in the first dynamic image to create multiple processed images; and a display control unit that causes a first display device to display the multiple processed images.

[0012] Several exemplary ophthalmic observation devices further include an instruction receiving unit that receives an instruction for selecting at least one processed image from the plurality of processed images displayed on the first display device. The image processing unit applies second image processing based on at least one value of the image parameters corresponding to the at least one processed image to a second dynamic image generated by the dynamic image generation unit after the selection of the at least one processed image using the instruction receiving unit. The display control unit causes the second display device to display the second dynamic image to which the second image processing has been applied.

[0013] In several exemplary ophthalmic observation devices, when the instruction receiving unit selects one of the plurality of processed images, the image processing unit applies image processing using a value of the image parameter corresponding to the one processed image as the second image processing to the second dynamic image.

[0014] In several exemplary ophthalmic observation devices, when two or more processed images are selected from the plurality of processed images using the instruction receiving unit, the image processing unit applies image processing using a value of the image parameter corresponding to one of the two or more processed images as the second image processing to the second dynamic image.

[0015] In several exemplary ophthalmic observation devices, when two or more processed images are selected from the plurality of processed images using the instruction receiving unit, the image processing unit determines a value based on two or more values ​​of the image parameters corresponding to the two or more processed images respectively, and applies the image processing using the one value as the second image processing to the second dynamic image.

[0016] In several exemplary ophthalmic observation devices, a recording unit is also included for recording the value of the image parameter used in the second image processing.

[0017] In several exemplary ophthalmic observation devices, an identifier receiving unit is also included that receives an identifier of a user, and the recording unit records the value of the image parameter in association with the identifier received by the identifier receiving unit.

[0018] In several exemplary ophthalmic observation devices, an attribute information acquisition unit is further included for acquiring attribute information representing the attributes of the examined eye to a medical procedure, and the recording unit records the value of the image parameter in association with the attribute information acquired by the attribute information acquisition unit.

[0019] In several exemplary ophthalmic observation devices, a selection unit is also included to select at least one value from the values ​​of the image parameters previously recorded by the recording unit, and the image processing unit applies image processing based on the at least one value selected by the selection unit to a third dynamic image generated by the dynamic image generation unit.

[0020] In several exemplary ophthalmic observation devices, the recording unit records the shooting conditions when the second dynamic image is generated in association with the value of the image parameter, the selection unit further selects the shooting conditions associated with the at least one value selected by the selection unit, and includes a determination unit that determines the value of the image parameter based on the at least one value selected by the selection unit and the shooting conditions, and the image processing unit applies image processing using the value of the image parameter determined by the determination unit to the third dynamic image.

[0021] In several exemplary ophthalmic observation devices, the image processing unit applies the first image processing to a portion of the still image included in the first dynamic image, i.e., a local image, to create a plurality of locally processed images as the plurality of processed images, and the display control unit causes the first display device to display a plurality of images, each including the plurality of locally processed images.

[0022] In several exemplary ophthalmic observation devices, the image processing unit includes a first local image determination unit that applies the segmentation of an image used to determine a predetermined region of the examined eye to the still image included in the first dynamic image to determine the local image.

[0023] In several exemplary ophthalmic observation devices, the first local image determination unit applies the segmentation to the second dynamic image and sequentially determines local images of still images included in the second dynamic image, and the image processing unit sequentially applies the second image processing to the local images determined from the still images included in the second dynamic image.

[0024] In several exemplary ophthalmic observation devices, the display control unit causes the first display device or the second display device to display the first dynamic image or a still image included in the first dynamic image, and further includes a graphical user interface for specifying a local area in the displayed first dynamic image or the still image included in the first dynamic image, and the image processing unit sets the local image according to the local area specified using the user interface.

[0025] In several exemplary ophthalmic observation devices, the image processing unit includes a second local image determination unit that sequentially determines local images corresponding to the local region in a still image included in the second dynamic image, and the image processing unit sequentially applies the second image processing to the local images determined from the still image included in the second dynamic image.

[0026] In several exemplary ophthalmic observation devices, the display control unit causes the first display device to display two or more of the plurality of processed images or thumbnails of the two or more processed images.

[0027] In several exemplary ophthalmic observation devices, the display control unit causes the first display device to sequentially display two or more processed images or thumbnails of the plurality of processed images.

[0028] In several exemplary ophthalmic observation devices, a monitoring unit is also included for monitoring the activity of the examined eye, and the display control unit changes the display state of the plurality of processed images according to the output from the monitoring unit.

[0029] In several exemplary ophthalmic observation devices, an abnormality detection unit is also included for detecting abnormalities in the examined eye, and the display control unit changes the display state of the plurality of processed images based on the output from the abnormality detection unit.

[0030] In several exemplary ophthalmic observation devices, the image parameters include one or more of the following: hue parameter, brightness parameter, contrast parameter, gain parameter, gamma parameter, color temperature parameter, white balance parameter, RGB balance parameter, gray balance parameter, edge enhancement parameter, shadow enhancement parameter, sharpening parameter, and high dynamic range parameter.

[0031] Several exemplary embodiments are ophthalmic image processing apparatuses that process images of an eye being examined, including: a dynamic image receiving unit that receives a first dynamic image of the eye being examined; an image processing unit that applies first image processing, using multiple different values ​​of predetermined image parameters, to a still image included in the first dynamic image to create multiple processed images; and a display control unit that causes a first display device to display the multiple processed images.

[0032] Several exemplary ophthalmic image processing apparatuses further include an instruction receiving unit that receives an instruction for selecting at least one processed image from the plurality of processed images displayed on the first display device, the image processing unit applying second image processing based on at least one value of the image parameters corresponding to the at least one processed image to a second dynamic image of the examined eye received by the dynamic image receiving unit after the selection of the at least one processed image based on the instruction, and the display control unit causing the second display device to display the second dynamic image to which the second image processing has been applied.

[0033] Several exemplary embodiments are ophthalmic image processing methods that process an image of an eye under examination, receive a first dynamic image of the eye under examination, apply a first image processing method using multiple different values ​​of predetermined image parameters to still images included in the first dynamic image to create multiple processed images, display the multiple processed images, receive an instruction for selecting at least one processed image from the displayed multiple processed images, after selecting the at least one processed image based on the instruction, receive a second dynamic image of the eye under examination, apply a second image processing method based on at least one value of the image parameters corresponding to the at least one processed image to the second dynamic image, and display the second dynamic image to which the second image processing method has been applied.

[0034] Several exemplary modes are a program that enables a computer to execute exemplary ophthalmic image processing methods.

[0035] Several exemplary modes are computer-readable, non-transitory recording media that record the programs of the exemplary modes.

[0036] Based on the illustrative approach, a new technique for facilitating ophthalmic observation can be provided. Attached Figure Description

[0037] Figure 1 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device (ophthalmic surgical microscope) according to an exemplary embodiment.

[0038] Figure 2 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0039] Figure 3 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0040] Figure 4 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0041] Figure 5 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0042] Figure 6 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0043] Figure 7 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0044] Figure 8 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0045] Figure 9 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0046] Figure 10 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0047] Figure 11 This is a flowchart illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0048] Figure 12A This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0049] Figure 12B This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0050] Figure 12C This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0051] Figure 12D This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0052] Figure 12E This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0053] Figure 13 This is a flowchart illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0054] Figure 14A This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0055] Figure 14B This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0056] Figure 14C This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0057] Figure 14D This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0058] Figure 14E This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0059] Figure 14F This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0060] Figure 14G This is a schematic diagram illustrating an example of the processing performed by an ophthalmic observation device according to an exemplary embodiment.

[0061] Figure 15 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0062] Figure 16 This is a schematic diagram illustrating an example of the structure of an ophthalmic observation device according to an exemplary embodiment.

[0063] Figure 17 This is a schematic diagram illustrating an example of the structure of an ophthalmic image processing apparatus according to an exemplary embodiment. Detailed Implementation

[0064] Referring to the accompanying drawings, several exemplary embodiments of the ophthalmic observation apparatus, ophthalmic image processing apparatus, ophthalmic image processing method, program, and recording medium are described in detail. Furthermore, the matters described in the documents cited in this specification, any prior art, and the exemplary embodiments can be combined.

[0065] The exemplary ophthalmic observation device is used to monitor the condition of the examined eye during medical procedures such as examination, surgery, and treatment. The exemplary ophthalmic observation device described below is a surgical microscope system; however, ophthalmic observation devices are not limited to surgical microscope systems. For example, an ophthalmic observation device can be any one of a slit-lamp microscope, fundus camera, refractometer, corneal astigmatism meter, tonometer, corneal endothelial microscope, wavefront aberrometer, microperimeter, SLO, and OCT device, or a system including more than one of these. More commonly, an ophthalmic observation device can be any ophthalmic device with observation capabilities.

[0066] The observation site using ophthalmic observation equipment can be any part of the examined eye, including any part of the anterior and / or posterior eye. Examples of anterior eye observation sites include the cornea, iris, anterior chamber, anterior chamber angle, lens, ciliary body, and zonules of Qin. Examples of posterior eye observation sites include the retina, choroid, sclera, and vitreous body. The observation site is not limited to ocular tissues; it can also include any part of the eyelids, meibomian glands, or orbit, which is observed in ophthalmology (and / or other specialties).

[0067] At least a portion of the functionality of the elements disclosed in this specification is implemented using a circuitry or processing circuitry. The circuitry or processing circuitry includes any of the following: a common processor, a special-purpose processor, an integrated circuit, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a programmable logic device (e.g., a SPLD (Simple Programmable Logic Device), a CPLD (Complex Programmable Logic Device), an FPGA (Field Programmable Gate Array)), conventional circuitry, and any combination thereof. A processor is considered to be a processing circuitry or circuitry including transistors and / or other circuitry. In this disclosure, circuitry, units, components, or similar terms are hardware that performs at least a portion of the disclosed functionality or hardware programmed to perform at least a portion of the disclosed functionality. The hardware may be the hardware disclosed in this specification or known hardware programmed and / or configured to perform at least a portion of the functions described herein. In the case where the hardware is a processor considered as a type of circuit structure, the circuit structure, unit, component, or similar term is a combination of hardware and software used to constitute the hardware and / or processor.

[0068] <Ophthalmic Observation Device>

[0069] Figure 1 The structure of an ophthalmic observation device is shown as an example.

[0070] The ophthalmic observation device 1 (surgical microscope system) of the embodiment includes an operating device 2, a display device 3, and a surgical microscope 10. In several embodiments, the surgical microscope 10 may also include at least one of the operating device 2 and the display device 3. In addition, in several embodiments, the display device 3 may not be included in the ophthalmic observation device 1. That is, the display device 3 may also be a peripheral device of the ophthalmic observation device 1.

[0071] <Operating Device 2>

[0072] Operating device 2 includes operating equipment and / or input devices. For example, operating device 2 may also include buttons, switches, mice, keyboards, trackballs, control panels, dials, etc. Typically, operating device 2 includes a foot switch, similar to that of a typical ophthalmic surgical microscope.

[0073] <Display Device 3>

[0074] Display device 3 displays images of the examined eye acquired by surgical microscope 10. Display device 3 includes display devices such as flat panel displays. Alternatively, display device 3 may also include various display devices such as touch panels. A typical display device 3 includes a large-screen display. Display device 3 includes one or more display devices. In cases where display device 3 includes two or more display devices, for example, one is a larger-screen display device and the other is a smaller-screen display device.

[0075] The operating device 2 and the display device 3 do not need to be separate devices. For example, a device that integrates operating and display functions, such as a touch panel, can be used as the display device 3. In this case, the operating device 2 includes the touch panel and a computer program. Operations on the operating device 2 are input to a processor (not shown) as electrical signals. Alternatively, operations and information input can be performed using a graphical user interface (GUI) displayed on the display device 3 and the operating device 2. In several ways, the functions of the operating device 2 and the display device 3 can also be implemented using a touch screen.

[0076] <Surgical Microscope 10>

[0077] The surgical microscope 10 is used to observe the eye (examined eye) of a patient in a supine position. The surgical microscope 10 captures images of the examined eye to generate digital image data. In particular, the surgical microscope 10 generates a dynamic image of the examined eye. The dynamic image (image) generated by the surgical microscope 10 is transmitted to a display device 3 via wired and / or wireless signal lines for display. The user (surgical operator) can perform surgery while observing the examined eye through the displayed image. In addition to such image observation, several types of surgical microscopes 10 also allow observation via conventional eyepiece lenses.

[0078] In several embodiments, the surgical microscope 10 includes a communication device for transmitting and receiving electrical signals between itself and the operating device 2. The operating device 2 receives user operations and generates corresponding electrical signals (operation signals). The operation signals are transmitted to the surgical microscope 10 via wired and / or wireless signal lines. The surgical microscope 10 performs processing corresponding to the received operation signals.

[0079] <Optical System of Surgical Microscope 10>

[0080] Hereinafter, for ease of explanation, the optical system structure of the surgical microscope 10 will be described as follows: the direction of the optical axis of the objective lens will be defined as the z-direction (e.g., the vertical direction during surgery, the up-down direction), a predetermined direction orthogonal to the z-direction will be defined as the x-direction (e.g., the horizontal direction during surgery, the left-right direction for the surgeon and the patient), and a direction orthogonal to both the z-direction and the x-direction will be defined as the y-direction (e.g., the horizontal direction during surgery, the front-back direction for the surgeon, the body axis direction for the patient).

[0081] Furthermore, the following description primarily focuses on the case where the observation optical system has a pair of left and right optical systems (optical systems capable of binocular observation). However, other types of observation optical systems can also have optical systems for monocular observation, and those skilled in the art will recognize that the structure described below can be applied to such systems.

[0082] Figure 2 An example of the structure of the optical system of the surgical microscope 10 is shown. Figure 2 This diagram shows a schematic top view of the optical system viewed from above and a schematic side view of the optical system viewed from the side. For the sake of simplicity, the illustration of the illumination optical system 30 positioned above the objective lens 20 is omitted.

[0083] The surgical microscope 10 includes an objective lens 20, a dichroic mirror DM1, an illumination optics system 30, and an observation optics system 40. The observation optics system 40 includes a zoom extender 50 and a camera 60. In several configurations, either the illumination optics system 30 or the observation optics system 40 includes the dichroic mirror DM1.

[0084] Objective 20 is configured to face the eye being examined. Objective 20 is configured such that its optical axis is along the z-direction. Objective 20 may also include two or more lenses.

[0085] The dichroic mirror DM1 couples the optical path of the illumination optical system 30 with the optical path of the observation optical system 40. The dichroic mirror DM1 is positioned between the illumination optical system 30 and the objective lens 20. The dichroic mirror DM1 transmits the illumination light from the illumination optical system 30 and guides it through the objective lens 20 to the eye being examined, and reflects and guides the reflected light from the eye being examined through the objective lens 20 to the camera 60 of the observation optical system 40.

[0086] The dichroic mirror DM1 coaxially couples the optical paths of the illumination optical system 30 and the observation optical system 40. That is, the optical axes of the illumination optical system 30 and the observation optical system 40 intersect on the dichroic mirror DM1. When the illumination optical system 30 includes a left-eye illumination optical system (31L) and a right-eye illumination optical system (31R), and the observation optical system 40 includes a left-eye observation optical system 40L and a right-eye observation optical system 40R, the dichroic mirror DM1 coaxially couples the optical paths of the left-eye illumination optical system (first illumination optical system 31L) with those of the left-eye observation optical system 40L, and coaxially couples the optical paths of the right-eye illumination optical system (first illumination optical system 31R) with those of the right-eye observation optical system 40R.

[0087] The illumination optical system 30 is an optical system used to illuminate the eye being examined via the objective lens 20. The illumination optical system 30 can be configured to illuminate the eye being examined using any one of two or more illumination lights with different color temperatures. Under the control of the control unit (200) described later, the illumination optical system 30 projects illumination light of a specified color temperature onto the eye being examined.

[0088] The illumination optical system 30 includes first illumination optical systems 31L and 31R and a second illumination optical system 32.

[0089] The optical axes OL and OR of the first illumination optical system 31L and 31R are respectively configured approximately coaxially with the optical axis of the objective lens 20. This enables so-called "0-degree illumination," allowing the acquisition of a transillumination image utilizing the diffuse reflection from the fundus. In this method, the transillumination image of the examined eye can be observed with both eyes.

[0090] The second illumination optical system 32 is configured such that its optical axis OS is off-center from the optical axis of the objective lens 20. The first illumination optical systems 31L and 31R, as well as the second illumination optical system 32, are configured such that the offset of the optical axis OS relative to the optical axis of the objective lens 20 is greater than the offset of the optical axes OL and OR relative to the optical axis of the objective lens 20. This enables so-called "angled illumination (oblique illumination)," preventing the intrusion of ghosting caused by corneal reflections, etc., while allowing binocular observation of the examined eye. Furthermore, it allows for detailed observation of the areas and tissues of the examined eye.

[0091] The first illumination optical system 31L includes a light source 31LA and a condenser lens 31LB. The light source 31LA outputs illumination light of a wavelength in the visible region, for example, with a color temperature of 3000K (Kelvin). The illumination light output from the light source 31LA passes through the condenser lens 31LB, is transmitted through the dichroic mirror DM1, and is incident on the eye being examined through the objective lens 20.

[0092] The first illumination optical system 31R includes a light source 31RA and a condenser lens 31RB. The light source 31RA also outputs illumination light of a wavelength in the visible region, for example, with a color temperature of 3000K. The illumination light output from the light source 31RA passes through the condenser lens 31RB, is transmitted through the dichroic mirror DM1, and is incident on the eye being examined through the objective lens 20.

[0093] The second illumination optical system 32 includes a light source 32A and a condenser lens 32B. The light source 32A outputs illumination light with a wavelength in the visible range, for example, a color temperature of 4000K to 6000K. The illumination light output from the light source 32A passes through the condenser lens 32B and, without passing through the dichroic mirror DM1, passes through the objective lens 20 and enters the eye being examined.

[0094] That is, the color temperature of the illumination light from the first illumination optical system 31L and 31R is lower than the color temperature of the illumination light from the second illumination optical system 32. With such a structure, the eye under examination can be observed using the first illumination optical system 31L and 31R with a warm color, and the structure and manner of the eye under examination can be observed in detail.

[0095] In several ways, the optical axes OL and OR can be moved relatively relative to the optical axis of the objective lens 20. This relative movement is in the direction intersecting the optical axis of the objective lens 20, and is represented by a displacement vector in at least one of the x and y components that is not zero. In several ways, the optical axes OL and OR can be moved independently. On the other hand, in several ways, the optical axes OL and OR can be moved as a unit. For example, the surgical microscope 10 includes a moving mechanism (31d) that moves the first illumination optical systems 31L and 31R independently or as a unit, by which the first illumination optical systems 31L and 31R are moved independently or as a unit in a direction intersecting the optical axis of the objective lens 20. This allows adjustment of the visual effect on the examined eye. In several ways, the moving mechanism is operated under the control of a control unit (200) described later.

[0096] In several ways, the optical axis OS can be moved relative to the optical axis of the objective lens 20. This relative movement is in the direction intersecting the optical axis of the objective lens 20, and is expressed as a displacement vector in at least one of the x and y components that is not zero. For example, the surgical microscope 10 includes a moving mechanism (32d) that moves the second illumination optical system 32 in a direction intersecting the optical axis of the objective lens 20. This allows for adjustment of the visual effect of the concavity and convexity of the examined eye area and tissue. In several ways, the moving mechanism is operated under the control of a control unit (200), described later.

[0097] As described above, in this configuration, the illumination optical system 30 is positioned directly above the objective lens 20 (at the position of the transmission direction of the dichroic mirror DM1), and the observation optical system 40 is positioned at the position of the reflection direction of the dichroic mirror DM1. For example, the observation optical system 40 may be configured such that the angle formed by the plane (xy plane) orthogonal to the optical axis of the observation optical system 40 and the optical axis of the objective lens 20 is ±20 degrees or less.

[0098] According to the structure of this method, the observation optical system 40, whose optical path length is generally longer than that of the illumination optical system 30, is configured to be approximately parallel to the xy plane. Therefore, unlike conventional surgical microscopes where the observation optical system is positioned vertically in front of the surgeon's eyes, it does not obstruct the surgeon's field of vision. Consequently, the surgeon can easily observe the image displayed on the front-facing display device 3. In other words, the visibility of display information (images, reflections, and other various reference information of the examined eye) during surgery is improved. Furthermore, since no housing is placed in front of the surgeon's eyes, it does not create a sense of pressure on the surgeon, thereby reducing the surgeon's burden.

[0099] The observation optical system 40 is an optical system for observing an image based on the reflected light from the illumination light incident from the examined eye via the objective lens 20. In this configuration, the observation optical system 40 provides the image to the imaging element of the camera 60.

[0100] As described above, the observation optical system 40 includes a left-eye observation optical system 40L and a right-eye observation optical system 40R. The structure of the left-eye observation optical system 40L is the same as that of the right-eye observation optical system 40R. In several ways, the optical configurations of the left-eye observation optical system 40L and the right-eye observation optical system 40R can also be changed independently of each other.

[0101] The zoom expander 50 is also referred to as a beam expander, variable beam expander, etc. The zoom expander 50 includes a left-eye zoom expander 50L and a right-eye zoom expander 50R. The structure of the left-eye zoom expander 50L is the same as that of the right-eye zoom expander 50R. In several ways, the optical configurations of the left-eye zoom expander 50L and the right-eye zoom expander 50R can also be changed independently.

[0102] The zoom extender 50L for the left eye includes multiple zoom lenses 51L, 52L, and 53L. At least one of the multiple zoom lenses 51L, 52L, and 53L is movable in the optical axis direction via a zoom mechanism (not shown).

[0103] Similarly, the zoom extender 50R for the right eye includes multiple zoom lenses 51R, 52R and 53R, at least one of which is movable in the optical axis direction via a zoom mechanism (not shown).

[0104] The zoom mechanism can be configured to move each zoom lens of the left-eye zoom extender 50L and each zoom lens of the right-eye zoom extender 50R independently or as a unit towards the optical axis. This changes the magnification when photographing the eye being examined. In several configurations, the zoom mechanism is operated under the control of the control unit (200), described later.

[0105] The video camera 60 is a device that generates digital image data by capturing an image formed by the observation optical system 40; typically, it is a digital camera (digital video camera). The video camera 60 includes a left-eye video camera 60L and a right-eye video camera 60R. The structure of the left-eye video camera 60L is the same as that of the right-eye video camera 60R. In several ways, the optical configurations of the left-eye video camera 60L and the right-eye video camera 60R can be changed independently of each other.

[0106] The left-eye camera 60L includes an imaging lens 61L and an image sensor 62L. The imaging lens 61L forms an image on the imaging surface of the image sensor 62L based on the reflected light passing through the left-eye zoom extender 50L. The image sensor 62L is a region sensor, typically a charge-coupled device (CCD) image sensor or a complementary metal-oxide-semiconductor (CMOS) image sensor. The image sensor 62L operates under the control of a control unit (200) described later.

[0107] The right-eye camera 60R includes an imaging lens 61R and an image sensor 62R. The imaging lens 61R forms an image on the imaging surface of the image sensor 62R based on the reflected light passing through the right-eye zoom extender 50R. The image sensor 62R is a region sensor, typically a CCD image sensor or a CMOS image sensor. The image sensor 62R operates under the control of a control unit (200) described later.

[0108] <Processing System>

[0109] This describes the processing system of the ophthalmic observation device 1. Figures 3 to 10 Several structural examples of the processing system are shown. At least two or more of the structural examples described below can be locally combined. Furthermore, the structure of the processing system is not limited to these examples.

[0110] The control unit 200 controls each part of the ophthalmic observation device 1. The control unit 200 includes a main control unit 201 and a storage unit 202. The main control unit 201 includes a processor that controls each part of the ophthalmic observation device 1. For example, in order to realize the functions of this method, the processor can read and execute programs stored in the storage unit 202 or other storage devices, and can also utilize (reference, process, calculate, etc.) data and information stored in the storage unit 202 or other storage devices.

[0111] The main control unit 201 is capable of controlling the light sources 31LA, 31RA and 32A of the illumination optical system 30, the imaging elements 62L and 62R of the observation optical system 40, the moving mechanisms 31d and 32d, the zoom mechanisms 50Ld and 50Rd, the operating device 2, the display device 3, etc.

[0112] The control of light source 31LA includes turning the light source on and off, adjusting the light intensity, and adjusting the aperture. The control of light source 31RA includes turning the light source on and off, adjusting the light intensity, and adjusting the aperture. The main control unit 201 can exclusively control light sources 31LA and 31RA. The control of light source 32A includes turning the light source on and off, adjusting the light intensity, and adjusting the aperture.

[0113] When the lighting optical system 30 includes a light source capable of changing color temperature, the main control unit 201 can change the color temperature of the output lighting light by controlling the light source.

[0114] The control of image sensor 62L includes exposure adjustment, gain adjustment, and imaging rate adjustment. The control of image sensor 62R includes exposure adjustment, gain adjustment, and imaging rate adjustment. Furthermore, the main control unit 201 can control image sensors 62L and 62R so that their imaging timing is synchronized or the difference between their imaging timings is within a predetermined time. Moreover, the main control unit 201 can control the readout of digital data obtained from image sensors 62L and 62R.

[0115] The moving mechanism 31d moves the light sources 31LA and 31RA independently or integrally in a direction intersecting the optical axis of the objective lens 20. The main control unit 201 can move the optical axes OL and OR independently or integrally relative to the optical axis of the objective lens 20 by controlling the moving mechanism 31d.

[0116] The moving mechanism 32d moves the light source 32A independently or integrally in a direction intersecting the optical axis of the objective lens 20. The main control unit 201 can move the optical axis OS relative to the optical axis of the objective lens 20 by controlling the moving mechanism 32d.

[0117] In several ways, the main control unit 201 can coordinately control the moving mechanisms 31d and 32d.

[0118] The zoom mechanism 50Ld moves at least one of the multiple zoom lenses 51L to 53L of the left-eye zoom extender 50L in the optical axis direction. The main control unit 201 can change the magnification of the left-eye observation optical system 40L by controlling the zoom mechanism 50Ld.

[0119] Similarly, the zoom mechanism 50Rd moves at least one of the multiple zoom lenses 51R to 53R of the right-eye zoom expander 50R in the optical axis direction. The main control unit 201 can change the magnification of the right-eye observation optical system 40R by controlling the zoom mechanism 50Rd.

[0120] Control of the operating device 2 includes operation permission control, operation prohibition control, transmission control and / or reception control of operation signals from the operating device 2, etc. The main control unit 201 receives the operation signals generated by the operating device 2 and executes the control corresponding to the received signals.

[0121] Control of the display device 3 includes information display control, etc. The main control unit 201, acting as a display control unit, enables the display device 3 to display images based on digital image data generated by the imaging elements 62L and 62R. Typically, the display device 3 can display moving images (images) based on digital image data (image signals) generated by the imaging elements 62L and 62R, and can also display still images (frames) included in the moving images. Furthermore, the main control unit 201 can enable the display device 3 to display images (moving images, still images, etc.) obtained by processing the digital image data generated by the imaging elements 62L and 62R. Additionally, the main control unit 201 can enable the display device 3 to display any information generated by the ophthalmic observation device 1, and any information acquired by the ophthalmic observation device 1 from external sources.

[0122] Furthermore, the main control unit 201, acting as a display control unit, creates a left-eye image based on digital image data generated by the imaging element 62L, and a right-eye image based on digital image data generated by the imaging element 62R. This enables the display device 3 to display the created left-eye and right-eye images in a stereoscopic manner. For example, the main control unit 201 can create a pair of parallax images based on the left-eye and right-eye images, allowing the display device 3 to display this pair of parallax images. Users (such as surgical operators) can recognize the pair of parallax images as stereoscopic images using known stereoscopic observation methods. Any stereoscopic observation method applicable to this method can be used, such as a naked-eye stereoscopic observation method, a stereoscopic observation method using auxiliary tools (polarized glasses, etc.), a stereoscopic observation method utilizing image processing (image synthesis, rendering, etc.) of the left-eye and right-eye images, a stereoscopic observation method simultaneously displaying a pair of parallax images, a stereoscopic observation method switching between displaying a pair of parallax images, or any combination of two or more of these stereoscopic observation methods.

[0123] The data processing unit 210 performs various data processing operations. The following describes several examples of the processing operations that the data processing unit 210 can perform.

[0124] <Detailed Examples of Processing Systems>

[0125] Several examples of the processing that the data processing unit 210 can perform, along with the associated elements, are explained. Figures 4 to 10 Examples of the structure of the data processing unit 210 (and associated elements) are shown respectively. At least partially, they can be combined. Figures 4 to 10 Any two or more of the structural examples shown. The data processing unit 210 (its elements) includes a processor that operates according to predetermined software (program), and is implemented through hardware and software cooperation.

[0126] Figure 4 The data processing unit 210A shown is Figure 3 An example of a data processing unit 210. In this example, the data processing unit 210A includes an image processing unit 211. The image processing unit 211 applies image processing (first image processing) to the still images included in the dynamic image (first dynamic image, image) of the examined eye generated by the surgical microscope 10, using multiple different values ​​of predetermined image parameters. Thus, multiple processed images based on the still images are created. The processed image creation process based on the image processing unit 211 can be performed in parallel with the acquisition of the dynamic image of the examined eye based on the surgical microscope 10.

[0127] The types of image parameters used for the first image processing can be arbitrary. For example, the image parameters can include one or more of the parameters of the types exemplified below: hue parameter (parameter for hue transformation); brightness parameter (parameter for brightness transformation); contrast parameter (parameter for contrast transformation); gain parameter (parameter for gain change); gamma parameter (parameter for gamma correction (correction of the response characteristics of image gray levels)); color temperature parameter (parameter for color temperature transformation); white balance parameter (parameter for white balance transformation); RGB balance parameter (parameter for balance transformation between R value, G value, and B value); gray balance parameter (parameter for gray balance transformation); edge enhancement parameter (parameter for edge enhancement); shadow enhancement parameter (parameter for shadow enhancement); sharpening parameter (parameter for sharpening); high dynamic range parameter (parameter for HDR synthesis). The image parameters applicable to this example are not limited to the types exemplified here. More commonly, any parameter that can be used to change the visual effect of an image (such as display control parameters, image presentation parameters, image correction parameters, image adjustment parameters, etc.) can be used.

[0128] In the first image processing, the image processing unit 211 can apply multiple image processings using multiple different values of one image parameter to the still images included in the first moving image respectively. For example, the image processing unit 211 can apply N image processings (hue transformation) using N values of the hue parameter to the still image (N is an integer greater than or equal to 2). Thereby, N processed images are produced with different hues. The same applies when using image parameters other than the hue parameter.

[0129] In addition, in the first image processing, the image processing unit 211 can apply multiple image processings using combinations of multiple different values of two different image parameters to the still images included in the first moving image respectively. For example, the image processing unit 211 can apply N×M image processings (hue transformation and brightness transformation) composed of combinations of N values of the hue parameter and M values of the brightness parameter to the still image (N and M are integers greater than or equal to 2 respectively). Thereby, N×M processed images are produced with different combinations of hue and brightness. The same applies when using combinations other than the combination of the hue parameter and the brightness parameter. In addition, the number of combined image parameters can be any number greater than or equal to 2. In addition, it is not necessary to use all the multiple values prepared for each image parameter. For example, when preparing N values of the hue parameter and M values of the brightness parameter, combinations of N1 values of the hue parameter and M1 values of the brightness parameter can be considered. Here, N1≤N and M1≤M, and N1<N and / or M1<M.

[0130] The multiple processing images produced in this way are displayed on the display device 3 by the main control unit 201. Here, the main control unit 201 can either make the display device 3 display the multiple processing images produced by the image processing unit 211 themselves, or make the display device 3 display thumbnails (reduced images) of these processing images.

[0131] The method of displaying multiple processing images can be arbitrary. In several ways, the main control unit 201 can display two or more processing images (or their thumbnails) from the multiple processing images in an arranged manner. Here, the main control unit 201 can also perform the process of arranging and displaying a first group selected from the multiple processing images in an arranged manner, and the process of switching from the arrangement display of the first group to the arrangement display of a second group in accordance with a predetermined trigger. The trigger for switching the group of arranged display is issued manually or automatically.

[0132] In several ways, the main control unit 201 can sequentially display two or more processing images (or their thumbnails) from a plurality of processing images. Switching between displayed processing images can be done manually or automatically.

[0133] When displaying a thumbnail, the main control unit 201 (or image processing unit 211 or data processing unit 210A) performs the process of creating a thumbnail of the processed image produced by the first image processing.

[0134] The user (e.g., a surgeon, an assistant who receives instructions from the surgeon, etc.) selects the desired processing image from a plurality of processing images (or thumbnails thereof) displayed on the display device 3.

[0135] The number of processed images selected can be arbitrary. If one processed image is selected, it is provided to subsequent processing. If two or more processed images are selected, all or part of them are provided to subsequent processing. For example, the image processing unit 211A (or the data processing unit 210A) can be configured to select one processed image from two or more selected processed images according to a predetermined algorithm.

[0136] The ophthalmic observation device 1 includes an element (instruction receiving unit) for receiving user instructions for selecting processed images. The element functioning as the instruction receiving unit can be arbitrary. For example, the user can give instructions using the operating device 2. In several ways, the instruction receiving unit may include a voice recognition unit that detects and recognizes voice instructions, a gaze recognition unit that detects and recognizes gaze instructions, a brainwave recognition unit that detects and recognizes brainwave instructions, a finger recognition unit that detects and recognizes finger instructions, a biosignal recognition unit that detects and recognizes arbitrary biosignal instructions, etc.

[0137] When a user selects a processing image, the ophthalmic observation device 1 performs processing based on the selected image. Several examples of this processing are described below.

[0138] illustrate Figure 5 The structure example shown. Figure 5 The image processing unit 211A shown is Figure 4 An example of an image processing unit 211. In this example, the image processing unit 211A includes an image processing unit 2111 and a motion image processing unit 2112.

[0139] Image processing unit 2111, for example, performs reference Figure 4 At least a portion of the process described herein.

[0140] The dynamic image processing unit 2112 processes the dynamic image based on the processing image selected by the user from a plurality of processing images displayed on the display device 3. For example, the dynamic image processing unit 2112 first acquires the image parameters corresponding to the processing image selected by the user. That is, the dynamic image processing unit 2112 acquires the values ​​of the image parameters used in the first image processing for creating the processing image selected by the user. Then, the dynamic image processing unit 2112 applies image processing (second image processing) based on the values ​​of these image parameters to the dynamic image. The dynamic image for which the second image processing is applied is at least the dynamic image generated by the surgical microscope 10 after the user selects the processing image (referred to as the second dynamic image). Thus, a dynamic image that has undergone the same image processing as the processing image selected by the user is obtained. Furthermore, if more than one still image is included in the dynamic image (first dynamic image) acquired before the user selects the processing image, the same image processing can be applied to at least a portion of the more than one still image.

[0141] The main control unit 201 causes the display device 3 to display a second dynamic image that has undergone the second image processing. Typically, the ophthalmic observation device 1 can perform real-time application of the second image processing to the dynamic image (second dynamic image) acquired through the dynamic image acquisition and real-time display of the dynamic image with the second image processing applied while capturing a dynamic image of the examined eye using a surgical microscope 10. Thus, the user can observe in real-time a dynamic image that has undergone the following image processing: using the same image parameter values ​​as the processed image selected by the user.

[0142] As described above, the number of processed images selected by the user can be arbitrary. For example, if only one processed image is selected, the dynamic image processing unit 2112 can apply the image processing that uses the value of the image parameter corresponding to the selected processed image (one value) as a second image processing to the second dynamic image.

[0143] On the other hand, when two or more processing images are selected, the dynamic image processing unit 2112 selects one processing image from the two or more selected processing images, and applies the image processing using the value of the image parameter corresponding to that processing image (one value) as the second image processing to the second dynamic image. Alternatively, the dynamic image processing unit 2112 selects one value from two or more values ​​of the image parameter corresponding to the two or more selected processing images, and applies the image processing using that one value as the second image processing to the second dynamic image. These selection processes performed by the dynamic image processing unit 2112 are executed according to a predetermined algorithm. For example, the dynamic image processing unit 2112 may also be configured to analyze two or more processing images selected by the user and calculate a predetermined image quality evaluation value, and select a processing image with the best image quality evaluation value. Alternatively, when two or more image parameters are used in the first image processing, the dynamic image processing unit 2112 may also be configured to select the processing image or the image parameter value according to the priority between the preset image parameters. In this case, for example, the dynamic image processing unit 2112 is configured to select the optimal value (e.g., the highest contrast value) from multiple values ​​of the highest priority image parameter.

[0144] In other methods where two or more processing images are selected, the dynamic image processing unit 2112 determines a value based on two or more values ​​of image parameters corresponding to the selected two or more processing image segments, and can apply the image processing using that value as a second image processing to the second dynamic image. The calculation process that determines a value based on two or more values ​​can be arbitrary, such as a statistical calculation. The statistical value obtained by the statistical calculation can be any kind of statistical value such as average, median, maximum, minimum, etc. In addition, the calculation process to be applied can be preset or set each time processing is performed. As an example of the latter, the dynamic image processing unit 2112 can determine the type of calculation process (statistical calculation) to be applied based on one or more factors such as the type of image parameters used in the first image processing, the number of processing images selected by the user, the type of image acquired by the surgical microscope 10, and the stage of the medical procedure (surgery in this example).

[0145] Next, refer to Figure 6 . Figure 6 The data processing unit 210B shown is Figure 3 The data processing unit 210B in this example includes an image processing unit 211 and a parameter processing unit 212.

[0146] The image processing unit 211 may also be configured to perform the same processing as the image processing unit 211 or the image processing unit 211A.

[0147] The parameter processing unit 212 performs processing related to image parameters. The image processing unit 211 can perform processing using the output from the parameter processing unit 212. For example, the parameter processing unit 212 is configured to record the values ​​of image parameters used in the second image processing of the dynamic image processing unit 2112 (recording unit), and the image processing unit 211 is configured to perform new image processing (e.g., new first image processing, new second image processing) using the values ​​of image parameters recorded by the parameter processing unit 212. Furthermore, the parameter processing unit 212, as the recording unit, can be configured to record only the values ​​of image parameters used in the second image processing, or it can be configured to record values ​​other than those used in the second image processing. As an example of the latter, the parameter processing unit 212, as the recording unit, can record the values ​​of image parameters corresponding to each processed image selected by the user, or it can record one or more values ​​of image parameters corresponding to a portion of multiple processed images selected by the user, or it can record the values ​​of image parameters that have been manually adjusted.

[0148] The following describes the structure of the parameter processing unit 212 and several examples of the processing performed using the parameter processing unit 212.

[0149] Figure 7 The parameter processing unit 212A shown is Figure 6 An example of a parameter processing unit 212. The parameter processing unit 212A includes a parameter recording unit 2121 and a parameter selection unit 2122. When using the parameter processing unit 212A of this example, the ophthalmic observation device 1 may have an identifier receiving unit 221 and an attribute information acquisition unit 222. Furthermore, in several embodiments, the ophthalmic observation device 1 may have only one of the identifier receiving unit 221 and the attribute information acquisition unit 222.

[0150] The identifier receiving unit 221 receives the user's identifier (user ID). Typically, the user is the surgical operator. Alternatively, the user ID can be represented, for example, by a string or image (e.g., a barcode, 2D barcode, etc.) assigned to the surgical operator (doctor) within the medical institution, the surgical operator's name, or the surgical operator's biometric information (e.g., face, fingerprint, palm print, iris pattern, voice, etc.).

[0151] The identifier receiving unit 221 may include any device (hardware, software) for receiving such user IDs, such as operating device 2, camera, barcode scanner, biometric information scanner, microphone, processor, etc.

[0152] The parameter recording unit 2121 functions as the recording unit, recording the values ​​of image parameters in association with the user ID received by the identifier receiving unit 221. This allows it to determine which user selected which image parameter value. For example, the user ID can be used as a search query to search for the value of an image parameter.

[0153] The attribute information acquisition unit 222 acquires attribute information representing the attributes of the medical procedure performed on the examined eye. In other words, the attribute information acquisition unit 222 acquires attribute information representing various attributes related to the medical procedure (surgery in this example) performed on the examined eye using the ophthalmic observation device 1.

[0154] As attributes of surgery, these include surgical types corresponding to the surgical site (e.g., anterior eye surgery, posterior eye surgery, corneal surgery, anterior chamber angle surgery, ciliary body surgery, retinal surgery, vitreous surgery, etc.) and surgical types corresponding to diseases (e.g., cataract surgery, glaucoma surgery, corneal transplantation, retinal cell transplantation, retinal detachment surgery, laser photocoagulation, etc.). The same applies to medical procedures other than surgery (e.g., examinations, treatments, screenings, etc.).

[0155] The attributes of medical procedures are not limited to the types of medical procedures. For example, attribute information can include information representing multiple stages of a surgery. For instance, the stages of cataract surgery include alignment, incision preparation, ocular viscoelastic injection, CCC (corneal capillary closure), lens emulsification and drainage, lens cortical drainage, IOL (intraocular lens) insertion, IOL centering, ocular viscoelastic removal, and incision closure. Information representing such surgical stages can be included in the attribute information. Furthermore, the stages included in the attribute information can be preset or determined by the surgeon. Additionally, two or more stages applying the same image parameter values ​​can be grouped together (stage grouping).

[0156] Attribute information can be represented, for example, by strings or images (such as barcodes, 2D barcodes, etc.) assigned to medical procedures within the medical institution, or by the names of the medical procedures.

[0157] The attribute information acquisition unit 222 may include any device (hardware or software) for receiving such attribute information, such as the operating device 2, camera, barcode scanner, biometric information scanner, microphone, processor, etc.

[0158] Alternatively, the attribute information acquisition unit 222 can be configured to automatically determine the current stage by analyzing images (images) generated by the surgical microscope 10, operations performed using the operating device 2, and images displayed on the display device 3. Furthermore, if the surgical procedure is predetermined, the attribute information acquisition unit 222 can also be configured to automatically determine the current stage by referring to the procedures performed up to the current stage.

[0159] The main control unit 201 enables the display device 3 to display information representing the stage automatically determined by the attribute information acquisition unit 222. The user can judge whether the displayed information is correct and input the result of this judgment using the operation device 2 or the like. With this structure, the stage automatically determined by the attribute information acquisition unit 222 can be confirmed, thus preventing the recording of incorrect stages.

[0160] As part of the recording unit, parameter recording unit 2121 records the values ​​of image parameters in association with the attribute information acquired by attribute information acquisition unit 222. This allows it to determine which image parameter value the user selected during which medical procedure (type, stage, etc.). For example, the image parameter value can be searched using the type and stage of surgery as a search query. Furthermore, the attribute information can include any information such as the disease's progress, the date of the medical procedure, and the surgeon's skill level.

[0161] With both an identifier receiving unit 221 and an attribute information acquisition unit 222 provided, the parameter recording unit 2121, acting as a recording unit, can record the value of the image parameter in association with both the user ID received by the identifier receiving unit 221 and the attribute information acquired by the attribute information acquisition unit 222. This allows it to determine which user selected which image parameter value in which medical procedure (type, stage, etc.). For example, the value of the image parameter can be searched using the user ID and / or the type and stage of the procedure as a search query.

[0162] The parameter selection unit 2122 selects at least one value from the image parameter values ​​previously recorded by the parameter recording unit 2121. For example, the parameter selection unit 2122 can perform the selection of image parameter values ​​through the search query.

[0163] The image processing unit 211 (211A) is capable of applying image processing based on the values ​​of image parameters selected by the parameter selection unit 2122 to the dynamic image (third dynamic image) generated by the surgical microscope 10. The third dynamic image can be any dynamic image, such as the second dynamic image, a dynamic image obtained through other medical procedures on the same subject, or a dynamic image obtained through medical procedures on other subjects.

[0164] When the user ID is used as a search query to select the value of the image parameter, the image processing that uses the value of the image parameter previously used by the user (surgical operator, etc.) can be applied to the third dynamic image, so that the third dynamic image with the display mode (color, brightness, contrast, etc.) preferred by the user can be easily provided.

[0165] When selecting image parameter values ​​using attribute information of medical procedures as a search query, image processing that uses image parameter values ​​previously used in accordance with the type and stage of medical procedures can be applied to third dynamic images. Therefore, it is possible to easily provide third dynamic images with display methods (color, brightness, contrast, etc.) that match the type and stage of medical procedures.

[0166] Figure 8 The parameter processing unit 212B shown is Figure 6 Example of parameter processing unit 212. Parameter processing unit 212B and Figure 7 The parameter processing unit 212A similarly includes a parameter recording unit 2121 and a parameter selection unit 2122, and also includes a shooting condition recording unit 2123, a shooting condition selection unit 2124, and a parameter determination unit 2125. The structure and operation of the parameter recording unit 2121 and the parameter selection unit 2122 can be compared with... Figure 7 The situation is the same, so its explanation is omitted.

[0167] Even when using the parameter processing unit 212B of this example, the ophthalmic observation device 1 may include an identifier receiving unit 221 and an attribute information acquisition unit 222. Furthermore, in several embodiments, the ophthalmic observation device 1 may only include either the identifier receiving unit 221 or the attribute information acquisition unit 222. The structure and operation of the identifier receiving unit 221 and the attribute information acquisition unit 222 can be similar to... Figure 7 The situation is the same, so its explanation is omitted.

[0168] The imaging condition recording unit 2123 records the imaging conditions when the surgical microscope 10 generates the second dynamic image, in association with the values ​​of the image parameters recorded by the parameter recording unit 2121. In this example, the combination of the parameter recording unit 2121 and the imaging condition recording unit 2123 functions as a recording unit.

[0169] Shooting conditions may include, for example, lighting conditions, observation conditions, environmental conditions, and conditions of the eye being examined. Lighting conditions include elements related to the lighting optical system 30, such as light quantity (output light quantity and intensity from light sources 31LA, 31RA, and 32A), aperture value, and lighting mode (0-degree illumination, angled illumination, etc.). Observation conditions are elements related to the observation optical system 40, such as aperture value, magnification, and control values ​​(gain, etc.) of the imaging elements 62L and 62R. Environmental conditions include conditions related to the environment during shooting, such as the brightness of the room where the surgical microscope 10 is located. Conditions of the eye being examined include conditions related to the eye being examined, such as the presence or absence of disease, the type and severity of the disease, pupil diameter, and the degree of opacity in the affected area (cornea, lens, vitreous humor, etc.).

[0170] In this example, information is recorded by both the parameter recording unit 2121 and the shooting condition recording unit 2123, therefore, in addition to... Figure 7 In addition to the information considered in the examples, the shooting conditions can also be taken into account.

[0171] The shooting condition selection unit 2124 selects shooting conditions associated with the values ​​of image parameters selected by the parameter selection unit 2122 from shooting conditions previously recorded by the shooting condition recording unit 2123. Therefore, in addition to obtaining the values ​​of the image parameters selected by the parameter selection unit 2122, it is also possible to obtain the shooting conditions associated with those image parameter values.

[0172] The parameter determination unit 2125 determines the value of the image parameter based on the value of the image parameter selected by the parameter selection unit 2122 and the shooting conditions selected by the shooting condition selection unit 2124.

[0173] Regarding controllable conditions in the surgical microscope 10, such as illumination and observation conditions, the main control unit 201 can control the surgical microscope 10 (illumination optical system 30, observation optical system 40, etc.) according to the selected imaging conditions. For example, the main control unit 201 can control the surgical microscope 10 to reproduce the selected imaging conditions. As a result, it is possible to reproduce the image display mode when the image parameters selected by the parameter selection unit 2122 are applied.

[0174] Regarding imaging conditions other than controllable conditions in the surgical microscope 10 (environmental conditions, conditions of the examined eye, etc.), for example, the parameter determination unit 2125 performs processing based on the values ​​of the selected image parameters and the imaging conditions. This processing can be computational processing based on a predetermined algorithm.

[0175] In several methods, a correspondence (graph, table, etc.) between changes in shooting conditions (environmental conditions, conditions of the eye being examined, etc.) and the values ​​of image parameters is prepared in advance. The parameter determination unit 2125 compares the current shooting conditions with the selected shooting conditions and determines the values ​​of the image parameters based on the comparison result (the amount of change in shooting conditions, etc.) and the correspondence.

[0176] In several approaches, instead of preparing the aforementioned correspondence, a system (artificial intelligence engine) that uses machine learning to learn the relationship between changes in shooting conditions (environmental conditions, conditions of the examined eye, etc.) and the values ​​of image parameters. This artificial intelligence engine, for example, includes a neural network created through machine learning by taking the shooting conditions and the values ​​of the image parameters as input and taking the new values ​​of the image parameters as output. This artificial intelligence engine is trained, for example, to output optimal image parameter values ​​corresponding to changes in shooting conditions based on the assigned values ​​of the image parameters. The parameter determination unit 2125 includes such an artificial intelligence engine, to which the selected image parameter values ​​and shooting conditions are input. The image parameter values ​​output from this artificial intelligence engine become the result obtained by the parameter determination unit 2125.

[0177] The image processing unit 211 (211A) is capable of applying image processing using the values ​​of image parameters determined by the parameter determination unit 2125 to the dynamic image (third dynamic image) generated by the surgical microscope 10. As described above, the third dynamic image can be any dynamic image, such as the second dynamic image, a dynamic image obtained through other medical procedures on the same subject, or a dynamic image obtained through medical procedures on other subjects.

[0178] According to this example, the values ​​of image parameters can be adjusted (corrected) according to the differences in shooting conditions. Therefore, regardless of the differences in shooting conditions, a third moving image can be easily provided in a way that is close to the previously achieved good display method (color, brightness, contrast, etc.).

[0179] Next, the explanation Figure 9 The example shown illustrates this. In this example, by applying image processing only to a portion of the still images (frames) included in the moving image, the resources required for image processing are reduced, the time is shortened, and the image quality of the area of ​​interest in the observed eye is optimized.

[0180] Figure 9 The image processing unit 211B shown creates multiple localized processed images by applying first image processing to a portion (partial image) of a still image included in the first dynamic image generated by the surgical microscope 10. In this example, multiple localized processed images are processed as multiple processed images. The main control unit 201 causes the display device 3 to display multiple images, each including the created multiple localized processed images. Each displayed image can be either the corresponding localized processed image or a wider image including the corresponding localized processed image. This wider image can be, for example, an image in which the localized processed image is replaced in the corresponding still image. In this way, by displaying multiple localized processed images, the user can select the localized processed image in which the area of ​​interest is best depicted.

[0181] Figure 9 The image processing unit 211B shown is Figure 4 An example of the image processing unit 211. The image processing unit 211B in this example is... Figure 5 Similarly, the image processing unit 211A includes a processed image creation unit 2111 and a dynamic image processing unit 2112. Unless otherwise specified, the processed image creation unit 2111 and the dynamic image processing unit 2112 in this example can be respectively connected to... Figure 5 The image processing unit 2111 and the dynamic image processing unit 2112 in the example are the same.

[0182] The image processing unit 211B also includes a local image determination unit 2113A. The local image determination unit 2113A determines a local image in the still image by applying segmentation of the image used to determine a predetermined region of the examined eye to the still image included in the first moving image. Typically, segmentation is a process used to determine local regions in an image. Segmentation can include any known image processing technique, such as image processing like edge detection and / or segmentation utilizing machine learning (e.g., deep learning).

[0183] In this example, the image processing unit 2111 applies first image processing to the local image determined by the local image determination unit 2113A. That is, the scope of application of the first image processing of the image processing unit 2111 in this example is limited to the local image determined by the local image determination unit 2113A.

[0184] Furthermore, the local image determination unit 2113A can also apply the same segmentation to the second dynamic image generated by the surgical microscope 10 after the user selects the processing image. More specifically, the local image determination unit 2113A first applies segmentation to each still image (or each still image selected through a culling process, etc., hereinafter the same) included in the second dynamic image to determine the image of the same region of interest.

[0185] In several ways, the segmentation of the still images included in the second moving image can be performed using the same method as the segmentation of the still images included in the first moving image. Thus, images of the same region of interest are determined based on each still image included in the second moving image.

[0186] In several ways, the ophthalmic observation device 1 has the function of analyzing dynamic images generated by the surgical microscope 10 and monitoring the activity of the examined eye. The local image determination unit 2113A stores the position of the examined eye (referred to as the reference position) when a still image (applicable to segmentation, called the reference still image) included in the first dynamic image is obtained. Additionally, the local image determination unit 2113A stores information representing the range (local image range) of local images within the still images included in the first dynamic image. When generating a second dynamic image, each still image (called the object still image) included in the second dynamic image and the position of the examined eye (called the object position) when the object still image is obtained are input to the local image determination unit 2113A. The local image determination unit 2113A calculates the offset of the object position relative to the reference position, and determines the range of local images in the object still image by shifting the local image range only by this offset. According to this example, even for difficult-to-segment areas (e.g., areas with small brightness or color differences from the surroundings, smaller areas, etc.), local images in the second dynamic image can be tracked in real time (frames are obtained sequentially).

[0187] In this example of processing, the local image determination unit 2113A can sequentially determine local images of the still images included in the second dynamic image by applying segmentation to the second dynamic image. Furthermore, the image processing unit 210B (dynamic image processing unit 2112) can sequentially apply second image processing to the local images determined based on the still images included in the second dynamic image. Therefore, it is also possible to apply the values ​​of image parameters suitable for the user-selected local processed image to the second dynamic image, thereby enabling the observation of a second dynamic image in which the area of ​​interest is optimally depicted.

[0188] Next, the explanation Figure 10 The example shown. With Figure 9 Similarly, in this example, by applying image processing only to a portion of the still images (frames) included in the moving image, the resources required for image processing are reduced, the time is shortened, and the image quality of the area of ​​interest in the observed eye is optimized.

[0189] Figure 10The image processing unit 211C shown creates multiple localized processed images by applying first image processing to a portion (partial image) of a still image included in the first dynamic image generated by the surgical microscope 10. In this example, multiple localized processed images are processed as multiple processed images. The main control unit 201 causes the display device 3 to display multiple images, each including the created multiple localized processed images. Each displayed image can be either the corresponding localized processed image or a wider-area image including the corresponding localized processed image. This wider-area image can, for example, be an image in which the localized image in the corresponding still image is replaced with the localized processed image. In this way, by displaying multiple localized processed images, the user can select the localized processed image in which the area of ​​interest is best depicted. This is consistent with... Figure 9 The example is the same, but this example differs from others in its method of determining local images. Figure 9 The examples are different.

[0190] Figure 10 The image processing unit 211C shown is Figure 4 An example of the image processing unit 211. The image processing unit 211C in this example is... Figure 5 The image processing unit 211A similarly includes a processed image creation unit 2111 and a dynamic image processing unit 2112. Unless otherwise specified, the processed image creation unit 2111 and the dynamic image processing unit 2112 in this example can be respectively connected to... Figure 5 The image processing unit 2111 and the dynamic image processing unit 2112 in the example are the same. The image processing unit 211C also includes a local image determination unit 2113B.

[0191] in addition, Figure 10 The control unit 200A shown is Figure 3 Example of control unit 200. In this example, control unit 200A and... Figure 3 The control unit 200 similarly includes a main control unit 201 and a storage unit 202. Unless otherwise specified, the main control unit 201 and the storage unit 202 in this example can be respectively connected to... Figure 3 The main control unit 201 and storage unit 202 in the example are the same. The control unit 200A also includes a graphical user interface (GUI) control unit 203.

[0192] The main control unit 201 causes the display device 3 to display a first dynamic image generated by the surgical microscope 10 (or a still image included in the first dynamic image, hereinafter the same). The GUI control unit 203 causes the display device 3 to display a GUI for specifying a local area in the first dynamic image. This GUI is, for example, an image of a graphic having a shape and size similar to the area of ​​interest. As an example, if the area of ​​interest is the pupil, the GUI is a circular or elliptical image. The user can change the position, size, and shape of the GUI, for example, by using the operating device 2. Thus, for example, the user can align the GUI with the outer edge of the pupil image in the first dynamic image. When adjusting the GUI for the image of the area of ​​interest, the GUI control unit 203 can track the GUI in a way that matches the movement of the examined eye in the first dynamic image. The local image determination unit 2113B determines the range defined by the GUI (e.g., the range surrounded by the GUI (and a predetermined range of its periphery)) as a local area.

[0193] In this example, the image processing unit 2111 applies first image processing to the local image determined by the local image determination unit 2113B. That is, the scope of application of the first image processing of the image processing unit 2111 in this example is limited to the local image determined by the local image determination unit 2113B.

[0194] Furthermore, the local image determination unit 2113B can, for example, determine a local image in the second dynamic image generated by the surgical microscope 10 after the user selects a processing image, using the same method as the tracking.

[0195] In this example of processing, the local image determination unit 2113B can sequentially determine local images corresponding to the areas of interest in the still images included in the second dynamic image by applying segmentation to the second dynamic image. Furthermore, the image processing unit 210C (dynamic image processing unit 2112) can sequentially apply second image processing to the local images determined based on the still images included in the second dynamic image. Therefore, the values ​​of image parameters applicable to the user-selected local processed image can be applied to the second dynamic image, allowing observation of a second dynamic image where the area of ​​interest is optimally depicted. Additionally, in this example, the range specified by the user is set in the local image, thus allowing the range corresponding to the user's preferences to be represented in a manner appropriate to the user's preferences.

[0196] <Actions and Usage>

[0197] Here are a few examples illustrating the operation and usage of the ophthalmic observation device 1.

[0198] <First example>

[0199] Reference Figures 11-12EThis is the first example illustrating the operation and usage of the ophthalmic observation device 1. Furthermore, any of the described processing, operation, and usage methods can be combined in this example.

[0200] (S1: Begin generating and displaying live images)

[0201] First, the user performs a predetermined operation using the operating device 2, causing the ophthalmic observation device 1 to begin generating and displaying a live image of the examined eye. Specifically, the surgical microscope 10 illuminates the examined eye through the illumination optics system 30 while generating digital image data (image) of the examined eye using the imaging elements 62L and 62R. The generated image (live image 301) is displayed in real time on the display device 3 (see reference). Figure 12A In other words, the image acquired by the surgical microscope 10 is displayed as a live image on the display device 3. The user can perform surgery while observing this live image. This live image is equivalent to the first dynamic image.

[0202] (S2: Shift to image processing mode)

[0203] Next, the user performs a predetermined operation using the operating device 2 to switch the operation mode of the ophthalmic observation device 1 to the image processing mode. The image processing mode is an operation mode used to process the live images displayed by the ophthalmic observation device 1.

[0204] (S3: Capture Frame)

[0205] When the motion mode is switched to the image processing mode, the ophthalmic observation device 1 captures frames of live images (still images).

[0206] The number of frames to be captured is arbitrary. When only one frame is captured, that frame is provided for the following processing. When two or more frames are captured, the ophthalmic observation device 1 (e.g., data processing unit 210) can select one frame from these frames or generate a still image. For example, the ophthalmic observation device 1 calculates the image quality evaluation values ​​of the two or more captured frames, and by comparing these image quality evaluation values, it can select one frame. As another example, the ophthalmic observation device 1 displays the two or more captured frames on the display device 3, and the operation device 2 can select a user-specified frame. Furthermore, as another example, the ophthalmic observation device 1 can create a single frame by synthesizing the two or more captured frames in a predetermined image processing step.

[0207] (S4: First Image Processing)

[0208] The data processing unit 210 (image processing unit 211, processed image production unit 2111) applies first image processing, which uses multiple different values ​​of more than one image parameter, to the frame captured in step S3. Thus, multiple processed images based on the captured frame are produced.

[0209] exist Figure 12B In the example shown, for each frame (still image) 302 captured from the live image 301, K image processes included in the first image processing are applied. I (I types) image parameters (I is an integer greater than or equal to 1) are used in the K image processes. Furthermore, J values ​​(J is an integer greater than or equal to 2) are prepared for each image parameter (the i-th image parameter; i = 1, ..., I). The number J of prepared values ​​can be equal to or different from all the image parameters. In the former case, it is K = I × J. In the latter case, it is K = Σ[J(i)]. Here, J(i) represents the number of values ​​in the i-th image parameter, and Σ is the sum related to i. P(i, j) represents the j-th value of the i-th image parameter. Through this first image processing, multiple processed images 303-k are obtained. Here, k = 1, ..., K, where K is an integer greater than or equal to 2.

[0210] Here, the main control unit 201 or the data processing unit 210 records, for each of the multiple processed images, the values ​​(one or more values) of the image parameters used to create that processed image. For example, in Figure 12B In the example, for each processed image 303-k, one or more values ​​P(i, j) of one or more image parameters used to create that processed image 303-k are associated. The information obtained by associating such processed images with the values ​​of image parameters is called associated information.

[0211] (S5: Create thumbnails for processed images)

[0212] Next, the data processing unit 210 (image processing unit 211) creates a thumbnail of each of the multiple processed images created in step S4.

[0213] (S6: Display multiple thumbnails)

[0214] Next, the main control unit 201 causes the display device 3 to display the multiple thumbnails created in step S5. In this example, such as Figure 12C As shown, multiple thumbnails (thumbnail groups) 303 arranged in a predetermined manner are displayed on the live image 301.

[0215] The number of thumbnails displayed simultaneously is arbitrary. For example, all thumbnails created in step S5 can be displayed at the same time, or a portion of all thumbnails created in step S5 can be displayed (e.g., a predetermined number). Furthermore, the number of thumbnails displayed simultaneously can be changed. When only a portion of all thumbnails created in step S5 is displayed simultaneously, an operation for switching the displayed thumbnails can be performed. This operation can be, for example, page switching or scrolling.

[0216] (S7: Select thumbnail)

[0217] The user compares the multiple thumbnails displayed in step S6 and selects the desired thumbnail. This selection is performed using the indication receiver (e.g., a foot switch included in operating device 2). Figure 12D In the example shown, a thumbnail 304 is selected from a plurality of thumbnails 303.

[0218] (S8: Second Image Processing)

[0219] The main control unit 201 or the data processing unit 210 determines the value of the image parameter associated with the processed image corresponding to the thumbnail selected in step S7 by referring to the associated information.

[0220] Furthermore, the data processing unit 210 (image processing unit 211, dynamic image processing unit 2112) processes the live image generated by the surgical microscope 10 in real time using determined image parameter values ​​(second image processing). The live image processed in real time is displayed in real time on the display device 3 via the main control unit 201 (end). Moreover, the live image generated and displayed in this stage is equivalent to the second dynamic image (…). Figure 12E (Live image 305).

[0221] The user can perform surgery while observing a live image 305 that has undergone the same image processing as the image they selected.

[0222] With the parameter processing unit 212 (recording unit) provided, the values ​​of previously applied image parameters can be read and applied again. At this time, it is possible to selectively read and apply the values ​​of image parameters previously used by the user, and to selectively read and apply the values ​​of image parameters that match the attributes (e.g., type of surgery) of the currently performed medical procedure. Furthermore, it is possible to selectively read and apply the values ​​of image parameters that match the stage of the currently performed medical procedure, and to sequentially reapply multiple values ​​that match each of the multiple stages of the medical procedure.

[0223] In the case of anterior eye surgery, image parameters such as hue, brightness, contrast, and gain are used. This allows users, for example during cataract surgery, to observe the cornea, iris, pupil, and lens in their preferred display mode, and to clearly understand the incision site, lateral incision, anterior capsular incision site in CCC, the emulsification and suction status of the lens, and the position and orientation of the IOL. Furthermore, this optimized image display mode can be easily and quickly implemented.

[0224] In vitrectomy and posterior eye surgeries, image parameters such as gamma, gain, color temperature, white balance, and RGB balance are used. This allows for the highlighting of vitreous floaters, opacities, and thickened membranes, thereby improving surgical efficiency and reducing surgical errors.

[0225] <Second Example>

[0226] Reference Figures 13-14G This is a second example illustrating the operation and usage of the ophthalmic observation device 1. Furthermore, any of the described processing, operation, and usage methods can be combined in this example.

[0227] (S11: Begin generating and displaying live images)

[0228] First, the user performs a predetermined operation using the operating device 2, causing the ophthalmic observation device 1 to begin generating and displaying a live image of the examined eye. Specifically, the surgical microscope 10 illuminates the examined eye through the illumination optics system 30 while generating digital image data (image) of the examined eye using the imaging elements 62L and 62R. The generated image (live image 401) is displayed in real time on the display device 3 (see reference). Figure 14A In other words, the image acquired by the surgical microscope 10 is displayed as a live image on the display device 3. The user can perform surgery while observing this live image. This live image is equivalent to the first dynamic image.

[0229] (S12: Shift to image processing mode)

[0230] Next, the user performs a predetermined operation using the operating device 2 to switch the operation mode of the ophthalmic observation device 1 to the image processing mode. The image processing mode is an operation mode used to process the live images displayed by the ophthalmic observation device 1.

[0231] (S13: Capture Frame)

[0232] When the motion mode is switched to the image processing mode, the ophthalmic observation device 1 captures frames of live images (still images).

[0233] The number of frames to be captured is arbitrary. When only one frame is captured, that frame is provided for the following processing. When two or more frames are captured, the ophthalmic observation device 1 (e.g., data processing unit 210) can select one frame from these frames or generate a still image. For example, the ophthalmic observation device 1 calculates the image quality evaluation values ​​of the two or more captured frames and can select one frame by comparing these image quality evaluation values. As another example, the ophthalmic observation device 1 displays the two or more captured frames on the display device 3, and the operation device 2 can select a user-specified frame. Furthermore, as another example, the ophthalmic observation device 1 can create a single frame by synthesizing the two or more captured frames in a predetermined image processing step.

[0234] (S14: Set local image)

[0235] Next, the ophthalmic observation device 1 (image processing unit 211, local image determination unit 2113A or 2113B) sets a local image of the frame captured in step S13.

[0236] This step can be performed automatically or manually. If performed automatically, Figure 9 The local image determination unit 2113A determines the local image by applying segmentation to the frame captured in step S13 (by...). Figure 14B The image area surrounded by frame 402 is shown. In manual operation, for example, the main control unit 201 causes the display device 3 to display the frame captured in step S13, and the GUI control unit 203 causes a predetermined GUI to be displayed on the frame. The user uses the GUI to specify a local area of ​​the frame. The local image determination unit 2113B determines the range determined by the GUI (e.g., the range surrounded by the GUI (and a predetermined range around it)) as the local area (by...). Figure 14B (The image area surrounded by box 402 shown). Thus, for example, we obtain Figure 14C The partial image 403 is shown.

[0237] (S15: First Image Processing)

[0238] The data processing unit 210 (image processing unit 211, image processing unit 2111) applies a first image processing step using multiple different values ​​of one or more image parameters to the local image obtained in step S14. As a result, multiple locally processed images are created based on this local image.

[0239] exist Figure 14DIn the example shown, K image processes included in the first image processing are applied to local image 403. I (I types) image parameters (I is an integer greater than or equal to 1) are used in the K image processes. Additionally, J values ​​(J is an integer greater than or equal to 2) are prepared for each image parameter (the i-th image parameter; i = 1, ..., I). Furthermore, the number J of prepared values ​​can be equal to or different from all image parameters. In the former case, it is K = I × J. In the latter case, it is K = Σ[J(i)]. Here, J(i) represents the number of values ​​in the i-th image parameter, and Σ is the sum related to i. Q(i, j) represents the j-th value of the i-th image parameter. Through this first image processing, multiple locally processed images 404-k are obtained. Here, k = 1, ..., K, where K is an integer greater than or equal to 2.

[0240] Here, the main control unit 201 or the data processing unit 210 records, in association, the values ​​(one or more values) of the image parameters used to create each of the plurality of local processed images (or plurality of processed images). For example, in Figure 14D In the example, for each local processed image 404-k (or the corresponding processed image), one or more values ​​Q(i, j) of one or more image parameters used to create that local processed image 404-k are associated. The information obtained by associating such a local processed image (or processed image) with the values ​​of image parameters is called associated information.

[0241] (S16: Create thumbnails of locally processed images)

[0242] Next, the data processing unit 210 (image processing unit 211) creates a thumbnail of each of the multiple local processed images created in step S15.

[0243] (S17: Display multiple thumbnails)

[0244] Next, the main control unit 201 causes the display device 3 to display the multiple thumbnails created in step S16. In this example, such as Figure 14E As shown, multiple thumbnails (thumbnail groups) 405 arranged in a predetermined manner are displayed on the live image 401.

[0245] The number of thumbnails displayed simultaneously is arbitrary. For example, all thumbnails created in step S16 can be displayed simultaneously, or a portion of all thumbnails created in step S16 can be displayed simultaneously (e.g., a predetermined number). Furthermore, the number of thumbnails displayed simultaneously can be changed. When only a portion of all thumbnails created in step S16 is displayed simultaneously, an operation for switching the displayed thumbnails can be performed. This operation can be, for example, page switching or scrolling.

[0246] (S18: Select thumbnail)

[0247] The user compares the multiple thumbnails displayed in step S17 and selects the desired thumbnail. This selection is performed using the indication receiver (e.g., a foot switch included in operating device 2). Figure 14F In the example shown, a thumbnail 406 is selected from a plurality of thumbnails 405.

[0248] (S19: Second Image Processing)

[0249] The main control unit 201 or the data processing unit 210 determines the value of the image parameter associated with the local processed image corresponding to the thumbnail selected in step S18 by referring to the associated information.

[0250] Furthermore, the data processing unit 210 (image processing unit 211, dynamic image processing unit 2112) processes the live image generated by the surgical microscope 10 in real time using the determined image parameter values ​​(second image processing). The second image processing is applicable to applications such as... Figure 14B The area of ​​the live image (its frame) corresponding to frame 402 can also be the entire live image (its frame). In the former case, for example, the area of ​​the live image corresponding to frame 402 is determined sequentially by the tracking. The live image processed in real time is displayed in real time on the display device 3 by the main control unit 201 (end). Furthermore, the live image generated and displayed in this stage is equivalent to the second dynamic image. Figure 12E The live image 407 is an example of a live image displayed in this way. In live image 407, the second image processing is applied only to the area corresponding to frame 402 (the area surrounded by frame 408).

[0251] Users can perform surgery while observing a live image 407 that applies the same image processing to the image they selected.

[0252] When the parameter processing unit 212 (recording unit) is provided, the values ​​of previously applied image parameters can be read and applied again. At this time, it is possible to selectively read and apply the values ​​of image parameters previously used by the user, and to selectively read and apply the values ​​of image parameters that match the attributes (e.g., type of surgery) of the currently performed medical procedure. Furthermore, it is possible to selectively read and apply the values ​​of image parameters that match the stage of the currently performed medical procedure, and to sequentially reapply multiple values ​​that match each of the multiple stages of the medical procedure.

[0253] In the case of anterior eye surgery, image parameters such as hue, brightness, contrast, and gain are used. This allows users, for example during cataract surgery, to observe the iris, pupil, and lens in their preferred display mode, and to clearly understand the incision site, lateral incision, anterior capsule incision site in CCC, the emulsification and suction status of the lens, and the position and orientation of the IOL. Furthermore, this optimized image display mode can be easily and quickly implemented.

[0254] In vitrectomy and posterior eye surgeries, image parameters such as gamma, gain, color temperature, white balance, and RGB balance can be used to highlight vitreous floaters, opacities, and thickened membranes, thereby improving surgical efficiency and reducing surgical errors.

[0255] <Variation Example>

[0256] Several variations of the exemplary embodiments described above are explained.

[0257] exist Figure 12C , Figure 12D , Figure 14E , Figure 14F In the example shown, a portion of the image of the eye being examined is hidden by multiple thumbnails (processed image, locally processed image). To eliminate this, multiple thumbnails can be displayed in areas where the image of the eye being examined is not currently displayed. Therefore, examples of possible processing include reducing the display size of the multiple thumbnails, reducing the display size of each thumbnail, changing the arrangement of the multiple thumbnails, and reducing the number of thumbnails displayed simultaneously. Furthermore, by detecting the range of the image of the eye being examined, multiple thumbnails can be displayed in areas other than this range.

[0258] Figure 15 Examples of structures that can be used to achieve such actions are shown. Figure 15 The data processing unit 210C is Figure 3 This is an example of a data processing unit 210. In this example, the data processing unit 210C includes a monitoring processing unit 213 in addition to the image processing unit 211. The image processing unit 211 performs the same structure and operation as in the described embodiment.

[0259] The monitoring processing unit 213 performs data processing (monitoring unit) for monitoring the activities of the examined eye. For example, the monitoring processing unit 213 analyzes frames of dynamic images generated by the surgical microscope 10 to detect feature points of the examined eye. Feature points can be, for example, any feature point such as the pupil (center, outer edge, etc.), corneal ring (outer edge of the iris), anterior chamber angle, optic disc, macula, or blood vessels. The monitoring processing unit 213 detects the movement of feature points (temporal changes in the position of feature points) in the dynamic images in real time by sequentially analyzing frames generated as real-time dynamic images (first dynamic images).

[0260] Based on the output from the monitoring processing unit 213, the main control unit 201 can change the display state of multiple processed images (multiple partial processed images, multiple thumbnails) displayed together with the real-time dynamic image. This display state change control, for example, causes multiple processed images to be displayed in areas where the image of the inspected eye is not currently displayed. For example, the main control unit 201 can perform controls such as reducing the display size of the multiple processed images, reducing the display size of each processed image, changing the arrangement of the multiple processed images, reducing the number of thumbnails displayed simultaneously, and stopping the display of the multiple processed images (controlling not to display the multiple processed images), etc.

[0261] According to this variation, the display state of multiple processed images can be changed dynamically in response to the activity of the eye being examined, so that multiple processed images do not obstruct the observation of the image of the eye being examined. Furthermore, the structure (monitoring unit) for detecting the activity of the eye being examined is not limited to the aforementioned structure or processing, and can be arbitrary.

[0262] Figure 16 Other variations are shown. Figure 15 The variation focuses on the activity of the examined eye; however, this variation focuses on abnormalities occurring in the examined eye.

[0263] Figure 16 The data processing unit 210D is Figure 3 This is an example of a data processing unit 210. In this example, the data processing unit 210D includes an anomaly detection unit 214 in addition to the image processing unit 211. The image processing unit 211 performs the same structure and operation as in the described embodiment.

[0264] The anomaly detection unit 214 performs data processing for detecting anomalies in the examined eye. The type of anomaly being detected can be arbitrary, such as bleeding or wounds. The anomaly detection unit 214 detects anomalies, for example, by analyzing frames of a dynamic image generated by the surgical microscope 10. For example, bleeding detection is performed based on color changes in the image (such as an increase in red areas).

[0265] In several ways, the anomaly detection unit 214 may also include a system (artificial intelligence engine) created through machine learning, wherein the machine learning uses a training dataset including surgical images, etc. This artificial intelligence engine, for example, includes a neural network (typically a convolutional neural network) created by machine learning, which takes surgical images as input and outputs a predetermined probability of an anomaly occurring. In this example, the anomaly detection unit 214 sequentially inputs frames of a real-time dynamic image (first dynamic image) generated by the surgical microscope 10 into the artificial intelligence engine. The artificial intelligence engine sequentially outputs the probability of an anomaly occurring based on the input frames. The anomaly detection unit 214 performs anomaly detection based on the sequentially output anomaly occurring probabilities. For example, if the sequentially output anomaly occurring probability exceeds a predetermined threshold, the anomaly detection unit 214 determines that an anomaly has been detected. As another example, the anomaly detection unit 214 can perform anomaly detection based on changes in the sequentially output anomaly occurring probabilities (e.g., rate of change, amount of change).

[0266] The main control unit 201 can change the display state of multiple processed images (multiple local processed images, multiple thumbnails) displayed together with the real-time dynamic image based on the output from the anomaly detection unit 214. This display state change control can, for example, cause multiple processed images to be displayed in areas where the image of the inspected eye is not displayed, or cause multiple processed images to be displayed in areas other than the areas where anomalies are detected. For example, the main control unit 201 can perform controls such as reducing the display size of the multiple processed images, reducing the display size of each processed image, changing the arrangement of the multiple processed images, reducing the number of thumbnails displayed simultaneously, and stopping the display of the multiple processed images (controlling not to display the multiple processed images), etc.

[0267] According to this variation, the display state of multiple processed images can be dynamically changed in response to an abnormality occurring in the examined eye, so that multiple processed images do not obstruct observation of the image of the examined eye (especially the location of the abnormality). Furthermore, the structure (monitoring unit) for detecting abnormalities in the examined eye is not limited to the aforementioned structure or process, and can be arbitrary.

[0268] The ophthalmic observation device 1 of the described embodiment is configured to apply first image processing to still images (frames) included in a first dynamic image to generate and display a plurality of processed images. Several exemplary embodiments may be configured to sequentially apply first image processing to each still image (frame) included in the first dynamic image to generate and display a plurality of processed images (a plurality of processed dynamic images). In such a manner, the plurality of processed dynamic images can be arranged or displayed sequentially. Furthermore, for at least a portion of the plurality of processed dynamic images, still images (processed still images, processed frames) included in the processed images can also be displayed. According to this method, it is effective when it is necessary to optimize the display mode of the active eye being examined, or when it is necessary to optimize the display mode of objects accompanying the movement (floaters, turbidity, blistering membranes, fluid flow, blood flow, etc.). On the other hand, to perform such processing in real time, a greater amount of resources than those required by the ophthalmic observation device 1 of the described embodiment are needed. Therefore, in order to reduce the processing load when processing dynamic images, any processing load reduction method can be applied, such as reducing the number of generated dynamic images, reducing the number of dynamic images (thumbs up) to be processed, reducing the number of dynamic images (thumbs up) to be displayed, and performing frame culling.

[0269] <Ophthalmic Image Processing Device>

[0270] An ophthalmic image processing apparatus according to an exemplary embodiment is described. The ophthalmic image processing apparatus is configured to process images of the examined eye. Any aspect (function, structure, processing, operation, usage, etc.) related to the ophthalmic observation apparatus 1 of the described embodiment can be combined with the following exemplary ophthalmic image processing apparatus.

[0271] Figure 17 This example illustrates the structure of the ophthalmic image processing device. Figure 17 The elements of the ophthalmic image processing apparatus 500, other than the dynamic image receiving unit 501, can be configured in the same way as the corresponding elements in the ophthalmic observation apparatus 1 of the described embodiment. Unless otherwise specifically mentioned, their detailed descriptions are omitted.

[0272] The control unit 502 of the ophthalmic image processing apparatus 500 corresponds to the control unit 200 of the ophthalmic observation apparatus 1 of the embodiment. The image processing unit 503 of the ophthalmic image processing apparatus 500 corresponds to the data processing unit 210 (image processing unit 211) of the ophthalmic observation apparatus 1 of the embodiment. The user interface (UI) 504 of the ophthalmic image processing apparatus 500 corresponds to the operation device 2 and the display device 3 of the ophthalmic observation apparatus 1 of the embodiment.

[0273] The dynamic image receiving unit 501 receives dynamic images of the examined eye. The dynamic image receiving unit 501 receives dynamic images directly or indirectly from the surgical microscope 10. For example, the dynamic image receiving unit 501 is connected to the surgical microscope 10 via a communication line or cable. Alternatively, the dynamic image receiving unit 501 is connected via a communication line or cable to a device (buffer) that (temporarily) stores dynamic images generated by the surgical microscope 10.

[0274] The motion image receiving unit 501 receives the first motion image of the embodiment described above. The control unit 502 sends the first motion image received by the motion image receiving unit 501 to the image processing unit 503.

[0275] The image processing unit 503 performs the processing described in the embodiment to apply first image processing, which uses multiple different values ​​of predetermined image parameters, to the still images included in the first moving image, and to create multiple processed images.

[0276] The control unit 502 (display control unit) causes the UI 504 (first display device) to display multiple processed images generated by the image processing unit 503.

[0277] In addition to this series of actions, the ophthalmic image processing apparatus 500 can perform the following processing in the same way as the ophthalmic observation apparatus 1 of the described embodiment. The user can give an instruction to select at least one processed image from a plurality of processed images displayed in the UI 504. This instruction can be given in the same manner as in the ophthalmic observation apparatus 1 of the described embodiment, for example, using the UI 504 (instruction receiving unit).

[0278] The image processing unit 503 applies second image processing based on at least one value of image parameters corresponding to at least one processed image specified by the user to the dynamic image (second dynamic image) of the examined eye received by the dynamic image receiving unit 501 after selection based on the user instruction. This processing (second image processing) is performed in the same manner as in the ophthalmic observation device 1 of the described embodiment.

[0279] The control unit 502 (display control unit) causes the UI 504 (second display device) to display a second dynamic image that has been applied to the second image processing.

[0280] According to such an ophthalmic image processing device 500, it can perform the same function and effect as the ophthalmic observation device 1 of the above embodiment. In addition, by combining matters related to the ophthalmic observation device 1 of the above embodiment (function, structure, processing, operation, usage, etc.) with the ophthalmic image processing device 500, the resulting ophthalmic image processing device 500 can perform the function and effect corresponding to the combined matters.

[0281] <Image Processing Methods in Ophthalmology>

[0282] Exemplary embodiments (such as the ophthalmic observation device 1 or the ophthalmic image processing device 500) provide a method for processing ophthalmic images (images of the examined eye). Any aspects related to the ophthalmic observation device 1 of the above embodiments can be combined with the following exemplary ophthalmic image processing method; similarly, any aspects related to the ophthalmic image processing device 500 of the above embodiments can be combined.

[0283] An exemplary ophthalmic image processing method first receives a first dynamic image of the eye being examined. Then, it applies first image processing, using multiple different values ​​of predetermined image parameters, to each still image included in the first dynamic image to create multiple processed images. Next, it displays the created multiple processed images. Then, it receives an instruction to select at least one processed image from the displayed multiple processed images. After selecting at least one processed image based on this instruction, it receives a second dynamic image of the eye being examined. Then, it applies second image processing based on at least one value of image parameters corresponding to the selected at least one processed image to the second dynamic image. Next, it displays the second dynamic image to which the second image processing has been applied. This series of steps constitutes the operation and usage of the ophthalmic observation device 1 of the described embodiment. Figure 11 , Figures 12A-12E , Figure 13 , Figures 14A to 14G (etc.) will be explained.

[0284] According to this ophthalmic image processing method, the same function and effect as the ophthalmic observation device 1 and ophthalmic image processing device 500 of the above embodiments can be achieved. Furthermore, by combining matters related to the ophthalmic observation device 1 or ophthalmic image processing device 500 of the above embodiments with the ophthalmic image processing method, the resulting ophthalmic image processing method can achieve the function and effect corresponding to the combined matters.

[0285] <program>

[0286] An exemplary embodiment provides a program that causes a computer to execute the ophthalmic image processing method. Matters relating to the ophthalmic observation device 1 or the ophthalmic image processing device 500 of the embodiment can be combined with such a program.

[0287] According to such a procedure, the same function and effect as the ophthalmic observation device 1 and ophthalmic image processing device 500 of the described embodiment can be achieved. Furthermore, by combining matters related to the ophthalmic observation device 1 or ophthalmic image processing device 500 of the described embodiment with the procedure, the resulting procedure can achieve the function and effect corresponding to the combined matters.

[0288] <Recording Medium>

[0289] An exemplary embodiment provides a computer-readable, non-transitory recording medium on which the program described above is recorded. Matters relating to the ophthalmic observation device 1 or the ophthalmic image processing device 500 of the described embodiment can be combined with such a recording medium. The non-transitory recording medium can be of any type, including, for example, a magnetic disk, an optical disk, a optical disc, a semiconductor memory, etc.

[0290] According to such a recording medium, it can perform the same functions and effects as the ophthalmic observation device 1 and ophthalmic image processing device 500 of the above embodiments. In addition, by combining matters related to the ophthalmic observation device 1 or ophthalmic image processing device 500 of the above embodiments with the recording medium, the resulting recording medium can perform functions and effects corresponding to the combined matters.

[0291] The described embodiments are merely illustrative of how to implement the present invention. Those who wish to implement the present invention can make any modifications, omissions, additions, substitutions, etc., within the scope of the spirit of the invention.

[0292] (Explanation of reference numerals in the attached diagram)

[0293] 1: Ophthalmic observation device; 2: Operating device; 3: Display device; 10: Surgical microscope; 30: Illumination optical system; 40: Observation optical system; 60: Camera; 200: Control unit; 201: Main control unit; 203: GUI control unit; 210: Data processing unit; 211: Image processing unit; 2111: Processed image production unit; 2112: Dynamic image processing unit; 212: Parameter processing unit; 2121: Parameter storage unit; 2122: Parameter selection unit; 2123: Shooting condition recording unit; 2124: Shooting condition selection unit; 2125: Parameter determination unit; 2113A, 2113B: Local image determination unit; 221: Identifier receiving unit; 222: Attribute information acquisition unit.

Claims

1. An ophthalmic observation device for observing an examined eye, wherein, The ophthalmic observation device includes: The dynamic image generation unit captures the examined eye and generates a first dynamic image; The image processing unit applies first image processing, which uses multiple different values ​​of predetermined image parameters, to each of the still images included in the first dynamic image to create multiple processed images. The display control unit causes the first display device to display the plurality of processing images; and The instruction receiving unit receives an instruction for selecting at least one processing image from the plurality of processing images displayed by the first display device. The image processing unit performs second image processing based on at least one value of the image parameters corresponding to the at least one processed image, applicable to the second dynamic image generated by the dynamic image generation unit after the at least one processed image is selected using the instruction receiving unit. The display control unit causes the second display device to display the second dynamic image that has undergone the second image processing. When the instruction receiving unit selects two or more of the plurality of processed images, the image processing unit determines a value based on two or more values ​​of the image parameters corresponding to the two or more processed images respectively, and applies the image processing using the one value as the second image processing to the second dynamic image.

2. The ophthalmic observation device according to claim 1, wherein, The ophthalmic observation device also includes: A recording unit is used to record the value of the image parameter used for the second image processing.

3. The ophthalmic observation device according to claim 2, wherein, The ophthalmic observation device also includes: The identifier receiving unit receives the user's identifier. The recording unit records the value of the image parameter in association with the identifier received by the identifier receiving unit.

4. The ophthalmic observation device according to claim 2, wherein, The ophthalmic observation device also includes: The attribute information acquisition unit acquires attribute information representing the attributes of the examined eye in relation to the medical procedure. The recording unit records the value of the image parameter in association with the attribute information obtained by the attribute information acquisition unit.

5. The ophthalmic observation device according to claim 2, wherein, The ophthalmic observation device also includes: The selection unit selects at least one value from the image parameter values ​​previously recorded by the recording unit. The image processing unit applies image processing based on the at least one value selected by the selection unit to the third dynamic image generated by the dynamic image generation unit.

6. The ophthalmic observation device according to claim 5, wherein, The recording unit records the shooting conditions when the second moving image is generated in association with the value of the image parameter. The selection unit also selects shooting conditions associated with the at least one value selected by the selection unit. The ophthalmic observation device also includes: The decision unit determines the value of the image parameters based on the at least one value selected by the selection unit and the shooting conditions. The image processing unit applies image processing using the values ​​of the image parameters determined by the decision unit to the third dynamic image.

7. The ophthalmic observation device according to claim 1, wherein, The image processing unit applies the first image processing to a portion (i.e., a local image) of the still image included in the first dynamic image to create multiple local processed images as the multiple processed images. The display control unit causes the first display device to display multiple images, each including one of the multiple partially processed images.

8. The ophthalmic observation device according to claim 7, wherein, The image processing unit includes a first local image determination unit that applies the segmentation of the image used to determine a predetermined region of the examined eye to the still image included in the first dynamic image to determine the local image.

9. The ophthalmic observation device according to claim 8, wherein, The first local image determination unit applies the segmentation to the second dynamic image, and sequentially determines local images of the still images included in the second dynamic image. The image processing unit sequentially applies the second image processing to the local image determined from the still image included in the second dynamic image.

10. The ophthalmic observation device according to claim 7, wherein, The display control unit causes the first display device or the second display device to display the first dynamic image or a still image included in the first dynamic image. The ophthalmic observation device also includes: A graphical user interface for specifying the first animated image or a local region within the still image included in the first animated image. The image processing unit sets the local image based on the local area specified using the user interface.

11. The ophthalmic observation device according to claim 10, wherein, The image processing unit includes a second local image determination unit that sequentially determines local images corresponding to the local regions in the still images included in the second dynamic image. The image processing unit sequentially applies the second image processing to the local image determined from the still image included in the second dynamic image.

12. The ophthalmic observation device according to claim 1, wherein, The display control unit causes the first display device to display two or more of the plurality of processing images or thumbnails of the two or more processing images.

13. The ophthalmic observation device according to claim 1, wherein, The display control unit causes the first display device to sequentially display two or more of the plurality of processing images or thumbnails of the two or more processing images.

14. The ophthalmic observation device according to claim 1, wherein, The ophthalmic observation device also includes: The monitoring unit is used to monitor the activity of the examined eye. The display control unit changes the display status of the plurality of processed images based on the output from the monitoring unit.

15. The ophthalmic observation device according to claim 1, wherein, The ophthalmic observation device also includes: An abnormality detection unit is used to detect abnormalities in the examined eye. The display control unit changes the display status of the plurality of processed images based on the output from the anomaly detection unit.

16. The ophthalmic observation device according to claim 1, wherein, The image parameters include one or more of the following: hue parameter, brightness parameter, contrast parameter, gain parameter, gamma parameter, color temperature parameter, white balance parameter, RGB balance parameter, gray balance parameter, edge enhancement parameter, shadow enhancement parameter, sharpening parameter, and high dynamic range parameter.

17. An ophthalmic image processing apparatus for processing images of an eye being examined, wherein, The ophthalmic image processing device includes: The dynamic image receiving unit receives the first dynamic image of the examined eye; The image processing unit applies first image processing, which uses multiple different values ​​of predetermined image parameters, to each of the still images included in the first dynamic image to create multiple processed images. The display control unit causes the first display device to display the plurality of processing images; and The instruction receiving unit receives an instruction for selecting at least one processing image from the plurality of processing images displayed by the first display device. The image processing unit applies second image processing based on at least one value of the image parameters corresponding to the at least one processed image to the second dynamic image of the examined eye received by the dynamic image receiving unit after the selection of the at least one processed image based on the instruction. The display control unit causes the second display device to display the second dynamic image that has undergone the second image processing. When the instruction receiving unit selects two or more of the plurality of processed images, the image processing unit determines a value based on two or more values ​​of the image parameters corresponding to the two or more processed images respectively, and applies the image processing using the one value as the second image processing to the second dynamic image.

18. An ophthalmic image processing method, wherein, an image of an eye being examined is processed, wherein, Receive the first dynamic image of the examined eye. For the still images included in the first dynamic image, a first image processing method using multiple different values ​​of predetermined image parameters is applied to each to create multiple processed images. Display the multiple processed images. Receive an instruction for selecting at least one processing image from the displayed plurality of processing images. After selecting the at least one processed image based on the instruction, a second dynamic image of the examined eye is received. The second image processing, based on at least one value of the image parameter corresponding to the at least one processed image, is applied to the second dynamic image. Display the second dynamic image that has undergone the second image processing. Upon receiving an instruction to select two or more of the plurality of processed images, a value is determined based on two or more values ​​of the image parameters corresponding to the two or more processed images respectively, and the image processing using the one value is applied as the second image processing to the second dynamic image.

19. A computer-readable, non-transitory recording medium having a program that causes a computer to perform the method of claim 18.

Images