Ophthalmic device, control method of ophthalmic device, and storage medium
By introducing an automatic alignment adjustment and multi-examination protocol control unit into the ophthalmic device, the problem of user operation complexity is solved, and the usability and shooting efficiency of the device are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CANON KK
- Filing Date
- 2021-10-18
- Publication Date
- 2026-06-09
AI Technical Summary
Existing ophthalmic devices require complex alignment adjustments before capturing images, which is inconvenient for users. Furthermore, conditions need to be redefined when retrying the image capture, resulting in low device usability.
An ophthalmic device has been designed, comprising an examination unit, a drive unit, and a control unit. It features automatic alignment and adjustment and allows users to select and execute a series of control processes, including OCT imaging and visible light fundus imaging, by storing multiple examination protocols, thus simplifying the operation process.
It improves the usability of ophthalmic devices, simplifies user operation, reduces the complexity of retrying photography, and improves shooting efficiency and accuracy.
Smart Images

Figure CN114376506B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an ophthalmic device, a control method for the ophthalmic device, and a storage medium. Background Technology
[0002] Devices for obtaining two-dimensional fundus images of the examined eye (hereinafter referred to as fundus camera devices) and devices for obtaining tomographic images of the examined eye using optical coherence tomography (OCT) with low-coherence light (hereinafter referred to as OCT devices) have been put into practical use as ophthalmic devices.
[0003] In these devices, alignment and other adjustments between the device and the examined eye are performed before image capture. In recent years, ophthalmic devices with automated functions that automatically perform these adjustments have been developed. By using these automated functions, users can easily use the ophthalmic device to capture images of the examined eye without complex adjustments.
[0004] Japanese Patent Application Publication No. 2014-39870 discloses an optical image measuring device with the function of acquiring fundus tomographic images and fundus images. The optical image measuring device disclosed in Japanese Patent Application Publication No. 2014-39870 also has automatic imaging operation functions such as automatic imaging and automatic focusing. In the optical image measuring device disclosed in Japanese Patent Application Publication No. 2014-39870, a scanning mode is pre-selected from multiple scanning modes of signal light used for scanning the fundus. When the automatic imaging operation function is enabled, fundus tomographic images and fundus images are automatically acquired based on the selected scanning mode. Summary of the Invention
[0005] According to one aspect of the invention, an ophthalmic device includes: an examination unit configured to examine an eye; a drive unit configured to drive the examination unit; and a control unit configured to initiate control of the examination unit and the drive unit based on an examination protocol in response to predetermined conditions, the examination protocol defining a series of control procedures for performing a plurality of examinations, the series of control procedures including alignment adjustments for aligning the examination unit with the eye being examined, wherein the control unit is configured to display the results of the plurality of examinations and display information on a display unit for accepting instructions to retry a portion of the plurality of examinations.
[0006] Further features of the invention will become clear from the following description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0007] Figure 1 This is a block diagram illustrating a schematic construction example of an ophthalmic device according to a first exemplary embodiment.
[0008] Figure 2 This is a block diagram illustrating a schematic construction example of the control unit of an ophthalmic device according to a first exemplary embodiment.
[0009] Figure 3 This is a flowchart illustrating an example of a measurement process according to a first exemplary embodiment.
[0010] Figure 4 This is a diagram illustrating an example of a camera image according to a first exemplary embodiment.
[0011] Figure 5 This is a diagram showing an example of a result screen according to a first exemplary embodiment.
[0012] Figure 6A This is a diagram illustrating an example of a result screen comprising multiple screens according to a first exemplary embodiment. Figure 6B This is a diagram illustrating another example of a result screen comprising multiple screens according to a first exemplary embodiment.
[0013] Figure 7 This is a diagram showing an example of a result screen according to a first exemplary embodiment.
[0014] Figure 8A This is a diagram illustrating an example of a determined screen according to a first exemplary embodiment. Figure 8B This is a diagram illustrating an example of a determined screen according to a first exemplary embodiment.
[0015] Figure 9 This is a flowchart illustrating an example of a measurement process according to a second exemplary embodiment.
[0016] Figure 10 This is a diagram illustrating an example of a camera image according to a second exemplary embodiment.
[0017] Figure 11A This is a diagram illustrating an example of an anterior eye image obtained in an exemplary embodiment. Figure 11B This is a diagram illustrating an example of an image obtained by transforming an anterior eye image according to an exemplary embodiment.
[0018] Figures 12A to 12D Each of these is a diagram showing an example of a screen display according to a third exemplary embodiment.
[0019] Figure 13 This is a flowchart illustrating an example of an inspection protocol according to a third exemplary embodiment.
[0020] Figure 14 This is a flowchart illustrating an example of a first standby operation according to a third exemplary embodiment.
[0021] Figure 15This is a flowchart illustrating an example of a second standby operation according to a fourth exemplary embodiment.
[0022] Figure 16A and Figure 16B These are examples of output screens showing the variations.
[0023] Figure 17 This is a diagram showing an example of the output screen according to the variant. Detailed Implementation
[0024] Exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Please note that the dimensions, materials, shapes, and relative positions of the components described in the following exemplary embodiments are optional and can be modified depending on the construction of the apparatus to which the exemplary embodiments are applied and various conditions. In the drawings, reference numerals are consistently used to denote the same or functionally similar elements.
[0025] In a first exemplary embodiment, an ophthalmic device for performing both optical coherence tomography (OCT) imaging and visible light fundus imaging will be described as an example of an ophthalmic device according to an exemplary embodiment of the present invention. Conventional optical image measurement devices terminate operation after storing the acquired interferometric image and color fundus image. For example, in cases where imaging will be retried under the same conditions as a series of imaging operations already performed, or where a portion of a series of imaging operations needs to be retried, the user returns to the imaging condition specification step to specify the same conditions again, and then starts the same operation again from scratch, resulting in low availability. This exemplary embodiment aims to improve the usability of the ophthalmic device. The ophthalmic device according to this exemplary embodiment stores multiple examination protocols, each defining a series of control procedures including both OCT imaging and visible light fundus imaging, and the user can select one of the examination protocols and perform control based on the selected examination protocol.
[0026] The ophthalmic device according to this exemplary embodiment displays a fundus image obtained by fundus photography, an OCT image obtained by OCT photography, and a camera button on a display unit. The camera button is an example of display information for accepting instructions to perform at least one of fundus photography and OCT photography. When the camera button is activated, the ophthalmic device according to this exemplary embodiment performs the indicated examination under the same conditions as the examination defined in the selected examination protocol, instead of displaying a screen that prompts the user to select an examination protocol.
[0027] An ophthalmic system according to this exemplary embodiment can be constructed in such a way that the ophthalmic device (composite examination unit) according to this exemplary embodiment is connected to a network, receives commands (written examination instructions) from a doctor's personal computer, examines the eye to be examined based on the commands, and sends the examination results to the doctor's personal computer.
[0028] <Device Structure>
[0029] The ophthalmic apparatus used in this exemplary embodiment includes a fundus camera unit for capturing two-dimensional fundus images and an OCT camera unit for obtaining tomographic images of the examined eye using combined light (interference light), which is obtained by combining reflected light from the examined eye illuminated with measurement light with a reference light. An example of a fundus camera unit will be described using visible light to illuminate the fundus and capture color images; however, a color scanning laser ophthalmoscopy (SLO) camera that scans the fundus with visible light to obtain color fundus images may also be used.
[0030] In the following description, the continuous adjustment and imaging operations performed for photography will be collectively referred to as an examination. The information defining the continuous execution of multiple such examinations will be called an examination protocol. The operations performed during an examination may be measurements of eye characteristics rather than imaging. Examples of measurements include intraocular pressure, axial length, refractive power, corneal curvature, pupil diameter, and wavefront aberration measurements, as well as retinal thickness measurements in OCT examinations.
[0031] The term "protocol" generally refers to a process, system, or rule. In the field of communications, a protocol specifically refers to rules that define the format of information and the process of communication. As used herein, an inspection protocol refers to information that defines the process used to perform multiple inspections. An inspection protocol can define both inspection conditions and inspection procedures. Examples of imaging conditions when performing imaging as an inspection include scanning method, scan range, and the portion to be scanned.
[0032] The following will refer to Figure 1 Describing a first exemplary embodiment, Figure 1 A schematic construction and optical system of an ophthalmic device according to this exemplary embodiment are shown.
[0033] In the following description, the direction that is substantially the same as the line of sight of the examined eye E will be referred to as the Z-direction. The plane perpendicular to the Z-direction will be referred to as the XY plane. The X-direction refers to the horizontal direction, and the Y-direction refers to the vertical direction.
[0034] The ophthalmic device includes, as an example, an optical head unit 100, a spectrometer 200, and as an example, a control unit 300. Although the ophthalmic device is described as having these units housed internally, the control unit 300 and spectrometer 200 may be located externally to the main body of the ophthalmic device. A portion of the optical head unit 100 may be located externally to the ophthalmic device. The construction of the optical head unit 100, spectrometer 200, and control unit 300 will be described sequentially below.
[0035] <Construction of optical head unit 100 and spectrometer 200>
[0036] As an example of an examination unit, the optical head unit 100 includes an optical system for capturing images of the anterior ocular portion Ea of the examined eye E and a measurement optical system for capturing two-dimensional fundus images and three-dimensional tomographic images (OCT images) of the fundus Ef of the examined eye E. Various optical systems included in the optical head unit 100 will be described below.
[0037] The optical head unit 100 includes an objective lens 101 positioned opposite the eye being examined, E. A first dichroic mirror 102, serving as an optical path separation unit, is disposed on the optical axis L1 of the objective lens 101. The first dichroic mirror 102 divides the optical path on the optical axis L1 into an optical path (optical axis L3) for a fundus imaging system and an optical path (optical axis L5) for an OCT interferometer system based on wavelength bands.
[0038] A perforating lens 131, an imaging aperture 132, a focusing lens 133, an imaging lens 134, a third dichroic mirror 135, and an image sensor 136 are arranged on the optical axis L3 along the transmission direction of the first dichroic mirror 102. The perforating lens 131 has an opening at its center. The focusing lens 133 moves along the optical axis L3 to adjust the focus. The third dichroic mirror 135 branches the optical path on the optical axis L3 based on the wavelength band into the optical path guiding the image sensor 136 and the optical path guiding the fixation lamp 137. The image sensor 136 is a fundus image sensor that is sensitive to both visible and infrared light and is used for both moving image observation and still image capture. The fixation lamp 137 generates visible light to prompt the subject to perform visual fixation.
[0039] A corneal baffle 140, a relay lens 141, a focusing index unit 142, a lens 143, and an annular slit 144 are sequentially arranged on the optical axis L4 in the reflection direction of the perforated lens 131. The corneal baffle 140 has a light-blocking point at its center. The annular slit 144 has an annular slit opening. A lens baffle 145 and a second dichroic mirror 146 are arranged on the optical axis L4. The lens baffle 145 serves as a light-blocking member with a light-blocking point. The second dichroic mirror 146 has the characteristics of transmitting infrared light and reflecting visible light. The focusing index unit 142 can be moved along the optical axis L4 and can be inserted into or removed from the optical path on the optical axis L4.
[0040] A condenser lens 147 and a white light-emitting diode (LED) light source 148 are arranged in the reflection direction of the second dichroic mirror 146. The white LED light source 148 is an imaging light source comprising multiple white LEDs emitting pulsed visible light. A condenser lens 149 and an infrared LED light source 150 are arranged in the transmission direction of the second dichroic mirror 146. The infrared LED light source 150 is an observation light source comprising multiple infrared LEDs emitting constant infrared light. The objective lens 101, the second dichroic mirror 146, the optical components disposed between the objective lens 101 and the second dichroic mirror 146, and the condenser lenses 147 and 149 are included in an illumination optical system for illuminating the fundus. The fundus Ef of the examined eye E is illuminated by light from either the white LED light source 148 or the infrared LED light source 150 via the illumination optical system.
[0041] Lens 151, mirror 152, OCT X-scanner 153-1, OCT Y-scanner 153-2, and lenses 154 and 155 are arranged on the optical axis L5 in the reflection direction of the first dichroic mirror 102. Each of the OCT X-scanner 153-1 and OCT Y-scanner 153-2 includes, for example, a mirror and serves as a scanning unit for scanning the fundus Ef of the examined eye E with measurement light. The OCT X-scanner 153-1 and OCT Y-scanner 153-2 are arranged such that the vicinity of their central position becomes the focal point of lens 151. The vicinity of the central position is optically conjugate to the position of the pupil of the examined eye E. This configuration ensures that the optical path with the scanning unit as the object point is substantially parallel between objective lens 101 and lens 151. Therefore, the incident angle of the measurement light incident on the first dichroic mirror 102 located between the objective lens 101 and the lens 151 can remain the same while the OCT X-scanner 153-1 and the OCT Y-scanner 153-2 scan the fundus Ef with the measurement light. The OCT X-scanner 153-1 and the OCT Y-scanner 153-2 scan the measurement light in the main scanning direction and the sub-scanning direction orthogonal to the main scanning direction, respectively. However, the scanning direction is not limited to this.
[0042] The measurement light source 157 is a light source that emits light to obtain the measurement light to be input into the measurement optical path. In this exemplary embodiment, the measurement light in the OCT optical system is emitted from the fiber optic end as the light source. The fiber optic end is optically conjugate with the fundus Ef of the examined eye E. Lens 154 is a lens for focus adjustment and is driven by a motor (not shown) along the optical axis direction indicated by the arrow in the figure. The focus of the measurement light is adjusted in such a way that the measurement light emitted from the fiber optic end, which serves as the light source, forms an image on the fundus Ef. Lens 154, which serves as the focus adjustment unit, is disposed between the fiber optic end, which serves as the measurement light source, and the OCT X scanner 153-1 and OCT Y scanner 153-2, which serve as the scanning units. With such focus adjustment, an image of the measurement light emitted from the fiber optic end can be formed on the fundus Ef of the examined eye E, and the return light from the fundus Ef can be effectively returned to fiber optic 156-2.
[0043] exist Figure 1 In the figure, the optical path between OCT X scanner 153-1 and OCT Y scanner 153-2 is set in the plane of the figure. In fact, the optical path is guided to be orthogonal to the plane of the figure.
[0044] Next, the optical path from the measurement light source 157, the reference optical path, and the spectrometer 200 will be described. The measurement light source 157, optical coupler 156, optical fibers 156-1 to 156-4, lens 158, dispersion compensation glass 159, reference mirror 160, and spectrometer 200 are included in a Michelson interferometer system. Optical fibers 156-1 to 156-4 are single-mode optical fibers integrally connected by optical coupler 156. Light emitted from the measurement light source 157 is guided to optical coupler 156 via optical fiber 156-1. The light guided to optical coupler 156 is split into measurement light in optical fiber 156-2 and reference light in optical fiber 156-3. The measurement light passes through the optical path of the aforementioned OCT optical system to illuminate the fundus Ef of the examined eye E, which is the target of observation. The measurement light is reflected and scattered by the retina and returns to optical coupler 156 again through the same optical path.
[0045] Simultaneously, the reference light reaches the reference mirror 160 via fiber optic cable 156-3, lens 158, and dispersion-compensating glass 159, and is reflected. Dispersion-compensating glass 159 is inserted to match the dispersion of the reference light with that of the measurement light. The reference light reflected by the reference mirror 160 returns via the same optical path and reaches the optical coupler 156 again. The reference light and measurement light (return light) reaching the optical coupler 156 again are combined by the optical coupler 156. If the optical path lengths of the measurement light and the reference light are substantially the same, interference occurs between the measurement light and the reference light due to the combination. The reference mirror 160 is supported such that its position can be adjusted by a motor and drive mechanism (not shown) in the direction of the optical axis indicated by the arrow in the figure. By using the motor and adjustment mechanism, the optical path length of the reference light can be adjusted to the optical path length of the measurement light, which varies depending on the eye being examined, E. The combined light is guided to the spectrometer 200 via fiber optic cable 156-4.
[0046] The spectrometer 200 includes a lens 201, a diffraction grating 202, a lens 203, and a line sensor 204. Combined light emitted from optical fiber 156-4 is converted into substantially parallel light by lens 201, then split by diffraction grating 202 and focused onto line sensor 204 by lens 203. Individual elements of line sensor 204 output signals corresponding to the received light. Control unit 300 samples the signals at predetermined timings using image acquisition unit 304 (described below) and applies predetermined signal processing to the signals to generate a tomographic image.
[0047] Next, the surrounding environment of the measurement light source 157 will be described. In this exemplary embodiment, the measurement light source 157 uses a superluminescent diode (SLD) as a typical low-coherence light source. The light emitted from the measurement light source 157 has a center wavelength of 855 nm and a wavelength bandwidth of approximately 100 nm. Since bandwidth affects the resolution of the tomographic image obtained in the optical axis direction, bandwidth is an important parameter. Although an SLD is chosen as the type of light source here, the measurement light source 157 can be of any type, as long as it can emit low-coherence light. Amplified spontaneous emission (ASE) sources can also be used. Considering eye measurements, the center wavelength of the measurement light is ideally near-infrared. Since the center wavelength affects the lateral resolution of the obtained tomographic image, the center wavelength is ideally as short as possible. For these two reasons, light with a center wavelength of 855 nm is used in this exemplary embodiment.
[0048] Although a Michelson interferometer is used as the interferometer in this exemplary embodiment, a Mach-Zehnder interferometer can also be used. A Mach-Zehnder interferometer is ideally used if the difference in light intensity between the measurement light and the reference light is large. A Michelson interferometer is ideally used if the difference in light intensity is relatively small.
[0049] Stereo cameras 180-1 and 180-2, each including a lens and an image sensor, are respectively positioned on optical axes L6-1 and L6-2, which are different from the optical axis L1. Stereo cameras 180-1 and 180-2 are examples of observation units. For anterior eye observation, stereo cameras 180-1 and 180-2 are arranged substantially symmetrically with respect to the optical axis L1 on the XZ plane of the optical axis L1, and capture stereo images of the anterior eye portion Ea of the examined eye E substantially simultaneously in different directions. The pixel values obtained by stereo cameras 180-1 and 180-2 are output via control unit 300 to display unit 310, which is an example of a display unit. Display unit 310 may be a touch panel, to which the user can input commands by tapping. Here, the user's touch operation on the touch panel will be referred to as a tap. An anterior eye observation light source 125, positioned near the objective lens 101, illuminates the anterior eye portion Ea of the examined eye E.
[0050] The stereo cameras 180-1 and 180-2 are arranged substantially symmetrically with respect to the optical axis L1 on the XZ plane of the optical axis L1. However, the stereo cameras 180-1 and 180-2 can be arranged at a position offset along the Y direction to reduce vignetting effects from eyelashes and eyelids. The optical head unit 100 may include three or more stereo cameras 180 instead of two.
[0051] The optical head unit 100 includes a head drive unit 170, which serves as an example of a drive unit for driving the inspection unit. The head drive unit 170 includes three motors (not shown) and is configured such that the optical head unit 100 can move relative to the examined eye E in three dimensions (X, Y, and Z). This allows for alignment and adjustment of the optical head unit 100 relative to the examined eye E.
[0052] <Structure of Control Unit 300>
[0053] Next, we will refer to Figure 2 A schematic configuration of the control unit 300 is described. The control unit 300 includes a camera control unit 301, a storage unit 302 (as an example of a storage unit), an output control unit 303 (as an example of a notification unit), an image acquisition unit 304, and an image processing unit 305.
[0054] The camera control unit 301 is connected to the storage unit 302, the optical head unit 100, and the input unit 340. A touch panel can be provided, which allows the user to input commands by tapping, and can function as both the display unit 310 and the input unit 340. The camera control unit 301 receives input signals from the input unit 340 and controls the components of the optical head unit 100 based on a check protocol stored in the storage unit 302. The input unit 340 includes a mouse and keyboard (not shown).
[0055] Storage unit 302 stores the examination protocol, the generated image of the examined eye E, the image analysis results, the imaging conditions when the image was acquired, and information about the examined eye E. Storage unit 302 also stores various programs used to control the ophthalmic device.
[0056] According to this exemplary embodiment, an examination protocol is information defining a series of control processes for performing multiple examinations. Here, the examination includes adjustment operations (including alignment adjustments) and operations for capturing images. Multiple examination protocols are pre-stored in storage unit 302. The user selects one of the multiple examination protocols on an examination protocol selection screen (not shown) and issues instructions to execute control based on the selected examination protocol. The processing defined by the examination protocol may include information about imaging conditions such as the scanning mode and the portion to be scanned. The scanning mode and the portion to be scanned will be described below. The examination protocol may include control processes based on commands received via a network (not shown). In this case, the ophthalmic device may receive command information including patient information, examination mode, and portion to be examined via the network, store the command information in storage unit 302, and then send the information about the examination results stored in storage unit 302 to the personal computer that issued the instruction.
[0057] Image acquisition unit 304 is connected to storage unit 302, optical head unit 100, spectrometer 200, and image processing unit 305. Image acquisition unit 304 is also connected to stereo cameras 180-1 and 180-2 in optical head unit 100, and generates an anterior eye image of the examined eye E and sends this anterior eye image to image processing unit 305. Image acquisition unit 304 is also connected to image sensor 136 in optical head unit 100, and generates a fundus image of the examined eye E and sends this fundus image to image processing unit 305. Image acquisition unit 304 is also connected to line sensor 204 in spectrometer 200, and generates a tomographic image of the examined eye E and sends this tomographic image to image processing unit 305.
[0058] The image processing unit 305 processes the anterior eye image, fundus image, and tomographic image obtained by the image acquisition unit 304, and sends the processed image to the storage unit 302. The image processing unit 305 analyzes the anterior eye image, detects the relative position information between the examined eye E and the optical head unit 100, and sends the relative position information to the storage unit 302.
[0059] The output control unit 303 is connected to a display unit 310, such as a monitor, as an example of a display unit. The output control unit 303 can display anterior eye images, fundus images, and tomographic images of the examined eye E, as well as the analysis results obtained by the image processing unit 305, on the display unit 310. The display unit 310 (an example of a display unit) can be a touch panel that allows the user to input information via touch. A user's touch operation on the touch panel is called a tap. The output control unit 303 is connected to an audio output unit 350, such as an audio notification unit, and can provide the user with audio output of the analysis results obtained by the image processing unit 305 and issue warnings.
[0060] The control unit 300 may include a selection unit for selecting one of a plurality of inspection protocols based on user instructions.
[0061] The control unit 300 may include modules executed by a central processing unit (CPU) or a microprocessor unit (MPU), or circuitry that implements specific functions, such as an application-specific integrated circuit (ASIC). The storage unit 302 may be implemented using storage media such as memory and optical discs.
[0062] <Methods for displaying images of front eye movement>
[0063] Next, a method for displaying an anterior eye image of the examined eye E obtained by stereo cameras 180-1 and 180-2 according to this exemplary embodiment will be described.
[0064] Since the stereo cameras 180-1 and 180-2 are positioned on optical axes L6-1 and L6-2, which are different from the optical axis L1, the front eye image obtained by the image acquisition unit 304 is a horizontally distorted image viewed in the direction of the optical axes L6-1 and L6-2 (horizontal lateral direction).
[0065] The image processing unit 305 performs a projection transformation to transform the distorted frontal eye image into an image viewed from the optical axis L1 (from the front) (image transformation unit). The transformed coordinates (x', y') of each pixel in the image can be determined by the following equations (1) and (2) based on the coordinates (x, y) before transformation and the transformation coefficients (a, b, c, d, e, f, g, h):
[0066] as well as
[0067]
[0068] If there are four or more sets of corresponding points obtained before and after the transformation, the transformation coefficients (a, b, c, d, e, f, g, h) can be determined. These transformation coefficients can be determined through calibration during the assembly of the ophthalmic device or at startup.
[0069] Figure 11A An image of the fore-eye obtained by either stereo camera 180-1 or 180-2 is shown. Figure 11B The transformed image of the anterior eye is shown.
[0070] The image processing unit 305 transforms the captured anterior eye image at regular intervals and sends the transformed image to the storage unit 302. The output control unit 303 reads the transformed image from the storage unit 302 at regular intervals and displays the transformed anterior eye motion image on the display unit 310.
[0071] The image to be displayed here as a moving image can be at least one or both of the forearm images obtained by stereo cameras 180-1 and 180-2.
[0072] Figure 11A The pupil 320 in the image looks like a horizontally constricted basic ellipse, making it difficult for users to find pupil states such as pupil diameter. Figure 11B The pupil 321 in the image looks like a nearly perfect circle, and users can easily find pupil states such as pupil diameter.
[0073] When the transformed anterior eye movement image is displayed on the display unit 310, the output control unit 303 superimposes the image as shown in the figure. Figure 11B The circular alignment reference mark 322 (alignment reference mark generation unit) is shown. The position of the alignment reference mark indicates the alignment target position. The size of the circle indicates the minimum pupil diameter used for inspection.
[0074] This display of anterior eye motion images improves operability during the examination, both when the user is in standby and resuming operations based on pupil status, thus enabling the efficient acquisition of stable examination results.
[0075] <Methods for detecting relative position information>
[0076] Next, a method for detecting relative position information of the optical head unit 100 relative to the examined eye E using stereo cameras 180-1 and 180-2 according to this exemplary embodiment will be described.
[0077] The image processing unit 305 calculates the relative position information (position deviation) of the examined eye E and the optical head unit 100 in the X, Y, and Z directions by analyzing the features of the anterior eye image.
[0078] Image processing unit 305 binarizes the anterior eye image with a predetermined threshold and detects the pupil region. Then, image processing unit 305 calculates the centroid position of the detected pupil region. Image processing unit 305 calculates the positional deviation in the X and Y directions based on the difference between the calculated centroid position of the pupil region and a predetermined position in the anterior eye image. Here, the positional deviation in the X and Y directions can be calculated based on at least one or both of the anterior eye images obtained by stereo cameras 180-1 and 180-2.
[0079] Image processing unit 305 calculates the difference between the position of the center of gravity of the pupil region calculated from the front-eye image of stereo camera 180-1 and the position of the center of gravity of the pupil region calculated from the front-eye image of stereo camera 180-2. Then, image processing unit 305 uses the principle of triangulation to calculate the positional deviation and direction in the Z direction based on the difference between the center of gravity positions (parallax), the distance between stereo cameras 180-1 and 180-2, and the focal length.
[0080] Although the centroid of the pupil region is used as a feature of the anterior eye image, the positional deviation can be calculated based on the pupil center position. An index can be projected onto the cornea, and the positional deviation can be calculated based on this index. The anterior eye can be observed using a split prism inserted into the observation optics system.
[0081] <Methods for adjusting operations>
[0082] A method for adjustment operations according to this exemplary embodiment will be described. The ophthalmic device according to this exemplary embodiment performs alignment adjustment, focus adjustment, and coherence gate adjustment as one of the adjustment operations. The adjustment operations may include any adjustments for inspection, and are not limited to the three described above. For example, polarization operations may be performed to optimize OCT output sensitivity.
[0083] The first method described is an alignment adjustment method used in one of the operations of adjusting the optical head unit 100 relative to the examined eye E.
[0084] The camera control unit 301 sends a movement command to the head drive unit 170 to reduce the positional deviation calculated by the image processing unit 305. Then, the head drive unit 170 drives three motors (not shown) to move the position of the optical head unit 100 relative to the examined eye E in three dimensions (X, Y, and Z).
[0085] After the optical head unit 100 moves, the image processing unit 305 acquires an image of the anterior eye and detects the pupil region again. The image processing unit 305 determines whether the pupil of the examined eye E has moved into a pre-set specified range on the display screen. If it is determined that the pupil has moved into the specified range, the alignment adjustment ends. On the other hand, if the pupil of the examined eye E does not fall into the specified range, the aforementioned process is repeated.
[0086] Alternatively, a split prism can be inserted into the observation optics system, and the positional deviation can be calculated based on the split anterior eye image. An alignment index can be projected onto the anterior eye, and the optical head unit 100 can move based on the position where the index is projected. Multiple alignment indices, including coarse and fine alignment indices, can be used for progressive coarse and fine adjustments. Various such alignment adjustment methods can be combined.
[0087] Next, the focus adjustment for the fundus of the examined eye E, as one of the adjustment operations according to this exemplary embodiment, will be described.
[0088] Image processing unit 305 acquires a fundus image and calculates the contrast of the acquired fundus image. Camera control unit 301 moves focusing lens 133 to acquire a fundus image with high contrast. After moving focusing lens 133, image processing unit 305 acquires the fundus image and calculates the contrast again. If the contrast reaches or exceeds a preset reference level, focus adjustment ends. On the other hand, if the contrast falls below the reference level, the aforementioned process is repeated.
[0089] The fundus image used for focus adjustment can be a fundus image obtained through any fundus imaging technique, such as infrared fundus images and SLO (confocal laser scanning method using a near-infrared light source) images. In addition to contrast, methods for calculating the brightness of the entire fundus image or methods for converting the fundus image into frequency can also be used.
[0090] Focus adjustment can be performed using other methods. For example, a sensor for detecting phase difference in an image can be provided, and the optical head unit 100 can be moved based on this phase difference (phase difference autofocus). The camera pixels can be configured to have phase detection functionality. Slits and split prisms can be inserted in the optical path, and the focusing lens 133 can be moved in such a way that the split beams converge again. In the case of OCT imaging, the brightness and position of the two-dimensional tomographic image (OCT image) obtained by OCT imaging can be detected, and the lens 154 for focus adjustment can be moved in such a way that the brightness and position fall within an appropriate range. The above operations can be combined as a focus adjustment.
[0091] In the case of OCT imaging, coherence gate adjustment is further performed as one of the adjustment operations. The image processing unit 305 acquires an OCT image and detects the position of the tomographic image. Based on the position of the tomographic image, the camera control unit 301 drives the reference mirror 160 to adjust the optical path length of the reference light. After adjusting the optical path length, the image processing unit 305 acquires an OCT image again and detects the position of the tomographic image. If the tomographic image falls within a preset area, the coherence gate adjustment ends. If the tomographic image does not fall within this area, the aforementioned process is repeated. In coherence gate adjustment, instead of changing the optical path length of the reference light, a mirror can be inserted into the optical path of the measurement light, and the optical path length of the measurement light can be changed by changing the position of the mirror.
[0092] The aforementioned adjustments can be performed in different orders or simultaneously. For example, when coarse alignment adjustment is completed, fine alignment adjustment and focus adjustment can begin simultaneously. After the adjustments for alignment, focus, and coherence gate adjustment are completed, fine adjustments can be performed by repeating the adjustments for alignment, focus, and coherence gate adjustment.
[0093] <Operational process of a series of checks based on the inspection protocol>
[0094] Reference Figure 3 and Figure 4 This describes the operational process of a series of inspections based on an inspection protocol according to this exemplary embodiment. Here, the case of first performing OCT imaging and then performing visible light fundus imaging will be described as an example of the inspection protocol.
[0095] Before recording, in step S101, a selection unit for selecting an inspection protocol selects an inspection protocol based on user instructions. This selection unit may be included in the control unit 300. The inspection protocol selection screen displays multiple inspection protocols pre-stored in the storage unit 302. Each inspection protocol defines a series of control procedures for performing multiple inspections, including alignment adjustments.
[0096] Users can input commands by tapping on the display information that identifies the inspection protocol on the inspection protocol selection screen (not shown) displayed on the display unit 310. When displaying the information that identifies the inspection protocol, the name of the inspection protocol can be displayed. A camera icon, defined by the inspection protocol, can be displayed to graphically represent the camera conditions.
[0097] The examination protocol may include information such as the camera mode, scanning mode, and whether the examined eye E is the left or right eye. Users can preset the order and number of examinations. The preset order and number of examinations are stored in the storage unit 302. For example, the examination protocol can be selected or set to examine both the left and right eyes consecutively. From the examinations included in the examination protocol, excluding the last examination, the camera that displays the examination results on the display unit 310 after the examination is completed can be selected and set. For example, when control is performed based on the examination protocol used for OCT and fundus photography, after OCT photography, the OCT image can be displayed on the display unit 310, followed by fundus photography.
[0098] Possible imaging modes include fundus imaging, fundus fluorescein imaging, OCT imaging, and anterior eye imaging. Possible scanning modes include macular mode, glaucoma mode, optic disc mode, and OCT angiography (OCTA) mode. When switching scanning modes, the scanning method and the optimal fixation position for that scanning mode are set. Possible OCT scanning methods include three-dimensional (3D) scanning, radial scanning, cross scanning, circular scanning, and raster scanning.
[0099] Among the various inspection protocols, some may include imaging operations with the same imaging mode but different scanning modes, or may include additional imaging conditions. Examples of the multiple differential inspection protocols stored in storage unit 302 are inspection protocols for performing OCT 3D scanning and fundus imaging, and inspection protocols for performing OCT radial scanning and fundus imaging. In this exemplary embodiment, the case where 3D scanning is selected from the OCT scanning methods and fundus imaging is also performed will be described.
[0100] In step S102, the examination protocol selection screen (not shown) is transformed into the camera screen 1000, which serves as an example of the first screen, and the image acquisition unit 304 begins acquiring the anterior eye observation image. Here, the anterior eye observation image 1101, the fundus observation image 1201, and the tomographic image 1301 are displayed on the display unit 310. Figure 4 The camera feed is displayed on screen 1000. Before recording begins, the moving images are displayed as corresponding images 1101, 1201, and 1301.
[0101] The displayed anterior eye view image 1101 can be a corrected image, as if the anterior eye Ea were viewed from the front rather than at an angle. Any one or both of the anterior eye images obtained by stereo cameras 180-1 and 180-2 can be displayed as the anterior eye view image 1101.
[0102] As an example of the first frame, the camera frame 1000 does not necessarily need to simultaneously display the anterior eye observation image 1101, the fundus observation image 1201, and the tomographic image 1301. For example, while the user is adjusting the chin rest, only the anterior eye observation image 1101 can be displayed. The windows used to display the fundus observation image 1201 and the tomographic image 1301 can be blacked out or displayed statically until the image is acquired.
[0103] The camera view 1000, as an example of the first view, may include multiple views that are switched based on the adjustment of the chin rest and the progress of the adjustment operation. For example, only the anterior eye observation image 1101 may be displayed while the user is adjusting the chin rest and while alignment adjustments are being made. The fundus observation image 1201 and the tomographic image 1301 may be displayed while focus adjustments or coherence gate adjustments are being made.
[0104] If the user taps the shutter button 1003 on the camera screen 1000 (example of the first screen, marked "Yes" in step S102), the process proceeds to step S103 to begin control based on the inspection protocol. Although the operation of the shutter button 1003, as an example of a user instruction, is described as a predetermined condition for initiating control based on the inspection protocol, the predetermined condition is not limited to this. For example, control based on the inspection protocol can be initiated by using the output of a non-contact sensor or a contact sensor as a condition, where the non-contact sensor detects the subject approaching the ophthalmic device, and the contact sensor detects the subject's chin resting on a chin rest.
[0105] In step S103, alignment adjustment is performed as one of the adjustment operations. The camera control unit 301 sends a command to the head drive unit 170 to reduce the positional deviation calculated by the image processing unit 305. Then, the head drive unit 170 drives three motors (not shown) to move the position of the optical head unit 100 relative to the eye being examined E in three dimensions (X, Y, and Z). During this operation, the eye being examined E is illuminated with infrared light from the anterior eye observation light source 125. When the alignment adjustment is complete, the final alignment positions of various types of cameras are stored in the storage unit 302.
[0106] Next, focus adjustment is performed as one of the adjustment operations. The image processing unit 305 acquires a fundus image and calculates the contrast of the acquired fundus image. The camera control unit 301 moves the focusing lens 133 in a manner that increases the contrast of the fundus image.
[0107] Then, coherence gate adjustment is performed as one of the adjustment operations. The image processing unit 305 acquires the OCT image and detects the position of the tomographic image. Based on the position of the tomographic image, the camera control unit 301 drives the reference mirror 160 to adjust the optical path length of the reference light.
[0108] The adjustment operations in step S103 can be performed in different orders or simultaneously. For example, after coarse alignment adjustment is completed, fine alignment adjustment and focus adjustment can begin simultaneously. After the adjustment operations for alignment adjustment, focus adjustment, and coherence gate adjustment are completed, fine adjustment can be performed by repeating the adjustment operations for alignment adjustment, focus adjustment, and coherence gate adjustment.
[0109] The ophthalmic device can have the function of manually adjusting focus and alignment by the user without using the aforementioned automatic imaging function. The user moves the position of the optical head unit 100 relative to the examined eye E in the Z direction using slider 1103. The user also adjusts the focus using slider 1203 and performs coherence gate adjustment on the tomographic image 1301 using slider 1302. The user adjusts the scanning range displayed on the fundus observation image 1201. Then, the user presses the capture button 1003 to capture an image. Although the capture button 1003 functions as a button for receiving instructions to perform adjustment operations in the case of automatic imaging, after manual adjustment is completed, the capture button 1003 functions as a button for receiving instructions to acquire an image. When the capture button 1003 is pressed, the camera control unit 301 drives the OCT X scanner 153-1 and OCT Y scanner 153-2 to perform 3D scanning.
[0110] The camera screen 1000 may or may not include a stop button, which is an example of a display information used to pause control based on the inspection protocol. The camera screen 1000 may also include or may not include a restart button, which is an example of a display information used to resume control based on the inspection protocol if control is paused.
[0111] As an example of a first frame, the camera frame 1000 does not necessarily need to simultaneously display the anterior eye observation image 1101, the fundus observation image 1201, and the tomographic image 1301. The camera frame 1000 may include multiple frames that switch as the adjustment of the chin rest and the progress of the adjustment operation proceed. For example, while the user is adjusting the chin rest and while performing an alignment operation, only the anterior eye observation image 1101 may be displayed. The fundus observation image 1201 and the tomographic image 1301 may be displayed while focusing or coherence gate adjustment is being performed.
[0112] In step S104, an image of the examined eye E is captured. The image of the examined eye E is stored in storage unit 302. The image can be captured either after the aforementioned adjustment operation is completed or immediately after a preset countdown time. The ophthalmic device may have the function that allows the user to select one of these settings. Although this exemplary embodiment is described with reference to the case of obtaining an image of the examined eye E as an example of examination, eye characteristics such as refractive power can also be measured.
[0113] After the imaging or measurement is performed in step S104, the imaging control unit 301, which serves as an example of a control unit, moves the optical head unit 100, which serves as an example of an inspection unit, to a predetermined position. The optical head unit 100 is placed in a standby position, which is the position of the optical head unit 100 at the end of the imaging or measurement, or the position that the optical head unit 100 has moved from the position at the end of the imaging or measurement in the Z direction away from the subject.
[0114] Here, the standby position of the optical head unit 100 does not need to be exactly the same as its position in the X and Y directions at the end of the imaging or measurement. The optical head unit 100 can be placed in standby positions within a certain range to reduce the alignment adjustment time for the next inspection compared to the case where the optical head unit 100 is not in standby but returns to its initial position. This control method can be changed by settings, and after imaging or measurement is completed, the optical head unit 100 can be moved to its initial position when powered on.
[0115] Since the optical head unit 100 is placed in standby near the final alignment position after imaging or measurement is completed, the time spent on alignment adjustments for imaging or measurement, as indicated by the result screen described below, can be reduced. To reduce the burden on the subject, the optical head unit 100 can be removed from the subject.
[0116] The camera control unit 301 can be configured to move the position of the optical head unit 100 in the lateral direction, rather than in the aforementioned front-to-back direction, when switching between the left and right eyes for the next inspection. When the last inspection is of the right eye, by placing the optical head unit 100 in standby mode at the position at the end of the inspection or closer to the left eye, the time spent on the next alignment adjustment can be reduced. Conversely, when the last inspection is of the left eye, by placing the optical head unit 100 in standby mode at the position at the end of the inspection or closer to the right eye, the time spent on the next alignment adjustment can be reduced.
[0117] During standby, the relative position information between the examined eye E and the optical head unit 100 is continuously monitored, allowing the ophthalmic device to return to adjustment operations, imaging, or measurement at any time. Furthermore, by controlling the movement of the optical head unit 100 in a manner that allows it to be positioned within a detectable range, the alignment adjustment time for the next examination can be reduced.
[0118] An example of the detectable range of relative position information between the examined eye E and the optical head unit 100 is the range of the anterior eye image that the image processing unit 305 can detect. The range of the anterior eye image that the image processing unit 305 can detect refers to the range within the viewing angle of the observation unit where the anterior eye portion Ea of the examined eye E falls. Ideally, the anterior eye portion Ea of the examined eye E is kept within the viewing angle of the observation unit, thereby enabling rapid re-alignment.
[0119] If the relative position information is determined to be undetectable by the observation unit intended for the anterior eye region Ea of the examined eye E, the audio output unit 350, as an example of a notification unit, can issue a warning to the user. As another example, the display unit 310, as an example of a display unit, can display a warning message. The head drive unit 170 can be driven in such a way that the anterior eye region Ea of the examined eye E remains within the detection range of the observation unit.
[0120] In step S105, the camera control unit 301 determines whether all inspections included in the inspection protocol selected in step S101 have been completed. If not all inspections have been completed ("No" in step S105), the process returns to step S103. Then, the adjustment operation of step S103 and the imaging or measurement of step S104 are performed. For example, if the current control is based on an inspection protocol of performing OCT imaging followed by fundus imaging, and only the OCT imaging has been completed, the process proceeds to the adjustment operation and imaging for the subsequent fundus imaging. If all inspections included in the selected inspection protocol have been completed ("Yes" in step S105), the process proceeds to step S106 to display the inspection results.
[0121] In step S106, the camera control unit 301 displays a result screen as an example of a second screen. The result screen as an example of a second screen includes the results of the inspection included in the inspection protocol selected in step S101 and a camera button as an example of display information for accepting re-inspection instructions for some (parts) of multiple inspections. Figure 5 An example of a results screen is shown. The results screen displays a camera button as an example of displaying information for accepting a re-check instruction and a complete button as an example of displaying information for accepting completion instructions for each check. Therefore, you can select "Complete" or "Camera" on the results screen. Although in Figure 5 In the case of OCT images and fundus images, the display information for accepting re-examination instructions is the camera button. However, if the examination protocol includes refractive power measurement, the measurement button can be displayed as the display information for accepting re-examination instructions.
[0122] A camera button and an image are displayed in association with each other as examples of information displayed to receive re-inspection instructions. The result screen can include multiple screens. For example, screens can be switched using tabs. Icons can be used to display the inspection content, and tapping an icon opens the screen. Screens can be switched using swipe gestures on the touch panel. Figure 6A and Figure 6B An example of the resulting screens can be switched using tabs. Alternatively, a display time can be set for each screen, and the screen can be switched after a predetermined time has elapsed.
[0123] Screens can be generated for each inspection. When a re-inspection command is issued, the process proceeds to the step of adjustment operation or video recording or measurement, and the result screen can be displayed again after the inspection is completed.
[0124] The aforementioned camera and complete button can be in any form. The complete button doesn't always need to be displayed as long as it indicates that all checks have been completed. Checkboxes can be displayed in association with the corresponding images, along with a button for jointly performing the camera operation for the selected items. Figure 7 An example of a results screen including checkboxes is shown. The results screen can be constructed in such a way that completion or recording can be selected by dragging and dropping the icon or image corresponding to the check into a predetermined area.
[0125] The results screen can display the defining criteria of the image, such as numerical values indicating image quality, a status bar, a display showing the location and degree of halos or vignetting effects, and an image obtained by overlaying lines or boxes indicating the scan location onto a frontal fundus image. The results screen may include the ability to read and display the subject's past examination results.
[0126] In step S107, the camera control unit 301 determines whether all inspections have been instructed to be completed based on the status of receiving the re-inspection instruction in step S106. Here, the results of the instructed inspections and information about the inspections can be displayed on the display unit 310. Figure 8A The following confirmation screen is shown, displaying instructions for the corresponding check. Figure 8B A confirmation screen is shown below, displaying the results of the examination that has been instructed to be repeated. Alternatively, the results of completed examinations can be grayed out for distinction. Displaying the confirmation screen is not always necessary. After accepting the user's instruction in step S107, processing can immediately proceed to the examination operation. The ophthalmic device can be configured to inform the user about the next examination via voice guidance.
[0127] In step S107, if all checks included in the selected check protocol are instructed to be completed ("Yes" in step S107), the series of checks ends. Conversely, if any re-check instruction is given ("No" in step S107), the process proceeds to step S108 to perform the indicated check. Here, the obtained results of the indicated checks can be deleted from storage unit 302, and only one or more new results can be stored in storage unit 302. The obtained results of the indicated checks and the results obtained after accepting the instruction can be stored in storage unit 302.
[0128] In step S108, the camera control unit 301 performs an adjustment operation for the indicated inspection. Here, the optical head unit 100 can be driven to the final alignment position stored in the storage unit 302 in step S103. Starting the alignment adjustment from the stored alignment position can reduce the alignment adjustment time.
[0129] After completing step S108, the ophthalmic device performs a re-examination and displays the results screen again. During the re-examination, the ophthalmic device according to this exemplary embodiment performs the examination under the same conditions as the examination defined by the examination protocol selected in step S101. During the examination, the control unit 300 performs control based on the selected examination protocol. The series of examinations ends when all examinations included in the selected examination protocol have been indicated to be completed.
[0130] The ophthalmic device according to the exemplary embodiment can smoothly perform ophthalmic examinations to improve the usability of the ophthalmic device.
[0131] In a second exemplary embodiment, an ophthalmic device that performs both OCT imaging and visible light fundus imaging will be described as an example of an ophthalmic device according to an exemplary embodiment of the present invention. The ophthalmic device according to this exemplary embodiment stores multiple examination protocols, each defining a series of control procedures including both OCT imaging and visible light fundus imaging, and a user can select one of the multiple examination protocols and issue instructions based on the selected examination protocol to perform control.
[0132] The ophthalmic device according to this exemplary embodiment includes a stop button 1004 and a restart button 1005. The stop button 1004 displays information indicating acceptance of a pause control instruction based on a selected examination protocol. The restart button 1005 displays information indicating acceptance of a resumption control instruction if control has been paused. While performing control based on the selected examination protocol, the user can tap the stop button 1004 to pause the operation. By tapping the restart button 1005, the user can resume control based on the selected examination protocol without returning to the examination protocol selection screen.
[0133] The structure of the ophthalmic device, the structure of the control unit, the method for detecting relative position information, and the method for adjustment operation according to the second exemplary embodiment are similar to the structure of the ophthalmic device, the structure of the control unit, the method for detecting relative position information, and the method for adjustment operation according to the first exemplary embodiment. Therefore, further details will be omitted.
[0134] <Operational process of a series of checks based on the inspection protocol>
[0135] Reference Figure 9 and Figure 10 This describes the operational process of a series of checks based on an inspection protocol according to this exemplary embodiment.
[0136] Steps S201, S202, and S203 are similar to steps S101, S102, and S103 according to the first exemplary embodiment, respectively. Therefore, their description will be omitted. The difference between this exemplary embodiment and the first exemplary embodiment is that, in step S204, a stop button 1004 (an example of display information for accepting a pause control instruction) and a restart button 1005 (an example of display information for accepting a resume control instruction) are displayed on the display unit 310.
[0137] When the subject moves unsteadily or when the user determines that the intended examination cannot be performed, the ophthalmic device receives an instruction to pause protocol-based control in order to take images under favorable conditions. When conditions become favorable, the ophthalmic device can quickly perform the desired examination by receiving an instruction to resume protocol-based control without returning to the protocol selection screen.
[0138] In step S204, the camera control unit 301 determines whether to suspend control based on the inspection protocol. If the user taps the stop button 1004, the camera control unit 301 determines to suspend control based on the inspection protocol. Although in Figure 9Step S204 is shown as occurring between the adjustment operation and the camera or measurement, but this processing sequence is not restrictive. The stop button 1004 can accept user input at any step between S202 and S205. Upon tapping the stop button 1004 ("Yes" in step S204), the camera control unit 301 pauses control and processing proceeds to step S206.
[0139] When control is paused, the user can input a command by tapping the restart button 1005. In step S206, if the restart button 1005 is tapped ("Yes" in step S206), the camera control unit 301 resumes control based on the selected inspection protocol and proceeds to step S203. Here, the inspection to be resumed is an inspection under the same inspection conditions as the previously performed inspection. When control is resumed, the camera control unit 301 resumes control from the adjustment operation or camera operation that was in progress when control was paused. In the case of measurement, the camera control unit 301 similarly resumes control from the adjustment operation or measurement operation that was in progress when control was paused. For example, when the inspection protocol defines two camera operations and an adjustment operation is in progress for the first camera operation when control is paused, the camera control unit 301 resumes control from the adjustment operation for the first camera operation.
[0140] The adjustment operation can be resumed from the paused state. The adjustment operation can also be resumed from the stage prior to the paused adjustment operation. The camera control unit 301 can be configured to return to a stage preset by the user and resume control. Since the ongoing inspection protocol is stored in the storage unit 302, the camera control unit 301 can resume control without returning to the inspection protocol selection screen.
[0141] When the optical head unit 100 is in standby mode, relative position information can be continuously detected, similar to the first exemplary embodiment. The ophthalmic device may also include a notification unit that can notify the user of a warning when relative position information is undetectable. The head drive unit 170 can be driven to keep the eye E under examination within the detection range.
[0142] exist Figure 9 In this embodiment, the processing ends after the imaging or measurement is performed in step S205. However, steps S105 to S108 according to the first exemplary embodiment can be added to combine the first exemplary embodiment and the second exemplary embodiment.
[0143] According to this exemplary embodiment, the ophthalmic device can resume a paused examination without returning to the examination protocol selection screen. This improves the usability of the ophthalmic device.
[0144] In the first and second exemplary embodiments, an ophthalmic device capable of performing OCT imaging and fundus imaging to obtain examination results in image form has been described. However, the ophthalmic device can be configured to measure eye characteristics such as refractive power.
[0145] Although ophthalmic devices are described as including control equipment (control units) internally, the control unit capable of controlling the ophthalmic device can be located externally. Examples of such control equipment include personal computers and tablet terminals.
[0146] Furthermore, the control unit may include multiple control units. For example, one control unit may display the examination results on the display unit, while another control unit begins to control the optical unit and the drive unit. Such multiple control units may be arranged inside the ophthalmic device, or they may be arranged in a distributed manner inside and outside the ophthalmic device.
[0147] In a third exemplary embodiment, a description of an ophthalmic device will be given, which places the examination in standby mode for a predetermined time based on the preceding and subsequent examination contents in an examination protocol for performing multiple examinations. In such an ophthalmic device, there may be a situation where, when multiple examinations are performed automatically and continuously, the time spent on the entire series of examinations increases because each examination process is stopped too many times before it begins. There may also be a situation where the examination ends in imaging failure because it is impossible to stop the examination at any time during the examination. In view of this, this exemplary embodiment aims to efficiently perform continuous examinations and obtain stable examination results. The construction of the ophthalmic device according to the third exemplary embodiment, the construction of the control unit, the method for detecting relative position information, and the method for adjusting the operation are similar to the construction of the ophthalmic device according to the first exemplary embodiment, the construction of the control unit, the method for detecting relative position information, and the method for adjusting the operation. Therefore, their detailed description will be omitted. The operation process according to this exemplary embodiment and the operation processes described in the foregoing various exemplary embodiments can be performed at least partially in combination or by substitution without inconsistency.
[0148] Reference Figures 12A to 12D and Figure 13The inspection operation process according to this exemplary embodiment is described. Before recording, the user initially selects an inspection protocol on an inspection protocol selection screen (not shown) displayed on the display unit 310. The inspection protocol selection screen displays multiple inspection protocols pre-stored in the storage unit 302. Each of the multiple inspection protocols defines a series of control processes for performing multiple inspections, including alignment adjustments. On the inspection protocol selection screen, parameters such as the camera mode, camera parameters, and camera conditions (whether the inspected eye E is the left or right eye), the number of inspections, and the inspection order, as defined in the inspection protocol, can also be selected.
[0149] Next, the check protocol selection screen switches to the camera screen, and the image acquisition unit 304 begins acquiring the fore-eye image. Here, Figure 12A The screen 311 shown is displayed on the display unit 310. Screen 311 includes anterior eye image display area 323 (in this state, the anterior eye image is not yet displayed), fundus image display area 324, and tomographic image display area 325. Screen 311 includes an examination start button 326 (start receiving unit) for accepting an examination start command. Before the examination begins, an anterior eye motion image is displayed on the anterior eye image display area 323.
[0150] Next, the subject sits in front of the ophthalmic device. In this position, the user operates an operating unit (not shown) to operate a head drive unit 170 capable of driving the optical head unit 100 in three dimensions (X, Y, and Z). Specifically, the user moves the optical head unit 100 to display a portion of the pupil of the examined eye E on the anterior ocular image display area 323. Here, the examined eye E is illuminated with infrared light from the anterior ocular observation light source 125.
[0151] After a portion of the pupil of the eye being examined, E, is displayed on the anterior eye image display area 323, the user presses the examination start button 326 to begin control based on the examination protocol.
[0152] Figure 13 This is a flowchart of the control based on the inspection protocol performed by the camera control unit 301. Figure 13 An example of an examination protocol is shown for sequentially performing an examination including tomography and fundus photography on the left eye of a subject (first examination), and then sequentially performing an examination including tomography and fundus photography on the right eye (second examination). Operation based on this examination protocol will be described below.
[0153] In step S501, the camera control unit 301 begins control based on the inspection protocol. Processing proceeds to step S502.
[0154] In step S502, the camera control unit 301 aligns the optical head unit 100 with the left eye in the three-dimensional (X, Y, and Z) directions using the aforementioned method (alignment operation). After the alignment of the optical head unit 100 is completed, the process proceeds to step S503.
[0155] In step S503, the image acquisition unit 304 begins acquiring preview motion images of the fundus image and the tomographic image. Based on the acquired motion images, the camera control unit 301 controls the components of the optical head unit 100 to perform camera adjustments. These camera adjustments include focus and exposure adjustments for acquiring the fundus image, and focus and optical path length adjustments for acquiring the tomographic image. Figure 12B The screen 312 shown is displayed on the display unit 310. Screen 312 includes a standby button 327 (standby command receiving unit). The fundus image display area 324 displays fundus motion images, and the tomographic image display area 325 displays tomographic motion images.
[0156] After the camera adjustment is completed, the process proceeds to step S504.
[0157] In step S504, the camera control unit 301 performs tomographic imaging using infrared light under the imaging conditions defined in the inspection protocol. After the imaging is completed, the process proceeds to step S505.
[0158] In step S505, the camera control unit 301 performs fundus imaging using visible light under the imaging conditions defined in the inspection protocol. Before fundus imaging, the camera control unit 301 may perform appropriate image adjustments again. After imaging is completed, the process proceeds to step S506.
[0159] In step S506, the camera control unit 301 controls the head drive unit 170 to move the optical head unit 100 from the left eye to the right eye. The amount of movement can be a predetermined fixed amount or a movement based on past alignment results. After the movement is completed, the process proceeds to step S507.
[0160] In step S507, the camera control unit 301 initiates an alignment operation similar to that in step S502. When the pupil is successfully detected by the image processing unit 305, the process proceeds to step S508. The ophthalmic device can be configured to be user-operable in this operation if the pupil is not successfully detected.
[0161] In step S508, the camera control unit 301 pauses the inspection operation to dilate the pupil of the eye, which had constricted due to the previous inspection using visible light. The following operations are performed during standby. Figure 12CThe screen 313 shown is displayed on the display unit 310. The screen 313 includes a remaining standby time display section 328, a standby cancellation button 329 (standby cancellation switch), and a standby extension button 330 (standby extension switch). If the conditions for canceling standby as described below are met, the process proceeds to step S509.
[0162] In step S509, the camera control unit 301 performs an alignment operation similar to that in step S502. After alignment is completed, the process proceeds to step S510.
[0163] In steps S510 to S512, the camera control unit 301 performs operations similar to those in steps S503 to S505.
[0164] In step S513, a camera result inspection screen (not shown) is displayed on the display unit 310. The inspection process ends.
[0165] exist Figure 13 During the inspection process, the camera operation is described as being performed immediately in steps S504, S505, S511, and S512. However, images can be captured after a predetermined time has elapsed.
[0166] exist Figure 13 During the inspection process, the alignment operation is described as being performed in steps S502, S507, and S509. However, the alignment operation with the left eye can continue during steps S503 to S505. The alignment operation with the right eye can continue during steps S510 to S512.
[0167] exist Figure 13 During the inspection process, the inspection is described as being placed into standby mode in step S508. However, the purpose of standby mode is to temporarily suspend the execution of the camera (steps S511 and S512). Therefore, step S508 for placing the inspection into standby mode can be performed at any time during the alignment operation and camera adjustment (steps S507 to S510).
[0168] <First standby operation process during standby>
[0169] Reference Figure 14 The flowchart describes the standby operation in step S508 of the inspection performed after the inspection using visible light.
[0170] In step S601, the camera control unit 301 suspends the inspection operation and begins standby. Processing proceeds to step S602. The camera control unit 301 sets the standby time and counts down the remaining standby time during steps S602 to S609. The output control unit 303... Figure 12CThe remaining standby time is displayed in the remaining standby time display section 328 on the screen 313 shown. The output control unit 303 uses the audio output unit 350 to issue a notification of the remaining time via voice.
[0171] In step S602, the camera control unit 301 detects... Figure 12C If the standby cancel button 329 on screen 313 shown is pressed (Yes in step S602), the process proceeds to step S610. If the standby cancel button 329 is not pressed (No in step S602), the process proceeds to step S603.
[0172] In step S603, the camera control unit 301 detects... Figure 12C If the standby extension button 330 on screen 313 shown is pressed (Yes in step S603), the process proceeds to step S604. If the standby extension button 330 is not pressed (No in step S603), the process proceeds to step S605.
[0173] In step S604, the camera control unit 301 extends the remaining standby time by a predetermined time. The output control unit 303 updates... Figure 12C The remaining standby time display section 328 on screen 313 shows the time. Processing proceeds to step S605. The remaining time here can be extended by a predetermined fixed value. The amount of remaining time to be extended can be set individually. For example, each time the standby extension button 330 is pressed, the remaining time increases by 5 seconds.
[0174] In step S605, the camera control unit 301 determines whether the remaining time of the standby operation is 0 (whether the standby time has elapsed since the start of standby). If the remaining time of the standby operation is 0 (yes in step S605), the process proceeds to step S610. If the remaining time of the standby operation is not 0 (no in step S605), the process proceeds to step S606.
[0175] The standby time here is determined by the type or parameters of the preceding and subsequent examinations. This is because the state of pupil constriction differs depending on the previous examination. For example, if a lower amount of visible light was used in a previous examination, the pupil of the examined eye E is less likely to constrict. Therefore, the standby time can be determined based on the amount of visible light used in the previous examination.
[0176] The minimum pupil diameter varies depending on the next examination. For example, the minimum pupil diameter for fundus photography is approximately 3.3 mm to 4.0 mm. The minimum pupil diameter for tomography is approximately 2.5 mm. Therefore, the standby time can be determined based on the type of the next examination.
[0177] If less visible light was used in a previous examination and a smaller minimum pupil diameter is required for the next examination, the standby time can be reduced. Standby time can also be varied based on examination parameters such as the examination mode and the number of images to be captured. Standby time can be determined based on measurements obtained by measuring the pupil diameter at least before or after the examination.
[0178] The standby time here can be a value prepared in advance and stored in storage unit 302, or a value read from a condition table stored in storage unit 302. The standby time can be calculated using a formula.
[0179] In step S606, the image processing unit 305 obtains the anterior eye image and performs the aforementioned projection transformation on the anterior eye image. The process then proceeds to step S607.
[0180] In step S607, the image processing unit 305 detects the pupil position (pupil position detection information) using the aforementioned position information detection method. The process then proceeds to step S608.
[0181] In step S608, the image processing unit 305 determines whether the pupil position falls within the pupil detection range based on the pupil position detection information. If the pupil position is outside the pupil detection range ("No" in step S608), the process proceeds to step S609. If the pupil position falls within the pupil detection range ("Yes" in step S608), the process proceeds to step S602. The pupil detection range here can be set to a narrow range close to the center of the image, similar to the range in the aforementioned alignment method, and the alignment operation in step S507 can continue even during standby operation.
[0182] The pupil detection range can be set to the widest possible range on the anterior eye image display area 323, and alignment operations can be performed without losing the pupil's trajectory during standby operation. With this wide alignment range, the optical head unit 100 does not move excessively during standby operation.
[0183] In step S609, the camera control unit 301 controls the optical head unit 100 based on the pupil position detection information obtained in step S607, so as to set the pupil position within the pupil detection range determined in step S608. After the movement is completed, the process proceeds to step S602.
[0184] In step S610, the camera control unit 301 cancels the standby operation and resumes the inspection operation.
[0185] In this exemplary embodiment, an ophthalmic device has been described that, based on previous and subsequent examinations, places examinations in a protocol for performing multiple examinations into standby mode for a predetermined period of time after an examination using visible light. This solves the problem that, when multiple examinations are performed automatically and continuously, each examination is routinely and excessively stopped to wait until the pupil of the examined eye constricts. This increases the time of the entire series of examinations. In this exemplary embodiment, examinations are placed into standby mode based on previous and subsequent examinations. Since examinations are paused only when appropriate, stable examination results can be obtained efficiently.
[0186] <Standby button>
[0187] Next, we will describe the... Figure 12B The standby operation is triggered by the standby button 327 on screen 311. Figure 13 During the inspection process in steps S502 to S512, the standby button 327 can be pressed by the user at any time. When the standby button 327 is pressed, the camera control unit 301 pauses the inspection operation and performs a check. Figure 14 Steps S602 to S609 similarly include a standby operation for the alignment operation. The standby time here can be set by the user before the inspection begins. Figure 12D The standby time setting unit (standby time setting unit) displayed on the setting screen 314 shown in the figure can be set by entering or selecting a value. A predetermined fixed value can be used.
[0188] During standby operation, the display Figure 12C The screen shown is 313, and the user can press the standby extension button 330 and the standby cancel button 329. When the standby extension button 330 is pressed, the remaining standby time is extended by a predetermined time, as in steps S603 and S604. When the standby cancel button 329 is pressed, the standby operation is canceled to resume the check operation.
[0189] When the standby cancel button 329 is pressed after the camera adjustment timer is pressed (step S503 or S510), the standby extend button 330 can be assigned the function of a camera button for immediately capturing an image when pressed.
[0190] If multiple examinations are performed automatically and continuously and the ophthalmic device cannot be stopped at any time during the examination, the imaging may fail due to factors such as blinking of the examined eye E. In this exemplary embodiment, since the ophthalmic device can be stopped at any time, stable examination results can be obtained efficiently.
[0191] The operation process according to the fourth exemplary embodiment is based on an examination protocol for performing multiple examinations, wherein the examination is placed in standby mode for a predetermined time based on the measurement result of the pupil diameter. The construction of the ophthalmic device, the construction of the control unit, the method for detecting relative position information, and the method for adjusting the operation according to the fourth exemplary embodiment are similar to the construction of the ophthalmic device, the construction of the control unit, the method for detecting relative position information, and the method for adjusting the operation according to the first exemplary embodiment. Therefore, further details will be omitted. The operation process according to this exemplary embodiment and the operation processes described in the foregoing various exemplary embodiments can be performed, at least in part, in combination or by substitution, without inconsistency.
[0192] <Methods for measuring pupil diameter>
[0193] A description of the method used by the image processing unit 305 to measure the pupil diameter according to this exemplary embodiment will be given.
[0194] Similar to the method described in conjunction with the operation process of the third exemplary embodiment, the image processing unit 305 acquires an image of the anterior eye, performs a projection transformation on the image, and detects the pupil region. Then, the image processing unit 305 approximates the boundary coordinates of the pupil region with an ellipse and measures the length (minor axis) of the minor axis, which is the pupil diameter.
[0195] <Operation Process>
[0196] The operation process for inspection according to this exemplary embodiment is based on the inspection protocol with the operation process for inspection according to the third exemplary embodiment. Figure 13 Similar inspection protocols exist. Therefore, a detailed description of the inspection protocols will be omitted.
[0197] <Second standby operation process during standby>
[0198] Reference Figure 15 The flowchart describes the standby operation of step S508, which is performed after the inspection using visible light during the operation.
[0199] In step S701, the camera control unit 301 pauses the inspection operation and begins standby operation. Processing proceeds to step S702. The camera control unit 301 sets the standby time and counts down the remaining standby time during steps S702 to S707. The output control unit 303... Figure 12C The remaining standby time is displayed in the remaining standby time display section 328 on the screen 313 shown. The output control unit 303 uses the audio output unit 350 to issue a notification of the remaining time via voice.
[0200] In steps S702 to S705, with Figure 14 The steps S606 to S609 are the same. The camera control unit 301 detects the pupil position from the front eye image, determines whether the pupil position falls within the pupil detection range, and performs an appropriate alignment operation.
[0201] In step S704, if the detected pupil position falls within the pupil detection range (Yes in step S704), the process proceeds to step S706.
[0202] In step S706, the camera control unit 301 measures the pupil diameter (pupil diameter measurement information) using the aforementioned pupil diameter measurement method. The process then proceeds to step S707.
[0203] In step S707, the image processing unit 305 determines whether the measured pupil diameter is greater than or equal to a predetermined pupil diameter based on the pupil diameter measurement information. If the measured pupil diameter is less than the predetermined pupil diameter ("No" in step S707), the process returns to step S702. If the measured pupil diameter is greater than or equal to the predetermined pupil diameter ("Yes" in step S707), the process proceeds to step S708. The criteria for determining the pupil diameter here are based on the type of the next examination and the minimum pupil diameter expected for the next examination.
[0204] In step S708, the camera control unit 301 cancels the standby operation and resumes the inspection operation.
[0205] In this procedure, an ophthalmic device has been described that, based on pupil diameter measurements, places examinations in a standby state within a predetermined time after a visible light examination, according to an examination protocol that involves multiple examinations. This solves the problem that, when performing multiple examinations automatically and continuously, each examination is routinely and excessively stopped to wait until the pupil of the examined eye constricts. This increases the time of the entire series of examinations. In this procedure, because the examination is placed in standby state based on pupil diameter measurements, the examination is only paused when appropriate, thereby efficiently obtaining stable examination results.
[0206] In step S706, the image processing unit 305 can predict the time period until a predetermined pupil diameter is detected based on the measurement results of multiple pupil diameters at different times. For example, the image processing unit 305 calculates an approximate curve from the measurement results of multiple pupil diameters at different times and calculates the time until the predetermined pupil diameter is detected. Then, the image processing unit 305 can count the remaining time period until the predetermined pupil diameter is detected, and output the remaining time using the display unit 310 and / or the audio output unit 350.
[0207] The order of examinations in the examination protocol can be changed based on the pupil diameter measurement result obtained in step S706. For example, if an examination that can be performed with a pupil diameter smaller than the measured pupil diameter is scheduled after an examination placed in standby mode, then an examination that can be performed with a smaller pupil diameter can be performed first.
[0208] As described above, this exemplary embodiment can provide an ophthalmic device capable of efficiently obtaining stable examination results through continuous examination.
[0209] (Variant example: Result output)
[0210] When camera conditions are poor, a re-inspection is performed. However, the image obtained through re-inspection may not be the desired result. In the various exemplary embodiments described above, the image obtained through re-inspection and the image obtained before re-inspection can be displayed adjacent to each other for the user to select.
[0211] Figure 16A and Figure 16B Two examples of displaying the check results list are shown. In the examples, five checks were performed, with check 3 retried once and check 4 retried twice. Figure 16A In the diagram, the number of additional icons indicates the number of re-checks. Figure 16B In the text, the shaded line indicates that one or more re-checks were performed. When the field for check 4 is clicked, the following will be displayed: Figure 17 Images 1701, 1703, and 1705 are shown. A corresponding image from 1701, 1703, and 1705 can be selected by clicking one of selection buttons 1702, 1704, or 1706. A button 1708 can be displayed for issuing editing instructions, and a new image can be generated by clicking button 1708. In the example shown, images 1701 and 1703 are selected by selection buttons 1702 and 1704, generating a combined image 1707. The selected image can be represented by a colored box. The selected button can be displayed in a different color. If the examination is conducted via a network command, the re-examined image and the image before the re-examined image can be sent as examination results to the personal computer that issued the command, and the doctor can make selections on the personal computer's screen.
[0212] As other aspects and optional features of the invention related to the foregoing exemplary embodiments and variations, the following additional notes are disclosed:
[0213] (Additional Note 1-1)
[0214] An ophthalmic device includes: an examination unit configured to examine a subject eye; a drive unit configured to drive the examination unit; a selection unit configured to select an examination protocol from a plurality of different examination protocols based on instructions from a user, each of the plurality of different examination protocols defining a series of control processes for performing a plurality of examinations, the series of control processes including alignment adjustment for aligning the examination unit with the subject eye; and a control unit configured to initiate control of the examination unit and the drive unit based on the selected examination protocol in response to predetermined conditions, wherein the control unit is configured to display the results of the plurality of examinations and display information on a display unit for accepting instructions to perform at least one of the plurality of examinations based on the selected examination protocol.
[0215] (Additional notes 1-2)
[0216] An ophthalmic device includes: an examination unit configured to examine a subject eye; a drive unit configured to drive the examination unit; and a control unit configured to initiate control of the examination unit and the drive unit based on an examination protocol in response to predetermined conditions. The examination protocol defines a series of control procedures for performing multiple examinations, including alignment adjustments for aligning the examination unit with the subject eye. The control unit is configured to display the results of the multiple examinations and display information on a display unit for accepting instructions to retry a portion of the multiple examinations.
[0217] (Additional notes 1-3)
[0218] The ophthalmic device may further include a selection unit configured to select an examination protocol from a plurality of different examination protocols based on instructions from a user. Each of the multiple examination protocols defines a series of control procedures for performing multiple examinations, including alignment adjustments for aligning the examination unit with the eye being examined. The control unit may be configured to initiate control of the examination unit and the drive unit based on the selected examination protocol in response to predetermined conditions.
[0219] (Additional notes 1-4)
[0220] The display unit can be a touch panel, and the displayed information can be a button that the user can tap.
[0221] (Additional notes 1-5)
[0222] The control unit can display the result of one of the multiple tests and the display information for accepting instructions to perform a test on the display unit in a way that is related to each other.
[0223] (Additional notes 1-6)
[0224] The control unit can be configured to display on the display unit a first screen showing the results of multiple checks while an adjustment operation, including alignment adjustments, is in progress, and a second screen showing the results of multiple checks. The second screen may include multiple screens that can be switched via commands from the user.
[0225] (Additional notes 1-7)
[0226] The control unit can display a first screen and a second screen on the display unit. The first screen is displayed when an adjustment operation, including alignment adjustment, is in progress, and the second screen displays the results of multiple checks. The second screen may include multiple screens that can be switched via user commands or after a predetermined time.
[0227] (Additional notes 1-8)
[0228] The control unit can display the result of one of the multiple checks and the display information for accepting instructions to perform a check on one of the multiple screens included in the second screen.
[0229] (Additional notes 1-9)
[0230] A screen can be generated whenever one of the multiple checks ends.
[0231] (Additional notes 1-10)
[0232] The user can select one of the multiple checks included in the inspection protocol, excluding the last check. When the user-selected check is completed, the control unit can display a second screen showing the results of the multiple checks.
[0233] (Additional notes 1-11)
[0234] The control unit can, in response to a user instruction, suspend control of the inspection unit and drive unit based on the inspection protocol, and, in response to a user instruction, resume control of the inspection unit and drive unit based on the inspection protocol when control of the inspection unit and drive unit based on the inspection protocol is suspended.
[0235] (Additional notes 1-12)
[0236] An ophthalmic device includes: an examination unit configured to examine a subject eye; a drive unit configured to drive the examination unit; and a control unit configured to initiate control of the examination unit and the drive unit based on an examination protocol in response to predetermined conditions. The examination protocol defines a series of control procedures for performing multiple examinations, including alignment adjustments for aligning the examination unit with the subject eye. The control unit is configured to suspend control of the examination unit and the drive unit based on the examination protocol in response to a user instruction, and, if control of the examination unit and the drive unit based on the examination protocol is suspended, to resume control of the examination unit and the drive unit based on the examination protocol in response to a user instruction.
[0237] (Additional notes 1-13)
[0238] Upon receiving an instruction to resume control of the inspection unit and drive unit based on the inspection protocol, the control unit can resume control of the inspection unit and drive unit from a suspended inspection among multiple inspections defined by the inspection protocol.
[0239] (Additional notes 1-14)
[0240] Upon receiving an instruction to suspend control of the inspection unit and drive unit based on the inspection protocol, the control unit can put the inspection unit into standby mode at the position it was in when the instruction was received.
[0241] (Additional notes 1-15)
[0242] The ophthalmic device also includes a detection unit configured to detect the relative position information between the examined eye and the examination unit. Even if control of the examination unit and drive unit based on the examination protocol is suspended, the detection unit can continue to detect the relative position information.
[0243] (Additional notes 1-16)
[0244] The ophthalmic device may also include a notification unit configured to issue a warning if the detection unit fails to detect relative position information.
[0245] (Additional notes 1-17)
[0246] The control unit can control the drive unit to keep the eye being examined within the range of the relative position information detected by the detection unit.
[0247] (Additional notes 1-18)
[0248] The predefined conditions can be instructions from the user.
[0249] (Additional notes 1-19)
[0250] The control unit can modify the checks included in the inspection protocol based on instructions from the user.
[0251] (Additional notes 1-20)
[0252] After at least one of the multiple checks is completed, the control unit can place the check unit in a standby position at a predetermined location.
[0253] (Additional notes 1-21)
[0254] The standby position of the inspection unit can be the position at which the inspection unit is located at the end of at least one inspection, or the position to which the inspection unit moves from the position at the end of the at least one inspection along the optical axis of the optical system of the inspection unit in the direction in which the inspection unit moves away from the eye being inspected.
[0255] (Additional notes 1-22)
[0256] When the inspection protocol includes inspections of both the left and right eyes, and the at least one inspection is the last inspection of the right eye, the standby position of the inspection unit can be the position where the inspection unit was at the end of the at least one inspection, or the position where the inspection unit is closer to the left eye than the position where the at least one inspection was at the end of the at least one inspection.
[0257] (Additional notes 1-23)
[0258] The control unit can align the examination unit with both the subject's right and left eyes. The examination protocol can define the examination to be performed on both the right and left eyes.
[0259] (Additional notes 1-24)
[0260] The inspection unit can perform optical coherence tomography (OCT) to obtain information about the characteristics of the examined eye by combining the returned light from the eye illuminated by measurement light with a reference light. The inspection protocol can define imaging conditions under which OCT inspections, including multiple examinations, are performed. The control unit enables the inspection unit and drive unit to perform adjustment operations, including alignment adjustment, focus adjustment, and coherence gate adjustment.
[0261] (Additional notes 1-25)
[0262] The camera conditions can be at least one of the following: scanning method, part to be scanned, and scanning range.
[0263] (Additional notes 1-26)
[0264] An examination protocol can be a protocol that defines the execution of multiple different examinations, including fundus photography using visible light.
[0265] (Additional notes 1-27)
[0266] The examination protocol may be an examination protocol that defines the performance of an OCT examination and fundus photography using visible light. The OCT examination is used to obtain information about the characteristics of the examined eye by using combined light, which is obtained by combining the return light from the examined eye illuminated with measurement light with a reference light.
[0267] (Additional notes 1-28)
[0268] An ophthalmic device includes: an examination unit configured to perform OCT imaging for obtaining a tomographic image of an examined eye and fundus imaging using visible light, the OCT imaging using combined light obtained by combining a return light from the examined eye illuminated by measurement light with a reference light; a driving unit configured to drive the examination unit; a selection unit configured to select imaging conditions from a plurality of imaging conditions associated with OCT imaging based on an instruction from a user; and a control unit configured to issue an instruction to automatically and sequentially perform adjustment operations, OCT imaging and fundus imaging based on the selected imaging conditions, the adjustment operations including alignment adjustment, focus adjustment and coherence gate adjustment, the control unit being configured to display on a display unit an OCT image of the examined eye acquired by OCT imaging based on the selected imaging conditions, a fundus image of the examined eye acquired by fundus imaging, and display information for receiving instructions to perform at least one of OCT imaging and fundus imaging based on the selected imaging conditions.
[0269] (Additional notes 1-29)
[0270] A control method for an ophthalmic device, the ophthalmic device including an examination unit for examining an eye and a drive unit for driving the examination unit, the control method comprising: selecting an examination protocol from a plurality of different examination protocols based on instructions from a user, each examination protocol defining a series of control processes for performing a plurality of examinations, the series of control processes including alignment adjustment for aligning the examination unit with the eye being examined; initiating control of the examination unit and the drive unit based on the selected examination protocol in response to predetermined conditions; and displaying the results of the plurality of examinations and display information on a display unit for accepting instructions to perform at least one of the plurality of examinations based on the selected examination protocol.
[0271] (Additional notes 1-30)
[0272] A control method for an ophthalmic device, the ophthalmic device including an examination unit for examining an eye and a drive unit for driving the examination unit, the control method including initiating control of the examination unit and the drive unit based on an examination protocol in response to predetermined conditions, the examination protocol defining a series of control processes for performing multiple examinations, the series of control processes including alignment adjustment for aligning the examination unit with the eye being examined, and displaying the results of the multiple examinations on a display unit and display information for accepting instructions to retry a portion of the multiple examinations.
[0273] (Additional notes 1-31)
[0274] A control method for an ophthalmic device, the ophthalmic device including an examination unit for examining an eye and a drive unit for driving the examination unit, the control method including initiating control of the examination unit and the drive unit based on an examination protocol in response to predetermined conditions, the examination protocol defining a series of control processes for performing multiple examinations, the series of control processes including alignment adjustment for aligning the examination unit with the eye being examined, pausing control of the examination unit and the drive unit based on the examination protocol in response to a user instruction, and resuming control of the examination unit and the drive unit based on the examination protocol in response to a user instruction when control of the examination unit and the drive unit based on the examination protocol is paused.
[0275] (Additional notes 1-32)
[0276] A control method for an ophthalmic device, the device comprising an examination unit and a drive unit, the examination unit performing OCT imaging for obtaining a tomographic image of an examined eye and fundus imaging using visible light, the OCT imaging using combined light obtained by combining a return light from the examined eye illuminated by measurement light with a reference light, the drive unit driving the examination unit, the control method comprising selecting imaging conditions from a plurality of imaging conditions associated with OCT imaging based on instructions from a user, issuing instructions to automatically and sequentially perform adjustment operations, OCT imaging and fundus imaging based on the selected imaging conditions in response to predetermined conditions, the adjustment operations including alignment adjustment, focus adjustment and coherence gate adjustment, and displaying on a display unit an OCT image of the examined eye acquired by OCT imaging based on the selected imaging conditions, a fundus image of the examined eye acquired by fundus imaging, and display information for receiving instructions to perform at least one of OCT imaging and fundus imaging based on the selected imaging conditions.
[0277] (Additional notes 1-33)
[0278] The program enables computers to control ophthalmic devices.
[0279] (Additional note 2-1)
[0280] An ophthalmic device includes: an examination unit configured to examine an eye; an alignment unit configured to align the examination unit with the eye; a position information detection unit configured to detect relative position information between the eye and the examination unit; a storage unit configured to store an examination sequence defining a series of control processes for performing multiple examinations, the series of control processes including an alignment operation for aligning the examination unit with the eye; and a control unit configured to control the alignment of the alignment unit based on the relative position information and to control the examination unit to perform multiple examinations based on the stored examination sequence, wherein the control unit is configured to place the examination unit in standby mode for a predetermined time after performing a first examination using visible light among multiple pending examinations and before performing a second examination after the first examination.
[0281] (Additional note 2-2)
[0282] An ophthalmic device includes: an examination unit configured to optically examine an eye; an observation unit including at least two camera units configured to capture images of the anterior portion of the eye in a direction different from the optical axis of the examination unit; an alignment unit configured to align the examination unit with the eye; a transformation unit configured to transform images captured by each of the at least two camera units into images captured in the direction of the optical axis of the examination unit; and a control unit configured to display alignment reference marks superimposed on the transformed images, the alignment reference marks serving as a reference when aligning the examination unit with the eye.
[0283] (Additional notes 2-3)
[0284] Alignment reference marks can be marks indicating the size of the pupil diameter required for the examination.
[0285] (Additional notes 2-4)
[0286] The ophthalmic device may further include: a driving unit configured to drive an alignment unit; a position information detection unit configured to detect relative position information between the examined eye and the examination unit based on the output of the observation unit; a storage unit configured to store an examination sequence defining a series of control procedures for performing multiple examinations, the series of control procedures including an alignment operation for aligning the examination unit with the examined eye; a start receiving unit configured to receive an instruction to start the stored examination sequence; and a control unit configured to control the examination unit to perform multiple examinations, wherein, in response to an instruction, the control unit is configured to control the driving unit based on the relative position information and to control the examination unit to perform the examination based on the stored examination sequence, and after performing a first examination using visible light among multiple pending examinations, the examination unit is placed in standby mode for a predetermined time before performing a second examination after the first examination.
[0287] (Additional notes 2-5)
[0288] An ophthalmic device includes: an observation unit configured to observe the anterior eye of a subject's eye; an examination unit configured to examine the eye; an alignment unit configured to align the examination unit with the eye; a drive unit configured to drive the alignment unit; a position information detection unit configured to detect relative position information between the eye and the examination unit based on the output of the observation unit; a storage unit configured to store an examination sequence defining a series of control procedures for performing multiple examinations, the series of control procedures including an alignment operation for aligning the examination unit with the eye; a start receiving unit configured to receive an instruction to start the stored examination sequence; and a control unit configured to, in response to an instruction, control the drive unit based on the relative position information and control the examination unit based on the stored examination sequence to perform the examination, wherein the control unit is configured to place the examination unit in standby mode for a predetermined time after a first examination using visible light among multiple pending examinations, and before a second examination following the first examination.
[0289] (Additional notes 2-6)
[0290] The scheduled time can be a standby time, which varies depending on at least one of the types of the first and second checks and the check parameters.
[0291] (Additional notes 2-7)
[0292] The examination parameters can be information about the amount of visible light to be used in the first examination and the minimum pupil diameter for the second examination.
[0293] (Additional notes 2-8)
[0294] The control unit can be configured to place the inspection unit in standby mode in front of the eye being inspected for the second inspection.
[0295] (Additional notes 2-9)
[0296] The ophthalmic device may include a pupil diameter measuring unit configured to measure the pupil diameter of the examined eye based on an image captured by one of at least two imaging units. The control unit may be configured to perform a second examination if the pupil diameter of the examined eye measured by the pupil diameter measuring unit is a predetermined pupil diameter.
[0297] (Additional notes 2-10)
[0298] The ophthalmic device may further include: a prediction unit configured to predict the time until the pupil diameter of the examined eye reaches a predetermined pupil diameter based on the output of the pupil diameter measurement unit; a counting unit configured to count the remaining time until the pupil diameter of the examined eye reaches the predetermined pupil diameter; and a notification unit configured to notify of the remaining time counted by the counting unit.
[0299] (Additional Note 2-11)
[0300] The control unit can be configured such that, in the case where the pupil diameter of the eye being examined, as measured by the pupil diameter measuring unit during standby, is smaller than a predetermined pupil diameter, and the examination performed with the smaller pupil diameter is scheduled after the examination currently in standby, the examination performed with the smaller pupil diameter is performed first.
[0301] (Additional Note 2-12)
[0302] An ophthalmic device includes: an examination unit configured to examine an eye; an alignment unit configured to align the examination unit with the eye; a position information detection unit configured to detect relative position information between the eye and the examination unit; a storage unit configured to store an examination sequence defining a series of control processes for performing multiple examinations, the series of control processes including an alignment operation of aligning the examination unit with the eye; a control unit configured to control the alignment unit to perform alignment based on the relative position information and to control the examination unit to perform examinations based on the stored examination sequence; and a receiving unit configured to receive instructions to pause the operation and to place the operation in standby mode for a predetermined time while the examination sequence is running.
[0303] (Additional note 2-13)
[0304] The ophthalmic device may also include a setting unit configured to set a predetermined time.
[0305] (Additional Note 2-14)
[0306] The control unit can be configured to place a standby inspection unit in front of the eye being inspected, which is scheduled to be aligned next or is currently in alignment processing.
[0307] (Additional Note 2-15)
[0308] The ophthalmic device may also include an extension switch configured to give a command to extend the standby time by a predetermined time. Each time the extension switch is pressed, the standby time can be extended by the predetermined time.
[0309] (Additional note 2-16)
[0310] The ophthalmic device may also include a cancel switch configured to issue a command to cancel standby.
[0311] (Additional note 2-17)
[0312] The examination unit is a composite examination unit constructed to perform multiple different examinations. Examinations using visible light can be those that illuminate the fundus of the examined eye and obtain fundus images of the examined eye.
[0313] (Additional note 2-18)
[0314] The ophthalmic device may further include: a counting unit configured to count the remaining time of standby time; and a notification unit configured to issue a notification of the remaining time counted by the counting unit.
[0315] (Additional note 2-19)
[0316] The notification unit can be a display unit configured to show the remaining time.
[0317] (Additional notes 2-20)
[0318] The notification unit can be a unit configured to issue a notification of the remaining time via voice.
[0319] (Additional Note 2-21)
[0320] The position information detection unit can be configured to continue detecting relative position information during standby. The control unit can be configured to control the alignment unit to keep the position of the eye being detected within the detection range of the position information detection unit when the position of the eye being detected approaches the limit of the detection range of the position information detection unit.
[0321] (Additional Note 2-22)
[0322] A control method for an ophthalmic device, the ophthalmic device including an examination unit for examining an eye and an alignment unit for aligning the examination unit with the eye, the control method including: detecting relative position information about the eye and the examination unit; storing an examination sequence defining a series of control procedures for performing multiple examinations, the series of control procedures including an alignment operation for aligning the examination unit with the eye; controlling the alignment unit to perform alignment based on the relative position information, and controlling the examination unit to perform examinations based on the stored examination sequence, wherein, after performing a first examination using visible light among multiple pending examinations, the examination unit is placed in standby mode for a predetermined time before performing a second examination after the first examination.
[0323] (Additional note 2-23)
[0324] A method for controlling an ophthalmic device comprising at least two camera units, each camera unit configured to capture an image of the anterior segment of the eye in a direction different from the optical axis of an examination unit configured to perform an optical examination on the eye being examined, the control method comprising: aligning the examination unit with the eye being examined; transforming the images captured by each of the at least two camera units into images captured in the direction of the optical axis of the examination unit; and displaying alignment reference marks superimposed on the transformed images, the alignment reference marks serving as a reference during alignment.
[0325] (Additional notes 2-24)
[0326] A method for controlling an ophthalmic device, the ophthalmic device comprising: an observation unit configured to observe the anterior portion of an eye of a subject; an examination unit configured to examine the eye; an alignment unit configured to align the examination unit with the eye; a drive unit configured to drive the alignment unit; a position information detection unit configured to detect relative position information between the eye and the examination unit based on the output of the observation unit; a storage unit configured to store an examination sequence defining a series of control procedures for performing multiple examinations, the series of control procedures including an alignment operation for aligning the examination unit with the eye; a start receiving unit configured to receive an instruction to begin the stored examination sequence; and a control unit configured to, in response to an instruction, control the drive unit based on the relative position information and control the examination unit to perform examinations based on the stored examination sequence, the method comprising placing the examination unit in standby mode for a predetermined time after a first examination using visible light in a plurality of pending examinations and before a second examination following the first examination.
[0327] (Additional note 3-1)
[0328] An ophthalmic device configured to automatically perform multiple examinations on an eye using an examination unit, comprising: an examination device configured to sequentially perform multiple examinations using the examination unit; a receiving unit configured to receive a specific instruction between a first examination and a second examination; and a control unit configured to control the examination device based on the instruction.
[0329] (Additional note 3-2)
[0330] The ophthalmic device may further include: an alignment unit configured to align the examination unit with the eye being examined; a position information detection unit configured to detect relative position information between the eye being examined and the examination unit; and a storage unit configured to store an examination sequence defining a series of control procedures for performing multiple examinations, the series of control procedures including an alignment operation for aligning the examination unit with the eye being examined. The control unit may be configured to control the alignment unit to align based on the relative position information and to control the examination unit to perform examinations based on the stored examination sequence, and to place the examination unit in standby mode after a first examination using visible light among multiple pending examinations and before a second examination following the first examination.
[0331] (Additional note 3-3)
[0332] The ophthalmic device may further include: an observation unit comprising at least two camera units, each camera unit configured to capture an image of the anterior segment of the examined eye in a direction different from the optical axis of the examination unit; an alignment unit configured to align the examination unit with the examined eye; a transformation unit configured to transform the image captured by each of the at least two camera units into an image captured in the direction of the optical axis of the examination unit; and an output control unit configured to display alignment reference marks superimposed on the transformed image, the alignment reference marks serving as a reference when aligning the examination unit with the examined eye.
[0333] (Additional notes 3-4)
[0334] A method for controlling an ophthalmic device configured to automatically perform multiple examinations on an eye using an examination unit, the method comprising: performing multiple examinations sequentially using the examination unit, receiving specific instructions between a first examination and a second examination, and controlling the examination device based on the instructions.
[0335] (Additional notes 3-5)
[0336] The ophthalmic device further includes: a drive unit configured to drive an examination unit; and a selection unit configured to select an examination protocol from multiple different examination protocols based on instructions from a user. Each examination protocol defines a series of control procedures for performing multiple examinations, including alignment adjustments for aligning the examination unit with the examined eye. The control unit may be configured to initiate control of the examination unit and the drive unit based on the selected examination sequence in response to predetermined conditions, and to display the results of multiple examinations and display information on a display unit for accepting instructions to perform at least one of the multiple examinations based on the selected examination sequence.
[0337] Other embodiments
[0338] The present invention can also be implemented as follows: a computer of a system or apparatus that reads and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more fully referred to as a "non-transitory computer-readable storage medium") to perform one or more functions of the above-described embodiments, and / or, the system or apparatus includes one or more circuits (e.g., application-specific integrated circuits (ASICs)) for performing one or more functions of the above-described embodiments; and a method executed by the computer of the system or apparatus, for example, reading and executing the computer-executable instructions from the storage medium to perform one or more functions of the above-described embodiments, and / or, controlling the one or more circuits to perform one or more functions of the above-described embodiments. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessor unit (MPU)) and may include a network of separate computers or separate processors to read and execute the computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or storage medium. For example, storage media may include one or more of the following: hard disk, random access memory (RAM), read-only memory (ROM), memory of a distributed computing system, optical disc (e.g., compact disc (CD), digital versatile optical disc (DVD), or Blu-ray disc (BD)). TM ), flash memory devices, memory cards, etc.
[0339] The embodiments of the present invention can also be implemented by providing the software (program) that performs the functions of the above embodiments to the system or device via a network or various storage media, and the computer or central processing unit (CPU) or microprocessor unit (MPU) of the system or device reads and executes the program.
[0340] While the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.
Claims
1. An ophthalmic device comprising: An examination unit, configured to examine the eye being examined; A driving unit, configured to drive the inspection unit; The selection unit is configured to select an inspection protocol from a plurality of different inspection protocols based on instructions from the user. Each of the plurality of different inspection protocols defines a series of control processes for performing a plurality of inspections. The series of control processes includes alignment adjustments for aligning the inspection unit with the eye being inspected. as well as The control unit is configured to initiate control of the inspection unit and the drive unit based on a selected inspection protocol in response to predetermined conditions. The control unit displays the results of the plurality of checks and display information on the display unit for accepting instructions to retry a portion of the plurality of checks.
2. The ophthalmic device according to claim 1, in, The display unit is a touch panel, and The displayed information includes buttons that are tapped by the user.
3. The ophthalmic device according to claim 1, wherein, The control unit displays, in association with each other, the result of one of the plurality of checks and display information for accepting instructions to perform said check on the display unit.
4. The ophthalmic device according to claim 1, in, The control unit displays a first screen and a second screen on the display unit. The first screen is displayed while an adjustment operation, including alignment adjustment, is in progress. The second screen displays the results of the plurality of checks. The second screen includes multiple screens that can be switched by user commands or after a predetermined time.
5. The ophthalmic device according to claim 4, wherein, The control unit displays the result of one of the multiple screens included in the second screen, along with display information for accepting instructions to perform the one check, on one of the multiple check results.
6. The ophthalmic device according to claim 5, wherein, A screen is generated whenever one of the multiple checks ends.
7. The ophthalmic device according to claim 4, in, The user selects one of the multiple checks included in the inspection agreement, excluding the last check, and When the checks selected by the user are completed, the control unit displays the second screen, which shows the results of the multiple checks.
8. The ophthalmic device according to claim 1, wherein, The control unit, in response to a user instruction, suspends control of the inspection unit and drive unit based on the inspection protocol, and, in response to a user instruction, resumes control of the inspection unit and drive unit based on the inspection protocol when control of the inspection unit and drive unit based on the inspection protocol is suspended.
9. The ophthalmic device according to claim 8, wherein, Upon receiving an instruction to resume control of the inspection unit and drive unit based on the inspection protocol, the control unit resumes control of the inspection unit and drive unit from the suspended inspection among the plurality of inspections defined by the inspection protocol.
10. The ophthalmic device according to claim 8, wherein, When the control unit receives an instruction to pause the control of the inspection unit and the drive unit based on the inspection protocol, it puts the position of the inspection unit at the time the instruction was received into standby mode.
11. The ophthalmic device according to claim 8, further comprising: A detection unit, configured to detect relative positional information between the examined eye and the detection unit. The detection unit continues to detect the relative position information even when the control of the inspection unit and the drive unit based on the inspection protocol is suspended.
12. The ophthalmic device of claim 11, further comprising: A notification unit is configured to issue a warning if the detection unit fails to detect the relative position information.
13. The ophthalmic device according to claim 11, wherein, The control unit controls the drive unit to keep the eye being examined within the range of the relative position information detected by the detection unit.
14. The ophthalmic device according to claim 1, wherein, The predetermined conditions include instructions from the user.
15. The ophthalmic device according to claim 1, wherein, The control unit modifies the checks included in the inspection protocol based on instructions from the user.
16. The ophthalmic device according to claim 1, wherein, After at least one of the plurality of checks is completed, the control unit places the check unit in a standby position at a predetermined location.
17. The ophthalmic device according to claim 16, wherein, The standby position of the inspection unit is the position where the inspection unit is at the end of the at least one inspection, or the position that the inspection unit moves from the position where the inspection unit is at the end of the at least one inspection along the optical axis of the optical system of the inspection unit in the direction in which the inspection unit moves away from the eye being inspected.
18. The ophthalmic device according to claim 16, wherein, When the inspection protocol includes inspection of both the left and right eyes and the at least one inspection is the last inspection of the right eye, the standby position of the inspection unit is the position of the inspection unit at the end of the at least one inspection, or the position of the inspection unit is closer to the left eye than the position of the inspection unit at the end of the at least one inspection.
19. The ophthalmic device according to claim 1, in, The control unit aligns the examination unit with the subject's right and left eyes, and The inspection protocol defines that inspections are performed on both the right and left eyes.
20. The ophthalmic device according to claim 1, in, The examination unit performs optical coherence tomography (OCT) to obtain information about the characteristics of the examined eye by combining the returned light from the eye illuminated by the measurement light with a reference light. The inspection protocol defines the imaging conditions for performing optical coherence tomography (OCT) examinations, which are included in the plurality of inspections. The control unit enables the inspection unit and the drive unit to perform adjustment operations including alignment adjustment, focus adjustment and coherence gate adjustment.
21. The ophthalmic device according to claim 20, wherein, The imaging conditions are at least one of the scanning method, the part to be scanned, and the scanning range.
22. The ophthalmic device according to claim 1, wherein, The examination protocol defines both optical coherence tomography (OCT) and fundus photography using visible light. The OCT is used to obtain information about the characteristics of the examined eye by using combined light, which is obtained by combining the returned light from the examined eye illuminated with measurement light with a reference light.
23. An ophthalmic device comprising: An examination unit, configured to examine the eye being examined; The drive unit is configured as a drive checking unit; The selection unit is configured to select an inspection protocol from a plurality of different inspection protocols based on instructions from the user. Each of the plurality of different inspection protocols defines a series of control processes for performing a plurality of inspections. The series of control processes includes alignment adjustments for aligning the inspection unit with the eye being inspected. as well as The control unit is configured to initiate control of the inspection unit and the drive unit based on a selected inspection protocol in response to predetermined conditions. The control unit suspends control of the inspection unit and the drive unit based on the inspection protocol in response to a user's instruction, and resumes control of the inspection unit and the drive unit based on the inspection protocol in response to a user's instruction when control of the inspection unit and the drive unit based on the inspection protocol is suspended.
24. A control method for an ophthalmic device, the ophthalmic device comprising an examination unit for examining an eye and a drive unit for driving the examination unit, the control method comprising: Based on instructions from the user, an inspection protocol is selected from multiple different inspection protocols, each of which defines a series of control processes for performing multiple inspections, including alignment adjustments for aligning the inspection unit with the eye being inspected. In response to predetermined conditions, control of the inspection unit and drive unit is initiated based on the selected inspection protocol; as well as The results of the plurality of checks and display information for accepting instructions to retry a portion of the plurality of checks are displayed on the display unit.
25. A control method for an ophthalmic device, the ophthalmic device comprising an examination unit for examining an eye and a drive unit for driving the examination unit, the control method comprising: Based on instructions from the user, an inspection protocol is selected from multiple different inspection protocols, each of which defines a series of control processes for performing multiple inspections, including alignment adjustments for aligning the inspection unit with the eye being inspected. In response to predetermined conditions, control of the inspection unit and drive unit is initiated based on the selected inspection protocol; In response to a user command, control of the inspection unit and drive unit based on the inspection protocol is suspended; as well as If control of the inspection unit and drive unit based on the inspection protocol is suspended, control of the inspection unit and drive unit based on the inspection protocol is resumed in response to a user instruction.
26. A non-transitory computer-readable storage medium storing a program for causing a computer to perform the control method according to claim 24.
27. A non-transitory computer-readable storage medium storing a program for causing a computer to perform the control method according to claim 25.