Ophthalmological device, method for controlling ophthalmological device, and recording medium

Abstract

An ophthalmological device 1 according to an exemplary embodiment of the present invention executes, by an imaging control unit 71, combined control involving: control of a focal position changing unit 31 for sequentially applying a plurality of different focal positions; control of an exposure amount changing unit 32 for sequentially applying to a plurality of different exposure amounts; and control for causing an imaging optical system 3 to acquire time-series images. Thus, for each of the plurality of different focal positions, a first image group corresponding to a plurality of different exposure amounts is acquired. Furthermore, the ophthalmological device 1 generates, by an image processing unit 81, a single image by combining the plurality of first image groups obtained for the plurality of different focus positions.

Description

Ophthalmic apparatus, method for controlling an ophthalmic apparatus, and recording medium

[0001] The present disclosure relates to an ophthalmic apparatus, a method for controlling an ophthalmic apparatus, and a recording medium.

[0002] Diagnostic imaging plays an important role in the field of ophthalmology. Various types of ophthalmic devices (ophthalmic imaging devices) are used in ophthalmic diagnostic imaging, such as slit lamp microscopes, fundus cameras, scanning laser ophthalmoscopes (SLOs), and optical coherence tomography (OCT). Ophthalmic devices with imaging functions are not limited to these ophthalmic imaging devices; imaging functions are also incorporated into ophthalmic examination and measurement devices such as refractometers, keratometers, tonometers, specular microscopes, wavefront analyzers, and microperimeters.

[0003] One of the most widely and frequently used ophthalmic devices is the slit lamp microscope. Slit lamp microscopes are primarily used to photograph the anterior segment of the eye. In addition to photographing cross sections using slit light illumination, slit lamp microscopes are also used to photograph the anterior segment from the front. In addition to slit lamp microscopes, frontal photography of the anterior segment can also be performed using fundus cameras, various ophthalmic examination devices, and various ophthalmic measurement devices.

[0004] Japanese Patent Application Laid-Open No. 2023-3456

[0005] One object of the present disclosure is to improve the quality of images obtained by frontal imaging of the anterior segment of the eye.

[0006] an exposure amount change unit that changes the exposure amount used to photograph the anterior eye segment; a focus position change control that controls the focus position change unit to sequentially change the focus position of the photographing optical system to a plurality of different focus positions; an exposure amount change control that controls the exposure amount change unit to sequentially change the exposure amount used to photograph the anterior eye segment to a plurality of different exposure amounts; and an imaging control that controls the photographing optical system to acquire time-series images, thereby causing the photographing optical system to acquire a first group of images corresponding to the plurality of different exposure amounts for each of the plurality of different focus positions; and an image processing unit that combines the plurality of first image groups acquired for the plurality of different focus positions to generate a single image.

[0007] Another exemplary aspect of the embodiment is an ophthalmologic apparatus including: an illumination optical system that projects illumination light onto an anterior segment of the eye to be examined; an imaging optical system that images the anterior segment onto which the illumination light is projected from the front; a focus position changing unit that changes a focus position of the imaging optical system; an exposure amount changing unit that changes an exposure amount for imaging the anterior segment; a focus position change control that controls the focus position changing unit to sequentially change the focus position of the imaging optical system to a plurality of different focus positions; an exposure amount change control that controls the exposure amount changing unit to sequentially change the exposure amount for imaging the anterior segment to a plurality of different exposure amounts; and an imaging control that controls the imaging optical system to acquire time-series images; and an imaging control unit that causes the imaging optical system to acquire a second group of images corresponding to a plurality of pairs of focus positions and exposure amounts, including pairs based on a one-to-one correspondence between the plurality of different focus positions and the plurality of different exposure amounts; and an image processing unit that synthesizes the second group of images to generate a single image.

[0008] Yet another exemplary aspect of the embodiment is an ophthalmologic apparatus including: an illumination optical system that projects illumination light onto an anterior segment of the eye to be examined; an imaging optical system that images the anterior segment onto which the illumination light is projected from the front; a focus position changing unit that changes a focus position of the imaging optical system; focus position change control that controls the focus position changing unit to sequentially change the focus position of the imaging optical system to a plurality of different focus positions; and imaging control that controls the imaging optical system to acquire time-series images at a constant exposure amount, by combining these components; an imaging control unit that causes the imaging optical system to acquire a third group of images corresponding to the plurality of different focus positions; and an image processing unit that synthesizes the third group of images to generate a single image, wherein the third group of images includes only one image for each of the plurality of different focus positions.

[0009] Yet another exemplary aspect of the embodiment is an ophthalmologic apparatus including: an illumination optical system that projects illumination light onto an anterior segment of the test eye; an imaging optical system that images the anterior segment onto which the illumination light is projected from the front; an exposure amount change unit that changes the exposure amount when imaging the anterior segment; exposure amount change control that controls the exposure amount change unit to sequentially change the exposure amount when imaging the anterior segment to a plurality of different exposure amounts; and imaging control that controls the imaging optical system to acquire time-series images at a constant focus position; an imaging control unit that causes the imaging optical system to acquire a fourth group of images corresponding to the plurality of different exposure amounts; and an image processing unit that combines the fourth group of images to generate a single image, wherein the fourth group of images includes only one image for each of the plurality of different exposure amounts.

[0010] Yet another exemplary aspect of an embodiment is a method for controlling an ophthalmic apparatus for photographing an anterior segment of a test eye, the ophthalmic apparatus including an illumination optical system that projects illumination light onto the anterior segment, an imaging optical system that photographs the anterior segment onto which the illumination light is projected from the front, a focus position changing unit that changes the focus position of the imaging optical system, an exposure amount changing unit that changes the exposure amount used to photograph the anterior segment, and a processor, the method including a photographing step of causing the imaging optical system to acquire a first group of images corresponding to the plurality of different exposure amounts for each of the plurality of different focus positions by having the processor execute combined control of focus position change control that controls the focus position changing unit to sequentially change the focus position of the imaging optical system to a plurality of different focus positions, exposure amount change control that controls the exposure amount changing unit to sequentially change the exposure amount used to photograph the anterior segment to a plurality of different exposure amounts, and imaging control that controls the imaging optical system to acquire time-series images, and an image processing step of synthesizing the plurality of first image groups acquired for the plurality of different focus positions by the photographing step to generate a single image.

[0011] Yet another exemplary aspect of an embodiment is a method for controlling an ophthalmic apparatus for photographing an anterior segment of a subject's eye, the ophthalmic apparatus including: an illumination optical system that projects illumination light onto the anterior segment; an imaging optical system that photographs the anterior segment onto which the illumination light is projected from the front; a focus position changing unit that changes the focus position of the imaging optical system; an exposure amount changing unit that changes the exposure amount used to photograph the anterior segment; and a processor; the method includes a photographing step of causing the imaging optical system to acquire a second group of images corresponding to a plurality of pairs of focus positions and exposure amounts, including pairs based on a one-to-one correspondence between the plurality of different focus positions and the plurality of different exposure amounts, by having the processor execute combined control of a focus position change control that controls the focus position change unit to sequentially change the focus position of the imaging optical system to a plurality of different focus positions; an exposure amount change control that controls the exposure amount change unit to sequentially change the exposure amount used to photograph the anterior segment to a plurality of different exposure amounts; and an imaging control that controls the imaging optical system to acquire time-series images; and an image processing step of synthesizing the second group of images acquired by the photographing step to generate a single image.

[0012] Yet another exemplary aspect of an embodiment is a method for controlling an ophthalmic apparatus for photographing an anterior segment of a test eye, the ophthalmic apparatus including an illumination optical system that projects illumination light onto the anterior segment, an imaging optical system that photographs the anterior segment onto which the illumination light is projected from the front, a focus position changing unit that changes the focus position of the imaging optical system, and a processor, wherein the method includes a photographing step of causing the imaging optical system to acquire a third group of images corresponding to the plurality of different focus positions by having the processor execute combined control of focus position change control that controls the focus position changing unit to sequentially change the focus position of the imaging optical system to a plurality of different focus positions and photographing control that controls the imaging optical system to acquire time-series images at a constant exposure amount, and an image processing step of synthesizing the third group of images acquired by the photographing step to generate a single image, wherein the third group of images includes only one image for each of the plurality of different focus positions.

[0013] Yet another exemplary aspect of the embodiment is a method for controlling an ophthalmic device for photographing an anterior segment of a test eye, the ophthalmic device including an illumination optical system that projects illumination light onto the anterior segment, an imaging optical system that photographs the anterior segment onto which the illumination light is projected from the front, an exposure amount change unit that changes the exposure amount when photographing the anterior segment, and a processor, wherein the method includes a photographing step in which the imaging optical system acquires a fourth group of images corresponding to the plurality of different exposure amounts by having the processor execute combined control of exposure amount change control that controls the exposure amount change unit to sequentially change the exposure amount when photographing the anterior segment to a plurality of different exposure amounts and imaging control that controls the imaging optical system to acquire time-series images at a constant focus position, and an image processing step in which the fourth group of images acquired by the photographing step are combined to generate a single image, and the fourth group of images includes only one image for each of the plurality of different exposure amounts.

[0014] Yet another exemplary aspect of the embodiment is a program that causes a computer to execute the method according to any of the exemplary aspects.

[0015] Yet another exemplary aspect of the embodiment is a computer-readable non-transitory recording medium having a program according to any of the exemplary aspects recorded thereon.

[0016] According to the embodiment, it is possible to improve the quality of the image obtained by photographing the anterior segment from the front.

[0017] FIG. 1 is a schematic diagram showing the configuration of an ophthalmic apparatus according to a non-limiting aspect of an embodiment. FIG. 2 is a schematic diagram showing the configuration of an ophthalmic apparatus according to a non-limiting aspect of an embodiment. FIG. 3 is a schematic diagram showing the configuration of an ophthalmic apparatus according to a non-limiting aspect of an embodiment. FIG. 4 is a schematic diagram showing the configuration of an ophthalmic apparatus according to a non-limiting aspect of an embodiment. FIG. 5 is a flowchart showing the operation performed by an ophthalmic apparatus according to a non-limiting aspect of an embodiment. FIG. 6 is a schematic diagram showing the configuration of an ophthalmic apparatus according to a non-limiting aspect of an embodiment. FIG. 7 is a flowchart showing the operation performed by an ophthalmic apparatus according to a non-limiting aspect of an embodiment. FIG. 8 is a schematic diagram showing the configuration of an ophthalmic apparatus according to a non-limiting aspect of an embodiment. FIG. 9 is a flowchart showing the operation performed by an ophthalmic apparatus according to a non-limiting aspect of an embodiment. FIG. 1 is a schematic diagram showing an operation performed by an ophthalmic apparatus according to a non-limiting aspect of an embodiment; FIG. 2 is a schematic diagram showing a configuration of an ophthalmic apparatus according to a non-limiting aspect of an embodiment; FIG. 3 is a schematic diagram showing an operation performed by an ophthalmic apparatus according to a non-limiting aspect of an embodiment; FIG. 4 is a schematic diagram showing a configuration of an ophthalmic apparatus according to a non-limiting aspect of an embodiment; FIG. 5 is a flowchart showing an operation performed by an ophthalmic apparatus according to a non-limiting aspect of an embodiment; FIG. 6 is a schematic diagram showing an operation performed by an ophthalmic apparatus according to a non-limiting aspect of an embodiment;

[0018] Non-limiting embodiments of the present disclosure will now be described.

[0019] Any known technology can be combined with the embodiments. For example, any matter described in the documents cited in this disclosure can be combined with any aspect of the embodiments. Furthermore, at least one of any known document related to the technical field of the present disclosure, any known technology in a technical field similar to the technical field of the present disclosure, and any known technology in a technical field other than the technical field of the present disclosure can be combined with any aspect of the embodiments.

[0020] For example, the disclosures in Patent Document 1 (JP 2023-3456 A) can be incorporated by reference into the present disclosure. More generally, any technical matter disclosed by the applicant of the present application regarding technology related to the present disclosure (matters disclosed in patent applications, papers, etc.) can be incorporated by reference into the present disclosure.

[0021] Any two or more aspects of the embodiments may be at least partially combined.

[0022] At least a portion of the functionality of any aspect described in this disclosure is implemented using circuitry or processing circuitry. The circuitry or processing circuitry may be a general-purpose processor, a special-purpose processor, an integrated circuit, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), a Field Programmable Gate Array (FPGA)), or a combination of these devices configured and / or programmed to perform at least some of the disclosed functions. Array), conventional circuitry, and any combination thereof. A processor is considered to be processing circuitry or circuitry, including transistors and / or other circuitry. In this disclosure, circuitry, unit, means, or similar terms refers to hardware that performs at least a portion of the disclosed functions or hardware that is programmed to perform at least a portion of the disclosed functions. The hardware may be hardware disclosed herein or known hardware that is programmed and / or configured to perform at least a portion of the described functions. In the case of a processor, where the hardware can be considered to be a type of circuitry, circuitry, unit, means, or similar terms refers to a combination of hardware and software, and the software is used to configure the hardware and / or processor.

[0023] The present disclosure aims to improve the quality of an image obtained by frontal imaging of the anterior segment of the eye. More specifically, the present disclosure aims to improve the quality of a frontal image of the anterior segment of the eye by achieving both an expanded dynamic range and an expanded depth of field.

[0024] Expanding the dynamic range has various advantages. In ophthalmic imaging, which is the subject of the present disclosure, the following advantages are particularly apparent. In ophthalmic imaging that uses light, two or more eye regions with significantly different optical characteristics may be depicted in a single image. For example, a single image may be acquired depicting the cornea and the crystalline lens, which have significantly different optical reflectances, or a single image may be acquired depicting the cornea and the sclera, which have significantly different optical scattering characteristics. The dynamic range of a typical image sensor is insufficient to represent each of these two or more eye regions at an appropriate brightness. By expanding the dynamic range, an embodiment of the present disclosure can generate a single anterior front image in which each eye region is depicted at an appropriate brightness.

[0025] In addition, extending the depth of field also has various advantages. In ophthalmic imaging, which is the subject of the present disclosure, the following advantages are particularly apparent. In ophthalmic diagnoses, it is desirable to make a comprehensive judgment by referring to the conditions of multiple eye regions, rather than by referring to the condition of only one eye region. To achieve this, it is desirable to depict multiple eye regions without blurring. However, with the depth of field achieved by a typical optical system, it is difficult to clearly depict each of multiple eye regions. For example, it is difficult to obtain a single image in which both the cornea, which is located at a shallow position, and the crystalline lens, which is located at a deep position, are in focus. By extending the depth of field, embodiments of the present disclosure can generate a single frontal image of the anterior eye segment in which each eye region is depicted with high resolution.

[0026] In this way, by combining an expanded dynamic range with an expanded depth of field, embodiments of the present disclosure enable the generation of a single frontal image of the anterior segment in which multiple regions of the anterior segment are depicted with appropriate brightness and without blur.

[0027] Note that the effects of the embodiments according to the present disclosure are not limited to the ability to generate a single image that is excellent in both dynamic range and depth of field. It will be understood by those skilled in the art that each aspect described below produces effects according to its characteristics (configuration, operation, action, use, etc.).

[0028] Next, an overview of some exemplary aspects of the embodiment will be described. The expansion of the dynamic range is related to the exposure amount, and the expansion of the depth of field is related to the focal position. Here, the exposure amount is a physical quantity indicating the amount of light detected by a camera (image sensor) (in other words, the brightness of an image obtained by the camera), and is determined by shooting conditions such as the brightness of the lighting, the F-number of the lens, and the shutter speed.

[0029] In the first and second aspects of the embodiment, multiple shots are taken while varying both the exposure and the focus position, and a single image is generated from the group of images collected thereby. In the third aspect, multiple shots are taken at a constant exposure (same exposure) while varying the focus position, and a single image is generated from the group of images collected thereby. In the fourth aspect, multiple shots are taken at a constant focus position (same focus position) while varying the exposure position, and a single image is generated from the group of images collected thereby. These four aspects will be outlined in more detail below.

[0030] An overview and some non-limiting features of a first aspect of an embodiment will be described below. The ophthalmic apparatus according to this aspect includes an illumination optical system, a photographing optical system, a focus position changing unit, an exposure amount changing unit, a photographing control unit, and an image processing unit.

[0031] The illumination optical system is configured to project illumination light onto an anterior segment of the subject's eye. The photographing optical system is configured to photograph the anterior segment of the eye onto which the illumination light is projected from the front. The illumination optical system and the photographing optical system may be configured in the same manner as those of any type of ophthalmic device.

[0032] Frontal imaging performed by the imaging optical system is an operation of imaging the anterior segment of the eye from a position in front of the subject's eye. This frontal imaging is performed, for example, when the optical axis of the imaging optical system is approximately aligned with the axis of the subject's eye. In another example, this frontal imaging may be performed when the magnitude of deviation of the optical axis of the imaging optical system from the axis of the subject's eye is within an acceptable range. This acceptable range can be set arbitrarily, and may be, for example, a range of acceptable alignment error, a range determined based on the relationship with another imaging mode (e.g., the slit imaging mode (cross-sectional imaging mode) of a slit lamp microscope), a range determined based on the positions of multiple eye regions to be imaged, or the like.

[0033] The focal position changing unit is configured to change the focal position of the photographing optical system. The focal position changing unit may have any known configuration, and may include, for example, a focus lens.

[0034] The exposure amount changing unit is configured to change the exposure amount when photographing the anterior segment of the eye. The exposure amount changing unit may have any known configuration.

[0035] In some examples, the exposure amount changer is configured to change the exposure amount by changing the length of time (exposure time) during which a camera (image sensor) of the photographing optical system detects light. Such an exposure amount changer includes an exposure time changer having a configuration for changing the exposure time of the photographing optical system. The exposure time changer may include, for example, any of a mechanically controlled shutter, an electronically controlled shutter, a hybrid mechanically and electronically controlled shutter, and a light amount adjustment filter.

[0036] In some examples, the exposure amount changing unit is configured to change the exposure amount by changing the intensity of illumination light projected onto the anterior segment by the illumination optical system. Such an exposure amount changing unit includes an illumination intensity changing unit having a configuration for changing the intensity of the illumination light. The illumination intensity changing unit may include, for example, either a drive circuit for the illumination light source or a light amount adjustment filter.

[0037] The photographing control unit is configured to be able to execute focus position change control, exposure amount change control, and photographing control.

[0038] The focus position change control is a process of controlling the focus position change unit so as to change the focus position of the photographing optical system to a plurality of different focus positions in sequence.

[0039] The exposure amount change control is a process of controlling the exposure amount change unit to sequentially change the exposure amount in photographing the anterior segment to a plurality of different exposure amounts. In some examples, the photographing control unit is configured to execute exposure time change control to control the exposure time change unit to sequentially change the exposure time of the photographing optical system to a plurality of different exposure times. In addition, in some examples, the photographing control unit is configured to execute illumination intensity change control to control the illumination intensity change unit to sequentially change the intensity of illumination light projected onto the anterior segment to a plurality of different intensities.

[0040] The imaging control is a process of controlling the imaging optical system to acquire time-series images of the anterior segment. The time-series images acquired by the imaging control are a plurality of images acquired at different times. Examples of time-series images include moving images, video, time-lapse images, and slideshow images.

[0041] The photographing control unit is configured to cause the photographing optical system to acquire, for each of a plurality of different focal positions, a plurality of images (first image groups) corresponding to the respective focal positions and a plurality of different exposure amounts by combining the focus position change control, the exposure amount change control, and the photographing control. By this combined control, a plurality of first image groups corresponding to the plurality of different focal positions are acquired.

[0042] Let z1, z2, ..., zM (M pieces) be multiple different focal positions and e1, e2, ..., eN (N pieces) be multiple different exposure amounts. The first image group corresponding to each focal position zm (m = 1, 2, ..., M) is made up of an image corresponding to exposure amount e1, an image corresponding to exposure amount e2, ..., an image corresponding to exposure amount eN. In this example, each first image group is made up of N images, and the entire M first image groups are made up of M x N images. Here, M is an integer greater than or equal to 2, and N is an integer greater than or equal to 2.

[0043] The image processing unit is configured to synthesize a plurality of first images acquired at a plurality of different focal positions to generate a single image (a single anterior eye front image).

[0044] According to the first aspect configured as described above, the dynamic range can be expanded based on the first image groups corresponding to each focal position (each depth position) and corresponding to a plurality of different exposure amounts, and the depth of field can be expanded based on the plurality of first image groups corresponding to a plurality of different focal positions (a plurality of different depth positions). As a result, the first aspect can generate a single anterior-segment front image in which multiple regions of the anterior segment are depicted with appropriate brightness and without blur. Therefore, the first aspect can improve the quality of images obtained by frontal imaging of the anterior segment.

[0045] In a first aspect, the image processing unit may be configured to perform a process of generating a plurality of high dynamic range images corresponding to a plurality of different focus positions by applying high dynamic range synthesis to each of a plurality of first image groups, and a process of generating a single image by applying focus synthesis (depth stacking) to the generated plurality of high dynamic range images. That is, the image processing unit may be configured to generate a single frontal image of the anterior eye segment by combining the high dynamic range synthesis corresponding to each focus position with the focus synthesis corresponding to the plurality of different focus positions. This example provides one example of the process performed by the image processing unit of the first aspect. Note that the high dynamic range image is an image (dynamic range expanded image) having a dynamic range wider than the dynamic range of the original image.

[0046] Another example of processing performed by the image processing unit of the first aspect will be described. In this example, machine learning is utilized. The image processing unit of this example includes a pre-created machine learning model. Training data (teacher data, learning data) generated by photographing one or more objects (e.g., a living eye, an enucleated eye, an eye model, or other object) is used to create the machine learning model. This training data includes, for example, a plurality of pairs of images (training image groups) collected through multiple photographing operations performed using a plurality of different focus positions and a plurality of different exposure amounts, and a composite image (training composite image) of the training image groups. The machine learning of this example is performed by training a neural network including a convolutional neural network (CNN) using this training data. This constructs a machine learning model that receives input of a plurality of anterior-segment frontal images corresponding to a plurality of different focus positions and a plurality of different exposure amounts, and outputs a composite image (a single anterior-segment frontal image) of the plurality of anterior-segment frontal images.

[0047] The configuration and construction method of the machine learning model are not limited to this example. The machine learning model applicable to the image processing unit of the first aspect may have any configuration and construction method as long as it functions to apply image processing that combines high dynamic range compositing or a similar process with focus compositing or a similar process to a plurality of images corresponding to combinations of a plurality of different focus positions and a plurality of different exposure amounts.

[0048] In a first aspect, the image processing unit may include an image selection unit and an image synthesis unit. The image selection unit is configured to select one or more images from each of a plurality of first image groups acquired by the imaging optical system under combination control executed by the imaging control unit. This results in a plurality of images being selected from the plurality of first image groups. The selected plurality of images are referred to as a selected image group. The image synthesis unit is configured to synthesize the selected image group to generate a single image (a single anterior eye front image).

[0049] The plurality of first image groups acquired by the imaging optical system under the combination control executed by the imaging control unit may include images that are unsuitable for image processing such as image synthesis or for image diagnosis. For example, the plurality of first image groups may include images captured during blinking, images containing unwanted light, etc. Synthesizing all of such plurality of first image groups may contribute to a decrease in the quality of the generated single image.

[0050] This example is configured to combine a group of selected images extracted from the plurality of first image groups, rather than combining all of the plurality of first image groups. Therefore, this example makes it possible to more reliably generate a single anterior-segment front image in which multiple regions of the anterior segment are depicted with appropriate brightness and without blur. This has the effect of enabling a single anterior-segment front image of good quality to be obtained, even when the subject's eye is diseased. More generally, it is possible to reduce the influence of differences in the condition of the subject's eye and imaging conditions. In other words, it is possible to improve robustness.

[0051] In the first aspect, when the image selection unit selects two or more images from each of one or more first image groups among the plurality of first image groups, that is, when the image selection unit selects two or more images from any of the first image groups, the image synthesis unit may be configured to perform the following process. Note that this example provides one example of the process performed by the image processing unit of the first aspect.

[0052] The image composition unit of this example first applies high dynamic range composition to two or more images selected from each first image group, thereby generating a high dynamic range image, thereby obtaining one or more high dynamic range images corresponding to the one or more first image groups from which two or more images are selected.

[0053] Furthermore, the image synthesis unit of this example applies focus synthesis to an image group including one or more high dynamic range images corresponding to one or more first image groups from which two or more images are selected to generate a single image (a single frontal image of the anterior segment).

[0054] This example can also be expressed as follows: When the selected image group includes, for each of one or more focal positions among a plurality of different focal positions, two or more images corresponding to the focal positions, the image synthesis unit of this example performs a process of generating one or more high dynamic range images corresponding to the one or more focal positions by applying high dynamic range synthesis to the two or more images corresponding to the focal positions with which the two or more images are associated, thereby generating a high dynamic range image, and applying focus synthesis to the image group including the one or more high dynamic range images to generate a single image (a single frontal image of the anterior eye segment).

[0055] In a first aspect, the image selection unit may be configured to select two or more images from the first image group corresponding to focal positions included in a focal plane that intersects a plurality of different anterior segment tissues.

[0056] In other words, the image selection unit of this example is configured to select two or more images from the first image group corresponding to a certain focal position when images focused on multiple different eye regions (multiple different anterior ocular tissues) are obtained or have been obtained by imaging corresponding to that focal position. For example, imaging corresponding to a certain focal position may result in the following images: an image focused on both the cornea and the crystalline lens; an image focused on both the cornea and the iris; an image focused on the cornea and the anterior chamber; or an image focused on any three or all of the cornea, iris, crystalline lens, and anterior chamber. Note that the anterior chamber can be excluded from the factors used to determine image selection, except when observing floaters in the aqueous humor. Each of these anterior ocular tissues has its own unique optical properties (e.g., reflectance, scattering degree (scattering intensity), etc.). Therefore, when imaging with a constant exposure (the same exposure), each of these anterior ocular tissues is depicted with different brightnesses. According to this example, two or more images are selected from the first image group corresponding to a focal position where such a situation occurs or has occurred, and high dynamic range compositing is applied to the selected two or more images to generate a high dynamic range image. In this high dynamic range image, multiple different anterior ocular tissues are depicted at suitable brightnesses. In this example, two or more images are selected from the first image group corresponding to focal positions included in a focal plane that intersects multiple different anterior ocular tissues to generate a high dynamic range image, and focus compositing is applied to the image group including such one or more high dynamic range images to generate a single image (a single anterior ocular front image). According to this example, it is possible to generate a single anterior ocular front image in which multiple regions of the anterior ocular segment are depicted at suitable brightnesses and without blurring. In addition, high dynamic range compositing can be omitted for focal positions included in a focal plane that does not intersect multiple different anterior ocular tissues, thereby reducing processing resources and shortening processing time.

[0057] As described in the above example, the image selector selects one or more images from each of the plurality of first image groups. This example provides one example of how to determine the first image groups from which two or more images are to be selected among the plurality of first image groups.

[0058] The first image group from which two or more images are selected may be determined in advance, may be determined in parallel with the combination control (collection of the multiple first image groups) executed by the imaging control unit, or may be determined after the multiple first image groups have been acquired. Some examples of determining the first image group from which two or more images are selected will be described below.

[0059] In one example, the anatomical structure of the eye can be taken into account, either from a standard eye model or from an image of the anterior segment of the eye, such as a three-dimensional image generated by applying an optical coherence tomography scan (OCT scan) to the anterior segment of the eye or a three-dimensional image obtained by applying a slit scan using a Scheimpflug slit lamp microscope to the anterior segment of the eye.

[0060] In this example, multiple planes perpendicular to the eye axis (Z axis) are set, the number of anterior segment tissues that intersect with each plane (number of intersections) is calculated, and information (correspondence information) indicating the correspondence between the Z coordinate (depth position) of each plane and the number of intersections is stored.

[0061] During or after imaging of the anterior segment, the coordinates (Z coordinates) of each focal position applied in the imaging are compared with the Z coordinates of multiple planes to identify a plane corresponding to each focal position. For each focal position, the number of intersections corresponding to the identified plane is referred to as the number of anterior segment tissues intersecting the focal plane containing the focal position. If the number of intersections is two or more, two or more images are selected from the first image set corresponding to the focal position.

[0062] Correspondence information for a normal eye (a healthy eye) and correspondence information for an affected eye (an eye in which the shape or position of the anterior ocular tissue has changed) can be prepared and used selectively or in an integrated manner. Furthermore, multiple pieces of correspondence information for an affected eye can be prepared. For example, multiple pieces of correspondence information for multiple diseases can be prepared, or multiple pieces of correspondence information for multiple stages of progression of a single disease can be prepared.

[0063] In another example, a range of Z coordinates in which a plane perpendicular to the ocular axis intersects with a plurality of different anterior segment tissues is predetermined based on the anatomical structure of the eye, and during or after image capture, it is possible to determine whether or not to select two or more images from the first image set corresponding to the focal position by determining whether the Z coordinate of each focal position falls within the range.

[0064] As described above, in some examples, the image selection unit may include a determination unit that determines whether a plurality of different anterior ocular tissues intersect the focal plane, and the determination unit is configured to determine, for each of a plurality of different focal positions, whether a focal plane including the focal position intersects the plurality of different anterior ocular tissues by analyzing an image corresponding to the focal position.

[0065] For example, the determination unit applies segmentation to an image included in the first image group corresponding to a certain focal position, thereby dividing the image into a plurality of segments (partial regions), and determines whether the segments contain images of a plurality of different anterior ocular tissues. This determination may be performed based on, for example, at least one of a Z coordinate of the focal position, optical properties of the anterior ocular tissues, and an anatomical structure of the anterior ocular segment. This allows the determination unit to determine whether a focal plane including the focal position intersects with a plurality of different anterior ocular tissues.

[0066] For each focal position determined by the determination unit as having a corresponding focal plane that intersects with multiple different anterior segment tissues, the image selection unit selects two or more images from the first image group corresponding to the focal position, and the image synthesis unit applies high dynamic range synthesis to the two or more images selected from the first image group to generate a high dynamic range image. On the other hand, for each focal position determined by the determination unit as having a corresponding focal plane that does not intersect with multiple different anterior segment tissues, the image selection unit selects one image from the first image group corresponding to the focal position. The image synthesis unit generates a single image (a single anterior segment front image) by applying focus synthesis to an image group including one or more high dynamic range images generated in the above manner and one or more images selected in the above manner. This series of processes is an example of processing that is executed when there are both one or more focal positions determined as having a corresponding focal plane that intersects with multiple different anterior segment tissues and one or more focal positions determined as having a corresponding focal plane that does not intersect with multiple different anterior segment tissues. If all of the plurality of different focus positions fall into the former category, a high dynamic range image corresponding to each focus position may be generated, whereas if all of the plurality of different focus positions fall into the latter category, one image may be selected from the first set of images corresponding to each focus position.

[0067] It is possible that no image is selected from any of the multiple first image groups corresponding to multiple different focus positions. For example, all images included in the first image group corresponding to a certain focus position may be unsuitable for image processing or image diagnosis. In this case, the range corresponding to the focus position from which no image was selected may be interpolated. The interpolated range may be the range in focus when capturing an image at that focus position (the range corresponding to the depth of field). The interpolation process is performed, for example, based on one or two first image groups corresponding to one or two focus positions adjacent to the focus position from which no image was selected. Instead of or in addition to the interpolation process, the user may be notified to recapture the image at that focus position, or the recapture at that focus position may be automatically performed. Furthermore, when the interpolation process is performed, additional information indicating that the interpolation process has been performed or additional information indicating the interpolated range may be generated. The generated additional information is attached to any image (e.g., a single frontal image of the anterior segment).

[0068] When two or more images are selected from one first image group, the number of images to be selected may be predetermined or may be determined by processing. The number of images to be selected may be determined based on the Z coordinate of the focal position or the anatomical structure of the anterior segment. Furthermore, the number of images to be selected may be determined based on the number of types of anterior segment tissues that intersect the corresponding focal plane.

[0069] An overview and some non-limiting features of a second aspect of the embodiment will be described. Similar to the first aspect, the ophthalmologic apparatus according to this aspect includes an illumination optical system, a photographing optical system, a focus position changing unit, an exposure amount changing unit, a photographing control unit, and an image processing unit. Of these elements, elements other than the photographing control unit may be the same as those of the first aspect.

[0070] The photographing control unit of this aspect is configured to be able to execute focus position change control, exposure amount change control, and photographing control, as in the first aspect. The photographing control unit is configured to cause the photographing optical system to acquire a second group of images corresponding to a plurality of pairs of focus positions and exposure amounts, including pairs based on a one-to-one correspondence between a plurality of different focus positions and a plurality of different exposure amounts (i.e., a bijection defined between a set having these focus positions as elements and a set having these exposure amounts as elements), by combining the focus position change control, exposure amount change control, and photographing control.

[0071] Let z1, z2, ..., zM (M pieces) be the multiple different focal positions, e1, e2, ..., eM (M pieces) be the multiple different exposure amounts, and let (z1, e1), (z2, e2), ..., (zM, eM) be the multiple pairs based on a one-to-one correspondence between the multiple different focal positions and the multiple different exposure amounts. The second image group includes an image corresponding to the pair (z1, e1), an image corresponding to the pair (z2, e2), ..., an image corresponding to the pair (zM, eM). The second image group may further include images other than these (described below). The second image group of this aspect consists of at least M images.

[0072] In the first aspect, multiple image groups (multiple first image groups) corresponding to multiple different focal positions are acquired, whereas in the second aspect, one image group (second image group) is acquired.

[0073] The image processor is configured to combine the second set of images to generate a single image (a single frontal anterior segment image).

[0074] According to the second aspect configured as above, the dynamic range and the depth of field can be expanded based on the second image group corresponding to the plurality of pairs, so that a single frontal image of the anterior segment can be generated in which multiple regions of the anterior segment are depicted with appropriate brightness and without blur. Therefore, the second aspect can improve the quality of images obtained by frontal imaging of the anterior segment.

[0075] Furthermore, the second aspect can improve image quality using a smaller number of second image groups than the first image groups of the first aspect, and therefore has the further advantages of reducing the number of times images are taken, shortening the image taking time, reducing processing resources, shortening processing time, etc. Note that there may be cases where the degree of improvement in image quality is superior to that of the first aspect.

[0076] In a second aspect, each pair of a focal position and an exposure amount may be a combination of a focal position corresponding to one anterior segment tissue among a plurality of different anterior segment tissues and an exposure amount that is predetermined based on the strength of scattering (magnitude of scattering) of the anterior segment tissue.

[0077] Here, the focal position corresponding to the anterior segment tissue may be set to, for example, any position in the anterior segment tissue or any position near the anterior segment tissue. Furthermore, the scattering intensity of the anterior segment tissue may be, for example, a measurement value or a statistical value thereof obtained by measuring a living eye, or a measurement value or a statistical value thereof obtained by anatomical measurement. The exposure dose based on the scattering intensity of the anterior segment tissue may be determined by further referring to any information, such as the configuration and / or characteristics of the illumination optical system or the configuration and / or characteristics of the imaging optical system.

[0078] According to this example, multiple pairs of focal positions and exposure amounts corresponding to the scattering degrees of multiple different anterior segment tissues can be prepared as shooting conditions, and a second group of images can be collected using these shooting conditions to generate a single frontal image of the anterior segment.

[0079] Instead of using such default imaging conditions, imaging conditions may be individually set based on data obtained from the anterior segment of the subject's eye. For example, the ophthalmologic apparatus according to the second aspect may further include an anterior segment profile acquisition unit configured to acquire a depth profile of the anterior segment of the subject's eye. The anterior segment profile acquisition unit is configured to generate a depth profile and / or to receive a depth profile from an external device.

[0080] The depth profile of the anterior segment indicates the distribution of anterior segment data in the axial direction (Z direction) of the eye. For example, the depth profile may include any of the following: a cross-sectional image of the anterior segment obtained by photographing the anterior segment with a slit lamp microscope; data generated by applying an OCT scan or an ultrasound scan to the anterior segment (e.g., A-scan data, A-scan image, B-scan image, 3D image, etc.); or axial length data obtained by applying an axial length measurement to the subject's eye.

[0081] The ophthalmologic apparatus of this example may further include a photographing condition determination unit configured to determine multiple pairs of focus position and exposure amount based on the depth direction profile acquired by the anterior segment profile acquisition unit.

[0082] The imaging condition determination unit may be configured to identify depth positions of multiple different anterior segment tissues based on the depth profile. The depth profile includes data indicating the positions of the anterior segment tissues and images of the anterior segment tissues. For example, A-scan data of the anterior segment exhibits peaks indicating the positions of the cornea (anterior corneal surface, posterior corneal surface) and peaks indicating the positions of the lens (anterior capsule, posterior capsule). The anterior segment image also includes images of the cornea, anterior chamber, iris, and lens. The imaging condition determination unit can identify depth positions (Z coordinates) of multiple different anterior segment tissues by applying predetermined processing (e.g., signal processing such as peak detection, image processing such as segmentation) to the depth profile.

[0083] Furthermore, the imaging condition determination unit can determine a plurality of pairs of a focal position and an exposure dose based on the depth positions of a plurality of different anterior segment tissues identified based on the depth profile and the scattering intensity information of a plurality of different anterior segment tissues. The scattering intensity information may be, for example, a standard value of the scattering intensity of the anterior segment tissues or may be the scattering intensity of the anterior segment tissues obtained by a physical method such as OCT.

[0084] According to this example, multiple pairs of focus positions and exposure amounts can be determined as imaging conditions based on data on the anterior segment of the subject's eye, and a single anterior segment front image can be generated by collecting a second image group using these imaging conditions. Therefore, compared to the example using default imaging conditions, this example has the advantage of being able to perform imaging under conditions tailored to the individual subject's eye. On the other hand, compared to the example using default imaging conditions, the example using default imaging conditions has the advantage of requiring fewer processing resources and a shorter processing time.

[0085] As mentioned above, the plurality of pairs of focus positions and exposure amounts may include only pairs based on a one-to-one correspondence between different focus positions and different exposure amounts, or may include other pairs. The latter case will be described below. In the following example, multiple images are acquired for a certain focus position.

[0086] In a second aspect, the shooting control unit may be configured to cause the shooting optical system to acquire two or more images corresponding to two or more different exposure amounts for each of one or more focus positions among the plurality of different focus positions, wherein the second image group in this example includes two or more images for each of the one or more focus positions among the plurality of different focus positions and one image for each of the other one or more focus positions.

[0087] The imaging control unit of this example may be configured to cause the imaging optical system to acquire two or more images corresponding to two or more different exposure amounts for focal positions included in a focal plane that intersects multiple different anterior segment tissues, which may be configured similarly to the similar example of the first aspect.

[0088] More specifically, the image processing unit of this example may include a determination unit configured to determine, for each of a plurality of different focal positions, whether a focal plane including the focal position intersects with a plurality of different anterior segment tissues. The configuration and operation of the determination unit may be similar to those of the similar example in the first aspect.

[0089] The image processing unit of this example may be configured to generate a high dynamic range image for each focal position at which two or more images are acquired by applying high dynamic range synthesis to the two or more images corresponding to that focal position, and to generate a single image (a single frontal image of the anterior segment) based on an image group including one or more high dynamic range images generated for one or more focal positions at which two or more images are acquired. This configuration may be similar to that of the similar example in the first aspect.

[0090] According to this example, a high dynamic range image corresponding to one of a plurality of different depth positions can be acquired. This makes it possible to improve the imaging quality of a plurality of anterior segment tissues with different scattering intensities, for example. Furthermore, it becomes possible to automatically determine the depth position to which high dynamic range synthesis is applied when photographing the anterior segment of each subject's eye.

[0091] As in the similar example of the first aspect, a focus position (default focus position) to which high dynamic range synthesis is applied may be set in advance, and two or more images corresponding to two or more different exposure amounts may be acquired by shooting at that focus position, and a high dynamic range image may be generated from the two or more acquired images.

[0092] In the second aspect, the method of processing (image synthesis) for synthesizing the second image group to generate a single anterior-segment front image may be any method. For example, image synthesis may be performed using a machine learning model. The image processing unit in this example includes a machine learning model. This machine learning model is trained to receive input of multiple images with different focus positions and exposure amounts, and to output a synthesized image of the multiple images, thereby generating a single anterior-segment front image from the second image group.

[0093] The machine learning model of this example may have any configuration. For example, the machine learning model may include a convolutional neural network constructed by performing supervised learning using predetermined training data. This training data may include, for example, a plurality of pairs of a training image group having different focus positions and exposure amounts and a training composite image that is a composite image of the training image group. The machine learning model of this example may be the same model as the similar example in the first aspect.

[0094] In an example that does not utilize machine learning, the image processor may be configured to generate a single image (a single frontal anterior segment image) by applying image processing, including both focus stacking and high dynamic range stacking, to the second set of images, as described in more detail below.

[0095] An overview and some non-limiting features of a third aspect of the embodiment will be described. The ophthalmologic apparatus according to this aspect includes an illumination optical system, an imaging optical system, a focal position changing unit, an imaging control unit, and an image processing unit. Of these elements, the elements other than the imaging control unit may be the same as those of the first aspect.

[0096] Unlike the first aspect (and the second aspect), the ophthalmic apparatus of this aspect does not necessarily include an exposure amount change unit. Although the ophthalmic apparatus of this aspect may include an exposure amount change unit, the control of the exposure amount change unit is not essentially used in the control for collecting the image group to which image composition is applied.

[0097] The shooting control unit of this aspect is configured to be able to execute focus position change control similar to that of the first aspect and shooting control different from that of the first aspect. In the focus position change control, the focus position change unit is controlled to sequentially change the focus position of the shooting optical system to multiple different focus positions. In the shooting control, the shooting optical system is controlled to acquire time-series images with a constant exposure amount. The shooting control unit combines the focus position change control and the shooting control to cause the shooting optical system to acquire a third image group corresponding to multiple different focus positions. The third image group consists of multiple images corresponding to multiple different focus positions. The third image group includes only one image for each of the multiple different focus positions. If the multiple different focus positions are z1, z2, ..., zM (M images), the third image group consists of M images.

[0098] The image processor is configured to combine the third group of images to generate a single image (a single frontal anterior segment image).

[0099] According to the third aspect configured as above, it is possible to expand the dynamic range and the depth of field based on the third image group corresponding to the plurality of different focal positions, and therefore it is possible to generate a single frontal image of the anterior segment in which the plurality of regions of the anterior segment are depicted with suitable brightness and without blur. Therefore, the third aspect makes it possible to improve the quality of the image obtained by frontal imaging of the anterior segment.

[0100] Moreover, the third aspect can improve image quality using a smaller number of third image groups than the first image groups of the first aspect, and therefore has the further advantages of reducing the number of times images are taken, shortening the image taking time, reducing processing resources, shortening processing time, etc. Note that there may be cases where the first aspect is superior in terms of the degree of improvement in image quality.

[0101] Furthermore, the third aspect is configured to collect the third image group without performing the exposure amount change control in the first and second aspects, thereby simplifying the control for collecting the image group.

[0102] In a third aspect, the image processing unit may be configured to generate a plurality of (pseudo) high dynamic range images corresponding to a plurality of different focal positions by generating a (pseudo) high dynamic range image from each of a plurality of images included in a third image group corresponding to a plurality of different focal positions, and to apply focus stacking to the generated plurality of high dynamic range images to generate a single image (a single frontal image of the anterior eye segment).

[0103] The process of generating a pseudo-high dynamic range image from a single image may be performed using any image processing technology. In some examples, the image processing unit includes a machine learning model. The machine learning model is trained to receive an input image having a first dynamic range and output an image having a second dynamic range greater than the first dynamic range. Multiple high dynamic range images are generated by inputting each of the third set of images to the machine learning model. Any machine learning technique may be used in this example, including, for example, a technique based on the number or region of input exposures, a technique based on the number of learning tasks, a technique based on new sensor data, a technique based on new learning strategies, and a technique based on applications.

[0104] An overview and some non-limiting features of a fourth aspect of the embodiment will be described. The ophthalmologic apparatus according to this aspect includes an illumination optical system, a photographing optical system, an exposure amount changing unit, a photographing control unit, and an image processing unit. Of these elements, the elements other than the photographing control unit may be the same as those of the first aspect.

[0105] Unlike the first aspect (and the second aspect), the ophthalmologic apparatus of this aspect does not necessarily include a focus position changing unit. Although the ophthalmologic apparatus of this aspect may include a focus position changing unit, the control of the focus position changing unit is not essentially used in the control for collecting a group of images to which image synthesis is applied.

[0106] The photographing control unit of this aspect is configured to be able to execute exposure amount change control similar to that of the first aspect and photographing control different from that of the first aspect. In the exposure amount change control, the exposure amount change unit is controlled to sequentially change the exposure amount in photographing the anterior segment to multiple different exposure amounts. In the photographing control, the photographing optical system is controlled to acquire time-series images at a constant focus position. The photographing control unit combines the exposure amount change control and the photographing control to cause the photographing optical system to acquire a fourth image group corresponding to multiple different exposure amounts. The fourth image group consists of multiple images corresponding to multiple different exposure amounts. The fourth image group includes only one image for each of the multiple different exposure amounts. If the multiple different exposure amounts are e1, e2, ..., eN (N images), the fourth image group consists of N images.

[0107] The image processor is configured to combine the fourth group of images to generate a single image (a single anterior frontal image).

[0108] According to the fourth aspect configured as above, the dynamic range and the depth of field can be expanded based on the fourth image group corresponding to the plurality of different exposure amounts, so that a single frontal image of the anterior segment can be generated in which the plurality of regions of the anterior segment are depicted with appropriate brightness and without blur. Therefore, the fourth aspect can improve the quality of the image obtained by frontal imaging of the anterior segment.

[0109] Moreover, the fourth aspect can improve image quality using a smaller number of fourth image groups than the first image groups of the first aspect, and therefore has the further advantages of reducing the number of times images are taken, shortening the image taking time, reducing processing resources, shortening processing time, etc. Note that there may be cases where the first aspect is superior in terms of the degree of improvement in image quality.

[0110] Furthermore, the fourth aspect is configured to collect the fourth image group without performing the focus position change control in the first and second aspects, thereby simplifying the control for collecting the image group.

[0111] In a fourth aspect, the image processing unit may be configured to generate a plurality of (pseudo) extended depth-of-field images corresponding to a plurality of different exposure amounts by generating a (pseudo) extended depth-of-field image from each of a plurality of images included in a fourth image group corresponding to a plurality of different exposure amounts. The extended depth-of-field image is an image (deep focus image, pan focus image) having a depth of field deeper than the depth of field of the original image. Furthermore, the image processing unit may be configured to generate a single image (a single frontal image of the anterior segment) by applying high dynamic range blending to the plurality of extended depth-of-field images corresponding to the plurality of different exposure amounts.

[0112] The process of generating a pseudo-extended depth-of-field image from a single image may be performed using any image processing technique. In some examples, the image processing unit includes a machine learning model. The machine learning model is trained to receive an input of a single image having a first depth of field and output an image having a second depth of field greater than the first depth of field. By inputting each image of the fourth image group into the machine learning model, multiple extended depth-of-field images are generated. Any machine learning technique may be applied to this example.

[0113] This disclosure will describe several exemplary aspects of the embodiments outlined above, including first to fourth aspects. Furthermore, this disclosure will mainly describe exemplary aspects of an ophthalmic device, an exemplary aspect of a method for controlling an ophthalmic device, an exemplary aspect of a program, and an exemplary aspect of a recording medium. However, the categories of aspects of the embodiments are not limited to these. For example, it will be understood by those skilled in the art that embodiments according to the present disclosure can provide various aspects of medical methods, various aspects of imaging methods, various aspects of data processing methods, and the like.

[0114] <Ophthalmic Apparatus> A non-limiting aspect of an ophthalmic apparatus according to an embodiment will be described below. An example of the configuration of the ophthalmic apparatus according to this aspect is shown in FIGS.

[0115] The ophthalmic apparatus 1 is used to photograph the anterior segment of the subject's eye E, and includes an illumination optical system 2, an imaging optical system 3, an optical path coupling element 4, a movement mechanism 6, a control unit 7, a data processing unit 8, a communication unit 9, a user interface (UI) 10, a focal position changing unit 31, and an exposure amount changing unit 32. Reference numeral 5 denotes the optical axis of the optical system of the ophthalmic apparatus 1. The cornea of ​​the subject's eye E is designated by reference numeral Co, the iris by reference numeral Ir, and the crystalline lens by reference numeral Cr.

[0116] The communication unit 9 performs data communication between the ophthalmologic apparatus 1 and other devices. That is, the communication unit 9 transmits data to other devices and receives data transmitted from other devices. The communication unit 9 may perform any data communication method. For example, the communication unit 9 includes one or more of various communication interfaces, such as a communication interface conforming to the Internet, a communication interface conforming to a dedicated line, a communication interface conforming to a LAN, and a communication interface conforming to short-range communication. The data communication may be wired communication or wireless communication.

[0117] The data transmitted or received by the communication unit 9 may be encrypted data. The control unit 7 and / or the data processing unit 8 may include either or both of an encryption processing unit that encrypts data transmitted by the communication unit 9 and a decryption processing unit that decrypts data received by the communication unit 9.

[0118] The user interface 10 includes any user interface device such as a display device, an operation device, etc. By using the user interface 10, users such as a doctor, a patient, and an assistant can operate the ophthalmic apparatus 1 and input information to the ophthalmic apparatus 1. At least a part of the user interface 10 may be a peripheral device of the ophthalmic apparatus 1.

[0119] The display device displays various types of information under the control of the control unit 7. The display device may include a flat panel display such as a liquid crystal display (LCD). The operation device includes a device for operating the ophthalmic apparatus 1 and a device for inputting information. The operation device includes, for example, a button, a switch, a lever, a dial, a handle, a knob, a mouse, a keyboard, a trackball, an operation panel, etc. A device in which the display device and the operation device are integrated, such as a touch screen, may also be used.

[0120] The direction along the axis of the subject's eye E is defined as the Z direction, and the plane perpendicular to the Z direction is defined as the XY plane. The left-right direction (horizontal direction) for the subject is defined as the X direction, and the direction perpendicular to both the X and Z directions (up-down direction, body axis direction) is defined as the Y direction.

[0121] The optical path combining element 4 combines the optical path (illumination optical path) of the illumination optical system 2 with the optical path (imaging optical path) of the imaging optical system 3. The optical path combining element 4 may be, for example, a beam splitter such as a half mirror or a dichroic mirror. The optical axis of the optical path from the optical path combining element 4 toward the subject's eye E, in other words, the optical axis common to the illumination optical path and the imaging optical path, is the optical axis 5. That is, in this example, the illumination optical system 2 and the imaging optical system 3 are coaxially combined by the optical path combining element 4.

[0122] 1, the optical axis 2a of the illumination optical system 2 is arranged parallel to the optical axis 5, and the optical axis 3a of the photographing optical system 3 is arranged perpendicular to the optical axis 5, but the positional relationship between the illumination optical system 2 and the photographing optical system 3 is not limited to this. For example, the optical axis 2a of the illumination optical system 2 may be arranged perpendicular to the optical axis 5, and the optical axis 3a of the photographing optical system 3 may be arranged parallel to the optical axis 5.

[0123] One or more elements of the illumination optical system 2 and / or one or more elements of the photographing optical system 3 may be arranged closer to the subject's eye E than the optical path coupling element 4. For example, an optical system configuration may be adopted in which the illumination optical system 2 and the photographing optical system 3 share an objective lens arranged closer to the subject's eye E than the optical path coupling element 4.

[0124] It is also possible to employ an optical system configuration in which the illumination optical system 2 and the photographing optical system 3 are not coaxially coupled. For example, like a typical slit lamp microscope, the illumination optical system 2 and the photographing optical system 3 may be configured independently of each other, and further, the relative positions of the illumination optical system 2 and the photographing optical system 3 may be changeable.

[0125] In anterior segment imaging, the ophthalmic apparatus 1 (illumination optical system 2 and imaging optical system 3) is aligned with the subject's eye E. For example, alignment adjusts the position of the ophthalmic apparatus 1 so that the optical axis 5 coincides with the axis of the subject's eye E (XY alignment). Alignment also adjusts the position of the ophthalmic apparatus 1 so that the distance between the subject's eye E and the ophthalmic apparatus 1 coincides with a predetermined working distance (Z alignment). The manner of alignment is not limited to this example. Alignment is performed automatically or manually.

[0126] The illumination optical system 2 has a configuration for projecting illumination light onto the anterior segment of the subject's eye E. The illumination optical system 2 may have a configuration similar to that of an illumination optical system of a known ophthalmic device. Although not shown in the figure, the illumination optical system 2 includes an illumination light source, a lens, etc.

[0127] In a non-limiting example in which the ophthalmic apparatus 1 is a slit lamp microscope, the illumination optical system 2 includes an illumination light source, a positive lens, a slit forming unit, an objective lens, etc. Illumination light output from the illumination light source passes through the positive lens and is projected onto the slit forming unit. Light (slit light) that passes through the slit formed by the slit forming unit is refracted by the objective lens and projected onto the anterior segment of the subject's eye E. When the ophthalmic apparatus 1 is a slit lamp microscope, the optical path coupling element 4 is not necessary. When the ophthalmic apparatus 1 is another anterior segment imaging modality, a known configuration of that modality can be adopted for the illumination optical system 2.

[0128] The photographing optical system 3 is configured to photograph the anterior segment from the front onto which illumination light is projected by the illumination optical system 2. The photographing optical system 3 photographs the anterior segment from the front, for example, after the XY alignment (and Z alignment) described above has been performed. By photographing the anterior segment from the front, a front image of the anterior segment is obtained. The front image is an image expressed on the XY plane, and its pixel positions are defined by coordinates in the XY coordinate system.

[0129] The photographing optical system 3 may have a configuration similar to that of a photographing optical system of a known ophthalmic apparatus. Although not shown, the photographing optical system 3 includes a lens, an image sensor, etc. The photographing optical system 3 may function as a video camera capable of capturing moving images.

[0130] The photographing optical system 3 may be configured to be able to photograph images from a direction other than the front, in addition to photographing from the front.

[0131] The ophthalmologic apparatus 1 may be configured to construct a front image from images other than a front image. For example, the imaging optical system 3 may be configured to acquire an image (anterior segment oblique image) from an oblique position relative to the subject's eye E, and the data processing unit 8 may be configured to apply image processing such as projection to the anterior segment oblique image to generate a front image of the anterior segment. In another example, the illumination optical system 2 and the imaging optical system 3 may be configured to collect a data set by applying a slit scan or OCT scan using a Scheimpflug optical system to the anterior segment, and the data processing unit 8 may be configured to construct a three-dimensional anterior segment image from the data set and generate a front image of the anterior segment from the three-dimensional anterior segment image. In these cases, the front image of the anterior segment generated by the data processing unit 8 can be used as a substitute for the front image of the anterior segment acquired by the imaging optical system 3 in this example.

[0132] In a non-limiting example in which the ophthalmic apparatus 1 is a slit lamp microscope, the photographing optical system 3 includes an objective lens, a variable magnification optical system, an imaging lens, an image sensor, etc. Light incident on the photographing optical system 3 is refracted by the objective lens and the variable magnification optical system, and an image is formed on the imaging surface of the image sensor by the imaging lens. The image sensor may be an area sensor having a two-dimensional imaging area, such as a charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor. If the ophthalmic apparatus 1 is an anterior segment imaging modality of another type, the photographing optical system 3 may employ a known configuration of that modality.

[0133] The focal position changing unit 31 has a configuration for changing the focal position of the photographic optical system 3. The focal position changing unit 31 may have any configuration that can be employed to move the focal position. At least one element of the focal position changing unit 31 may be provided in the photographic optical system 3.

[0134] In some examples, the focus position changing unit 31 may be configured to move the focus position of the photographing optical system 3 by moving at least a part of the photographing optical system 3. In addition, in some examples, the focus position changing unit 31 may be configured to move the focus position of the photographing optical system 3 by changing the focal length of the photographing optical system 3. In addition, in some examples, the focus position changing unit 31 may be configured to move the focus position of the photographing optical system 3 by adding an optical element (such as a lens) to the photographing optical system 3 or by removing an optical element from the photographing optical system 3.

[0135] In one example, the focus position changing unit 31 may be configured to move the focus position by moving the entire photographing optical system 3 in a direction along the photographing optical axis. In another example, the focus position changing unit 31 may be configured to move the focus position by moving the objective lens of the photographing optical system 3 in a direction along the photographing optical axis. In yet another example, the focus position changing unit 31 may be configured to move the focus position by moving a focusing lens provided between the objective lens and the imaging lens in a direction along the photographing optical axis to change the focal length. In either example, the focus position changing unit 31 includes a mechanism for moving a movable object.

[0136] The exposure amount changing unit 32 has a configuration for changing the exposure amount in anterior eye imaging. The exposure amount changing unit 32 may have any configuration that can be employed to change the exposure amount. At least one element of the exposure amount changing unit 32 may be provided in the illumination optical system 2 or the imaging optical system 3.

[0137] In some examples, the exposure amount changing unit 32 may be configured to change the exposure amount by using the illumination optical system 2. In addition, in some examples, the exposure amount changing unit 32 may be configured to change the exposure amount by using the imaging optical system 3.

[0138] In one example, the exposure amount changing unit 32 changes the exposure amount by changing the exposure time of the image sensor (camera) of the photographing optical system 3. In this example, the exposure amount changing unit 32 includes an exposure time changing unit 32A having a configuration for changing the exposure time of the photographing optical system 3 (see FIG. 3A ). The exposure time changing unit 32A may include, for example, a mechanical shutter or an electronic shutter provided in the photographing optical path. Furthermore, the exposure time changing unit 32A may include an electronically controlled light amount adjustment filter arranged in the photographing optical path, or a light amount adjustment filter that can be inserted into or removed from the photographing optical path.

[0139] In another example, the exposure amount changing unit 32 changes the exposure amount by changing the intensity of the illumination light projected onto the anterior segment by the illumination optical system 2. The exposure amount changing unit 32 in this example includes an illumination intensity changing unit 32B having a configuration for changing the intensity of the illumination light (see FIG. 3B ). The illumination intensity changing unit 32B may include, for example, a light source driving circuit that drives the light source (illumination light source) of the illumination optical system 2, an electronically controlled light amount adjusting filter arranged in the illumination light path, or a light amount adjusting filter that can be inserted into or removed from the illumination light path.

[0140] The movement mechanism 6 is configured to integrally move the illumination optical system 2 and the photographing optical system 3. For example, the movement mechanism 6 may be capable of moving the illumination optical system 2 and the photographing optical system 3 in the X direction, Y direction, and Z direction. In cases where the ophthalmic apparatus 1 is a slit lamp microscope, for example, the movement mechanism 6 may be capable of moving the illumination optical system 2 and the photographing optical system 3 independently of each other.

[0141] The control unit 7 is configured to control each unit of the ophthalmologic apparatus 1. For example, the control unit 7 controls elements of the illumination optical system 2 (illumination light source, optical elements, mechanisms, etc.), elements of the imaging optical system 3 (optical elements, mechanisms, etc.), the focal position changing unit 31, the exposure amount changing unit 32, the movement mechanism 6, the data processing unit 8, the communication unit 9, the user interface 10, etc.

[0142] The control unit 7 includes a processor, a main storage device, an auxiliary storage device, etc. Computer programs such as various control programs are stored in the auxiliary storage device. These computer programs may be stored in a computer or storage device accessible to the ophthalmic apparatus 1. The functions of the control unit 7 are realized by cooperation between software such as the control programs and hardware such as the processor.

[0143] The control unit 7 includes an imaging control unit 71 (see FIGS. 2, 3A, and 3B). The imaging control unit 71 in this example performs a combination of control of the focus position changing unit 31, control of the exposure amount changing unit 32, and control of the imaging optical system 3. The control of the focus position changing unit 31 is performed to sequentially change the focus position of the imaging optical system 3 to a plurality of different focus positions (focus position change control). The control of the exposure amount changing unit 32 is performed to sequentially change the exposure amount for imaging the anterior segment to a plurality of different exposure amounts (exposure amount change control). The control of the imaging optical system 3 is performed to acquire time-series images of the anterior segment (imaging control).

[0144] By combining the focus position change control, the exposure amount change control, and the shooting control, for each of the plurality of different focus positions switched by the focus position change control, a plurality of images (first image groups) corresponding to the plurality of different exposure amounts switched by the exposure amount change control are collected by the shooting optical system 3. In this way, a plurality of first image groups corresponding to the plurality of different focus positions are acquired.

[0145] The data processing unit 8 is configured to perform various types of data processing. The data to be processed may be either data acquired by the ophthalmologic apparatus 1 or data input from an external source.

[0146] The data processing unit 8 includes a processor, a main storage device, an auxiliary storage device, etc. Computer programs such as various data processing programs are stored in the auxiliary storage device. These computer programs may be stored in a computer or storage device accessible to the ophthalmic apparatus 1. The functions of the data processing unit 8 are realized by cooperation between software such as the data processing programs and hardware such as the processor.

[0147] The data processing unit 8 includes an image processing unit 81 (see FIG. 2 ). The image processing unit 81 generates a single image (a single anterior-segment front image) by combining a plurality of first image groups corresponding to a plurality of focal positions acquired under the above-described combination control executed by the imaging control unit 71. Below, several examples of the image processing unit 81 will be described together with operations of the ophthalmologic apparatus 1 that can be performed by the image processing unit 81.

[0148] A first example of the image processing unit 81 will be described. The image processing unit 81 of this example first applies high dynamic range compositing to each of the first image groups acquired under the above-described combination control. This generates a plurality of high dynamic range images corresponding to the first image groups, respectively, i.e., a plurality of high dynamic range images corresponding to a plurality of different focus positions, respectively.

[0149] Furthermore, the image processing unit 81 of this example applies focus stacking to each of the generated multiple high dynamic range images to generate a single front image of the anterior segment.

[0150] In this way, the image processing unit 81 of this example is configured to generate a single frontal image of the anterior segment in which multiple parts of the anterior segment are depicted with appropriate brightness and without blurring, by combining high dynamic range synthesis corresponding to each focus position with focus synthesis corresponding to multiple different focus positions.

[0151] 4A and 4B, an example of an operation that can be performed using the image processing unit 81 of this example will be described. It is assumed that preparatory operations such as alignment and focusing have been performed in advance.

[0152] As shown in Figure 4A, first, the photography control unit 71 executes combined control of focus position change control for the focus position change unit 31, exposure amount change control for the exposure amount change unit 32, and photography control for the photography optical system 3, so that the ophthalmic device 1 acquires multiple frontal images of the anterior segment of the subject's eye E (multiple first image groups) (S1).

[0153] The combined control in step S1 may further include illumination control for the illumination optical system 2. The illumination control may include any control related to the projection of illumination light onto the anterior segment of the subject's eye E. For example, the illumination control may include any of the following controls: turning on / off the output of illumination light; changing the light characteristics (intensity, amount of light, wavelength, polarization, etc.) of the illumination light; or changing the projection conditions of the illumination light (projection position, projection angle, shape of the projection area, dimensions of the projection area, etc.).

[0154] The image group 100 in Fig. 4B is an example of the first group of M images acquired in step S1. In Fig. 4B, "LDR" indicates an image with a low dynamic range, and "HDR" indicates an image with a high dynamic range. Here, the high and low dynamic ranges may be relative to each other. Furthermore, "LDR" may mean that the image is not generated using high dynamic range synthesis or similar image processing, and "HDR" may mean that the image is generated using high dynamic range synthesis or similar image processing.

[0155] In this example, to acquire the image group 100 consisting of M first image groups, the following control conditions are applied: the condition for the focus position change control is to sequentially switch the focus position of the photographing optical system 3 to M different focus positions z1, z2, ..., zM (M is an integer of 2 or more); the condition for the exposure amount change control is to sequentially switch the exposure amount for photographing the anterior segment to three different exposure amounts e1, e2, e3; the condition for the photographing control is to perform (at least) one photographing for each of 3M pairs (zm, en) (m = 1 to M, n = 1 to 3) corresponding to all combinations of the M different focus positions z1, z2, ..., zM and the three different exposure amounts e1, e2, e3.

[0156] For example, the photographing control unit 71 first controls the focal position changing unit 31 to set the focal position of the photographing optical system 3 to a first focal position z1, and then controls the exposure amount changing unit 32 to sequentially switch the exposure amount among three different exposure amounts e1, e2, and e3, while controlling the photographing optical system 3 to photograph the anterior segment three times. As a result, three images LDR(z1, e1), LDR(z1, e2), and LDR(z1, e3) corresponding to the first focal position z1 are obtained. These three images LDR(z1, en) (n = 1 to 3) constitute a first image group corresponding to the first focal position z1.

[0157] Next, the photographing control unit 71 controls the focal position changing unit 31 to switch the focal position of the photographing optical system 3 from the first focal position z1 to the second focal position z2, and controls the exposure amount changing unit 32 to sequentially switch the exposure amount among three different exposure amounts e1, e2, and e3, while controlling the photographing optical system 3 to photograph the anterior segment three times. As a result, three images LDR(z2, e1), LDR(z2, e2), and LDR(z2, e3) corresponding to the second focal position z2 are obtained. These three images LDR(z2, en) (n = 1 to 3) constitute a first image group corresponding to the second focal position z2.

[0158] Similarly, the photographing control unit 71 controls the focal position changing unit 31 to switch the focal position of the photographing optical system 3 from the mth focal position zm to the m+1th focal position z(m+1), and also controls the photographing optical system 3 to photograph the anterior segment three times while controlling the exposure amount changing unit 32 to sequentially switch the exposure amount to three different exposure amounts e1, e2, and e3.

[0159] By performing the above three photographing operations for each of the M focal positions z1 to zM, three images LDR(zm, e1), LDR(zm, e2), and LDR(zm, e3) corresponding to three different exposure amounts e1, e2, and e3 are obtained for each focal position zm (m = 1 to M). That is, an image group 100 consisting of M first image groups corresponding to the M focal positions z1 to zM, respectively, is obtained, as shown in FIG. 4B .

[0160] The control procedure for collecting the M first image group (100) is not limited to this example. For example, while a first exposure amount e1 is being applied, M images LDR(zm, e1) (m = 1 to M) corresponding to the first exposure amount e1 may be acquired by performing M shootings while sequentially applying M different focal positions z1 to zM, while a second exposure amount e2 is being applied, M images LDR(zm, e2) (m = 1 to M) corresponding to the second exposure amount e2 may be acquired by performing M shootings while sequentially applying M different focal positions z1 to zM, while a third exposure amount e3 is being applied, M images LDR(zm, e3) (m = 1 to M) corresponding to the third exposure amount e3 may be acquired by performing M shootings while sequentially applying M different focal positions z1 to zM.

[0161] Once the acquisition of the multiple first image groups (S1) is completed, the image processing unit 81 applies high dynamic range synthesis to each of the multiple first image groups to generate multiple high dynamic range images corresponding to multiple different focus positions (S2).

[0162] In the example of Fig. 4B, the image processing unit 81 generates a high dynamic range image HDR(zm) by applying high dynamic range synthesis to three images LDR(zm, en) (n = 1 to 3) corresponding to each focal position zm (m = 1 to M). As a result, M high dynamic range images HDR(zm) (m = 1 to M) corresponding to M different focal positions z1 to zM are obtained (see reference numeral 101 in Fig. 4B).

[0163] Note that the process of step S1 and the process of step S2 can be performed partially in parallel. For example, immediately after three images LDR(zm, en) (n = 1 to 3) corresponding to the first focal position z1 are acquired, high dynamic range composition of these images may be performed while capturing an image corresponding to the second focal position z2.

[0164] Once the generation of multiple high dynamic range images corresponding to multiple different focus positions (S2) is complete, the image processing unit 81 applies focus stacking to these high dynamic range images to generate a single frontal image of the anterior segment (S3).

[0165] In the example of Figure 4B, the image processing unit 81 generates a single anterior segment front image HDR(z1-zM) by applying focus stacking to M high dynamic range images HDR(zm) (m = 1 to M) corresponding to M different focus positions z1 to zM, respectively (see reference numeral 102 in Figure 4B).

[0166] Each high dynamic range image HDR(zm) is a dynamic range expanded image generated from three images LDR(zm, e1), LDR(zm, e2), and LDR(zm, e3) corresponding to three different exposures, but its depth of field is only a narrow range (i.e., a shallow depth of field) that includes the mth focal plane corresponding to the mth focus position zm.

[0167] In contrast, the anterior eye front image HDR(z1-zM) 102 generated by applying focus stacking to M high dynamic range images HDR(1) to HDR(M) indicated by the reference numeral 101 is not only an expanded dynamic range image but also an expanded depth of field image having a deep depth of field that includes all of the M different focal planes corresponding to the M different focal positions z1 to zM, respectively.

[0168] The control unit 7 of this example performs control to output the anterior eye front image in which both the dynamic range and the depth of field have been expanded, generated in step S3 (S4).

[0169] For example, the control unit 7 can control the communication unit 9 to transmit the anterior eye front image to another device, control the user interface 10 (display device) to display the anterior eye front image, save the anterior eye front image in a storage device (not shown), control a printing device (not shown) to print the anterior eye front image, etc. This completes the operation of this example (END).

[0170] According to the ophthalmic device 1 of this example, the dynamic range can be expanded based on a group of first images corresponding to each focal position and corresponding to a plurality of different exposure amounts, and the depth of field can be expanded based on a plurality of groups of first images corresponding to a plurality of different focal positions.

[0171] 4B , the dynamic range can be expanded based on the first image groups LDR(zm, en) (n = 1 to 3) corresponding to each focus position zm and corresponding to three different exposure amounts en (n = 1 to 3), and the depth of field can be expanded based on an image group 100 made up of M first image groups corresponding to M different focus positions z1 to zM. This expansion of the depth of field is performed based on M dynamic range expanded images corresponding to M different focus positions z1 to zM.

[0172] In this way, the ophthalmic device 1 of this example is capable of generating a single frontal image of the anterior segment in which multiple parts of the anterior segment are depicted with appropriate brightness and without blur, thereby improving the quality of the image obtained by frontal photography of the anterior segment.

[0173] A second example of the image processing unit 81 will be described. As shown in Fig. 5A , the image processing unit 81 of this example generates a single anterior eye front image from a plurality of first image groups using a machine learning model 82 created in advance.

[0174] An example of an operation that can be performed using the image processing unit 81 of this example will be described with further reference to FIGS. 5B and 5C.

[0175] As shown in FIG. 5B , first, training data to be used for constructing the machine learning model 82 is prepared (S11). The training data is created, for example, based on images of a living eye (and / or images of a model eye (the same applies below)) and includes a plurality of pairs of a training image group and a training composite image. The training image group is, for example, a plurality of images collected by photographing a living eye multiple times under a plurality of shooting conditions including a plurality of different combinations of a plurality of different focus positions and a plurality of different exposure amounts. In other words, the training image group is a collection of images corresponding to pairs of focus positions and exposure amounts. The training composite image is, for example, an image obtained by combining the training image group. The dynamic range of the training composite image is wider than the dynamic range of each training image, and the depth of field thereof is deeper than the depth of field of each training image.

[0176] Furthermore, machine learning model 82 is constructed by applying machine learning to a neural network using such training data (S11). The neural network may be any mathematical model available for image processing, including, for example, a convolutional neural network.

[0177] An example of machine learning for constructing the machine learning model 82 will be described. A device for performing machine learning (machine learning model construction device) may be provided in the ophthalmologic apparatus 1 or in a separate device (computer).

[0178] The model construction unit 200 shown in FIG. 5C is an example of a machine learning model construction device, and includes a learning processing unit 201 and a neural network 202.

[0179] In this example, neural network 202 includes a convolutional neural network. Neural network 202 may be constructed using, for example, a known open-source neural network architecture. An example of the structure of a convolutional neural network is shown in FIG. 5C.

[0180] An image is input to the input layer. A plurality of pairs of convolutional layers and pooling layers are arranged behind the input layer. In the example shown in FIG. 5C, three pairs of convolutional layers and pooling layers are provided, but the number of pairs is arbitrary.

[0181] The convolution layer performs convolution operations to extract features (such as contours) from an image. A convolution operation is a multiplication and accumulation operation of a filter function (weighting coefficients, filter kernel) of the same dimension as the input image on the input image. The convolution layer applies the convolution operation to multiple portions of the input image. More specifically, the convolution layer multiplies each pixel value of the partial image to which the filter function has been applied by the value (weight) of the filter function corresponding to that pixel to calculate the product, and then calculates the sum of the products across multiple pixels of this partial image. The resulting sum-of-products value is assigned to the corresponding pixel in the output image. By performing the multiplication and accumulation operation while shifting the location (partial image) to which the filter function is applied, the convolution operation result for the entire input image is obtained. This convolution operation generates multiple images in which various features are extracted using multiple weighting coefficients. In other words, multiple filtered images such as smoothed images and edge images are obtained. The multiple images generated by the convolution layer are called feature maps.

[0182] The pooling layer compresses (e.g., thins out) the feature map generated by the immediately preceding convolutional layer. More specifically, the pooling layer calculates statistical values ​​of predetermined neighboring pixels of a pixel of interest in the feature map at predetermined pixel intervals (stride), and outputs an image with dimensions smaller than the input feature map. The statistical value applied to the pooling operation is, for example, the maximum value (max pooling) or the average value (average pooling).

[0183] A convolutional neural network can extract many features from an input image by processing it using multiple pairs of convolutional layers and pooling layers.

[0184] A fully connected layer is provided after the last pair of convolutional and pooling layers. Although two fully connected layers are provided in the example structure shown in FIG. 5C, any number of fully connected layers can be used. The fully connected layer performs processing such as image classification, image segmentation, and regression using features compressed by a combination of convolution and pooling. An output layer that provides output results is provided after the last fully connected layer.

[0185] In some exemplary embodiments, the convolutional neural network may not include a fully connected layer (e.g., a fully convolutional network (FCN)), or may include a support vector machine, a recurrent neural network (RNN), or the like. Furthermore, the machine learning performed on the neural network 202 may include transfer learning. Furthermore, the model construction unit 200 (learning processing unit 201) may be configured to be able to apply fine tuning to the neural network 202.

[0186] The learning processing unit 201 applies machine learning using training data to the neural network 202 (S11). When the neural network 202 includes a convolutional neural network, the parameters adjusted by the learning processing unit 201 include, for example, filter coefficients of the convolutional layer and connection weights and offsets of the fully connected layer.

[0187] As described above, the training data may include a set (dataset) of pairs of a training image group consisting of a plurality of images collected by photographing the anterior segment of a living eye multiple times under a plurality of photographing conditions, including a plurality of combinations of a plurality of focus positions and a plurality of exposure amounts, and a training composite image generated from the training image group. Since each pair included in such a training image group is the same type of image as the image input to the image processing unit 81 (machine learning model 82), it is considered that the quality (accuracy, precision, etc.) of the output of the image processing unit 81 can be improved compared to when machine learning is performed using training data consisting only of a different type of image.

[0188] However, the types of images included in the training data are not limited to the same types of images as the images input to the image processing unit 81, and may include, for example, images acquired by any ophthalmic modality (such as a slit lamp microscope, a fundus camera, an OCT device, or a surgical microscope), images acquired by an imaging diagnostic modality of another medical department (such as an ultrasound diagnostic device, an X-ray diagnostic device, an X-ray CT device, or a magnetic resonance imaging device), images generated by processing actual images, pseudo-images generated by a computer, etc. Furthermore, the number of images included in the training data may be increased using techniques such as data expansion and data augmentation.

[0189] The machine learning method for constructing the machine learning model 82 may be any method, such as supervised learning, semi-supervised learning, unsupervised learning, and reinforcement learning, or at least a partial combination of two or more of these. In some exemplary embodiments, supervised learning (or semi-supervised learning) is performed using training data in which input images are labeled with final output labels. For example, supervised learning can be performed using a group of training images as input images and training synthetic images as labels.

[0190] The machine learning model 82 constructed in this manner functions to receive input of a plurality of frontal anterior eye images (first image group) corresponding to a plurality of different combinations of a plurality of different focus positions and a plurality of different exposure amounts, and to output a composite image (an image with both an expanded dynamic range and depth of field) of the input plurality of frontal anterior eye images.

[0191] The technique used to construct the machine learning model 82 is not limited to the example shown here. For example, any technique such as a support vector machine, a Bayesian classifier, boosting, k-means, kernel density estimation, principal component analysis, independent component analysis, self-organizing map, random forest, or generative adversarial network (GAN) can be used to construct the machine learning model.

[0192] The machine learning model 82 constructed as described above is loaded into the image processing unit 81 (S12). That is, the image processing unit 81 can use the machine learning model 82. The machine learning model 82 may be included in the image processing unit 81, or may be included in a computer accessible by the image processing unit 81. The steps up to this point are preparation steps performed before the anterior segment of the subject's eye E is photographed.

[0193] Next, an anterior segment of the subject's eye E is photographed (S13). Specifically, similar to step S1 in Fig. 4A, the photographing control unit 71 executes a combined control of the focus position change control for the focus position change unit 31, the exposure amount change control for the exposure amount change unit 32, and photographing control for the photographing optical system 3, whereby the ophthalmologic apparatus 1 acquires a plurality of anterior segment front images of the subject's eye E (a plurality of first image groups).

[0194] The plurality of first image groups acquired in step S13 are sent to the image processing unit 81. The image processing unit 81 inputs the plurality of first image groups to the machine learning model 82 (S14). The machine learning model 82 generates a single anterior eye front image based on the plurality of input first image groups. This anterior eye front image has both an expanded dynamic range and an expanded depth of field compared to the input first image group.

[0195] The control unit 7 performs control to output the anterior eye front image generated in step S14 (S15), similar to step S4 in Fig. 4A. This completes the operation of this example (END).

[0196] As described above, the ophthalmologic apparatus 1 of the present example is configured to generate an anterior-segment front image with an expanded dynamic range and depth of field by inputting a plurality of first image groups corresponding to a plurality of different combinations of a plurality of different focus positions and a plurality of different exposure amounts into a machine learning model. The ophthalmologic apparatus 1 of the present example configured in this manner can generate an anterior-segment front image in which a plurality of regions of the anterior segment are depicted with appropriate brightness and without blur, thereby improving the quality of the image obtained by capturing the anterior segment frontally.

[0197] A third example of the image processing unit 81 will be described. In this example, instead of synthesizing all images included in the plurality of first image groups collected by anterior eye imaging, only a plurality of images selected from the plurality of first image groups are synthesized to generate a frontal image of the anterior eye with an expanded dynamic range and depth of field.

[0198] The first image group may include images captured during blinking or images contaminated with unwanted light, and combining these images together will result in a decrease in the quality of the resulting image. One of the purposes of this example is to address such problems.

[0199] The ophthalmologic apparatus 1 according to this embodiment acquires multiple first image groups by sequentially capturing multiple in-focus regions (depth regions corresponding to the depth of field) corresponding to multiple different focal positions, respectively. However, the number of types of anterior ocular tissue included in each in-focus region may vary. For example, the in-focus region for an image captured with the focal position positioned at or near the corneal apex includes only the cornea, while the in-focus region for an image captured with the focal position positioned at or near the anterior surface of the crystalline lens includes the crystalline lens and iris. Even when a focal region includes two or more types of anterior ocular tissue, there may be cases where a clear depiction of only one of the anterior ocular tissues is desired or sufficient. However, because each anterior ocular tissue has its own unique optical properties (e.g., reflectance, scattering, etc.), the brightness of an image captured with a constant exposure varies for each anterior ocular tissue. Therefore, for example, there may be cases where a single image capture with one exposure is sufficient at one focal position, while two or more images with two or more exposures are desired at another focal position. In other words, it is conceivable that high dynamic range compositing is not necessary at one focus position, but is desirable at another focus position. More generally, the number of images required for one focus position may differ from the number of images applied to another focus position. Alternatively, the number of exposures (number of shots) applied to one focus position may differ from the number of exposures (number of shots) applied to another focus position. One objective of this example is to achieve improvements in this respect.

[0200] 6A, the image processing unit 81 of this example includes an image selection unit 83 and an image synthesis unit 85. The image selection unit 83 includes a determination unit .

[0201] The image selection unit 83 selects one or more images from each of a plurality of first image groups acquired by the photographing optical system 3 under a combination of focus position change control, exposure amount change control, and photographing control executed by the photographing control unit 71. That is, the image selection unit 83 selects one or more images from the first image groups corresponding to each focus position applied in the focus position change control. A set of images selected from the plurality of first image groups by the image selection unit 83 is called a selected image group. The image synthesis unit 85 synthesizes the selected image groups to generate a single anterior eye front image with an expanded dynamic range and depth of field.

[0202] The image selection unit 83 may perform any process to extract a selected image group from the multiple first image groups. In this example, the image selection unit 83 selects one image from the first image group corresponding to a focal position included in a focal plane that intersects only one type of anterior ocular tissue, and selects two or more images from the first image group corresponding to a focal position included in a focal plane that intersects two or more different anterior ocular tissues. That is, the image selection unit 83 changes the number of images selected from the first image group depending on the number of types of anterior ocular tissue that intersect with the focal plane. The number of types of anterior ocular tissue that intersect with the focal plane may be the number of types of anterior ocular tissue that intersect with the focal plane itself, or the number of types of anterior ocular tissue included in the depth of field range (focus area) that includes the focal plane. In this example, the image selection unit 83 performs this image selection process using the determination unit 84, but the image selection process is not limited to this.

[0203] The determination unit 84 is configured to determine whether a focal plane including each of a plurality of different focal positions applied in the anterior segment imaging intersects with a plurality of different anterior segment tissues by analyzing an image corresponding to the focal position. This image analysis may include any image analysis method that can be employed to identify an image of the anterior segment, such as image segmentation (image segmentation using a machine learning model and / or image segmentation using an algorithm that does not use machine learning).

[0204] An example of an operation that can be performed using the image processing unit 81 of this example will be described with further reference to FIGS. 6B and 6C.

[0205] 6B, first, an anterior segment of the subject's eye E is photographed (S21). Specifically, similar to step S1 in Fig. 4A, the photographing control unit 71 executes a combination of focus position change control for the focus position changing unit 31, exposure amount change control for the exposure amount changing unit 32, and photographing control for the photographing optical system 3, whereby the ophthalmologic apparatus 1 acquires a plurality of anterior segment front images (a plurality of first image groups) of the subject's eye E. The plurality of first image groups acquired in step S21 are sent to the image processing unit 81.

[0206] The image selection unit 83 selects one or more images from each of the first image groups acquired for the different focus positions in step S21 (S22), thereby obtaining a selected image group.

[0207] In step S22, the determination unit 84 determines, for each focal position applied in the anterior segment imaging in step S21, whether a focal plane including that focal position intersects with two or more different anterior segment tissues by analyzing the image corresponding to that focal position. The determination unit 84 makes this determination by analyzing one or more images included in the corresponding first image group for each focal position. For example, the determination unit 84 may analyze each image included in the first image group to determine whether the image is good or bad (e.g., whether the image was obtained during a blink, whether the image contains an image of unnecessary light, etc.), and make this determination for one or more images determined to be good.

[0208] In this example, the image selection unit 83 may be configured to select one image from the first image group corresponding to a focal position included in a focal plane determined by the determination unit 84 to intersect only one type of anterior ocular tissue, and to select two or more images from the first image group corresponding to a focal position included in a focal plane determined by the determination unit 84 to intersect two or more different anterior ocular tissues. In the latter case, the number of images selected may be any number, for example, a predetermined number. Alternatively, the image selection unit 83 may be configured to determine the number of images to be selected from the first image group based on the number of anterior ocular tissues intersecting the focal plane. In either case, the image selection unit 83 may be configured to select only images that are not inappropriate images (such as images obtained during blinking or images containing unwanted light).

[0209] The selected image group obtained in step S22 is sent to the image composition unit 85. If there is a first image group in which two or more images are selected (S23: Yes), the image composition unit 85 applies high dynamic range composition to the two or more images selected from the first image group (S24). High dynamic range composition may be performed using a machine learning model and / or an algorithm that does not use machine learning. If one image has been selected from each of all first image groups (S23: No), step S24 is not executed.

[0210] Through the above processing, multiple anterior-segment front images are obtained, each corresponding to a different focal position applied to the anterior-segment imaging in step S21. That is, one anterior-segment image is obtained corresponding to each focal position. When the high dynamic range synthesis in step S24 is performed on one or more first image groups, one or more of the multiple obtained anterior-segment front images are high dynamic range images. Considering the structure of the anterior segment, a portion of the cornea and a portion of the sclera are located at almost the same depth, and a portion of the lens and a portion of the iris are located at almost the same depth. Therefore, in practice, an anterior-segment front image corresponding to any focal position is a high dynamic range image.

[0211] The image synthesis unit 85 applies focus synthesis to the multiple anterior-segment front images to generate a single anterior-segment front image (S25). Focus synthesis may be performed using a machine learning model and / or an algorithm that does not use machine learning. The anterior-segment front image generated in step S25 is an image with an extended depth of field and (in practice) an extended dynamic range.

[0212] The image synthesis unit 85 of this example may be configured to perform image processing (image generation) that integrates the high dynamic range synthesis of step S24 and the focus synthesis of step S25 using a machine learning model.

[0213] The control unit 7 performs control to output the anterior eye front image generated in step S25 (S26), similar to step S4 in Fig. 4A. This completes the operation of this example (END).

[0214] A specific example of the operation of FIG. 6B will be described with further reference to FIG. 6C. In this example, an image group (plurality of first image groups) indicated by reference numeral 110 in FIG. 6C is obtained by photographing the anterior segment of the eye in step S21. The image group 110 is a collection of images similar to the image group 100 in FIG. 4B, and is made up of M first image groups corresponding to M focal positions z1 to zM, respectively. Each first image group is made up of three images LDR(zm, e1), LDR(zm, e2), and LDR(zm, e3) corresponding to three different exposure amounts e1, e2, and e3, respectively.

[0215] In the example of FIG. 6C, in step S22, the image selection unit 83 obtains a selected image group 111 from the image group 110. The selected image group 111 performs the following image selection process: select one image LDR(z1, e1) from three images LDR(z1, e1), LDR(z1, e2), LDR(z1, e3) included in the first image group corresponding to the first focus position z1; select two images LDR(z2, e1), LDR(z2, e2) from three images LDR(z2, e1), LDR(z2, e2), LDR(z2, e3) included in the first image group corresponding to the second focus position z2; select one image LDR(zM, e3) from three images LDR(zM, e1), LDR(zM, e2), LDR(zM, e3) included in the first image group corresponding to the Mth focus position zM.

[0216] Here, it is assumed that one image is selected from each of the M-3 first image groups corresponding to the third to (M-1)th focal positions, respectively, and the number of images included in the selected image group 111 is M+1.

[0217] In the example of Figure 6C, in step S24, the image synthesis unit 85 applies high dynamic range synthesis to two images LDR(z2, e1) and LDR(z2, e2) corresponding to the second focus position z2 to generate a high dynamic range image HDR(z2).

[0218] This results in an image group 112. The image group 112 includes a high dynamic range image HDR(z2) corresponding to the second focal position z2 and images LDR(zm) corresponding to each of the other focal positions zm (m = 1 to M, m ≠ 2). Here, the images LDR(zm) in the image group 112 are the same as the images LDR(zm, en) in the selected image group 111 (m = 1 to M, m ≠ 2; n is 1, 2, or 3).

[0219] In the example of FIG. 6C, in step S25, the image synthesis unit 85 applies focus synthesis to M images included in the image group 112 to generate an anterior eye front image HDR(z1-zM) 113.

[0220] The anterior eye front image HDR(z1-zM) 113 is an image in which the dynamic range in the depth region corresponding to the second focal position z2 is expanded and the depth of field is expanded across the depth regions corresponding to multiple different focal positions z1 to zM.

[0221] Therefore, in the anterior eye front image HDR(z1-zM) 113, multiple types of anterior eye tissues present in the depth region corresponding to the second focal position z2 are each depicted with suitable brightness and without blur, and one anterior eye tissue present in the depth region corresponding to another focal position zm (m ≠ 2) is also depicted with suitable brightness and without blur.

[0222] In this way, the ophthalmic device 1 of this example is capable of generating a single frontal image of the anterior segment in which multiple parts of the anterior segment are depicted with appropriate brightness and without blur, thereby improving the quality of the image obtained by frontal photography of the anterior segment.

[0223] In this example, a plurality of images (a plurality of first image groups) corresponding to a plurality of different combinations of a plurality of different focus positions and a plurality of different exposure amounts are collected, and a selected image group is extracted from the plurality of first image groups to perform image synthesis. This example has the unique effect of being able to generate a synthetic image while excluding defective images.

[0224] On the other hand, when a plurality of different focal positions to be used for anterior segment imaging are predetermined, the number of imaging times at each focal position (i.e., the number and value of exposure doses to be applied) can be predetermined based on the relationship between these focal positions and the structure of the anterior segment. As a result, for example, in the case of the specific example of FIG. 6C , two imaging times (exposure doses e1 and e2) are performed at the second focal position z2, and one imaging time (default exposure dose) is performed at each of the other focal positions zm, thereby obtaining a group of images equivalent to the selected image group 111. This example can achieve unique effects such as a reduction in imaging time, a reduction in the burden on the subject, and an improvement in examination throughput.

[0225] A description will be given of another aspect of the ophthalmic apparatus 1. The configuration of the ophthalmic apparatus 1 of this aspect may be similar to the configuration shown in Figures 1 to 3B.

[0226] In the above-described aspect, the ophthalmologic apparatus 1 acquires image groups (plurality of first image groups) corresponding to all combinations of a plurality of different focus positions and a plurality of different exposure amounts by executing a combination of focus position change control, exposure amount change control, and imaging control. Therefore, in the above-described aspect, the anterior segment is imaged (at least) a number of times equal to the product of the number of the plurality of different focus positions and the number of the plurality of different exposure amounts.

[0227] In contrast, in this embodiment, the ophthalmic apparatus 1 executes a combination of focus position change control, exposure amount change control, and imaging control to acquire a group of images (second image group) corresponding to a plurality of pairs of focus positions and exposure amounts that correspond to each other through a one-to-one correspondence between the plurality of different focus positions and the plurality of different exposure amounts. In other words, the ophthalmic apparatus 1 executes a combination of focus position change control, exposure amount change control, and imaging control to acquire a second image group consisting of a plurality of images corresponding to a plurality of pairs of focus positions and exposure amounts that correspond to each other through a bijection defined between a set of a plurality of different focus positions and a set of a plurality of different exposure amounts. Here, because a bijection is defined between the set of focus positions and the set of exposure amounts, the number of elements in both sets is equal. Therefore, the number of imaging operations performed in this embodiment for anterior eye imaging is equal to the number of elements in these sets. If the number of focus positions (number of exposure amounts) applied to anterior eye imaging is the same between the above embodiment and this embodiment, the number of imaging operations in this embodiment is less than that in the above embodiment.

[0228] An example of an operation that can be performed by the ophthalmologic apparatus 1 of this embodiment will be described with further reference to FIGS. 7A and 7B.

[0229] As shown in Fig. 7A , first, an anterior segment of the subject's eye E is photographed (S31). The photographing control unit 71 of this embodiment executes a combination control of the focus position change control for the focus position change unit 31, the exposure amount change control for the exposure amount change unit 32, and photographing control for the photographing optical system 3, similar to step S1 of Fig. 4A . However, unlike step S1 of Fig. 4A , the combination control is executed to acquire a second group of images corresponding to a plurality of pairs of focus positions and exposure amounts that correspond to each other in one-to-one correspondence between a plurality of different focus positions and a plurality of different exposure amounts.

[0230] 7B , the ophthalmologic apparatus 1 of this embodiment performs anterior ocular segment imaging multiple times while sequentially applying multiple pairs (zm, em) of focal positions and exposure amounts, thereby acquiring a second image group 120 consisting of multiple images LDR(z1, e1), LDR(z2, e2), ..., LDR(zM, eM) corresponding to the multiple pairs (zm, em). The second image group acquired in step S31 is sent to the image processing unit 81.

[0231] The image processing unit 81 synthesizes the second image group acquired in step S31 to generate a single anterior-segment front image (S32). This image synthesis process includes both a process for expanding the dynamic range and a process for expanding the depth of field. In the example of FIG. 7B , the image synthesis process is applied to the second image group 120 to generate an anterior-segment front image HDR(z1-zM) 121.

[0232] For example, the image processing unit 81 may be configured to perform image processing (image generation) that integrates high dynamic range synthesis and focus stacking using a machine learning model. In another example, the image processing unit 81 may be configured to apply high dynamic range synthesis to the second image group to generate an expanded dynamic range image (a high dynamic range image), apply focus stacking to the second image group to generate an expanded depth of field image, and then combine the expanded dynamic range image and the expanded depth of field image to generate a single frontal image of the anterior eye segment. The image synthesis processing that can be performed in step S32 is not limited to these examples.

[0233] The anterior-segment front image with both the expanded dynamic range and depth of field generated in step S32 is sent to the control unit 7. The control unit 7 performs control to output this anterior-segment front image (S33), similar to step S4 in Fig. 4A. This completes the operation of this example (END).

[0234] In this way, the ophthalmologic apparatus 1 of this embodiment can generate a single frontal image of the anterior segment in which multiple regions of the anterior segment are depicted with appropriate brightness and without blur, thereby improving the quality of images obtained by frontal imaging of the anterior segment. Another advantage is that the number of times the anterior segment is imaged can be reduced.

[0235] In the present embodiment described above, the photographing control unit 71 is configured to cause the photographing optical system 3 to acquire a second image group corresponding to a plurality of pairs (a plurality of pairs of focal positions and exposure amounts) determined by one-to-one correspondence between the plurality of different focal positions and the "plurality of different exposure amounts," by combining focus position change control that controls the focus position change unit 31 to sequentially change the focus position of the photographing optical system 3 to a plurality of different focus positions, exposure amount change control that controls the exposure amount change unit 32 to sequentially change the exposure amount for photographing the anterior segment to "a plurality of different exposure amounts," and photographing control that controls the photographing optical system 3 to acquire time-series images. That is, the plurality of images included in the second image group correspond to different exposure amounts, respectively.

[0236] In contrast, in some examples, two (or three or more) images included in the second image group may correspond to the same exposure amount. In this case, the photographing control unit 71 is configured to cause the photographing optical system 3 to acquire a second image group corresponding to multiple pairs (multiple pairs of focal positions and exposure amounts) determined by one-to-one correspondence between the multiple different focal positions and the multiple exposure amounts. Typically, the number of different exposure amounts is equal to or less than the number of different focal positions. According to this example, it is possible to improve image quality with a small number of photographs.

[0237] 8, the exposure amounts corresponding to the focal positions z1, z2, and z(M-2) are all the same value e1, and the exposure amounts corresponding to the focal positions z3, z4, and z(M-1) are all the same value e2. In this example, the second image groups 130 are combined to generate a single frontal anterior eye image with an expanded dynamic range and depth of field.

[0238] In this aspect, each pair of a focal position and an exposure amount may be a combination of a focal position corresponding to one of a plurality of different anterior segment tissues and an exposure amount predetermined based on the scattering strength of the anterior segment tissue. By setting multiple pairs according to this example, it is possible to prepare default imaging conditions according to the scattering strength of a plurality of different anterior segment tissues (such as the cornea, the lens, and the iris), thereby facilitating and simplifying anterior segment imaging.

[0239] Instead of or in addition to preparing default photographing conditions, the ophthalmologic apparatus 1 of this embodiment may have a function of determining photographing conditions (one or more pairs of a focal position and an exposure amount). An example of the configuration of the ophthalmologic apparatus 1 having the photographing condition determination function is shown in Fig. 9. The configuration of Fig. 9 is obtained by adding an anterior eye profile acquisition unit 40 and a photographing condition determination unit 86 to the configuration of Fig. 2.

[0240] The anterior segment profile acquisition unit 40 acquires a depth profile of the anterior segment of the subject's eye E. The depth profile may be any information indicating the distribution of anterior segment data in the axial direction (Z direction). For example, the depth profile may be any of an anterior segment cross-sectional image acquired with a slit lamp microscope, anterior segment data acquired with an OCT scan (A-scan data, A-scan image, B-scan image, 3D image, etc.), anterior segment data acquired with an ultrasound scan (A-scan data, A-scan image, B-scan image, 3D image, etc.), and anterior segment data acquired by axial length measurement.

[0241] The anterior segment profile acquisition unit 40 includes, for example, a depth profile generation unit configured to generate a depth profile and / or a depth profile reception unit configured to receive a depth profile from an external device. The depth profile generation unit may be, for example, an element of the data processing unit 8. The function of the depth profile reception unit can be realized using, for example, a communication unit 9 that receives a depth profile from an external device, a reading device that reads a depth profile recorded on a recording medium, or an image scanner that converts a depth profile recorded on a print medium into digital data.

[0242] The photographing condition determination unit 86 determines multiple pairs of focus positions and exposure amounts to be applied when photographing the anterior segment of the subject's eye E based on the depth direction profile of the anterior segment of the subject's eye E acquired by the anterior segment profile acquisition unit 40.

[0243] The depth profile acquired by the anterior segment profile acquisition unit 40 includes information indicating the positions of various anterior segment tissues in the subject's eye EE (tissue position information) and images of various anterior segment tissues (tissue images). For example, A-scan data of the anterior segment includes peaks indicating the positions of the cornea (anterior corneal surface, posterior corneal surface) and peaks indicating the positions of the crystalline lens (anterior capsule, posterior capsule). Furthermore, the anterior segment image includes images of the cornea, the anterior chamber, the iris, the crystalline lens, etc.

[0244] When the depth profile includes tissue position information, such as A-scan data, the imaging condition determination unit 86 identifies the depth positions of multiple different anterior segment tissues of the subject's eye E, for example, using processing for detecting tissue position information from the depth profile. This processing includes signal processing such as peak detection. Furthermore, the imaging condition determination unit 86 determines multiple pairs of focal position and exposure dose based on the multiple depth positions identified for each of the multiple different anterior segment tissues of the subject's eye E and the scattering intensity information of these anterior segment tissues. The scattering intensity information may be, for example, a standard value of the scattering intensity of the anterior segment tissue, a measured value of the scattering intensity of the anterior segment tissue obtained by measurement using OCT, or the like.

[0245] When the depth profile includes a tissue image, such as an image obtained by a slit lamp microscope or an OCT device, the imaging condition determination unit 86 identifies the depth positions of a plurality of different anterior segment tissues of the subject's eye E, for example, by using a process for detecting a tissue image (the position of the tissue image) from the depth profile. This process includes image processing such as segmentation. Furthermore, the imaging condition determination unit 86 determines a plurality of pairs of a focal position and an exposure amount based on the plurality of depth positions identified for the plurality of different anterior segment tissues of the subject's eye E and the scattering intensity information of these anterior segment tissues.

[0246] According to this example, it is possible to determine a plurality of pairs of focal position and exposure amount as imaging conditions based on the anterior segment data (depth profile) of the subject's eye E, and to collect the second image group using these imaging conditions. Therefore, it is possible to perform imaging under conditions suited to each individual subject's eye.

[0247] As described above, the plurality of pairs of focus positions and exposure amounts may include only a plurality of pairs determined by one-to-one correspondence between a plurality of different focus positions and a plurality of different exposure amounts, or may include other pairs. In the former case, one image is acquired for each focus position. In the latter case, for example, two or more images may be acquired for a certain focus position. Note that the example of FIG. 8 (where the number of different focus positions applied to anterior segment imaging is greater than the number of different exposure amounts applied) corresponds to another example of the latter case.

[0248] In this embodiment, an example in which two or more images are acquired for a certain focal position will be described. In this example, the shooting control unit 71 causes the shooting optical system to acquire two or more images corresponding to two or more different exposure amounts for each of one or more focal positions among the plurality of different focal positions. In this example, the second image group includes two or more images for each of the one or more focal positions among the plurality of different focal positions, and includes one image for each of the other one or more focal positions.

[0249] The imaging control unit 71 of this example may be configured to cause the imaging optical system to acquire two or more images corresponding to two or more different exposure amounts for focal positions included in a focal plane that intersects multiple different anterior segment tissues. This configuration may be similar to the similar example described above.

[0250] To this end, the image processing unit 81 of this example may include a determination unit 87 (see FIG. 10A ). The determination unit 87 is configured to determine, for each of a plurality of different focal positions, whether a focal plane including the focal position intersects with a plurality of different anterior ocular tissues. The configuration and operation of the determination unit 87 of this example may be similar to the configuration and operation of the determination unit 84 of FIG. 6A . To make the above determination, the determination unit 87 of this example may be configured to analyze images corresponding to each focal position (such as an anterior ocular segment image of the subject's eye E, an anterior ocular segment image of another eye, or an anterior ocular segment image of a standard eyeball model).

[0251] The imaging control unit 71 of this example executes control (combined control of focal position change control, exposure amount change control, and imaging control) to cause the imaging optical system 3 to acquire a second image group based on the determination result obtained by the determination unit 87. As a result, for focal positions corresponding to focal planes determined to intersect with multiple different anterior segment tissues, two or more imaging sessions corresponding to two or more different exposure amounts are performed to acquire two or more images. On the other hand, for focal positions corresponding to focal planes determined to intersect with only one anterior segment tissue, one or more imaging sessions corresponding to one exposure amount are performed to acquire one image. The multiple images collected by this series of imaging form the second image group of this example.

[0252] A specific example of this example will be described. The second image group 140 in Fig. 10B includes the following images: for the focus position z1, it includes two images LDR(z1, e1) and LDR(z1, e2) corresponding to two different exposure amounts e1 and e2; for the focus position z2, it includes three images LDR(z2, e1), LDR(z2, e2), and LDR(z2, e3) corresponding to three different exposure amounts e1, e2, and e3; for the focus position z3, it includes one image LDR(z3, e2) corresponding to one exposure amount e2; and for the focus position zM, it includes two images LDR(zM, e2) and LDR(zM, e3) corresponding to two different exposure amounts e2 and e3. Although not shown, for focus positions other than these, it includes one or more images corresponding to one or more exposure amounts.

[0253] In this example, the image processing unit 81 applies high dynamic range synthesis to two or more corresponding images for focal positions corresponding to focal planes determined to intersect multiple different anterior segment tissues to generate a high dynamic range image. The generated high dynamic range image is used for focus synthesis. On the other hand, one image acquired for a focal position corresponding to a focal plane determined to intersect only one anterior segment tissue is used directly for focus synthesis. The image group 141 in Figure 10B consists of multiple images obtained through this processing.

[0254] Furthermore, the image processing unit 81 of this example generates a single frontal anterior segment image by applying focus stacking to an image group including one or more high dynamic range images generated in the above manner. The image HDR(z1-zM) 142 in FIG. 10B corresponds to the single frontal anterior segment image generated in this manner.

[0255] According to this example, it is possible to generate a frontal image of the anterior segment with both an expanded dynamic range and depth of field, and it is also possible to automatically determine the focal position (depth region) to which high dynamic range synthesis is applied when photographing the anterior segment of each subject's eye.

[0256] One variant, similar to a similar example of the above-mentioned aspect, involves pre-setting a focus position (default focus position) at which high dynamic range synthesis is applied, acquiring two or more images corresponding to two or more different exposures when photographed at that focus position, generating a high dynamic range image from the two or more acquired images, and generating a single frontal image of the anterior segment based on a group of images including the generated high dynamic range images.

[0257] In this aspect, any method may be used for generating a single anterior-segment front image from the second image group (image synthesis). For example, image synthesis can be performed using a machine learning model. The image processing unit 81 of this example includes a machine learning model (see FIG. 5A ). The machine learning model of this example receives input of multiple images with different focus positions and exposure amounts, and is trained to output a synthesized image of the multiple input images. When the second image group of this example is input to this machine learning model, a single anterior-segment front image, which is a synthesized image of the second image group, is output.

[0258] The machine learning model in this example may have any configuration. For example, the machine learning model may include a convolutional neural network constructed by performing supervised learning using predetermined training data. This training data may include, for example, a plurality of pairs of training images with different focus positions and exposure amounts and training composite images that are composite images of the training image group.

[0259] In another example, the technique described in the following document can be used: Xin Huang et al., "Inverting the Imaging Process by Learning an Implicit Camera Model", https: / / doi.org / 10.48550 / arXiv.2304.12748, submitted on 25 April 2023.

[0260] Image synthesis may be performed without using machine learning. For example, the image processing unit 81 in this example may be configured to generate a single anterior segment front image by applying image processing including both focus synthesis and high dynamic range synthesis to the second group of images. The image processing in this example can be performed using, for example, the technique described in the following document: Qinchun Qian and Bahadir K. Gunturk, "Extending depth of field and dynamic range from differently focused and exposed images," Multidimensional Systems and Signal Processing, 27, 493-509 (2016), https: / / doi.org / 10.1007 / s101603-016-01603-3. org / 10.1007 / s11045-015-0315-x.

[0261] A description will now be given of yet another aspect of the ophthalmic apparatus 1. The configuration of the ophthalmic apparatus 1 of this aspect may be similar to the configuration shown in Figures 1 to 3B, but may not include the exposure amount changing unit 32 as shown in Figure 11.

[0262] In the above-described embodiment, the ophthalmic apparatus 1 acquires an image group by performing a combination of focus position change control, exposure amount change control, and imaging control. In contrast, the ophthalmic apparatus 1 of the present embodiment acquires an image group (third image group) by performing a combination of focus position change control and imaging control without performing exposure amount change control.

[0263] In the focus position change control of this aspect, the photography control unit 71 controls the focus position change unit 31 to sequentially change the focus position of the photographic optical system 3 to a plurality of different focus positions. Also, in the photography control of this aspect, the photography control unit 71 controls the photographic optical system 3 to acquire time-series images at a constant exposure amount. The anterior eye segment photography of this aspect differs from the above aspect in that it is performed at a constant exposure amount.

[0264] The imaging control unit 71 combines such focus position change control and imaging control to cause the imaging optical system 3 to acquire a third image group corresponding to a plurality of different focus positions. The third image group is made up of a plurality of images corresponding to a plurality of different focus positions. The third image group includes only one image for each focus position. If the plurality of different focus positions are z1, z2, ..., zM (M number of images), the third image group is made up of M images. The image processing unit 81 combines the third image group to generate a single frontal image of the anterior eye segment.

[0265] An example of an operation that can be performed by the ophthalmologic apparatus 1 of this embodiment will be described with further reference to FIGS. 12A and 12B.

[0266] As shown in Fig. 12A , first, an anterior segment of the subject's eye E is photographed (S41). Unlike step S1 in Fig. 4A , the photographing control unit 71 of this embodiment executes combined control of focal position change control for the focal position changing unit 31 and photographing control for the photographing optical system 3. A constant exposure amount (the same exposure amount) is applied to the multiple photographs thus taken. This exposure amount may be a predetermined default exposure amount, or may be an exposure amount set based on various photographing conditions (such as the state of the subject's eye E and the main area to be observed). In step S41, the ophthalmologic apparatus 1 of this embodiment acquires a third group of images corresponding to multiple different focal positions.

[0267] 12B , the ophthalmologic apparatus 1 of this embodiment performs M imaging operations at M focal positions z1, z2, ..., zM with a constant exposure amount e1 to acquire a third image group 150 including M images LDR(z1, e1), LDR(z2, e1), ..., LDR(zM, e1) corresponding to the M focal positions z1, z2, ..., zM, respectively. The third image group acquired in step S41 is sent to the image processing unit 81.

[0268] The image processing unit 81 generates a plurality of high dynamic range images corresponding to a plurality of different focal positions from the third image group acquired in step S41 (S42). In the example of Fig. 12B, a high dynamic range image group 151 consisting of M high dynamic range images HDR(z1), HDR(z2), ..., HDR(zM) is generated from M images LDR(z1, e1), LDR(z2, e1), ..., LDR(zM, e1). That is, for each of a plurality of focal positions zm (m = 1 to M), a high dynamic range image HDR(zm) is generated from the image LDR(zm, e1) corresponding to focal position zm.

[0269] The high dynamic range image generation in step S42 differs from general high dynamic range synthesis, which synthesizes multiple images obtained by photographing at multiple different exposures, in that it is a process of expanding the dynamic range of a single image (in other words, a process of generating a pseudo high dynamic range image).

[0270] Various methods have been proposed for such processing, many of which utilize machine learning techniques. In this embodiment, for example, the technique described in the following document can be used: Lin Wang and Kuk-Jin Yoon, "Deep Learning for HDR Imaging: State-of-the-Art and Future Trends," JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015, https: / / doi.org / 10.48550 / arXiv.2110.10394. As described in the document, other techniques that can be used in this embodiment include techniques based on the number or area of ​​input exposures, techniques based on the number of learning tasks, techniques based on novel sensor data, techniques based on novel learning strategies, and techniques based on applications.

[0271] The image processing unit 81 of this embodiment includes a machine learning model. This machine learning model is trained to receive an input of an image having a first dynamic range and output an image having a second dynamic range greater than the first dynamic range. By inputting each image of the third image group into this machine learning model, multiple high dynamic range images corresponding to multiple different focus positions are obtained.

[0272] Next, the image processing unit 81 applies focus stacking to the multiple high dynamic range images generated in step S42 to generate a single anterior segment front image (S43). This anterior segment front image is an image in which both the dynamic range and the depth of field are expanded. In the example of FIG. 12B , focus stacking is applied to the high dynamic range image group 151 to generate an anterior segment front image HDR(z1-zM) 152.

[0273] The anterior eye front image generated in step S43 is sent to the control unit 7. The control unit 7 performs control to output this anterior eye front image (S44), similar to step S4 in Fig. 4A. This completes the operation of this example (END).

[0274] In this way, the ophthalmologic apparatus 1 of this embodiment can generate a single anterior-segment front image in which multiple anterior-segment regions are depicted with appropriate brightness and without blur, thereby improving the quality of images obtained by frontal imaging of the anterior segment. Another advantage is that the number of times the anterior segment is imaged can be reduced. Furthermore, since exposure dose change control is not performed during anterior-segment imaging, another advantage is that control can be facilitated and simplified.

[0275] A description will now be given of yet another aspect of the ophthalmic apparatus 1. The configuration of the ophthalmic apparatus 1 of this aspect may be similar to the configuration shown in Figures 1 to 3B, but may not include the focal position changing unit 31 as shown in Figure 13.

[0276] 2, 9, and 10A, the ophthalmic apparatus 1 acquires an image group by performing a combination of focus position change control, exposure amount change control, and photography control. Also, in the aspect shown in Fig. 11, the ophthalmic apparatus 1 acquires an image group by performing a combination of focus position change control and photography control. In contrast, the ophthalmic apparatus 1 of the present aspect shown in Fig. 13 acquires an image group (fourth image group) by performing a combination of exposure amount change control and photography control without performing focus position change control.

[0277] In the exposure amount change control of this embodiment, the photography control unit 71 controls the exposure amount change unit 32 to sequentially change the exposure amount for anterior eye photography to a plurality of different exposure amounts. Also, in the photography control of this embodiment, the photography control unit 71 controls the photography optical system 3 to acquire time-series images at a constant focal position. The anterior eye photography of this embodiment differs from the above embodiment in that it is performed at a constant focal position.

[0278] The photographing control unit 71 combines such exposure amount change control and photographing control to cause the photographing optical system 3 to acquire a fourth image group corresponding to a plurality of different exposure amounts. The fourth image group is made up of a plurality of images corresponding to a plurality of different exposure amounts. The fourth image group includes only one image for each exposure amount. If the plurality of different exposure amounts are e1, e2, ..., eN (N images), the fourth image group is made up of N images. The image processing unit 81 combines the fourth image group to generate a single anterior eye front image.

[0279] An example of an operation that can be performed by the ophthalmologic apparatus 1 of this embodiment will be described with further reference to FIGS. 14A and 14B.

[0280] As shown in Fig. 14A , first, an anterior segment of the subject's eye E is photographed (S51). Unlike step S1 in Fig. 4A , the photographing control unit 71 of this embodiment executes combined control of exposure amount change control for the exposure amount change unit 32 and photographing control for the photographing optical system 3. A fixed focal position (the same focal position) is applied to multiple photographs thus taken. This focal position may be a predetermined default focal position, or may be a focal position set based on various photographing conditions (such as the state of the subject's eye E and the main area to be observed). In step S51, the ophthalmologic apparatus 1 of this embodiment acquires a fourth group of images corresponding to multiple different exposure amounts.

[0281] 14B , the ophthalmologic apparatus 1 of this embodiment performs N imaging operations at N exposure amounts e1, e2, ..., eN at a constant focal position z1 to acquire a fourth image group 160 including N images LDR(z1, e1), LDR(z1, e2), ..., LDR(z1, eN) corresponding to the N exposure amounts e1, e2, ..., eN, respectively. The fourth image group acquired in step S51 is sent to the image processing unit 81.

[0282] The image processing unit 81 generates a plurality of depth-of-field extended images corresponding to a plurality of different exposure amounts from the fourth image group acquired in step S51 (S52). In the example of FIG. 14B , a depth-of-field extended image group 161 consisting of N depth-of-field extended images LDR(z1-zM,e1), LDR(z1-zM,e2), ..., LDR(z1-zM,eN) is generated from N images LDR(z1,e1), LDR(z1,e2), ..., LDR(z1,eN). That is, for each of a plurality of exposure amounts en (n = 1 to N), a depth-of-field extended image LDR(z1-zM,en) is generated from the image LDR(z1,en) corresponding to the exposure amount en. Here, the image LDR(z1,en) corresponding to the exposure amount en has a depth of field corresponding to the focal position z1, and the depth-of-field expanded image LDR(z1-zM,en) generated from this image LDR(z1,en) has a deep depth of field corresponding to M different focal positions z1 to zM.

[0283] The depth of field extension in step S52 is a process of extending the depth of field of a single image (in other words, a process of generating a pseudo-depth-extended image), unlike general depth of field extension (focus synthesis) which combines multiple images obtained by photographing at multiple different focus positions.

[0284] Various methods have been proposed for such processing, many of which utilize machine learning techniques. In this aspect, for example, a technique for extending the depth of field using a phase mask and deep learning described in the following document can be employed: Lingbo Jin et al. "Deep learning extended depth-of-field microscope for fast and slide-free histology", The Proceedings of the National Academy of Science (PNAS), December 14, 2020, 117 (52) 33051-33060, https: / / doi.org / 10.1007 / s10240-018-0180-3 org / 10.1073 / pnas. 2013571117.

[0285] The image processing unit 81 of this embodiment includes a machine learning model. This machine learning model is trained to receive an input of an image having a first depth of field and output an image having a second depth of field deeper than the first depth of field. By inputting each image of the fourth image group into this machine learning model, a plurality of extended depth-of-field images corresponding to a plurality of different exposure amounts are obtained.

[0286] Next, the image processing unit 81 applies high dynamic range compositing to the plurality of extended depth-of-field images generated in step S52 to generate a single anterior-segment front image (S53). This anterior-segment front image is an image in which both the dynamic range and the depth of field have been extended. In the example of FIG. 14B , a front anterior-segment image HDR(z1-zM) 162 is generated by applying high dynamic range compositing to the extended depth-of-field image group 161.

[0287] The anterior eye front image generated in step S53 is sent to the control unit 7. The control unit 7 performs control to output this anterior eye front image (S54), similar to step S4 in Fig. 4A. This completes the operation of this example (END).

[0288] In this way, the ophthalmologic apparatus 1 of this embodiment can generate a single anterior-segment front image in which multiple anterior-segment regions are depicted with appropriate brightness and without blur, thereby improving the quality of images obtained by frontal imaging of the anterior segment. Another advantage is that the number of times the anterior segment is imaged can be reduced. Furthermore, since no focus position change control is performed during anterior-segment imaging, another advantage is that control can be facilitated and simplified.

[0289] Some non-limiting aspects of the ophthalmic apparatus according to the embodiment have been described above. Any two or more aspects can be at least partially combined.

[0290] Other Aspects Embodiments of the present disclosure are not limited to ophthalmic devices. Examples of embodiments other than ophthalmic devices include a method for controlling an ophthalmic device, a method for processing images acquired by an ophthalmic device, a method for acquiring and processing eye images, a program, and a recording medium. Similar to the embodiments of the ophthalmic device, these embodiments can also improve the image quality in anterior segment imaging.

[0291] Some embodiments provide a method for controlling an ophthalmic apparatus. A first aspect of the control method for an ophthalmic apparatus will be described. The ophthalmic apparatus of this aspect can be used to photograph an anterior segment of a subject's eye, and includes an illumination optical system, an imaging optical system, a focus position changer, an exposure amount changer, and a processor. The illumination optical system projects illumination light onto the anterior segment of the subject's eye. The imaging optical system photographs the anterior segment onto which the illumination light is projected from the front. The focus position changer changes the focus position of the imaging optical system. The exposure amount changer changes the exposure amount used to photograph the anterior segment. The method of this aspect includes an imaging step and an image processing step. The imaging step is performed by the processor executing combined control of focus position change control, exposure amount change control, and imaging control. In the focus position change control, the processor controls the focus position changer to sequentially change the focus position of the imaging optical system to a plurality of different focus positions. In the exposure amount change control, the processor controls the exposure amount changer to sequentially change the exposure amount used to photograph the anterior segment to a plurality of different exposure amounts. In the photographing control, the processor controls the photographing optical system to acquire time-series images. In the photographing step of this aspect, the processor executes these combined controls to cause the photographing optical system to acquire first image groups corresponding to multiple different exposure amounts for each of multiple different focus positions. In the image processing step, the processor synthesizes the multiple first image groups acquired for the multiple different focus positions in the photographing step to generate a single image.

[0292] A second aspect of a control method for an ophthalmic apparatus will be described. The ophthalmic apparatus of this aspect can be used to capture images of the anterior segment of a subject's eye, and similarly to the first aspect, includes an illumination optical system, an imaging optical system, a focus position changing unit, an exposure amount changing unit, and a processor. The method of this aspect includes an imaging step and an image processing step. The imaging step is performed by the processor executing a combination of focus position change control, exposure amount change control, and imaging control. Each control is similar to that of the first aspect. The imaging step of this aspect causes the processor to execute these combination controls, thereby causing the imaging optical system to acquire a second group of images corresponding to a plurality of pairs of focus positions and exposure amounts, including pairs based on a one-to-one correspondence between a plurality of different focus positions and a plurality of different exposure amounts. In the image processing step, the processor combines the second group of images acquired in the imaging step to generate a single image.

[0293] A third aspect of a control method for an ophthalmic apparatus will be described. The ophthalmic apparatus of this aspect can be used to capture images of the anterior segment of a subject's eye, and like the first aspect, includes an illumination optical system, an imaging optical system, a focus position changing unit, and a processor, but does not include an exposure amount changing unit. The method of this aspect includes an imaging step and an image processing step. The imaging step is performed by the processor executing a combined control of focus position change control and imaging control. Each control is similar to the first aspect, but the imaging control of this aspect is performed with a constant exposure amount (same exposure amount). The imaging step of this aspect causes the processor to execute this combined control, thereby causing the imaging optical system to acquire a third group of images corresponding to a plurality of different focus positions. The third group of images includes only one image for each of the plurality of different focus positions. In the image processing step, the processor combines the third group of images acquired in the imaging step to generate a single image.

[0294] A fourth aspect of the control method for an ophthalmic apparatus will be described. The ophthalmic apparatus of this aspect can be used to capture images of the anterior segment of a subject's eye, and like the first aspect, includes an illumination optical system, an imaging optical system, an exposure amount changer, and a processor, but does not include a focus position changer. The method of this aspect includes an imaging step and an image processing step. The imaging step is performed by the processor executing a combination of exposure amount change control and imaging control. Each control is similar to the first aspect, but the imaging control of this aspect is performed at a constant focus position (same focus position). The imaging step of this aspect causes the processor to execute this combination control, thereby causing the imaging optical system to acquire a fourth group of images corresponding to a plurality of different exposure amounts. The fourth group of images includes only one image for each of the plurality of different exposure amounts. In the image processing step, the processor combines the fourth group of images acquired in the imaging step to generate a single image.

[0295] Any of the items described in this disclosure can be combined with the method according to the embodiments.

[0296] Some embodiments provide a program. The program according to the embodiments causes a computer including a processor and a memory to execute the method according to any one of the first to fourth aspects. Any of the features described in the present disclosure can be combined with the program according to the embodiments.

[0297] Some embodiments provide a computer-readable non-transitory recording medium. The recording medium according to the embodiments has recorded thereon a program for causing a computer to execute the method according to any one of the first to fourth aspects. Any of the features described in the present disclosure can be combined with the recording medium according to the embodiments.

[0298] A computer-readable non-transitory recording medium that can be used as a recording medium in this embodiment may be a recording medium of any form, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0299] The embodiments and aspects described in the present disclosure are merely examples, and any modifications (omissions, substitutions, additions, etc.) within the scope of the present invention can be applied to the embodiments and aspects of the present disclosure.

[0300] REFERENCE SIGNS LIST 1 Ophthalmic apparatus 2 Illumination optical system 3 Photography optical system 31 Focus position changing unit 32 Exposure amount changing unit 32A Exposure time changing unit 32B Illumination intensity changing unit 40 Anterior eye profile acquisition unit 7 Control unit 71 Photography control unit 8 Data processing unit 81 Image processing unit 82 Machine learning model 83 Image selection unit 84 Determination unit 85 Image synthesis unit 86 Photography condition determination unit 87 Determination unit

Claims

1. An ophthalmic device comprising: an illumination optical system that projects illumination light onto an anterior segment of a subject's eye; an imaging optical system that images the anterior segment onto which the illumination light is projected from the front; a focus position changing unit that changes a focus position of the imaging optical system; an exposure amount changing unit that changes an exposure amount in imaging the anterior segment; an imaging control unit that causes the imaging optical system to acquire a first group of images corresponding to the multiple different exposure amounts for each of the multiple different focus positions by combining a focus position change control that controls the focus position change unit to sequentially change the focus position of the imaging optical system to a multiple different focus positions, an exposure amount change control that controls the exposure amount change unit to sequentially change the exposure amount in imaging the anterior segment to a multiple different exposure amounts, and an imaging control that controls the imaging optical system to acquire time-series images; and an image processing unit that synthesizes the multiple first image groups acquired for the multiple different focus positions to generate a single image.

2. The ophthalmic device of claim 1, wherein the image processing unit generates a plurality of high dynamic range images corresponding to the plurality of different focus positions by applying high dynamic range synthesis to each of the plurality of first image groups, and applies focus synthesis to the plurality of high dynamic range images to generate the single image.

3. The ophthalmic device of claim 1, wherein the image processing unit includes: an image selection unit that selects one or more images from each of the plurality of first image groups; and an image synthesis unit that synthesizes the selected images selected from the plurality of first image groups by the image selection unit to generate the single image.

4. The ophthalmic device of claim 3, wherein when the image selection unit selects two or more images from each of one or more first image groups among the plurality of first image groups, the image synthesis unit generates one or more high dynamic range images corresponding to the one or more first image groups by applying high dynamic range synthesis to two or more images selected from the first image group for each of the one or more first image groups to generate a high dynamic range image, and applies focus synthesis to an image group including the one or more high dynamic range images to generate the single image.

5. The ophthalmic device according to claim 4, wherein the image selection unit selects two or more images from the first image group corresponding to focal positions included in a focal plane that intersects a plurality of different anterior eye tissues.

6. The ophthalmic device of claim 5, wherein the image selection unit includes a determination unit that determines, for each of the plurality of different focal positions, whether a focal plane including the focal position intersects with a plurality of different anterior eye tissues by analyzing an image corresponding to the focal position.

7. An ophthalmic device comprising: an illumination optical system that projects illumination light onto an anterior segment of a subject's eye; an imaging optical system that images, from the front, the anterior segment onto which the illumination light is projected; a focus position changing unit that changes a focus position of the imaging optical system; an exposure amount changing unit that changes an exposure amount in imaging the anterior segment; an imaging control unit that causes the imaging optical system to acquire a second image group corresponding to a plurality of pairs of focus position and exposure amount including pairs based on a one-to-one correspondence between the plurality of different focus positions and the plurality of different exposure amounts, by combining a focus position change control that controls the focus position change unit to sequentially change the focus position of the imaging optical system to a plurality of different focus positions, an exposure amount change control that controls the exposure amount change unit to sequentially change the exposure amount in imaging the anterior segment to a plurality of different exposure amounts, and an imaging control that controls the imaging optical system to acquire time-series images; and an image processing unit that synthesizes the second image group to generate a single image.

8. The ophthalmic device of claim 7, wherein each of the plurality of pairs comprises a focal position corresponding to one of a plurality of different anterior tissues and an exposure amount determined in advance based on the scattering strength of the anterior tissue.

9. The ophthalmologic apparatus according to claim 7, further comprising: an anterior segment profile acquisition unit that acquires a depth direction profile of the anterior segment; and an imaging condition determination unit that determines the plurality of pairs based on the depth direction profile.

10. The ophthalmic device of claim 9, wherein the imaging condition determination unit identifies depth positions of a plurality of different anterior eye tissues based on the depth profile, and determines the plurality of pairs based on the depth positions of the plurality of different anterior eye tissues and scattering intensity information of the plurality of different anterior eye tissues.

11. An ophthalmic device according to any one of claims 7 to 10, wherein the shooting control unit causes the shooting optical system to acquire two or more images corresponding to two or more different exposure amounts for each of one or more focus positions among the plurality of different focus positions, the second image group includes the two or more images corresponding to each of the one or more focus positions, and the image processing unit applies high dynamic range synthesis to the two or more images corresponding to each of the one or more focus positions to generate a high dynamic range image, and generates the single image based on an image group including the one or more high dynamic range images corresponding to the one or more focus positions.

12. The ophthalmic device according to claim 11, wherein the imaging control unit causes the imaging optical system to obtain the two or more images for focal positions included in a focal plane that intersects with a plurality of different anterior eye tissues.

13. The ophthalmic device of claim 12, wherein the image processing unit includes a judgment unit that judges, for each of the plurality of different focal positions, whether a focal plane including that focal position intersects with a plurality of different anterior eye tissues, and the shooting control unit executes control to cause the shooting optical system to acquire the second group of images based on the judgment result obtained by the judgment unit.

14. An ophthalmic device according to any one of claims 7 to 10, wherein the image processing unit includes a machine learning model that has been trained to receive input of a plurality of images having different focus positions and exposure amounts, and output a composite image of the plurality of images, and the machine learning model generates the single image from the second group of images.

15. The ophthalmic device of claim 14, wherein the machine learning model includes a convolutional neural network that has undergone supervised learning using training data including a plurality of pairs of a training image group differing in both focus position and exposure amount and a training synthetic image that is a synthetic image of the training image group.

16. The ophthalmic device according to any one of claims 7 to 10, wherein the image processing unit generates the single image by applying image processing including both focus stacking and high dynamic range stacking to the second group of images.

17. An ophthalmic device comprising: an illumination optical system that projects illumination light onto an anterior segment of a subject's eye; an imaging optical system that images, from the front, the anterior segment onto which the illumination light is projected; a focus position changing unit that changes a focus position of the imaging optical system; an imaging control unit that causes the imaging optical system to acquire a third image group corresponding to the plurality of different focus positions by combining a focus position change control that controls the focus position changing unit to sequentially change the focus position of the imaging optical system to a plurality of different focus positions and an imaging control that controls the imaging optical system to acquire time-series images with a constant exposure amount; and an image processing unit that synthesizes the third image group to generate a single image, wherein the third image group includes only one image for each of the plurality of different focus positions.

18. The ophthalmic device of claim 17, wherein the image processing unit generates a plurality of high dynamic range images corresponding to the plurality of different focus positions by generating a high dynamic range image from each of the plurality of images included in the third image group, and applies focus stacking to the plurality of high dynamic range images to generate the single image.

19. The ophthalmic device of claim 18, wherein the image processing unit includes a machine learning model that has been trained to receive an input of an image having a first dynamic range and output an image having a second dynamic range greater than the first dynamic range, and the machine learning model generates the plurality of high dynamic range images from the third group of images.

20. An ophthalmic device comprising: an illumination optical system that projects illumination light onto an anterior segment of a test eye; an imaging optical system that images the anterior segment onto which the illumination light is projected from the front; an exposure amount change unit that changes the exposure amount in imaging the anterior segment; an imaging control unit that causes the imaging optical system to acquire a fourth image group corresponding to the multiple different exposure amounts by combining an exposure amount change control that controls the exposure amount change unit to sequentially change the exposure amount in imaging the anterior segment to multiple different exposure amounts and an imaging control that controls the imaging optical system to acquire time-series images at a constant focus position; and an image processing unit that synthesizes the fourth image group to generate a single image, wherein the fourth image group includes only one image for each of the multiple different exposure amounts.

21. The ophthalmic device of claim 20, wherein the image processing unit generates a plurality of extended depth of field images corresponding to the plurality of different exposure amounts by generating an extended depth of field image from each of the plurality of images included in the fourth image group, and applies high dynamic range synthesis to the plurality of extended depth of field images to generate the single image.

22. The ophthalmic device of claim 21, wherein the image processing unit includes a machine learning model that has been trained to receive an input of an image having a first depth of field and output an image having a second depth of field deeper than the first depth of field, and the machine learning model generates the plurality of extended depth of field images from the fourth group of images.

23. An ophthalmic device according to any one of claims 1, 7 and 20, wherein the exposure amount changing unit includes an exposure time changing unit for changing the exposure time of the photographing optical system, and the photographing control unit, in the exposure amount changing control, executes exposure time change control for controlling the exposure time changing unit so as to sequentially change the exposure time of the photographing optical system to the plurality of different exposure times.

24. An ophthalmic device according to any one of claims 1, 7 and 20, wherein the exposure amount change unit includes an illumination intensity change unit for changing the intensity of the illumination light, and the photography control unit executes illumination intensity change control in the exposure amount change control to control the illumination intensity change unit so as to sequentially change the intensity of the illumination light to a plurality of different intensities.

25. A method for controlling an ophthalmic apparatus for photographing an anterior segment of a test eye, the ophthalmic apparatus including: an illumination optical system that projects illumination light onto the anterior segment; an imaging optical system that photographs the anterior segment onto which the illumination light is projected from the front; a focus position changing unit that changes a focus position of the imaging optical system; an exposure amount changing unit that changes an exposure amount in photographing the anterior segment; and a processor; the method including: an imaging step of causing the imaging optical system to acquire a first group of images corresponding to the multiple different exposure amounts for each of the multiple different focus positions by having the processor execute combined control of focus position change control that controls the focus position changing unit to sequentially change the focus position of the imaging optical system to multiple different focus positions, exposure amount change control that controls the exposure amount changing unit to sequentially change the exposure amount in photographing the anterior segment to multiple different exposure amounts, and imaging control that controls the imaging optical system to acquire time-series images; and an image processing step of synthesizing the multiple first image groups acquired for the multiple different focus positions by the imaging step to generate a single image.

26. A method for controlling an ophthalmic apparatus for photographing an anterior segment of a subject's eye, the ophthalmic apparatus including: an illumination optical system that projects illumination light onto the anterior segment; an imaging optical system that photographs the anterior segment onto which the illumination light is projected from the front; a focus position changing unit that changes a focus position of the imaging optical system; an exposure amount changing unit that changes an exposure amount in photographing the anterior segment; and a processor; the method including: an imaging step of causing the imaging optical system to acquire a second group of images corresponding to a plurality of pairs of focus position and exposure amount including pairs based on a one-to-one correspondence between the plurality of different focus positions and the plurality of different exposure amounts, by having the processor execute combined control of focus position change control that controls the focus position changing unit to sequentially change the focus position of the imaging optical system to a plurality of different focus positions, exposure amount change control that controls the exposure amount changing unit to sequentially change the exposure amount in photographing the anterior segment to a plurality of different exposure amounts, and imaging control that controls the imaging optical system to acquire time-series images; and an image processing step of synthesizing the second group of images acquired by the imaging step to generate a single image.

27. A method for controlling an ophthalmic device for photographing an anterior segment of a test eye, the ophthalmic device including: an illumination optical system that projects illumination light onto the anterior segment; an imaging optical system that photographs the anterior segment onto which the illumination light is projected from the front; a focus position changing unit that changes a focus position of the imaging optical system; and a processor, the method including: an imaging step of causing the imaging optical system to acquire a third group of images corresponding to the multiple different focus positions by having the processor execute combined control of focus position change control that controls the focus position changing unit to sequentially change the focus position of the imaging optical system to multiple different focus positions and imaging control that controls the imaging optical system to acquire time-series images at a constant exposure amount; and an image processing step of synthesizing the third group of images acquired by the imaging step to generate a single image, the third group of images including only one image for each of the multiple different focus positions.

28. A method for controlling an ophthalmic device for photographing an anterior segment of a test eye, the ophthalmic device including: an illumination optical system that projects illumination light onto the anterior segment; an imaging optical system that photographs the anterior segment onto which the illumination light is projected from the front; an exposure amount change unit that changes the exposure amount in photographing the anterior segment; and a processor; the method includes: an imaging step of causing the imaging optical system to acquire a fourth image group corresponding to the multiple different exposure amounts by having the processor execute combined control of exposure amount change control that controls the exposure amount change unit to sequentially change the exposure amount in photographing the anterior segment to multiple different exposure amounts and imaging control that controls the imaging optical system to acquire time-series images at a constant focus position; and an image processing step of synthesizing the fourth image group acquired by the imaging step to generate a single image, wherein the fourth image group includes only one image for each of the multiple different exposure amounts.

29. A non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute the method according to any one of claims 25 to 28.

Images