Ophthalmologic apparatus, method for controlling ophthalmologic apparatus, program, and recording medium

Abstract

To provide a novel technique for correcting image distortion occurring in ophthalmologic imaging using a Scheimpflug optical system.SOLUTION: A Scheimpflug optical system 2 of an ophthalmologic apparatus 1 includes an illumination system 21 and an imaging system 22L (22R) configured to satisfy a Scheimpflug condition. The illumination system 21 projects illumination light onto an anterior eye part of an eye E under examination. The imaging system 22L includes an image sensor 221 and images the anterior eye part. A storage unit 82 stores a coordinate transformation function F representing a relationship between a first coordinate system on an imaging surface 221a of the image sensor 221 and a second coordinate system on an object surface 211 of the Scheimpflug optical system 2. An image processing unit 81 generates a processed image by applying coordinate transformation using the coordinate transformation function F to an anterior eye part image generated by the Scheimpflug optical system 2. A display unit 11 displays the processed image.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an ophthalmic apparatus, a method for controlling an ophthalmic apparatus, a program, and a recording medium. [Background technology]

[0002] Diagnostic imaging plays an important role in the field of ophthalmology. Various types of ophthalmic devices (ophthalmic imaging devices) are used in ophthalmic imaging diagnostics, such as slit lamp microscopes, fundus cameras, scanning laser ophthalmoscopes (SLO), and optical coherence tomography (OCT). Ophthalmic devices with imaging functions are not limited to these ophthalmic imaging devices; imaging functions are also incorporated into ophthalmic examination and measurement devices such as refractometers, keratometers, tonometers, specular microscopes, wavefront analyzers, and microperimeters.

[0003] While slit lamp microscopes have been primarily used for anterior segment observation, other modalities have also been proposed. For example, Patent Document 1 discloses an ophthalmic device using a Scheimpflug optical system. While Scheimpflug photography has the remarkable advantage of being able to obtain images with a deep depth of field, it suffers from the problem of image distortion. This image distortion includes distortion caused by the structure of the Scheimpflug optical system and distortion caused by the structure of the eye. Examples of image distortion caused by the Scheimpflug optical system include the trapezoidal distortion described in Patent Document 2 and refractive distortion caused by the refractive effect of a lens. Generally, refractive distortion caused by the refractive effect of a lens appears as barrel distortion or pincushion distortion. Another example of image distortion caused by the eye is refractive distortion caused by the refractive effect at the boundaries of ocular tissues. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-49320 [Patent Document 2] Japanese Patent Application Publication No. 2019-213733 Summary of the Invention [Problem to be solved by the invention]

[0005] One object of the present disclosure is to provide a new technique for correcting image distortion that occurs in ophthalmic imaging using a Scheimpflug optical system. [Means for solving the problem]

[0006] One exemplary aspect of an embodiment is an ophthalmic apparatus including a Scheimpflug optical system, a storage unit, an image processing unit, and a display unit. The Scheimpflug optical system includes an illumination system and an imaging system. The illumination system is configured to project illumination light onto an anterior segment of the subject's eye. The imaging system includes an image sensor and is configured to capture an image of the anterior segment of the subject's eye. The illumination system and the imaging system are configured to satisfy the Scheimpflug condition. The storage unit stores a coordinate transformation function. This coordinate transformation function represents a relationship between a first coordinate system on an imaging plane of the image sensor of the imaging system and a second coordinate system on an object plane of the Scheimpflug optical system. The image processing unit is configured to apply coordinate transformation using this coordinate transformation function to an anterior segment image generated by the Scheimpflug optical system. An image generated from the anterior segment image by this coordinate transformation is called a processed image. The display unit displays the generated processed image.

[0007] Another exemplary aspect of an embodiment is a method for controlling an ophthalmic apparatus. The ophthalmic apparatus includes a Scheimpflug optical system, a storage unit, a display unit, and a processor. The Scheimpflug optical system includes an illumination system and an imaging system. The illumination system is configured to project illumination light onto an anterior segment of the subject's eye. The imaging system includes an image sensor and is configured to capture an image of the anterior segment of the subject's eye. The illumination system and the imaging system are configured to satisfy the Scheimpflug condition. The method of this aspect causes the processor to store in the storage unit a coordinate transformation function representing a relationship between a first coordinate system on an imaging plane of the image sensor of the imaging system and a second coordinate system on an object plane of the Scheimpflug optical system. The method of this aspect further causes the processor to perform a process of applying a coordinate transformation using the coordinate transformation function to an anterior segment image generated by the Scheimpflug optical system to generate a processed image. The method of this aspect also causes the processor to perform a process of displaying the generated processed image on the display unit.

[0008] Yet another exemplary aspect of the embodiment is a program that causes a computer to execute the method according to the exemplary aspect.

[0009] Yet another exemplary aspect of the embodiment is a computer-readable non-transitory recording medium having a program according to the exemplary aspect recorded thereon. [Effects of the Invention]

[0010] According to the embodiment, it is possible to provide a new method for correcting image distortion that occurs in ophthalmic imaging using a Scheimpflug optical system. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating a configuration of an ophthalmologic apparatus according to a non-limiting aspect of an embodiment. [Figure 2] 1 is a diagram illustrating a configuration of an ophthalmologic apparatus according to a non-limiting aspect of an embodiment. [Figure 3] 1 is a diagram illustrating a configuration of an ophthalmologic apparatus according to a non-limiting aspect of an embodiment. [Figure 4] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 5] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 6] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 7] 10 is a flowchart illustrating a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 8] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 9] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 10] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 11] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 12] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 13] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 14] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 15] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 16] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 17] FIG. 10 is a diagram for explaining a coordinate transformation function according to a non-limiting aspect of the embodiment. [Figure 18] 10 is a flowchart illustrating an operation performed by an ophthalmologic apparatus according to a non-limiting aspect of an embodiment. [Figure 19]10 is a flowchart illustrating an operation performed by an ophthalmologic apparatus according to a non-limiting aspect of an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Non-limiting embodiments of the present disclosure will now be described.

[0013] Any known technology can be combined with the embodiments. For example, any matter described in a document cited in this disclosure can be combined with any aspect of the embodiments. Furthermore, at least one of any known document related to the technical field of the present disclosure, any known technology in a technical field similar to the technical field of the present disclosure, and any known technology in a technical field other than the technical field of the present disclosure can be combined with any aspect of the embodiments.

[0014] For example, the matters disclosed in Patent Document 1 (JP 2023-49320 A) can be incorporated by reference into the present disclosure. More generally, any technical matters disclosed by the applicant of the present application regarding the technology related to the present disclosure (matters disclosed in patent applications, papers, etc.) can be incorporated by reference into the present disclosure.

[0015] Any two or more of the various non-limiting aspects of the embodiments may be at least partially combined.

[0016] At least a portion of the functionality of any aspect described in this disclosure is implemented using circuitry or processing circuitry. The circuitry or processing circuitry may be a general-purpose processor, a special-purpose processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA)), or a combination of these configured and / or programmed to perform at least a portion of the disclosed functionality. The term "circuitry," "unit," "means," or the like refers to hardware that performs at least a portion of the disclosed functions or that is programmed to perform at least a portion of the disclosed functions. The hardware may be the hardware disclosed herein or may be known hardware that is programmed and / or configured to perform at least a portion of the described functions. In the case of a processor, where the hardware can be considered a type of circuitry, the term "circuitry," "unit," "means," or the like refers to a combination of hardware and software, where the software is used to configure the hardware and / or the processor.

[0017] <Outline of the embodiment> An object of the embodiment of the present disclosure is to provide a new method for correcting image distortion that occurs in ophthalmic imaging using a Scheimpflug optical system. By achieving this object, it becomes possible to provide high-quality images with reduced distortion in ophthalmic imaging using a Scheimpflug optical system. Furthermore, by using a Scheimpflug optical system, it becomes possible to obtain images with a large depth of field. In other words, this embodiment has the remarkable effect of providing eye images that achieve both faithful morphological depiction and a large depth of field. As a result, image diagnosis can be performed favorably. For example, image observation and image analysis can be performed accurately.

[0018] The objects and effects of the embodiments of the present disclosure are not limited to those described above. Although some non-limiting aspects of the embodiments are described below, those skilled in the art will understand that each of these aspects provides effects according to its characteristics (configuration, operation, action, use, etc.).

[0019] A first aspect of the embodiment is an ophthalmologic apparatus for photographing an anterior segment of an eye to be examined, which includes a Scheimpflug optical system, a storage unit, an image processing unit, and a display unit.

[0020] The Scheimpflug optical system includes an illumination system and an imaging system. The illumination system is configured to project illumination light onto the anterior segment of the eye to be examined. The imaging system includes an image sensor and is configured to capture an image of the anterior segment of the eye to be examined. The illumination system and the imaging system are configured to satisfy the Scheimpflug condition.

[0021] Generally, the Scheimpflug condition is a condition relating to an optical system (illumination system) that projects illumination light onto an object and an optical system (photography system) that photographs the object. It stipulates that the illumination system and the photography system be configured so that the object plane, the lens principal plane, and the film plane (imaging plane) intersect on the same straight line. In an optical system that satisfies the Scheimpflug condition, the object plane is not arranged parallel to the lens principal plane. A camera using such a Scheimpflug optical system (a Scheimpflug camera) can simultaneously focus and photograph objects at a wide depth range, from close to far distances. In the anterior eye imaging of this embodiment, for example, photography can be performed while focusing on the entire wide depth range, from the anterior surface of the cornea to the posterior surface of the lens. This achieves the effect of obtaining a high-resolution image that depicts the entire main observation range of the anterior eye.

[0022] The storage unit stores a coordinate transformation function. The coordinate transformation function includes information representing a relationship between a first coordinate system on an imaging plane of an image sensor of the imaging system and a second coordinate system on an object plane of the Scheimpflug optical system. The coordinate transformation function may be expressed in any form. For example, the coordinate transformation function may be a mathematical formula, a table (lookup table), a graph, or a chart, or may be a computer program created based on at least one of a mathematical formula, a table, a graph, and a chart. Alternatively, the coordinate transformation function may be a mathematical model (machine learning model) constructed using machine learning.

[0023] The image processing unit applies coordinate transformation using the coordinate transformation function stored in the storage unit to the anterior eye image generated by the Scheimpflug optical system. The image generated from the anterior eye image by this coordinate transformation is called a processed image. The generated processed image is displayed on the display unit.

[0024] According to the ophthalmic device of the first aspect, distortion of an anterior eye image with a deep depth of field obtained using a Scheimpflug optical system can be corrected by a new method using a coordinate transformation function between the imaging plane of the Scheimpflug optical system and the object plane, making it possible to generate an image with a deep depth of field in which the shape of the anterior eye is faithfully depicted.

[0025] The coordinate transformation function of this aspect includes at least information representing the relationship between a first coordinate system on the imaging plane of the image sensor and a second coordinate system on the object plane of the Scheimpflug optical system. Therefore, the coordinate transformation function of this aspect reflects at least information regarding image distortion (trapezoidal distortion, refractive distortion, etc.) caused by the structure of the Scheimpflug optical system. As will be described later, the coordinate transformation function of this aspect may further reflect information regarding image distortion (refractive distortion, etc.) caused by the object (eye) being photographed.

[0026] A second aspect of the embodiment provides an example of a coordinate transformation function in the ophthalmic apparatus of the first aspect.

[0027] In this embodiment, the first coordinate system on the imaging plane of the image sensor is a first pixel position coordinate system that represents pixel position coordinates on this imaging plane, and the second coordinate system on the object plane of the Scheimpflug optical system is a second pixel position coordinate system that represents pixel position coordinates in a raw image corresponding to this object plane.

[0028] The raw image, whose pixel position coordinates are expressed by the second pixel position coordinate system, is an image (original image) generated by the image sensor of the imaging system of the Scheimpflug optical system, is an image that depicts the morphology of the ocular tissue located on the object plane as is, and is an image (pre-correction image) before image distortion correction is performed. Therefore, the coordinate system on the object plane and the coordinate system in the raw image can be associated with each other, and can be regarded as the same.

[0029] The coordinate transformation function of this embodiment may be configured based on a plurality of coordinate transformation functions. As a non-limiting example, the coordinate transformation function of this aspect includes three coordinate transformation functions (first to third coordinate transformation functions).

[0030] The first coordinate transformation function represents the relationship between a first pixel position coordinate system that represents pixel position coordinates on the imaging surface of the image sensor and a first length coordinate system that defines lengths on the same imaging surface. In other words, the first coordinate transformation function defines coordinate transformation between two different coordinate systems defined on the same object (imaging surface).

[0031] The second coordinate transformation function represents the relationship between a first length coordinate system on the imaging plane of the image sensor and a second length coordinate system that defines a length on the object plane of the Scheimpflug optical system, i.e., the second coordinate transformation function defines a coordinate transformation between two different coordinate systems defined on two different objects (the imaging plane and the object plane), respectively.

[0032] The third coordinate transformation function represents the relationship between the second length coordinate system that defines the length in the object plane (raw image) of the Scheimpflug optical system and the second image position coordinate system that represents the pixel position coordinates in the same object plane (raw image). In other words, the third coordinate transformation function specifies the coordinate transformation between two different coordinate systems defined on the same object (object plane, raw image).

[0033] Here, the first pixel position coordinate system and the second pixel position coordinate system are discrete coordinate systems in which coordinate values are expressed by discrete values (integer values), and the first length coordinate system and the second length coordinate system are continuous coordinate systems in which coordinate values are expressed by continuous values (real values). Thus, the first coordinate transformation function defines the transformation between the discrete coordinate system and the continuous coordinate system both defined on the imaging plane of the image sensor, the second coordinate transformation function defines the transformation between the continuous coordinate system defined on the imaging plane of the image sensor and the continuous coordinate system defined on the object plane of the Scheimpflug optical system, and the third coordinate transformation function defines the transformation between the continuous coordinate system and the discrete coordinate system both defined on the object plane of the Scheimpflug optical system.

[0034] The second aspect provides an example of the configuration of the coordinate transformation function of this embodiment.

[0035] A third aspect of the embodiment provides an example of a configuration of a second coordinate transformation function in the ophthalmic apparatus of the second aspect.

[0036] As described above, the second coordinate transformation function represents the relationship between the first length coordinate system in the imaging plane of the image sensor and the second length coordinate system that defines the length in the object plane of the Scheimpflug optical system.

[0037] The second coordinate transformation function of this aspect includes a nonlinear projection transformation function determined by ray tracing based on predetermined simulation conditions. These simulation conditions include optical system conditions determined from the structure of the Scheimpflug optical system and ocular conditions determined from the structure of the eyeball model. The coordinate transformation function of this aspect reflects both information about image distortion caused by the structure of the Scheimpflug optical system and information about image distortion caused by the object (eye) being photographed.

[0038] The eyeball model of this embodiment may be any model that represents the shape or characteristics of the eyeball, such as an eye model, an image of a living eye, a theoretical eyeball model, or a practical eyeball model. The eyeball model of this embodiment may be any of a normal eye model, a healthy eye model, and a diseased eye model. By determining a nonlinear projective transformation function from each of multiple eyeball models, multiple second coordinate transformation functions (hence multiple coordinate transformation functions) corresponding to the multiple eyeball models can be prepared. In this case, it is possible to select a coordinate transformation function appropriate for the eye to be examined and perform image distortion correction.

[0039] A fourth aspect of the embodiment provides an example of optical system conditions related to the structure of the Scheimpflug optical system in the ophthalmic device of the third aspect. The optical system conditions of this aspect are determined based on at least the conditions of the imaging plane of the image sensor of the imaging system (imaging plane conditions), the conditions of the object plane of the Scheimpflug optical system (object plane conditions), and the conditions of the optical elements included in the Scheimpflug optical system (optical element conditions).

[0040] The imaging surface condition may be any condition related to the imaging surface, and may include, for example, at least one of a condition related to the type of image sensor (e.g., a charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor), a condition related to the configuration or structure of the imaging surface (e.g., dimensions, number of pixels, pixel size, shape), and a condition related to the arrangement of the imaging surface (e.g., position, orientation).

[0041] The object surface condition may be any condition related to the object surface, and may include, for example, at least one of the position, orientation, size, and shape of the object surface.

[0042] The optical element condition may be any condition related to any optical element or any optical element group constituting the Scheimpflug optical system. For example, the optical element condition may include at least one of the position, orientation, size, shape, and optical characteristics of the optical element (optical element group). Examples of optical characteristics include the refractive index, refractive power, focal length, and aberration of the lens or lens group.

[0043] A fifth aspect of the embodiment provides an example of an ophthalmic device according to the third or fourth aspect, for an ophthalmic condition.

[0044] The eye model used to determine the eye conditions of this embodiment includes one or more eye structure models. The eye structure model is a model of any eye structure. An eye structure is an element (tissue) that constitutes the eye or a partial tissue of the eye. Examples of eye structures include the cornea, iris, lens, anterior chamber, pupil, conjunctiva, vitreous body, retina, choroid, sclera, etc.

[0045] The ocular conditions of this embodiment are determined based on at least conditions of one or more ocular structure models. The conditions of the ocular structure models may include, for example, the position, orientation, size, shape, optical properties, etc. of the ocular structures. The ocular conditions may also include conditions related to the background of the ocular (e.g., air conditions such as the refractive index of air).

[0046] Considering that this embodiment is for performing anterior segment imaging, the conditions of the eye structure model according to this embodiment include at least one of the conditions of a cornea model, the conditions of an iris model, the conditions of a lens model, the conditions of an anterior chamber model, the conditions of a pupil model, the conditions of a vitreous body model, and the conditions of air.

[0047] The conditions for the cornea model include, for example, conditions for the anterior corneal surface model (position, orientation, dimensions, shape, etc. of the anterior corneal surface), conditions for the posterior corneal surface model (position, orientation, dimensions, shape, etc. of the posterior corneal surface), and the refractive index of the cornea. The conditions for the iris model include, for example, conditions for the anterior iris surface model (position, orientation, dimensions, shape, etc. of the anterior iris surface). The conditions for the lens model include, for example, conditions for the anterior lens surface model (position, orientation, dimensions, shape, etc. of the anterior lens surface), conditions for the posterior lens surface model (position, orientation, dimensions, shape, etc. of the posterior lens surface), and the refractive index of the lens. The conditions for the anterior chamber model include, for example, the dimensions of the anterior chamber (anterior chamber depth, etc.), the refractive index of the aqueous humor, and the like. The conditions for the pupil model include, for example, the position, orientation, dimensions (pupil diameter), shape, etc. The conditions for the vitreous body model include, for example, the refractive index of the vitreous body.

[0048] A sixth aspect of the embodiment provides an example of a nonlinear projection transformation function for determining a second coordinate transformation function used for coordinate transformation between an imaging plane and an object plane in an ophthalmic device of any of the third to fifth aspects.

[0049] As described above, the nonlinear projection transformation function is determined by ray tracing based on predetermined simulation conditions. In this aspect, actual ray tracing is performed using the output ray vector from the Scheimpflug optical system as an initial condition. By this actual ray tracing, paths of virtual rays within the eyeball model are determined using the output ray vector from the Scheimpflug optical system as an initial condition. The nonlinear projection transformation function of this aspect is determined based on the paths of virtual rays within the eyeball model determined in this manner.

[0050] A seventh aspect of the embodiment provides an example of a nonlinear projection transformation function for determining a second coordinate transformation function used for coordinate transformation between an imaging plane and an object plane in the ophthalmic device of the fifth aspect, and in particular provides an example of an end condition for ray tracing for determining this nonlinear projection transformation function.

[0051] As in the sixth aspect, the nonlinear projection transformation function of this aspect is determined based on the path of a virtual ray in the eyeball model obtained by actual ray tracing using the output ray vector from the Scheimpflug optical system as the initial condition.

[0052] The actual ray tracing of this embodiment ends when any of the following three conditions (first to third ending conditions) is satisfied.

[0053] The first termination condition is that the virtual ray reaches the object surface of the Scheimpflug optical system before reaching the surface of interest of the eye structure model. The surface of interest of the eye structure model is, for example, the surface (outer surface) of the eye structure. Examples of surfaces of interest include the anterior and posterior surfaces of the cornea, the anterior surface of the iris, and the anterior and posterior surfaces of the lens. In other words, a surface of interest can be said to be the interface between one eye structure and another. For example, the posterior surface of the cornea is the interface between the cornea and the anterior chamber, the anterior surface of the iris is the interface between the anterior chamber and the iris, the anterior surface of the lens is the interface between the anterior chamber and the lens, and the posterior surface of the lens is the interface between the lens and the vitreous body. The anterior surface of the cornea is the interface between the cornea and air. Light is refracted at these surfaces of interest (interfaces). Actual ray tracing includes calculations to determine the refraction state of the virtual ray at the surface of interest. The first termination condition means that the virtual ray under consideration intersects with the object plane at a position anterior to the plane of interest (closer to the Scheimpflug optics). That is, in actual anterior eye imaging, an actual ray corresponding to this virtual ray illuminates an anterior eye region located anterior to the plane of interest, and the illuminated anterior eye region is photographed in focus.

[0054] The second termination condition is that the distance from the intersection point of the virtual ray and the surface of interest to the object surface is equal to or greater than a threshold. The second termination condition means that the virtual ray under consideration does not affect anterior eye imaging. For example, if the distance from the intersection point of the virtual ray and the anterior corneal surface to the object surface is greater than the radius of a predetermined aperture, the virtual ray will not enter the eye. The threshold value for the distance between the intersection point and the object surface may be a value set in an eyeball model, for example. For example, considering that the average corneal diameter of an adult is approximately 12 mm horizontally and 11 mm vertically, an elliptical aperture of these dimensions is set as the threshold for the anterior corneal surface (the horizontal threshold is approximately 6 mm and the vertical threshold is approximately 5.5 mm).

[0055] The third termination condition is that the virtual ray does not intersect with the target surface, i.e., the coordinates of the intersection point in the calculation of real ray tracing are imaginary values.

[0056] An eighth aspect of the embodiment provides an example of a nonlinear projection transformation function for determining a second coordinate transformation function used for coordinate transformation between an imaging plane and an object plane in an ophthalmic device of the fifth or seventh aspect, and in particular provides an example of a ray tracing operation for determining this nonlinear projection transformation function.

[0057] As in the sixth aspect, the nonlinear projection transformation function of this aspect is determined based on the path of a virtual ray in the eyeball model obtained by actual ray tracing using the output ray vector from the Scheimpflug optical system as the initial condition.

[0058] The eyeball model of this aspect has multiple planes of interest set therein. The multiple planes of interest may be two or more planes of interest of one eye structure model (e.g., the anterior and posterior surfaces of the cornea), or may be two or more planes of interest of two or more eye structure models. Since anterior eye imaging is performed in this embodiment, the planes of interest of the eyeball model of this aspect include, for example, at least two of the anterior surface of the cornea, the posterior surface of the cornea, the anterior surface of the iris, the anterior surface of the lens, and the posterior surface of the lens.

[0059] The actual ray tracing of this aspect is performed using the exit ray vector from the Scheimpflug optical system as an initial condition and on the condition that a virtual ray passes through multiple surfaces of interest in sequence. In the actual ray tracing of this aspect, for each of the multiple surfaces of interest, the vector of a virtual ray (exit ray vector) that exits from that surface of interest is calculated based on the vector of a virtual ray that enters that surface of interest (incident ray vector) and conditions related to that surface of interest. By performing this calculation sequentially for the multiple surfaces of interest, the path of the virtual ray within the eyeball model based on the exit ray vector from the Scheimpflug optical system is determined.

[0060] As an example, a case will be described in which the multiple surfaces of interest in the eyeball model are the anterior corneal surface, the posterior corneal surface, the anterior lens, and the posterior lens. A virtual ray, whose initial condition is the exit ray vector from the Scheimpflug optical system, travels linearly through the air space a distance determined based on the working distance of the Scheimpflug optical system and the direction of the exit ray vector from the Scheimpflug optical system, and is incident on the anterior corneal surface. The incident ray vector of the virtual ray with respect to the anterior corneal surface is equal to the exit ray vector from the Scheimpflug optical system. In the actual ray tracing of this example, the exit ray vector of the virtual ray from the anterior corneal surface is calculated according to Snell's law based on the incident ray vector of the virtual ray with respect to the anterior corneal surface, the conditions of the anterior corneal surface model, and predetermined conditions of the eyeball model (the refractive index of air and the refractive index of the cornea).

[0061] A virtual ray emitted from the anterior surface of the cornea travels linearly inside the cornea a distance determined based on the corneal thickness in the eyeball model and the direction of the exit ray vector of the virtual ray from the anterior cornea, and then enters the posterior surface of the cornea. The incident ray vector of the virtual ray with respect to the posterior surface of the cornea is equal to the exit ray vector from the anterior surface of the cornea. In the actual ray tracing of this example, the exit ray vector of the virtual ray from the posterior surface of the cornea is calculated according to Snell's law based on the incident ray vector of the virtual ray with respect to the posterior surface of the cornea, the conditions of the posterior surface of the cornea model, and predetermined conditions of the eyeball model (the refractive index of the cornea and the refractive index of the aqueous humor).

[0062] A virtual ray emitted from the posterior surface of the cornea travels linearly within the anterior chamber a distance determined based on the anterior chamber depth in the eyeball model and the direction of the exit ray vector of the virtual ray from the posterior surface of the cornea, and is incident on the anterior surface of the lens. The incident ray vector of the virtual ray with respect to the anterior surface of the lens is equal to the exit ray vector from the posterior surface of the cornea. In actual ray tracing in this example, the exit ray vector of the virtual ray from the anterior surface of the lens is calculated according to Snell's law based on the incident ray vector of the virtual ray with respect to the anterior surface of the lens, the conditions of the anterior surface of the lens model, and predetermined conditions of the eyeball model (the refractive index of aqueous humor and the refractive index of the lens).

[0063] A virtual ray emitted from the anterior surface of the lens travels linearly within the lens for a distance determined based on the lens thickness (distribution of the distance between the anterior surface and the posterior surface of the lens) in the eyeball model and the direction of the exit ray vector of the virtual ray from the anterior surface of the lens, and then enters the posterior surface of the lens. The entrance ray vector of the virtual ray with respect to the posterior surface of the lens is equal to the exit ray vector from the anterior surface of the lens. In the actual ray tracing of this example, the exit ray vector of the virtual ray from the posterior surface of the lens is calculated according to Snell's law based on the entrance ray vector of the virtual ray with respect to the posterior surface of the lens, the conditions of the posterior surface of the lens, and predetermined conditions of the eyeball model (the refractive index of the lens and the vitreous refractive index). The virtual ray emitted from the anterior surface of the lens travels linearly within the vitreous body. This allows the path of the virtual ray to be obtained, passing through the anterior surface of the cornea, the posterior surface of the cornea, the anterior surface of the lens, and the posterior surface of the lens, in that order.

[0064] When the seventh and eighth aspects are combined, the actual ray tracing for determining the path of the virtual ray is terminated when any of the three termination conditions in the seventh aspect is satisfied, while sequentially calculating the change in the traveling direction of the virtual ray in a plurality of target planes of the eyeball model according to the method of the eighth aspect.

[0065] A ninth aspect of the embodiment provides an example of a nonlinear projection transformation function for determining a second coordinate transformation function used for coordinate transformation between an imaging plane and an object plane in an ophthalmic device of any of the third to fifth aspects, which is different from the sixth to eighth aspects.

[0066] In this aspect, the ray tracing for determining the nonlinear projection transformation function includes ray tracing based on optical system conditions determined from the structure of the Scheimpflug optical system, and ray tracing based on ocular conditions determined from the structure of the eyeball model.

[0067] Thus, the ninth aspect is characterized in that, in order to determine the nonlinear projection transformation function, ray tracing based on the optical system conditions and ray tracing based on the ocular conditions are performed separately.

[0068] A tenth aspect of the embodiment provides an example of ray tracing based on optical system conditions determined from the structure of the Scheimpflug optical system in the ophthalmic device of the ninth aspect.

[0069] In this embodiment, ray tracing based on optical system conditions includes two types of processing. The first processing is a first ray tracing for determining a first conversion function for trapezoidal distortion caused by the Scheimpflug optical system. The second processing is a processing for determining a second conversion function for image distortion (refractive distortion, lens distortion) caused by distortion aberration of the Scheimpflug optical system.

[0070] Thus, the tenth aspect is characterized in that, in order to determine the nonlinear projection transformation function, a process (first ray tracing) for determining the transformation function (first transformation function) used for trapezoidal distortion correction and a process for determining the transformation function (second transformation function) used for refractive distortion correction are executed separately.

[0071] An eleventh aspect of the embodiment provides a first example of ray tracing for determining a first transformation function for trapezoidal distortion caused by a Scheimpflug optical system in the ophthalmic device of the tenth aspect.

[0072] In this aspect, the first ray tracing includes paraxial ray tracing based on optical system conditions determined from the structure of the Scheimpflug optical system, with the output ray vector from the Scheimpflug optical system as the initial condition.

[0073] A twelfth aspect of the embodiment provides an example of a process for determining a second conversion function for image distortion caused by distortion aberration of the Scheimpflug optical system in the ophthalmic device of the tenth or eleventh aspect.

[0074] In this embodiment, the process for determining the second conversion function includes a process for introducing a mathematical formula used for refractive distortion correction, which may be any mathematical formula that can be used for correcting refractive distortion caused by distortion aberration of the optical system.

[0075] The mathematical formula introduced in this aspect is a mathematical formula that expresses the relationship between a coordinate system that does not take into account image distortion caused by the optical system's distortion and a coordinate system that takes into account the image distortion. In other words, the mathematical formula introduced is a mathematical formula that shows the correspondence between pixel positions when image distortion caused by the optical system's distortion is not taken into account and pixel positions when the image distortion is taken into account.

[0076] Examples of formulas that may be used in this embodiment include those disclosed in the following document: DUANE C. BROWN, 1971. Close-Range Camera Calibration. Photogrammetric Engineering, Vol. 37, No. 8, pp. 855-866 (Formulas (20) and (21)). For more information on these formulas, please also refer to the following document, which cites the same document: CHIKATSU, Hirofumi. 2017. General-Purpose Camera Calibration Using the Scheimpflug Principle. Photogrammetry and Remote Sensing, Vol. 56, No. 5, pp. 225-233.

[0077] In this embodiment, the introduced formula may be used as the second conversion function as is. Alternatively, the introduced formula may be modified in a predetermined manner to obtain a formula that is used as the second conversion function. This modification of the formula is performed, for example, based on the configuration and characteristics of the Scheimpflug optical system of the ophthalmic apparatus of this embodiment.

[0078] A thirteenth aspect of the embodiment provides an example, different from that of the twelfth aspect, of a process for determining a second conversion function related to image distortion caused by distortion aberration of the Scheimpflug optical system in an ophthalmic device of the tenth or eleventh aspect.

[0079] In this aspect, the process for determining the second transformation function may include ray tracing. This ray tracing includes actual ray tracing (second ray tracing) based on optical system conditions determined from the structure of the Scheimpflug optical system, with the output ray vector from the Scheimpflug optical system as an initial condition. The second ray tracing is performed, for example, in the same manner as in any of the sixth to eighth aspects.

[0080] A fourteenth aspect of the embodiment provides an example of ray tracing performed on an ophthalmic device according to any one of the tenth to thirteenth aspects, based on optical system conditions determined from the structure of the Scheimpflug optical system.

[0081] In this aspect, in ray tracing based on the optical system conditions, a composite function of a first conversion function for trapezoidal distortion correction and a second conversion function for refractive distortion correction is determined, and this composite function is used for both trapezoidal distortion correction and refractive distortion correction.

[0082] In general, the composite function of two functions α and β is written as α(β). The first and second conversion functions of this embodiment are represented by g1 and g2, respectively. The composite function obtained in this embodiment may be either g2(g1) or g1(g2).

[0083] A fifteenth aspect of the embodiment provides an example of ray tracing for determining a nonlinear projection transformation function for determining a second coordinate transformation function used for coordinate transformation between an imaging plane and an object plane in an ophthalmic device of the fourteenth aspect.

[0084] In this aspect, ray tracing is performed based on eyeball conditions determined from the structure of the eyeball model, and ray tracing (third ray tracing) is performed to determine a conversion function (third conversion function) related to image distortion caused by refraction by the eyeball. The third ray tracing is performed, for example, in the same manner as in the sixteenth to eighteenth aspects described below. The third conversion function is represented by g3.

[0085] In this aspect, for example, similarly to the fourteenth aspect, a composite function g2(g1) of a first conversion function g1 and a second conversion function g2 is calculated, and further, a composite function g3(g2(g1)) of this composite function g2(g1) and a third conversion function g3 is calculated. Note that the order in which the three conversion functions g1, g2, and g3 are combined is not limited to this and may be determined arbitrarily.

[0086] The composite function of the three transformation functions g1, g2, and g3 obtained in this embodiment is used to correct two types of image distortion caused by the Scheimpflug optical system (trapezoidal distortion correction and refractive distortion correction) and refractive distortion caused by the eyeball.

[0087] A sixteenth aspect of the embodiment provides an example of ray tracing based on eyeball conditions determined from the structure of an eyeball model in the ophthalmologic apparatus according to any one of the ninth to fifteenth aspects.

[0088] In this embodiment, ray tracing based on the eyeball condition includes ray tracing (third ray tracing) for determining a conversion function (third conversion function) related to image distortion caused by refraction by the eyeball.

[0089] Thus, the sixteenth aspect is characterized in that, in order to determine the nonlinear projective transformation function, ray tracing for determining a transformation function used to correct refractive distortion caused by the eyeball is performed separately from ray tracing for determining a transformation function used to correct image distortion caused by the Scheimpflug optical system.

[0090] A seventeenth aspect of the embodiment provides an example of a third ray tracing for determining a third conversion function related to refractive distortion caused by the eyeball, in the ophthalmic device of the sixteenth aspect.

[0091] In this aspect, the third ray tracing includes actual ray tracing based on eyeball conditions determined from the structure of the eyeball model, with the exit ray vector from the Scheimpflug optical system as the initial condition. The third ray tracing may be performed in the same manner as any of the sixth to eighth aspects, for example, or may be performed in a different manner.

[0092] An eighteenth aspect of the embodiment provides an example of a third ray tracing for determining a third conversion function related to refractive distortion caused by the eyeball, in the ophthalmic device of the seventeenth aspect.

[0093] In this aspect, the eyeball model for determining the eyeball condition includes a plurality of eye structure models. The eye structure models may be similar to those in the fifth aspect and aspects quoting therefrom. In this aspect, the eyeball condition is determined based on at least the conditions of the plurality of eye structure models.

[0094] In this aspect, the third ray tracing further includes a process (identification process) of identifying an eye structure model through which a virtual ray that reaches a predetermined point on the eye model passes, from among a plurality of eye structure models in the eye model. The number of eye structure models identified by the identification process is 0 or 1 or more.

[0095] In this aspect, as in the seventeenth aspect, actual ray tracing is performed based on eyeball conditions determined from the structure of the eyeball model, with the emergent ray vector from the Scheimpflug optical system being used as an initial condition. When one or more eye structure models are identified by the identification process, the actual ray tracing performs a process of determining an emergent ray vector corresponding to the incident ray vector incident on each target surface of each eye structure model identified by the identification process, and a process of determining an intersection between a virtual ray based on the emergent ray vector and an object surface.

[0096] For example, when a corneal model is identified by the identification process, actual ray tracing involves a process of determining an exit ray vector corresponding to an incident ray vector on the anterior surface of the cornea, and a process of determining an intersection between a virtual ray based on this exit ray vector and the object surface. If an intersection between a virtual ray based on the exit ray vector from the anterior surface of the cornea and the object surface exists, actual ray tracing in this example is complete. If an intersection between a virtual ray based on the exit ray vector from the anterior surface of the cornea and the object surface does not exist, actual ray tracing in this example involves determining an incident ray vector on the posterior surface of the cornea based on this virtual ray and the eyeball model, determining an exit ray vector corresponding to this incident ray vector, and determining an intersection between a virtual ray based on this exit ray vector and the object surface. For the processing performed in actual ray tracing in this embodiment, see also the seventh and eighth embodiments.

[0097] A nineteenth aspect of the embodiment is the ophthalmic apparatus of any one of the first to eighteenth aspects, further including a movement mechanism. The movement mechanism is configured to move the Scheimpflug optical system to a plurality of positions. The movement of the Scheimpflug optical system by the movement mechanism may be continuous or intermittent. For example, the movement mechanism can move the Scheimpflug optical system continuously (i.e., without stopping from the start point to the end point) on a path that passes through a plurality of positions. In other words, a plurality of positions are set on the movement path of the Scheimpflug optical system.

[0098] The Scheimpflug optical system of this aspect is arranged at a plurality of positions by a moving mechanism, and captures an image of the anterior segment of the subject's eye at each of the arranged positions, thereby collecting a plurality of anterior segment images corresponding to the plurality of positions.

[0099] The storage unit of this aspect stores a plurality of coordinate transformation functions corresponding to a plurality of positions at which the Scheimpflug optical system photographs the anterior segment. The correspondence relationship between the plurality of positions and the plurality of coordinate transformation functions may be one-to-one, one-to-many, or many-to-one.

[0100] The image processing unit of this aspect applies coordinate transformation using a corresponding coordinate transformation function to each of a plurality of anterior eye segment images collected by the Scheimpflug optical system corresponding to a plurality of positions.

[0101] For example, the image processing unit of this embodiment selects, for each of the multiple positions, an anterior-segment image corresponding to the position from the multiple anterior-segment images, selects a coordinate transformation function corresponding to the position from the multiple coordinate transformation functions, and applies coordinate transformation using the selected coordinate transformation function to the selected anterior-segment image. This generates a processed image corresponding to the position. By performing this series of processes for each of the multiple positions, multiple processed images corresponding to the multiple positions are generated.

[0102] In another example, the image processing unit of this aspect selects, for each of the plurality of anterior-segment images, a position corresponding to the anterior-segment image from among the plurality of positions, selects a coordinate transformation function corresponding to the selected position from among the plurality of coordinate transformation functions, and applies coordinate transformation using the selected coordinate transformation function to the anterior-segment image. This allows coordinate transformation using the coordinate transformation function associated with the position to be applied to the anterior-segment image, generating a processed image. By performing this series of processes on each of the plurality of anterior-segment images, a plurality of processed images corresponding to the plurality of positions are generated.

[0103] According to the nineteenth aspect, it is possible to generate an image with a deep depth of field in which the shape of the three-dimensional region of the anterior segment is faithfully depicted.

[0104] Any two or more of the features of the first to nineteenth aspects may be at least partially combined. Also, any feature described in the present disclosure may be at least partially combined with the first to nineteenth aspects. The ophthalmic device of the aspect obtained by such a combination not only exhibits the effects based on each of the combined features, but also exhibits a synergistic effect of the two or more combined features.

[0105] A twentieth aspect of the embodiment provides a method according to the first aspect of the invention for the ophthalmic apparatus. The method of this aspect is a method for controlling an ophthalmic apparatus for photographing an anterior segment of an eye to be examined. The ophthalmic apparatus controlled by the method of this aspect includes a Scheimpflug optical system, a memory unit, a display unit, and a processor. The Scheimpflug optical system includes an illumination system and an imaging system. The illumination system is configured to project illumination light onto the anterior segment of the eye to be examined. The imaging system includes an image sensor and is configured to photograph the anterior segment of the eye to be examined. The illumination system and the imaging system are configured to satisfy the Scheimpflug condition.

[0106] The method of this aspect causes the processor to execute a process of storing a coordinate transformation function in a storage unit, the coordinate transformation function representing the relationship between a first coordinate system on an imaging plane of an image sensor of an imaging system of a Scheimpflug optical system and a second coordinate system on an object plane of the Scheimpflug optical system.

[0107] Furthermore, the method of this aspect causes the processor to execute a process of applying a coordinate transformation using a coordinate transformation function stored in the memory unit to the anterior eye image generated by the Scheimpflug optical system to generate a processed image.

[0108] Additionally, the method of this aspect causes the processor to execute a process of displaying the generated processed image on the display unit.

[0109] The method according to the twentieth aspect has the same effects as the ophthalmologic apparatus according to the first aspect.

[0110] Any of the features of the first to nineteenth aspects can be at least partially combined with the method of the twentieth aspect. Furthermore, any of the features described in the present disclosure can be at least partially combined with the method of the twentieth aspect. The method of the aspect obtained by such a combination not only exhibits the effects of each of the combined features, but also exhibits a synergistic effect of the two or more combined features.

[0111] A twenty-first aspect of the embodiment is a program causing a computer to execute the method of the twentieth aspect. The computer of the twenty-first aspect includes the processor of the twentieth aspect.

[0112] The program according to the twenty-first aspect provides the same effects as the ophthalmologic apparatus according to the first aspect.

[0113] Any of the features of the first to nineteenth aspects can be at least partially combined with the program according to the twenty-first aspect. Furthermore, any of the features described in the present disclosure can be at least partially combined with the program according to the twenty-first aspect. The program according to the aspect obtained by such a combination not only exhibits the effects of each of the combined features, but also exhibits a synergistic effect of the two or more combined features.

[0114] A twenty-second aspect of the embodiment is a computer-readable non-transitory recording medium having the program of the twenty-first aspect recorded thereon.

[0115] The recording medium according to the twenty-second aspect has the same effects as the ophthalmologic apparatus according to the first aspect.

[0116] Any of the features of the first to nineteenth aspects can be at least partially combined with the recording medium of the twenty-second aspect. Furthermore, any of the features described in the present disclosure can be at least partially combined with the recording medium of the twenty-second aspect. The recording medium of the aspect obtained by such a combination not only exhibits the effects based on each of the combined features, but also exhibits a synergistic effect of the two or more combined features.

[0117] The present disclosure describes various non-limiting aspects, including aspects 1 to 22. The present disclosure mainly describes exemplary aspects of an ophthalmic apparatus, an exemplary aspect of a method for controlling an ophthalmic apparatus, an exemplary aspect of a program, and an exemplary aspect of a recording medium. However, possible categories of aspects of embodiments are not limited to these. For example, it will be understood by those skilled in the art that embodiments of the present disclosure can provide various aspects of a method for determining a coordinate transformation function, various aspects of a device for performing a process for determining a coordinate transformation function, various aspects of a coordinate transformation method, various aspects of a device for performing a coordinate transformation process, various aspects of a medical method, various aspects of an imaging method, various aspects of a data processing method, and the like.

[0118] <Ophthalmological equipment> Several non-limiting exemplary aspects of the ophthalmic apparatus according to the embodiment will be described. The ophthalmic apparatus according to the embodiment may include any type of anterior segment scanner. While the present disclosure describes in detail several exemplary aspects in which the configuration according to the embodiment is combined with a slit lamp microscope capable of performing anterior segment scanning, the aspects of the embodiment are not limited to these examples. For example, the aspects of the embodiment may be a slit lamp microscope that cannot perform anterior segment scanning, or an anterior segment imaging device other than a slit lamp microscope. Furthermore, at least a portion of the configuration according to the embodiment may be applied to an ophthalmic imaging device other than an anterior segment imaging device or a medical imaging device other than an ophthalmic imaging device.

[0119] 1 shows an example of the configuration of an ophthalmic apparatus according to one embodiment. The ophthalmic apparatus 1 of this embodiment is used for imaging the anterior segment of a subject's eye E, and includes a Scheimpflug optical system 2, a fixation optical system 3, a movement mechanism 6, a control unit 7, a data processing unit 8, a communication unit 9, and a user interface (UI) 10. The cornea of the subject's eye E is indicated by the symbol Co, the iris by the symbol Ir, and the crystalline lens by the symbol Cr.

[0120] According to the convention in the field of ophthalmology, the direction along the axis of the subject's eye E is defined as the Z direction (Z axis), and the plane perpendicular to the Z direction is defined as the XY plane. The left-right direction (horizontal direction) for the subject is defined as the X direction (X axis), and the direction perpendicular to both the X and Z directions (up-down direction, body axis direction) is defined as the Y direction (Y axis).

[0121] Reference symbol 2a denotes the optical axis of the optical system included in the Scheimpflug optical system 2. The optical axis 2a of the Scheimpflug optical system 2 (in this embodiment, the illumination optical axis 21a shown in FIG. 2) is arranged parallel to the Z axis. Reference symbol 3a denotes the optical axis of the fixation optical system 3 (referred to as the fixation optical axis). The fixation optical axis 3a is arranged so as to be inclined at an angle θ in the Y direction with respect to the Z axis (illumination optical axis 21a). In other words, the Scheimpflug optical system 2 and the fixation optical system 3 are arranged in a positional relationship such that the optical axis 2a and the fixation optical axis 3a form an angle θ in the Y direction. The inclination direction of the fixation optical axis 3a with respect to the optical axis 2a may be upward or downward. Furthermore, this inclination angle may be variable.

[0122] In another embodiment, the Scheimpflug optical system and the fixation optical system may be arranged so that the optical axis of the Scheimpflug optical system coincides with the optical axis of the fixation optical system, in which case, for example, the optical path of the Scheimpflug optical system and the optical path of the fixation optical system are coaxially coupled by an optical path coupling element (e.g., a half mirror).

[0123] The communication unit 9 performs data communication between the ophthalmologic apparatus 1 and other devices. That is, the communication unit 9 transmits data to other devices and receives data transmitted from other devices. The communication unit 9 may perform any data communication method. For example, the communication unit 9 includes one or more of various communication interfaces, such as a communication interface conforming to the Internet, a communication interface conforming to a dedicated line, a communication interface conforming to a LAN, and a communication interface conforming to short-range communication. The data communication may be wired communication or wireless communication.

[0124] The data transmitted or received by the communication unit 9 may be encrypted data. The control unit 7 and / or the data processing unit 8 may include either or both of an encryption processing unit that encrypts data transmitted by the communication unit 9 and a decryption processing unit that decrypts data received by the communication unit 9.

[0125] The user interface 10 may include any user interface device, such as a display device or an operation device. A user, such as a doctor, a patient, or an assistant, can use the user interface 10 to operate the ophthalmic apparatus 1 or input information to the ophthalmic apparatus 1. At least a part of the user interface 10 may be a peripheral device of the ophthalmic apparatus 1.

[0126] The user interface 10 of this embodiment includes a display unit 11 shown in Fig. 3. The display unit 11 displays various information under the control of the control unit 7. The display unit 11 may include a flat panel display such as a liquid crystal display (LCD).

[0127] The user interface 10 may include an operation device (not shown). The operation device includes a device for operating the ophthalmic apparatus 1 and a device for inputting information. For example, the operation device includes a button, a switch, a lever, a dial, a handle, a knob, a mouse, a keyboard, a trackball, an operation panel, etc. The user interface 10 may include a device in which a display device and an operation device are integrated, such as a touch screen.

[0128] The Scheimpflug optical system 2 includes an illumination system and an imaging system configured to satisfy the Scheimpflug condition, and performs Scheimpflug imaging of the anterior segment of the subject's eye E. The Scheimpflug optical system 2 is used to collect multiple anterior segment images (anterior segment image set, anterior segment image group) by applying an anterior segment scan to the subject's eye E. The illumination light used for the anterior segment scan in this embodiment is slit light, and is typically visible illumination light (visible slit light), but the type of illumination light is not limited to this.

[0129] FIG. 2 shows an example of the Scheimpflug optical system 2. FIG. 2 is a diagram (top view) of the Scheimpflug optical system 2 as viewed from above. The Scheimpflug optical system 2 of this example includes an illumination system 21 and a pair of imaging systems 22L and 22R. The illumination system 21 is configured to project illumination light (visible illumination light, slit light, visible slit light) onto the anterior segment of the subject's eye E. The imaging system 22L is configured to photograph the anterior segment onto which the visible illumination light is projected. Similarly, the imaging system 22R is configured to photograph the anterior segment onto which the visible illumination light is projected. The illumination system 21 and the pair of imaging systems 22L and 22R are moved by a moving mechanism 6. During anterior segment scanning, the illumination system 21 and the pair of imaging systems 22L and 22R are moved together in the X direction by the moving mechanism 6.

[0130] An optical axis (illumination optical axis) 21a of the illumination system 21 is arranged parallel to the Z axis, so that the illumination system 21 can project slit light onto the anterior segment of the eye E from the front direction.

[0131] The optical axis (photography optical axis) 22La of the photography system 22L is inclined at an angle θL in a first direction (+X direction or -X direction) defined by the X axis with respect to the illumination optical axis 21a. This allows the photography system 22L to photograph the anterior segment of the subject's eye E from a first oblique direction that forms a first angle (θL) in the width direction (X direction) of the slit light with respect to the front direction (Z direction). Furthermore, the illumination system 21 and the photography system 22L form a Scheimpflug optical system that satisfies the Scheimpflug condition.

[0132] The optical axis (photography optical axis) 22Ra of the photography system 22R is inclined by an angle θR with respect to the illumination optical axis 21a in a second direction (-X direction or +X direction) opposite to the first direction. The angles θL and θR may be equal to or different from each other. This allows the photography system 22R to photograph the anterior segment of the subject's eye E from a second oblique direction that forms a second angle (θR) in the width direction (X direction) of the slit light with respect to the front direction (Z direction). Furthermore, the illumination system 21 and the photography system 22R form a single Scheimpflug optical system that satisfies the Scheimpflug condition.

[0133] The illumination optical axis 21a and the two imaging optical axes 22La and 22Ra are all disposed on the ZX plane. The Scheimpflug optical system 2 is shown as a side view in FIG. 1 and as a top view in FIG. 2. The optical axis 2a in FIG. 1 corresponds to the illumination optical axis 21a and the two imaging optical axes 22La and 22Ra in FIG. 2. The fixation optical axis 3a in FIG. 1 is disposed so as to be inclined by an angle θ in the Y direction with respect to the illumination optical axis 21a and the two imaging optical axes 22La and 22Ra. The fixation optical axis 3a may be disposed so as to be inclined upward by the angle θ or downward by the angle θ with respect to the illumination optical axis 21a and the two imaging optical axes 22La and 22Ra.

[0134] A non-limiting example of the above-described configuration is described in Patent Document 1 (JP 2023-49320 A) by the present applicant. The Scheimpflug optical system 2 and fixation optical system 3 of this embodiment may have the same configuration as the slit lamp microscope described in Patent Document 1 (JP 2023-49320 A). In this case, the Scheimpflug optical system 2 has the following configuration.

[0135] The illumination system 21 includes an illumination light source that generates visible light, a slit forming unit that forms a slit opening for converting the generated visible light into slit light, and an objective lens that projects the formed slit light onto the anterior segment of the subject's eye E. The width and length directions of the slit opening correspond to the width direction (X direction) and length direction (Y direction) of the slit light, respectively.

[0136] The imaging system 22L includes an optical system including an objective lens, a variable magnification optical system, and an imaging lens, as well as an image sensor that detects light guided by this optical system. The image sensor is an area image sensor such as a CCD image sensor or a CMOS image sensor. The imaging system 22R has a configuration similar to that of the imaging system 22L.

[0137] As described above, the illumination system 21 and the photographing system 22L form a single Scheimpflug optical system. That is, the illumination system 21 and the photographing system 22L are configured so that an object plane including the illumination optical axis 21a of the illumination system 21, a principal plane of the optical system of the photographing system 22L, and an imaging plane of the image sensor of the photographing system 22L intersect on the same straight line. Similarly, the combination of the illumination system 21 and the photographing system 22R is configured so that an object plane including the illumination optical axis 21a of the illumination system 21, a principal plane of the optical system of the photographing system 22R, and an imaging plane of the image sensor of the photographing system 22R intersect on the same straight line.

[0138] The shape of the slit light projected onto the anterior segment by the illumination system 21 is a thin strip with the Y direction as the longitudinal direction (length direction of the slit light) and the X direction as the lateral direction (width direction of the slit light). The movement mechanism 6 moves the illumination system 21 and the imaging systems 22L and 22R integrally in the X direction. The movement mechanism 6 includes an actuator such as a motor and a mechanism for moving the illumination system 21 and the imaging systems 22L and 22R by the driving force generated by the actuator.

[0139] The Scheimpflug optical system 2 performs an anterior segment scan using slit light by combining the projection of slit light by the illumination system 21, multiple photographing (time-series photographing, video photographing, repeated photographing) by the photographing systems 22L and 22R, and the integral movement of the illumination system 21 and the photographing systems 22L and 22R by the movement mechanism 6. This allows the projection area (object plane) of the slit light projected from the front direction onto the anterior segment to be moved in the X direction while photographing multiple times from oblique directions by the photographing systems 22L and 22R, thereby collecting multiple anterior segment images (a series of anterior segment images, anterior segment image set). Each anterior segment image included in the collected anterior segment image set depicts a cross section of the anterior segment corresponding to the area of the anterior segment onto which the slit light was projected at the time of photographing.

[0140] When the configuration of Patent Document 1 (JP Patent Publication No. 2023-49320) is adopted, the fixation optical system 3 includes a fixation light source, a diffuser, a pinhole member, a first lens, a cross reticle plate, and a second lens in order to project fixation light onto the subject's eye E (fundus) from a direction inclined at an angle θ in the Y direction (upward or downward) with respect to the optical axis 2a of the Scheimpflug optical system 2.

[0141] The fixation light source generates green light as visible light. The generated visible light is diffused by a diffuser plate and then projected onto a pinhole member. The visible light that passes through an opening (pinhole) formed in the pinhole member is projected onto a cross reticle plate via a first lens. The visible light that passes through a cross-shaped light-transmitting portion formed on the cross reticle plate is guided to the subject's eye E via a second lens. The subject can visually recognize the cross-shaped fixation light (fixation target).

[0142] Returning to FIG. 1, the movement mechanism 6 moves the Scheimpflug optical system 2 and the fixation optical system 3. The movement mechanism 6 may be capable of moving the Scheimpflug optical system 2 and the fixation optical system 3 three-dimensionally (i.e., in the X, Y, and Z directions). This three-dimensional movement is used for alignment and tracking.

[0143] The control unit 7 controls each part of the ophthalmologic apparatus 1. For example, the control unit 7 controls elements of the illumination system 21 (illumination light source, optical elements, mechanisms, etc.), elements of the imaging systems 22L and 22R (image sensors, optical elements, mechanisms, etc.), the movement mechanism 6, the data processing unit 8, the communication unit 9, the user interface 10, etc.

[0144] The control unit 7 includes a processor, a main storage device, an auxiliary storage device, etc. The auxiliary storage device stores computer programs such as various control programs. These computer programs may be stored in a computer or storage device accessible to the ophthalmic apparatus 1. The functions of the control unit 7 are realized by cooperation between software such as the control programs and hardware such as the processor.

[0145] The data processing unit 8 executes various types of data processing. The data to be processed may be either data acquired by the ophthalmologic apparatus 1 or data input from the outside.

[0146] The data processing unit 8 includes a processor, a main storage device, an auxiliary storage device, etc. The auxiliary storage device stores computer programs such as various data processing programs. These computer programs may be stored in a computer or storage device accessible by the ophthalmic apparatus 1. The functions of the data processing unit 8 are realized by cooperation between software such as the data processing programs and hardware such as the processor.

[0147] One non-limiting example of the data processing unit 8 is shown in FIG. 3. The data processing unit 8 in this example includes an image processing unit 81 and a storage unit 82. The image processing unit 81 performs predetermined image processing based on information stored in the storage unit 82. The functions of the image processing unit 81 are realized by cooperation between software such as an image processing program and hardware such as a processor. The storage unit 82 may be any type of storage device, and may be a non-volatile memory, a volatile memory, or a combination thereof.

[0148] The storage unit 82 stores one or more coordinate transformation functions. The coordinate transformation function represents the relationship between a first coordinate system on the imaging plane of the image sensor of the imaging system 22L and a second coordinate system on the object plane of the Scheimpflug optical system 2. The relationship between the coordinate system (first coordinate system) on the imaging plane of the image sensor of the imaging system 22R and the second coordinate system on the object plane of the Scheimpflug optical system may also be defined by the same coordinate transformation function. Alternatively, when the relationship between the coordinate system (third coordinate system) on the imaging plane of the image sensor of the imaging system 22R and the second coordinate system on the object plane of the Scheimpflug optical system is defined by a different coordinate transformation function, both coordinate transformation functions may be determined by the same method.

[0149] The coordinate transformation function stored in the storage unit 82 may be any one of a mathematical formula, a table (lookup table), a graph, and a chart. Alternatively, the coordinate transformation function stored in the storage unit 82 may be a computer program created based on any one of a mathematical formula, a table (lookup table), a graph, and a chart.

[0150] In some examples, the coordinate transformation function stored in the storage unit 82 may be a mathematical model (machine learning model) constructed using machine learning. This machine learning model includes a neural network such as a convolutional neural network. The neural network may be constructed using, for example, a known open-source neural network architecture. The convolutional neural network may have a known structure including, for example, an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The neural network may include a support vector machine, a recurrent neural network, or the like. The machine learning model in this example is constructed by applying machine learning using training data to the neural network. The training data includes, for example, a large number of pairs of anterior eye segment images (input images) affected by image distortion and anterior eye segment images (ground truth images) in which the image distortion has been corrected. The input images may be images generated by photographing the anterior eye segment using any modality. Examples of such modalities include a slit lamp microscope, a Scheimpflug camera, an optical coherence tomography (OCT), a gonioscope, an ultrasound diagnostic device, etc. The ground truth image may be an image obtained by correcting image distortion in an input image using any known method. In the machine learning of this example, for example, an input image is input to a neural network, and the corresponding output image output from the neural network is compared with the ground truth image, thereby adjusting the parameters of the neural network. The machine learning method used to build the machine learning model of this example may be any method, and is not limited to the supervised learning described above. For example, semi-supervised learning, unsupervised learning, reinforcement learning, etc. may also be used. Furthermore, the machine learning model of this example may be built using a support vector machine, a Bayesian classifier, boosting, k-means, kernel density estimation, principal component analysis, independent component analysis, self-organizing map, random forest, generative adversarial network, etc.

[0151] Below, several non-limiting examples will be described, along with general matters regarding image distortion and its correction method, of methods for determining the coordinate transformation functions to be stored in the storage unit 82. In the following examples, a method for determining a coordinate transformation function that represents the relationship between the coordinate system on the imaging plane of the image sensor of the imaging system 22L and the coordinate system on the object plane of the Scheimpflug optical system 2 will be described, but the coordinate transformation function that represents the relationship between the coordinate system on the imaging plane of the image sensor of the imaging system 22R and the coordinate system on the object plane of the Scheimpflug optical system 2 can also be determined in a similar manner.

[0152] The process of determining the coordinate transformation function is executed by a computer. This computer may be provided in the ophthalmic apparatus 1 (for example, a coordinate transformation function determination unit can be provided in the data processing unit 8), or may be provided separately from the ophthalmic apparatus 1. The computer is configured to determine the coordinate transformation function by executing processing according to a predetermined program. In some examples, the computer is configured to perform machine learning to determine the coordinate transformation function.

[0153] In the following description of determining the coordinate transformation function, x, y, X, and Y coordinates are used, but these coordinates are introduced only for the purpose of the description and are different from the X, Y, and Z coordinates used in Figures 1 and 2 and the description related thereto. However, since the object surface of the Scheimpflug optical system 2 is a surface defined by the projection area of the slit light from the illumination system 2, when the alignment state with the eye E to be examined is appropriate, the coordinate system (XY coordinate system) of the object surface used in determining the coordinate transformation function corresponds to the YZ coordinate system in the XYZ coordinate system shown in Figures 1 and 2.

[0154] First, let us refer to Fig. 4. Fig. 4 shows the correspondence relationship between pixel positions of an image i before image distortion is corrected (pre-correction image) and an image I after image distortion is corrected (post-correction image).

[0155] The corrected image I presented on the left side of FIG. 4 indicates an image detected by the image sensor of the imaging system 22L (or imaging system 22R). The pixel position p in the corrected image I is indicated by (px, py). The pixel position p=(px, py) in the corrected image I corresponds to the position of a light receiving element in a group of light receiving elements (light receiving element array) arranged two-dimensionally on the imaging surface of the image sensor. In other words, the pixel position p=(px, py) in the corrected image I is a pixel position coordinate system (first pixel position coordinate system) that represents the pixel position coordinates on the imaging surface of the image sensor, and corresponds to the coordinate system (first coordinate system) defined on the imaging surface of the image sensor.

[0156] The pre-correction image i presented on the right side of Figure 4 depicts an image of the area of the anterior eye located on the object plane corresponding to the projection area of the slit light by the illumination system 2. The pixel position in the pre-correction image i is indicated by P = (Px, Py). The pixel position P = (Px, Py) of the pre-correction image i corresponds to a two-dimensional position on the object plane. In other words, the pixel position P = (Px, Py) of the pre-correction image i is a pixel position coordinate system (second pixel position coordinate system) that represents the pixel position coordinates in the image (raw image) corresponding to the object plane, and corresponds to the coordinate system (second coordinate system) defined on the object plane.

[0157] As shown in Figure 4, the shape of the image frame of the corrected image I is rectangular, corresponding to the shape of the imaging surface of the image sensor in the imaging system 22L, but the shape of the image frame of the precorrected image i is a distorted rectangle due to the effects of trapezoidal distortion and refraction distortion. If the coordinate transformation function from the first pixel position coordinate system in the corrected image I to the second pixel position coordinate system in the precorrected image i is F(px,py), the following relationship is obtained: I(px,py) = Map(i(Px,Py)) = Map(i(F(px,py))). Here, Map is a mapping determined according to image distortion, and is a linear mapping (bilinear mapping) when only trapezoidal distortion is considered, and is a nonlinear mapping when trapezoidal distortion and refraction distortion are considered.

[0158] The coordinate values px and py of pixel position p = (px, py) in corrected image I are integer values and correspond to the coordinate values Px and Py of pixel position P = F(px, py) = (Px, Py) in uncorrected image i. The coordinate transformation function F represents the transformation of pixel positions from the image sensor to the object surface in response to image distortion.

[0159] The coordinate values px and py of pixel coordinate p = (px, py) in corrected image I are integer values, but the coordinate values Px and Py of pixel position P = F(px, py) = (Px, Py) in pre-corrected image i are not necessarily integer values. Therefore, pixel coordinate interpolation is required. For example, if P = F(px, py) = (100.5, 203.77) (each coordinate value is expressed in pixel number), interpolation can be performed using the coordinate values of the four pixel positions (100, 203), (100, 204), (101, 203), and (101, 204) surrounding pixel position P. Note that the interpolation method is not limited to this example. For details of the matters described herein, please refer to the following document, for example: Guan, Haike et al., Distortion Correction and Panoramic Images by Image Processing, Ricoh Technical Report, No. 23, September 1997. (https: / / jp.ricoh.com / - / Media / Ricoh / Sites / jp_ricoh / technology / techreport / 23 / pdf / 048_055.pdf).

[0160] Several examples of methods for determining the coordinate transformation function F will be described. In the example shown in Fig. 5, the coordinate transformation using the coordinate transformation function F consists of a first transformation, a second transformation, and a third transformation. Below, the first transformation and the third transformation will be described, followed by a description of the second transformation.

[0161] The first transformation is a mapping from the corrected image I to itself, and is expressed using a first coordinate transformation function that defines the coordinate transformation from the first pixel position coordinate system (pxpy coordinate system) to the first length coordinate system (xy coordinate system).

[0162] The first pixel position coordinate system (pxpy coordinate system) is a coordinate system that defines pixel positions (arrangement of the light receiving element array of the image sensor) in the corrected image I, and its coordinate values are integers. The number of pixels arranged in the x direction in the corrected image I is defined as nx, and the number of pixels arranged in the y direction is defined as ny. In other words, the corrected image I is represented by nx × ny pixels arranged in a grid (matrix). The x-direction coordinate px in the first pixel position coordinate system (pxpy coordinate system) can take on values that are integers in the range of [1, nx], and the y-direction coordinate py can take on values that are integers in the range of [1, ny].

[0163] The first length coordinate system (xy coordinate system) is a coordinate system that defines the length in the corrected image I (the imaging surface of the image sensor), and its coordinate values are real values. The dimension in the x direction of the image frame of the corrected image I is defined as lx, and the dimension in the y direction is defined as ly. In other words, the dimension in the x direction of the imaging surface of the image sensor is defined as lx, and the dimension in the y direction is defined as ly. The dimension units may be any, for example, millimeters.

[0164] The first coordinate transformation function that defines the coordinate transformation from the first pixel position coordinate system (px / py coordinate system) to the first length coordinate system (x / y coordinate system) may be, for example, the following function: (x, y) = (px / dp, py / dp), where x is the length coordinate in the x direction, and the range of possible values for x is [-lx / 2, +lx / 2]; y is the length coordinate in the y direction, and the range of possible values for y is [-ly / 2, +ly / 2]; px is the pixel coordinate in the x direction, and the range of possible values for px is [-nx / 2, +nx / 2]; py is the pixel coordinate in the y direction, and the range of possible values for py is [-ny / 2, +ny / 2]; and dp is the array interval (pitch) of light receiving elements in the image sensor [millimeters / pixel].

[0165] The third transformation is a mapping from the uncorrected image i to itself, and is expressed using a third coordinate transformation function that defines the coordinate transformation from the second length coordinate system (XY coordinate system) to the second pixel position coordinate system (PxPy coordinate system).

[0166] The second length coordinate system (XY coordinate system) is a coordinate system that defines the length in the pre-correction image i (object surface), and its coordinate values are real values. The dimension in the X direction of the image frame of the pre-correction image i is defined as Lx, and the dimension in the Y direction is defined as Ly. In other words, the dimension in the X direction of the object surface is defined as Lx, and the dimension in the Y direction is defined as Ly. The dimension units may be any, for example, millimeters.

[0167] The second pixel position coordinate system (PxPy coordinate system) is a coordinate system that defines pixel positions (positions on the object plane) in precorrected image i, and its coordinate values are integers. As in the case of corrected image I, the number of pixels arranged in the X direction in precorrected image i is defined as nx, and the number of pixels arranged in the Y direction is defined as ny. In other words, as in corrected image I, precorrected image i is represented by nx × ny pixels arranged in a grid (matrix). The possible values of the X direction coordinate Px in the second pixel position coordinate system (PxPy coordinate system) are integers in the range of [1, nx], and the possible values of the Y direction coordinate py are integers in the range of [1, ny].

[0168] The third coordinate transformation function that defines the coordinate transformation from the second length coordinate system (XY coordinate system) to the second pixel position coordinate system (PxPy coordinate system) may be, for example, the following function: (Px, Py) = (X / dPX, Y / dPY), where X is the length coordinate in the X direction, and the range of possible X values is [-Lx / 2, +Lx / 2]; Y is the length coordinate in the Y direction, and the range of possible Y values is [-Ly / 2, +Ly / 2]; Px is the pixel coordinate in the X direction, and the range of possible Px values is [-nx / 2, +nx / 2]; Py is the pixel coordinate in the Y direction, and the range of possible Py values is [-ny / 2, +ny / 2]; and dPX and dPY are the array spacing (pitch) [millimeters / pixel] of the light receiving elements in the image sensor.

[0169] The second transformation is a mapping from the corrected image I to the pre-corrected image i, and is expressed using a second coordinate transformation function that defines the coordinate transformation from the first length coordinate system (xy coordinate system) to the second length coordinate system (XY coordinate system).

[0170] As mentioned above, the first length coordinate system (xy coordinate system) is a coordinate system that defines the length in the corrected image I (the imaging surface of the image sensor), and the second length coordinate system (xy coordinate system) is a coordinate system that defines the length in the pre-corrected image i (object surface).

[0171] The second coordinate transformation function that defines the coordinate transformation from the first length coordinate system (lxly coordinate system) to the second length coordinate system (LxLy coordinate system) is a nonlinear projective transformation function that takes into account the influence of trapezoidal distortion and refractive distortion caused by the structure of the Scheimpflug optical system 2, as well as the influence of refractive distortion caused by the structure of the eye that is the subject of imaging. When the influence of refractive distortion is not taken into account, the second coordinate transformation function is expressed as a linear projective transformation function, which can be considered as a special case of a nonlinear projective transformation function that also takes into account the influence of refractive distortion.

[0172] Such a nonlinear projection transformation function is determined, for example, by using ray tracing based on predetermined simulation conditions. The simulation conditions include, for example, optical system conditions determined from the structure of the Scheimpflug optical system 2 and ocular conditions determined from the structure of an eyeball model. The eyeball model may be any of an eye model (for example, a Gullstrand eye model), an image of a living eye, a theoretical eye model, and a physical eye model, or may be information other than these.

[0173] Below, two types of ray tracing methods for determining the nonlinear projective transformation function used as the second coordinate transformation function will be described. The first type uses single-stage ray tracing, which considers the Scheimpflug optical system 2 and the eyeball model as a single optical system. The second type uses multi-stage ray tracing, which performs separate ray tracing for each type of image distortion.

[0174] An example of a method for determining a nonlinear projection transformation function using single-stage ray tracing will be described with reference to Fig. 6. The XYZ coordinate system shown in Fig. 6 is a three-dimensional Cartesian coordinate system formed by adding a Z coordinate axis orthogonal to both the X and Y coordinate axes to a second length coordinate system (XY coordinate system) defined on the object surface 211, and as described above, is a coordinate system different from the XYZ coordinate systems shown in Figs. 1 and 2. The object surface 211 is disposed on the plane Z=0.

[0175] The object plane 211 corresponds to the projection area of the slit light from the illumination system 2. The symbol O indicates the central position (object surface center) of the object plane 211. Each position on the object plane 211 is expressed using an XY coordinate system. The origin of the XY coordinate system may be located at the object surface center O or at another position. The dimensions of the object plane 211 (i.e., the imaging range) are Lx×Ly [square millimeters].

[0176] The symbol EM indicates an eyeball model. The eyeball model EM is composed of eye structure models such as the cornea (anterior corneal surface, posterior corneal surface), lens (anterior lens surface, posterior lens surface), and iris (anterior iris surface). Eyeball conditions including the conditions of these eye structure models are set in the eyeball model EM.

[0177] Reference numeral 221 denotes an image sensor of the imaging system 22L. A first length coordinate system (xy coordinate system) is defined on an imaging surface 221a of the image sensor 221. Reference numeral o denotes the central position (center of the imaging surface) of the imaging surface 221a. Each position on the imaging surface 221a is expressed using the xy coordinate system. The origin of the xy coordinate system may be located at the center o of the imaging surface, or may be located at another position. The dimensions of the imaging surface 221a are lx×ly [square millimeters].

[0178] The symbol za denotes a coordinate axis corresponding to the optical axis 22La of the imaging system 22L. Two coordinate axes perpendicular to the za coordinate axis are the xa coordinate axis and the ya coordinate axis. The origin of the xayaza coordinate system is denoted by the symbol Oa. The za coordinate axis and the xa coordinate axis are located in the ZX plane defined by the XYZ coordinate system. Although not shown in the figure, the ya coordinate axis is perpendicular to the ZX plane and parallel to the Y coordinate axis. In this way, the zaxa coordinate system is a coordinate system obtained by rotating the ZX coordinate system around the Y coordinate axis.

[0179] The origin Oa of the x-, y-, and y-za coordinate system is set on an extension plane of the imaging surface 221a of the image sensor 221. The distance from the origin Oa to the imaging surface center o is defined as l, and the angle that the imaging surface 221a forms with the za coordinate axis is defined as a. Furthermore, the distance from the intersection of the extension plane of the object surface 211 and the za coordinate axis to the object surface center O is defined as L, and the angle that the object surface 211 forms with the za coordinate axis is defined as A.

[0180] Reference numerals 222 and 223 respectively denote the front and rear principal planes of the imaging system 22L. The position (za coordinate) of the front principal plane 222 is defined as ha, and the position (za coordinate) of the rear principal plane 223 is defined as Ha. The focal length of the imaging system 22L is defined as f [millimeters].

[0181] If the angle formed by a line connecting an arbitrary position (x, y) on the imaging plane 221a and the za coordinate position ha on the front principal plane 222 with the za coordinate axis is defined as wx, the angle wx is a variable determined by the angle of view of the imaging system 22L and the position (x, y) on the imaging plane 221a. A line (principal ray) V extending from the za coordinate position Ha on the rear principal plane 223 in a direction forming the same angle wx is directed toward the eyeball model EM. The angle formed by the principal ray V and the object plane 211 is defined as Wx.

[0182] Ray tracing in this example is performed under the conditions and settings described above. The chief ray V corresponds to the outgoing ray vector from the imaging system 22L. In ray tracing in this example, real ray tracing is performed using this outgoing ray vector V as the initial condition, and the paths of the rays (virtual rays) within the eyeball model EM are determined. The nonlinear projection transformation function in this example is determined based on the paths of the virtual rays that have been determined.

[0183] The actual ray tracing in this example is designed to terminate when, for example, any of the following three conditions (first to third termination conditions) is satisfied.

[0184] The first termination condition is that a virtual ray, whose initial condition is the output ray vector V, reaches the object surface 211 before reaching a surface of interest of the eyeball model EM (the anterior surface of the cornea, the posterior surface of the cornea, the anterior surface of the lens, the posterior surface of the lens, the anterior surface of the iris, etc.) (that is, that the virtual ray forms an image on the object surface 211). In other words, the virtual ray reaches the plane Z=0 defined in the XYZ coordinate system of FIG. 6 (that is, that the virtual ray forms an image on the plane Z=0) before reaching the surface of interest of the eyeball model EM. Note that although the actual image formation point is not located on the plane Z=0, this first termination condition is specified taking into consideration that the projection area of the slit light from the illumination system 2 is the plane Z=0.

[0185] The second termination condition is that when a virtual ray, whose initial condition is the exit ray vector V, intersects with any plane of interest of the eyeball model EM, the distance from this intersection point to the object surface 211 is equal to or greater than a predetermined threshold. This threshold corresponds to the size of the aperture of the eye diaphragm. This diaphragm may be, for example, the eyelid, the limbus, or the iris.

[0186] The third termination condition is that a virtual ray whose initial condition is the exit ray vector V does not intersect with any of the surfaces of interest of the eyeball model EM. The third termination condition corresponds to obtaining an imaginary solution as the coordinates of the intersection of the virtual ray and the surface of interest of the eyeball model EM in the calculation of real ray tracing.

[0187] In this example, a plurality of surfaces of interest are set in the eyeball model EM. In real ray tracing in this example, the path of a virtual ray, which has an output ray vector V as an initial condition, is determined by sequentially calculating the output ray vector from each of the plurality of surfaces of interest as this virtual ray passes through them in sequence. The order in which the virtual ray passes through the plurality of surfaces of interest is determined by the arrangement of the plurality of surfaces of interest, and is, for example, the anterior surface of the cornea, the posterior surface of the cornea, the anterior surface of the lens, and the posterior surface of the lens.

[0188] An example of a method for determining the nonlinear projection transformation function of this example will be described with further reference to FIG.

[0189] First, one position (x, y) on the imaging surface 221a of the image sensor 221 is set, and the output ray vector V is determined based on the simulation conditions shown in Fig. 6 (S1). The order in which the positions on the imaging surface 221a are set may be arbitrary, and may be determined in advance according to, for example, the arrangement order of the light receiving element array on the imaging surface 221a. The calculation in step S1 does not depend on the eyeball model EM, but is performed based on the optical system conditions related to the imaging system 22L of the Scheimpflug optical system 2. Actual ray tracing is started using the output ray vector V determined in step S1 as the initial condition (S2).

[0190] In the actual ray tracing of this example, it is determined whether the virtual ray based on the outgoing ray vector V reaches the plane Z=0 (object surface 211) before reaching the target surface of the eyeball model EM (S3).

[0191] In step S3, it is first determined whether or not the output light vector V from the imaging system 22L reaches the plane Z=0 before reaching the anterior corneal surface of the corneal model.

[0192] If the virtual ray reaches the plane Z=0 before reaching the anterior surface of the cornea (S3: Yes), the virtual ray forms an image in the external region (air region) of the eyeball model EM without entering the eyeball model EM (or before entering the eyeball model EM). In this case, the process proceeds to step S4. In step S4, it is determined whether real ray tracing has been completed for all positions (x, y) on the imaging surface 221a (S4).

[0193] If actual ray tracing has been completed for all positions (S4: Yes), the process proceeds to step S10, and actual ray tracing ends (S10).

[0194] On the other hand, if actual ray tracing has not been completed for all positions (S4: No), the process proceeds to step S5. In step S5, the next position on the imaging surface 221a is set (S5) according to the setting order described in step S1. Then, the process returns to step S1, a new output ray vector V is determined (S1), and actual ray tracing is started with this new output ray vector V as the initial condition (S2).

[0195] On the other hand, if the virtual ray reaches the anterior surface of the cornea without reaching the plane Z=0 (S3: No), the virtual ray enters the eyeball model EM, and the process proceeds to step S6, where the coordinates of the intersection of the virtual ray and the anterior surface of the cornea are calculated (S6).

[0196] The equation representing the virtual ray is (x,y,z)=t×(xa,ya,za)+(xb,yb,zb), and the equation representing the surface of interest is (x-(pi+Ri)). 2 +y 2 +(zs) 2 =Ri 2 Then, in step S6, a calculation is performed to determine the intersection point between these. Here, t represents the gradient of the virtual ray. Furthermore, pi represents the X coordinate of the i-th surface of interest, and Ri represents the radius of the i-th surface of interest. For example, i=1, 2, 3, 4 are set to the anterior surface of the cornea, the posterior surface of the cornea, the anterior surface of the lens, and the posterior surface of the lens, respectively. Furthermore, s represents the Z coordinate of the object surface 211 (the Z coordinate of the illumination system 21). When performing an anterior eye scan, a nonlinear projection transformation function is determined for each of multiple Z coordinates Z=s.

[0197] If the coordinates of the intersection between the virtual ray and the anterior corneal surface are imaginary solutions, that is, if there is no intersection between the virtual ray and the anterior corneal surface (S7: No), the process proceeds to step S10, and the actual ray tracing ends (S10).

[0198] On the other hand, if an intersection point between the virtual ray and the anterior corneal surface exists (S7: Yes), the process proceeds to step S8. In step S8, an exit ray vector from the anterior corneal surface is calculated based on the virtual ray incident on the anterior corneal surface (incident ray vector to the posterior corneal surface), the conditions of the anterior corneal surface (conditions of the anterior corneal surface model), and other conditions (air refractive index, corneal refractive index, etc.) (S8). This calculation is performed using, for example, Snell's law. This completes the process related to the virtual ray and the anterior corneal surface.

[0199] When step S8 is completed, the process proceeds to step S9. In step S9, it is determined whether or not there is a next surface of interest to be considered in the eyeball model EM (S9). When calculations related to the virtual ray have been completed for all surfaces of interest in the eyeball model EM (S9: No), the process proceeds to step S10, and real ray tracing ends (S10).

[0200] The eyeball model EM in this example has multiple surfaces of interest. If there is a next surface of interest (S9: Yes), the process returns to step S3. Note that the explanation up to this point has been about the processing related to the anterior corneal surface, which is the first surface of interest. The virtual ray refracted by the anterior corneal surface travels in the direction of the exit ray vector calculated in step S8. The second surface of interest is the posterior corneal surface. In this example, steps S1 to S9 are executed in the same manner as when the virtual ray represented by the exit ray vector V and the anterior corneal surface are considered.

[0201] This series of steps (steps S1 to S9) is executed, for example, in the order of the anterior surface of the cornea, the posterior surface of the cornea, the anterior surface of the lens, and the posterior surface of the lens. As a result, actual ray tracing is performed for all positions (x, y) on the imaging plane 221a, taking into account the structure of the Scheimpflug optical system 2 and the structure of the eyeball model EM. In other words, actual ray tracing is performed over the entire angle of view of the Scheimpflug optical system 2. Once the actual ray tracing is completed (S10), a nonlinear projective transformation function corresponding to each position (x, y) on the imaging plane 221a is determined (S11) based on the result. This concludes the processing of this example (END).

[0202] Figure 8 shows the correspondence between coordinates on the imaging surface 221a of the image sensor 221 (corrected image I) and coordinates on the object surface 211 (pre-corrected image i) based on the coordinate transformation function F(px,py) determined by actually performing the processing of Figure 7.

[0203] 8 shows the positional correspondence between the distribution of multiple coordinates (sensor coordinates) on imaging surface 221a of image sensor 221 and the distribution of multiple coordinates (object surface coordinates) on object surface 211. The position of each sensor coordinate is indicated by a diamond mark, and the position of each object surface coordinate is indicated by a filled-in triangle mark.

[0204] 8, region 301 indicates the external region (air region) of the eyeball model EM, region 302 indicates the region corresponding to the cornea model, region 303 indicates the region corresponding to the lens model, and region 304 indicates the region where the virtual ray does not pass due to a diaphragm (light blocking portion) of the iris model or the like.

[0205] This concludes the explanation of the method for determining a nonlinear projection transformation function using single-stage ray tracing.

[0206] Next, an example of a method for determining a nonlinear projective transformation function using multi-stage ray tracing will be described. In this example, ray tracing is performed based on optical system conditions determined from the structure of the Scheimpflug optical system 2, and ray tracing is performed based on ocular conditions determined from the structure of an eyeball model.

[0207] In this example, in ray tracing based on optical system conditions, a first ray tracing is performed to determine a first conversion function g1 related to the trapezoidal distortion caused by the Scheimpflug optical system 2, and processing is performed to determine a second conversion function g2 related to the image distortion (refractive distortion) caused by the distortion aberration of the Scheimpflug optical system 2.

[0208] More specifically, in this example, in the first ray tracing for determining the first transformation function g1, paraxial ray tracing is performed based on the optical system conditions, with the output ray vector from the Scheimpflug optical system 2 as the initial condition. Also, in this example, a predetermined mathematical formula expressing the relationship between a coordinate system that does not take image distortion into account and a coordinate system that takes image distortion into account is used as the second transformation function g2. The mathematical formula used as the second transformation function g2 may be the mathematical formula described in BROWN (or Chikazu) mentioned above, or may be another mathematical formula. The mathematical formula described in BROWN or Chikazu expresses the relationship between a coordinate system that does not take image distortion into account and a coordinate system that takes image distortion into account.

[0209] In another example, in the process for determining the second transformation function g2, actual ray tracing (second ray tracing) based on the optical system conditions may be performed, with the output ray vector from the Scheimpflug optical system 2 as the initial condition. The second ray tracing can be performed in the same manner as the actual ray tracing described above as single-stage ray tracing, so for this actual ray tracing, please refer to the description of single-stage ray tracing.

[0210] In this example, in ray tracing based on the eyeball condition, a third ray tracing is performed to determine a third conversion function g3 related to image distortion (refractive distortion) caused by refraction by the eyeball.

[0211] The nonlinear projective transformation function obtained in this example is a composite function g3(g2(g1)) of a first transformation function g1, a second transformation function g2, and a third transformation function g3. Here, the composite function g2(g1) is a function representing a two-stage coordinate transformation for correcting two types of image distortion (trapezoidal distortion and refractive distortion) caused by the structure of the Scheimpflug optical system 2. Furthermore, the final composite function g3(g2(g1)) is a function representing a three-stage coordinate transformation for correcting the trapezoidal distortion and refractive distortion caused by the structure of the Scheimpflug optical system 2 and the refractive distortion caused by the subject's eye E (eyeball).

[0212] 9, the first transformation function g1 is a coordinate transformation function from an image 311 expressed by coordinates (x, y) in the first length coordinate system described above to an image 312 expressed by coordinates (X1, Y1) in an X1Y1 coordinate system described later, and is used to correct trapezoidal distortion caused by the Scheimpflug optical system 2. The second transformation function g2 is a coordinate transformation function from the image 312 expressed by coordinates (X1, Y1) in the X1Y1 coordinate system to an image 313 expressed by coordinates (X2, Y2) in an X2Y2 coordinate system described later, and is used to correct refractive distortion caused by the Scheimpflug optical system 2. The third transformation function g3 is a coordinate transformation function from the image 313 expressed by coordinates (X2, Y2) in the X2Y2 coordinate system to an image 314 expressed by coordinates (X, Y) in the second length coordinate system (XY coordinate system) described above, and is used to correct refractive distortion caused by the subject's eye E.

[0213] In this example, the process of determining the first conversion function g1 is performed by paraxial ray tracing using the same optical system conditions as the ray tracing described with reference to Fig. 6. If the coordinates on the imaging plane 221a of the image sensor 221 are (x, y) and the coordinates on the object plane 211 are (X1, Y1), the following first conversion function g1 is obtained by this paraxial ray tracing.

[0214] X1=[{(-l) / (-(l×sin(a)+ha))}×{(-f×l×sin(a)) / (fl×sin(a))}]+[{(l+x)×cos(a)} / {sin(A)}]×[1 / {-(l+x)×sin(a)-ha}]×[{f×(-(l+x)×sin(a))} / {f-((l+x)×sin(a))}]

[0215] Y1=[y / {-(l+x)×sin(a)-ha}]×[{f×(-(l+x)×sin(a))} / {f-((l+x)×sin(a))}]

[0216] Thus, the first transformation function g1 in this example consists of two functions g1x and g1y: g1 = g1(x, y) = (g1x, g1y) = (X1, Y1). The function g1x takes the x coordinate as input and the X1 coordinate as output: X1 = g1x(x). Furthermore, the function g1y takes the x and y coordinates as input and the Y1 coordinate as output: Y1 = g1y(x, y).

[0217] The correspondence between coordinates (x, y) and coordinates (X1, Y1) = g1(x, y) according to the first transformation function g1 of this example is shown in Fig. 10. Fig. 10 shows the positional correspondence between the distribution of multiple coordinates (x, y) in the xy coordinate system and the distribution of coordinates (g1x(x), g1y(x, y)) in the X1Y1 coordinate system. Each coordinate (x, y) in the xy coordinate system is indicated by a diamond mark, and each coordinate (g1x(x), g1y(x, y)) in the X1Y1 coordinate system is indicated by a black circle mark.

[0218] In this example, the second conversion function g2 may be the mathematical formula disclosed in BROWN or Chikazu. The second conversion function g2 in this example converts coordinates (X1, Y1) in an X1Y1 coordinate system defined to express positions on the object surface 211 into coordinates (X2, Y2) in an X2Y2 coordinate system also defined to express positions on the object surface 211. The mathematical formulas disclosed in BROWN or Chikazu are used for calibration of a Scheimpflug camera, and are the following pair of mathematical formulas.

[0219] X2={(1+ka×r 2 +kb×r 4 )+2×(X1×pa+Y1×pb)}×X1+pa×r 2

[0220] Y2={(1+ka×r 2 +kb×r 4 )+2×(X1×pa+Y1×pb)}×Y1+pb×r 2

[0221] where r 2 =X1 2 +Y1 2 Furthermore, ka and kb are distortion correction coefficients in the radial direction, and pa and pb are distortion correction coefficients in the tangential direction.

[0222] Thus, the second transformation function g2 in this example consists of two functions g2x and g2y: g2 = g2(X1, Y1) = (g2x, g2y) = (X2, Y2). The function g2x takes the X1 coordinate and the Y1 coordinate as input and outputs the X2 coordinate: X2 = g2x(X1, Y1). Furthermore, the function g2y takes the X1 coordinate and the Y1 coordinate as input and outputs the Y2 coordinate: Y2 = g2y(X1, Y1).

[0223] FIG. 11 shows the correspondence between coordinates (X1,Y1) = g1(x,y) and coordinates (X2,Y2) = g2(g1(x,y)) using the second transformation function g2 of this example. FIG. 11 shows the positional correspondence between the distribution of coordinates (g1x(x), g1y(x,y)) in the X1Y1 coordinate system and the distribution of coordinates (g2x(g1x(x), g1y(x,y)), g2y(g1x(x), g1y(x,y))) in the X2Y2 coordinate system. Each coordinate g1(x,y) in the X1Y1 coordinate system is indicated by a black circle, and each coordinate g2(g1(x,y)) in the X2Y2 coordinate system is indicated by a rectangle.

[0224] In this example, the process of determining the third transformation function g3 (third ray tracing) is executed by actual ray tracing based on the eyeball condition, with the output ray vector from the Scheimpflug optical system 2 as the initial condition.

[0225] The eyeball model of this example includes a plurality of eye structure models, similar to the eyeball model EM in Fig. 6. Furthermore, the eyeball conditions of this example include, similar to the eyeball conditions based on the eyeball model EM, cornea model conditions, iris model conditions, lens model conditions, anterior chamber model conditions, pupil model conditions, vitreous body model conditions, air conditions, and the like.

[0226] In the third ray tracing, a process (referred to as an identification process) is performed to identify an eye structure model through which a virtual ray reaching a predetermined point on the eye model passes, from among a plurality of eye structure models in the eyeball model. Furthermore, in the third ray tracing, if one or more eye structure models are identified by this identification process, actual ray tracing including the following two processes is performed. In the first process of the actual ray tracing of this example, an exit ray vector corresponding to an incident ray vector on each target surface of each eye structure model identified by the identification process is calculated. That is, in the first process, an exit ray vector from the target surface is calculated based at least on the incident ray vector on the target surface. Furthermore, in the second process of the actual ray tracing of this example, an intersection between a virtual ray based on the exit ray vector from the target surface calculated in the first process and an object surface is calculated.

[0227] An example of an eyeball model used in the third ray tracing is shown in Fig. 12. In the description of eyeball model EM1 in Fig. 12, the +X direction may be referred to as the left, the -X direction as the right, the +Y direction as the up, and the -Y direction as the down.

[0228] In this example, a case will be described in which the slit light from the Scheimpflug optical system 2 passes through the center O of the eyeball model EM1. That is, in this example, a case will be described in which the object surface (plane Z=0) of the Scheimpflug optical system 2 passes through the center O. When performing an anterior eye scan, for example, the object surface (plane Z=0) of the Scheimpflug optical system 2 is positioned at a position away from the center O. As described above, even in this case, ray tracing can be performed in the same manner as in this example.

[0229] The eyeball model EM1 includes a cornea model Co and a lens model Cr. Corneal conditions such as the refractive index of the cornea, positional information (X coordinate, etc.) and morphological information (curvature, radius of curvature, dimensions, etc.) of the anterior corneal surface, and positional information (X coordinate, etc.) and morphological information (curvature, radius of curvature, dimensions, etc.) of the posterior corneal surface are assigned to the cornea model Co. Lens conditions such as the refractive index of the lens, positional information (X coordinate, etc.) and morphological information (curvature, radius of curvature, dimensions, etc.) of the anterior corneal surface, and positional information (X coordinate, etc.) and morphological information (curvature, radius of curvature, dimensions, etc.) of the posterior corneal surface are assigned to the lens model Cr.

[0230] The region to the right of the cornea model Co is an air region. Air conditions (such as the refractive index of air) are assigned to the air region. The region above the top end of the cornea model Co and the region below the bottom end are regions where virtual light rays are blocked by the eyelids or the like, that is, regions where virtual light rays do not pass (called blocking regions). The region between the cornea model Co and the lens model Cr is a region equivalent to the anterior chamber (anterior chamber region). Anterior chamber conditions (such as the anterior chamber depth and the refractive index of aqueous humor) are assigned to the anterior chamber region. The region to the left of the lens model Cr is a region equivalent to the vitreous body (vitreous region). Vitreous conditions (such as the refractive index of vitreous body) are assigned to the vitreous body region.

[0231] The parameter values (corneal conditions, lens conditions, etc.) in the eyeball model EM1 may be, for example, values in a Gullstrand eye model or the like, or may be values determined by applying image processing such as segmentation to an image obtained by photographing a living eye or an eye model. Furthermore, the third ray tracing may be performed using multiple eyeball models according to eye attributes (disease, race, age, etc.).

[0232] An overview of a series of steps executed in the third ray tracing of this example will be described. In the third ray tracing of this example, first, the aforementioned identification process is executed. In the identification process of this example, an eye structure model through which a virtual ray that reaches a predetermined point on the eyeball model EM1 passes is identified from among the multiple eye structure models included in the eyeball model EM1. In other words, the identification process of this example is a process of analyzing which eye structure model affects the depiction position of the predetermined point on the eyeball model EM1 when the eyeball model EM1 is photographed using the Scheimpflug optical system 2.

[0233] In the identification process of this example, a surface of interest through which a virtual ray that reaches a predetermined point on the eyeball model EM1 passes is identified from among multiple surfaces of interest included in the eyeball model EM1. In other words, in the identification process of this example, a surface of interest that affects the rendering position of the predetermined point on the eyeball model EM1 is identified.

[0234] The eyeball model EM1 has the anterior corneal surface, the posterior corneal surface, the anterior lens surface, and the posterior lens surface as surfaces of interest. Therefore, in the identification process of this example, it is determined whether a virtual ray that reaches a predetermined point on the eyeball model EM1 corresponds to any of the following cases: it reaches the point without passing through any surfaces of interest, it reaches the point by passing only through the anterior corneal surface, it reaches the point by passing only through the anterior and posterior corneal surfaces, it reaches the point by passing only through the anterior corneal surface, the posterior corneal surface, and the anterior lens surface, or it reaches the point by passing through the anterior corneal surface, the posterior corneal surface, the anterior lens surface, and the posterior lens surface.

[0235] Next, for a virtual ray that reaches a predetermined point on the eyeball model EM1, a ray tracing calculation is performed for each plane of interest identified in the identification process, and the intersection of this virtual ray with the object surface (plane Z=0) of the Scheimpflug optical system 2 is determined. Note that, as described above, in this example, the case where the object surface of the Scheimpflug optical system 2 passes through the center O is considered, and therefore the intersection with the plane Z=0 is determined, but if the object surface of the Scheimpflug optical system 2 does not pass through the center O, the intersection of the virtual ray with the object surface (plane Z=s) that corresponds to the position of the Scheimpflug optical system 2 in that case is calculated.

[0236] Note that if a virtual ray reaches a region above the upper end or below the lower end of the cornea model Co, the virtual ray does not contribute to anterior segment imaging, and therefore ray tracing calculation is not performed. In this case, the input coordinates (X2, Y2) to the third transformation function g3 may be set equal to the corresponding output coordinates (X, Y) (X = X2, Y = Y2).

[0237] 13 shows an incident virtual ray Lin with respect to the anterior corneal surface Cof and an outgoing virtual ray Lout refracted by the anterior corneal surface Cof. Symbol K denotes the intersection (incident position) of the incident virtual ray Lin and the anterior corneal surface Cof, and symbol Kn denotes the normal to the anterior corneal surface Cof at the incident position K. Point P2 denotes the intersection of the incident virtual ray Lin (its extension) and the X coordinate axis, and point P denotes the intersection of the outgoing virtual ray Lout and the X coordinate axis. Point P = (X2, Y2) in the eyeball model EM1 is located in the anterior chamber, and therefore, the light ray that reaches point P is affected by refraction from the anterior and posterior corneal surfaces.

[0238] The specific process of this example will now be described in more detail. In this example, the following parameters are taken into account:

[0239] The curvature of the ith surface of interest is defined as parameter Ri, where i=1, 2, 3, and 4 correspond to the anterior corneal surface, posterior corneal surface, anterior lens surface, and posterior lens surface, respectively.

[0240] The X coordinate of the ith surface of interest is defined as parameter pi, where i=1, 2, 3, and 4 correspond to the anterior surface of the cornea, the posterior surface of the cornea, the anterior surface of the lens, and the posterior surface of the lens, respectively.

[0241] When the i-th surface of interest is considered, the refractive index of the region in front (to the right) of this surface of interest is defined as parameter n. When the first surface of interest (anterior corneal surface) is considered, parameter n is the refractive index of air, when the second surface of interest (posterior corneal surface) is considered, parameter n is the refractive index of the cornea, when the third surface of interest (anterior crystalline lens) is considered, parameter n is the refractive index of aqueous humor, and when the fourth surface of interest (posterior crystalline lens) is considered, parameter n is the refractive index of the crystalline lens.

[0242] When the ith surface of interest is considered, the refractive index of the area behind (to the left of) this surface of interest is defined as parameter N. When the first surface of interest (anterior corneal surface) is considered, parameter n is the refractive index of the cornea, when the second surface of interest (posterior corneal surface) is considered, parameter n is the refractive index of the aqueous humor, when the third surface of interest (anterior crystalline lens) is considered, parameter n is the refractive index of the crystalline lens, and when the fourth surface of interest (posterior crystalline lens) is considered, parameter n is the refractive index of the vitreous body.

[0243] The aperture size of the ith surface of interest is denoted as Api, where i=1, 2, 3, and 4 correspond to the anterior corneal surface, posterior corneal surface, anterior lens surface, and posterior lens surface, respectively. See also Figure 14 for the parameter Api.

[0244] The X coordinate of the intersection of the i-th surface of interest and the line representing the aperture of the i-th surface of interest is defined as Bxi, where i=1, 2, 3, and 4 correspond to the anterior surface of the cornea, the posterior surface of the cornea, the anterior surface of the lens, and the posterior surface of the lens, respectively. See also Figure 14 for the parameter Bxi.

[0245] The index of the regions considered in the eyeball model EM1 is q: the air region is q=1, the cornea region is q=2, the anterior chamber region is q=3, the lens region is q=4, the vitreous region is q=5, and the blocking region is q=6.

[0246] Each region q can be calculated based on the eye structure model (assumed to be spherical) and the parameters Api and Bxi. As mentioned above, in this example, only the plane Z=0 (XY plane) is considered.

[0247] For example, the air region q=1 is located to the right of the anterior corneal surface, and is therefore expressed as a region that satisfies the following two conditions: [(X-(p1-R1)) 2 +Y 2 >R1 2 ];condition[X <Bx1]。

[0248] The anterior chamber region q=3 is located to the left of the posterior surface of the cornea (i=2) and to the right of the anterior surface of the lens (i=3). Therefore, it can be expressed as a region that satisfies the following three conditions: [(X-(p2-R2)) 2 +Y 2 <R2 2 ];Condition [(X-(p3-R3)) 2 +Y 2 >R3 2 ];condition[X <Bx3]。

[0249] Once these conditional expressions are found for each region q, ray tracing is performed as follows: For the air region q=1, there is no effect of refraction by the eyeball model EM1, so the output coordinates (X, Y) are set equal to the input coordinates (X2, Y2) (X=X2, Y=Y2) without ray tracing. For the cornea region q=2, one ray tracing is performed with the anterior corneal surface as the surface of interest. For the anterior chamber region q=3, two ray tracings are performed corresponding to two surfaces of interest, the anterior corneal surface and the posterior corneal surface. For the lens region q=4, three ray tracings are performed corresponding to three surfaces of interest, the anterior corneal surface, the posterior corneal surface, and the anterior lens surface. For the vitreous region q=5, four ray tracings are performed corresponding to four surfaces of interest, the anterior corneal surface, the posterior corneal surface, the anterior lens surface, and the posterior lens surface. For the blocking region q=6, (X, Y)=(X2, Y2) is assigned, or the ray tracing is ignored in this case.

[0250] Next, we will explain how to calculate the refraction state of the virtual ray in this example. Here again, the case of plane Z = 0 is considered. Furthermore, the parameters Ri, pi, n, N, Api, Bxi, and q are still used.

[0251] In addition to these parameters, in this example, as shown in Fig. 14, the incident ray vector is denoted by Vin, and the normal vector of the surface of interest (here, the anterior surface of the cornea) at the intersection of the incident ray vector Vin and the surface of interest is denoted by V_norm. The angle formed between the incident ray vector Vin and the normal vector V_norm is denoted by θ. Furthermore, the angle formed between the outgoing ray vector Vout from the surface of interest and the normal vector V_norm is denoted by Φ.

[0252] The incident ray vector Vin can be expressed as Vin = t × (X2 - Hx, Y2 - Hy, 0 - Hz) + (Hx, Hy, Hz) using the coordinates (X2, Y2, 0) and other characteristic points (Hx, Hy, Hz). Using this expression, the intersection point (X_inter, Y_inter, Z_inter) between the incident ray vector Vin and the surface of interest (anterior corneal surface) can be found. Note that the rear principal plane of the imaging system 22L of the Scheimpflug optical system 2 can be used as the characteristic point (Hx, Hy, Hz).

[0253] If the intersection calculation result is "No solution", or abs(Y_inter,Z_inter)>abs(sqrt(Api 2 ))(=Api), it is determined that the blocking region is q=6. Here, abs(α) indicates the absolute value of α, and sqrt(α) indicates the square root of α.

[0254] The normal vector V_norm can be expressed as t×((Ri+pi)-X_inter,0-Y_inter,Z_inter)+(X_inter,Y_inter,Z_inter). From this, θ=arccos[(V·V_norm) / (abs(V)×abs(V_norm))]. Here, arccos(α) represents the arc cosine of α. Furthermore, from Snell's law, Φ=arcsin(n×sinθ / N) can be obtained. Here, arcsin(α) represents the arc sine of α.

[0255] From the above, the output ray vector Vout=t×((n / N)×V+(n / N×cosθ-cosΦ)×V_norm)+(X_inter, Y_inter, Z_inter) can be calculated.

[0256] If there is a next plane of interest, the exit ray vector Vout is used as the incident ray vector Vin, and a similar ray tracing calculation is performed for this next plane of interest. By performing this ray tracing calculation a number of times according to the assigned index q and finding the value of t that makes the plane Z = 0, the coordinates (X, Y) can be found.

[0257] By tracing the third ray as described above, the third transformation function g3 of this example is determined. The third transformation function g3 of this example consists of two functions g3x and g3y: g3 = g3(X2, Y2) = (g3x, g3y) = (X, Y).

[0258] FIG. 15 shows the correspondence between coordinates (X2,Y2) = g2(X1,Y1) = g2(g1(x,y)) and coordinates (X,Y) = g3(X2,Y2) = g3(g2(g1(x,y))) using the third transformation function g3 of this example. FIG. 15 shows the positional correspondence between the distribution of coordinates g2(g1(x,y)) in the X2Y2 coordinate system and the distribution of coordinates g3(g2(g1(x,y))) in the XY coordinate system. Each coordinate g2(g1(x,y)) in the X2Y2 coordinate system is indicated by a rectangular mark, and each coordinate g3(g2(g1(x,y))) in the XY coordinate system is indicated by a filled-in triangle mark.

[0259] Figure 16 shows a comparison of the calculation results of the third transformation function g3 with a standard human eye model. As can be seen from Figure 16, the calculation results of the third transformation function g3 show that the contours of the cornea and the anterior chamber are different from those of the human eye model. As such, the shapes of the eye structures are depicted distorted in the captured image obtained using the Scheimpflug optical system 2.

[0260] Figure 17 shows a comparison between the shape of the human eye model (i.e., the true shape (x, y) of the human eye model being photographed) and the shape of the human eye model depicted in the photographed image affected by image distortion due to the composite function g3(g2(g1(x, y))) of the first, second, and third transformation functions g1, g2, and g3 (i.e., the shape of the image of the human eye model in the photographed image). The true shape of the human eye model is shown by a dashed line, and the shape of the image of the human eye model is shown by a solid line. In other words, when the Scheimpflug optical system 2 is used, the shape shown by the dashed line is distorted and depicted as the shape shown by the solid line.

[0261] In this example, after determining the correspondence relationship of such shapes (composite function g3(g2(g1(x,y)))), the third transformation shown in FIG. 5 is executed. As a result, the coordinate transformation function F of this example is determined. The determined coordinate transformation function F is stored in the storage unit 82. Alternatively, the determined coordinate transformation function F is stored in another storage device and provided to the ophthalmologic apparatus 1.

[0262] By using the determined coordinate transformation function F, the image processing unit 81 of the ophthalmic device 1 can correct the distortion of the image obtained by photographing the anterior segment of the subject's eye E using the Scheimpflug optical system 2, and generate a processed image that represents the actual shape of the anterior segment (or a shape close to it).

[0263] An example of the operation of the ophthalmologic apparatus 1 of this embodiment will be described with reference to FIG.

[0264] In this example, first, a process of determining a coordinate transformation function is executed (S21). This process is executed by the ophthalmologic apparatus 1 (the coordinate transformation function determination unit described above) or another computer. The method of determining the coordinate transformation function may be any method, and may be, for example, any of the exemplary methods described above.

[0265] The coordinate transformation function determined in step S21 is stored in the storage unit 82 by the control unit 7 (S22). Steps S21 and S22 are preparatory steps for the ophthalmologic apparatus 1 to perform image distortion correction.

[0266] Note that the coordinate transformation function can be updated by newly performing a process for determining the coordinate transformation function. Also, a new coordinate transformation function can be determined according to a new attribute (disease, race, age, etc.). In these cases, the control unit 7 stores the newly obtained coordinate transformation function in the storage unit 82.

[0267] After the above preparations, anterior eye imaging is performed using the ophthalmic apparatus 1 (S23). In anterior eye imaging, first, the subject's head is placed on a holding portion (forehead rest, chin rest) not shown of the ophthalmic apparatus 1. In response to an instruction operation performed using, for example, the user interface 10, the ophthalmic apparatus 1 starts projecting fixation light onto the subject's eye E, which is the target of anterior eye imaging, and aligns the Scheimpflug optical system 2 with the subject's eye E. Following the alignment, the ophthalmic apparatus 1 may start tracking to make the Scheimpflug optical system 2 follow the movement of the subject's eye E. Then, the ophthalmic apparatus 1 generates an image of the anterior eye of the subject's eye E using the Scheimpflug optical system 2.

[0268] The anterior-segment image of the subject's eye E generated in step S23 is sent by the control unit 7 to the image processing unit 81. The image processing unit 81 applies coordinate transformation using the coordinate transformation function stored in the storage unit 82 in step S22 to the anterior-segment image generated in step S23. This results in a processed image in which the trapezoidal distortion and refractive distortion caused by the Scheimpflug optical system 2 and the refractive distortion caused by the subject's eye E have been corrected (S24).

[0269] The control unit 7 causes the display unit 11 to display the processed image generated in step S24 (S25).

[0270] According to this example, the user can observe a processed image in which the morphology of the anterior segment of the subject's eye E is faithfully depicted and which has a deep depth of field. Furthermore, by providing the processed image obtained in step S24 for image analysis, it is possible to provide high-quality image diagnosis. For example, in addition to being able to determine accurate values of various anterior segment parameters based on the actual morphology of the anterior segment of the subject's eye E, it is also possible to perform anterior segment analysis over a wide depth range.

[0271] In step S25, the control unit 7 may display other information on the display unit 11 together with the processed image generated in step S24. For example, the ophthalmologic apparatus 1 can display the anterior eye image acquired in step S23 (i.e., the original image (raw image) of the processed image, which is the anterior eye image before image distortion correction) together with the processed image after image distortion correction. This makes it possible to compare the images before and after the image distortion correction performed in step S24.

[0272] A comparison of the images before and after image distortion correction can be performed by a computer, and the results of this image comparison can be used to improve the quality of current coordinate transformation functions or to determine new coordinate transformation functions.

[0273] Another example of the operation of the ophthalmologic apparatus 1 of this embodiment will be described with reference to Fig. 19. Unless otherwise specified, it is possible to combine the items related to the example of the operation of Fig. 18.

[0274] In this example, an anterior eye scan is applied to the subject's eye E. In the anterior eye scan, the Scheimpflug optical system 2 is translated relative to the subject's eye E to perform anterior eye imaging multiple times, thereby collecting multiple anterior eye images (anterior eye image sets) corresponding to multiple positions (multiple scan positions). The direction of translation of the Scheimpflug optical system 2 in the anterior eye scan is the X direction shown in FIGS. 1 and 2. Therefore, multiple anterior eye imaging is performed while translating the object plane of the Scheimpflug optical system 2 in the X direction. As mentioned in the description of FIG. 7, the multiple scan positions corresponding to multiple anterior eye imaging times correspond to the multiple Z coordinates Z=s (s=sj:j=1, 2,...J) introduced in the description of the method for determining the coordinate transformation function.

[0275] In this example, first, a process of determining a coordinate transformation function corresponding to each scan position Z=sj is executed (S31). The coordinate transformation function corresponding to each scan position Z=sj determined in step S31 is stored in the storage unit 82 by the control unit 7 (S32).

[0276] After the above preparations, an anterior eye scan is applied to the subject's eye E using the ophthalmologic apparatus 1 (S33). The set of anterior eye images of the subject's eye E generated in step S33 is sent to the image processing unit 81 by the control unit 7.

[0277] The image processing unit 81 reads out, for each anterior eye image included in the anterior eye image set, a coordinate transformation function corresponding to the scan position of the anterior eye image from the storage unit 82, and applies coordinate transformation using the read coordinate transformation function to the anterior eye image. As a result, image distortion of the anterior eye image (trapezoidal distortion and refractive distortion caused by the Scheimpflug optical system 2, and refractive distortion caused by the subject's eye E) is corrected to obtain a processed image. The image processing unit 81 applies this series of processes to all anterior eye images (or some of the anterior eye images) included in the anterior eye image set, thereby generating a plurality of processed images corresponding to a plurality of scan positions from the plurality of anterior eye images collected in the anterior eye scan in step S33 (S34).

[0278] The image processing unit 81 generates a three-dimensional image from the plurality of processed images generated in step S34 (S35).

[0279] For example, the image processing unit 81 first extracts an image area corresponding to the object surface from each processed image. Each extracted image area is a cross-sectional image at a position corresponding to the corresponding scan position, and is a cross-sectional image having a large depth of field. This allows multiple cross-sectional images corresponding to the multiple scan positions to be obtained. The multiple cross-sectional images are, for example, arranged parallel to each other at equal intervals.

[0280] The image processing unit 81 can construct a three-dimensional image based on a plurality of cross-sectional images extracted from a plurality of processed images. This three-dimensional image may be, for example, stack data, volume data (voxel data), etc.

[0281] Furthermore, the image processing unit 81 can apply predetermined rendering to the constructed 3D image, such as multiplanar reconstruction (MPR), maximum intensity projection (MIP), minimum intensity projection (MIP), and surface display (contour extraction, projection transformation, shading, etc.).

[0282] The control unit 7 causes the display unit 11 to display the three-dimensional image generated in step S35 (S36). For example, the control unit 7 can cause the display unit 11 to display a rendering image of the three-dimensional image.

[0283] Furthermore, the control unit 7 can selectively display the multiple processed images generated in step S34 on the display unit 11. For example, the control unit 7 can select, from the multiple processed images, a processed image corresponding to a scan position or cross-sectional position specified by the user using the user interface 10, and display the selected processed image on the display unit 11.

[0284] Furthermore, the control unit 7 can selectively display a plurality of cross-sectional images extracted from a plurality of processed images on the display unit 11. For example, the control unit 7 can select a cross-sectional image corresponding to a scan position or cross-sectional position designated by the user using the user interface 10 from the plurality of cross-sectional images and display the selected image on the display unit 11.

[0285] The image display modes that can be executed by the control unit 7 are not limited to these examples. The control unit 7 can execute any image display based on any data that can be acquired in the operation example shown in FIG.

[0286] According to this example, the user can observe a three-dimensional processed image that faithfully depicts the morphology of the anterior segment of the subject's eye E and has a deep depth of field. Furthermore, the data obtained in this example (one or more processed images, one or more cross-sectional images, three-dimensional images, rendered images, etc.) can be displayed or provided for image analysis. This makes it possible to provide high-quality image diagnosis. For example, it is possible to observe an image that represents the actual morphology of the anterior segment of the subject's eye E and determine accurate values of various anterior segment parameters based on this image, as well as perform anterior segment analysis of a wide three-dimensional region within the subject's eye E.

[0287] Some non-limiting aspects of the ophthalmic apparatus according to the embodiment have been described above. Any two or more aspects can be at least partially combined.

[0288] <Other aspects> The embodiments of the present disclosure are not limited to ophthalmic devices. Examples of embodiments other than ophthalmic devices include a method for controlling an ophthalmic device, a method for photographing the anterior segment of the eye, a program, and a recording medium. According to these embodiments, similar to the embodiments of the ophthalmic device, it is possible to provide a new technique for correcting image distortion that occurs in ophthalmic imaging using a Scheimpflug optical system.

[0289] Some embodiments provide a method for controlling an ophthalmic apparatus. The ophthalmic apparatus includes a Scheimpflug optical system, a storage unit, a display unit, and a processor. The Scheimpflug optical system includes an illumination system and an imaging system. The illumination system is configured to project illumination light onto an anterior segment of an eye to be examined. The imaging system includes an image sensor and is configured to capture an image of the anterior segment. The illumination system and the imaging system are configured to satisfy the Scheimpflug condition. The method according to this embodiment includes first to third steps. The first step causes the processor to store, in the storage unit, a coordinate transformation function that represents a relationship between a first coordinate system on an imaging plane of the image sensor and a second coordinate system on an object plane of the Scheimpflug optical system. The second step causes the processor to generate a processed image by applying a coordinate transformation using the coordinate transformation function stored in the storage unit in the first step to an anterior segment image generated by the Scheimpflug optical system. The third step causes the processor to display, on the display unit, the processed image generated in the second step.

[0290] Any of the items described in this disclosure can be combined with the method according to the embodiments.

[0291] Some embodiments provide a program. The program according to the embodiments causes a computer including a processor and a memory to execute the method according to the above-described embodiment. Any of the features described in the present disclosure can be combined with the program according to the embodiments.

[0292] Some embodiments provide a computer-readable non-transitory recording medium. The recording medium according to the embodiments stores a program that causes a computer to execute the method according to the above-described embodiments. Any of the features described in this disclosure can be combined with the recording medium according to the embodiments.

[0293] A computer-readable non-transitory recording medium that can be used as a recording medium in this embodiment may be a recording medium of any form, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0294] The embodiments and aspects described in the present disclosure are merely examples, and any modifications (omissions, substitutions, additions, etc.) within the scope of the present invention can be applied to the embodiments and aspects of the present disclosure. [Explanation of symbols]

[0295] 1 Ophthalmology equipment 2 Scheimpflug optics 21 Lighting system 22L, 22R shooting system 6 Moving mechanism 7 Control Unit 8 Data Processing Unit 81 Image processing section 82 Memory section 11 Display section

Claims

1. a Scheimpflug optical system including an illumination system that projects illumination light onto an anterior segment of the eye to be examined, and an imaging system that includes an image sensor and images the anterior segment, the illumination system and the imaging system being configured to satisfy the Scheimpflug condition; a storage unit that stores a coordinate transformation function that represents a relationship between a first coordinate system on an imaging plane of the image sensor and a second coordinate system on an object plane of the Scheimpflug optical system; an image processing unit that applies coordinate transformation using the coordinate transformation function to the anterior eye image generated by the Scheimpflug optical system to generate a processed image; a display unit that displays the processed image; 1. An ophthalmic device comprising:

2. the first coordinate system is a first pixel position coordinate system that represents pixel position coordinates on the imaging surface, the second coordinate system is a second pixel position coordinate system that represents pixel position coordinates in a raw image corresponding to the object surface, The coordinate transformation function is a first coordinate transformation function that represents a relationship between the first pixel position coordinate system and a first length coordinate system that defines a length on the imaging surface; a second coordinate transformation function that represents the relationship between the first length coordinate system and a second length coordinate system that defines a length on the object surface; a third coordinate transformation function that represents the relationship between the second length coordinate system and the second image position coordinate system; Including, The ophthalmic device of claim 1.

3. the second coordinate transformation function includes a nonlinear projection transformation function determined by ray tracing based on simulation conditions including optical system conditions determined from a structure of the Scheimpflug optical system and ocular conditions determined from a structure of an eyeball model. The ophthalmic device of claim 2.

4. the optical system conditions are determined based on at least a condition of the imaging plane, a condition of the object plane, and a condition of an optical element included in the Scheimpflug optical system. The ophthalmic apparatus of claim 3.

5. the eyeball model includes an eye structure model, the eyeball conditions are determined based on at least conditions of the eye structure model.

5. The ophthalmic apparatus according to claim 3 or 4.

6. the nonlinear projection transformation function is determined based on paths of virtual rays in the eyeball model obtained by actual ray tracing using an output ray vector from the Scheimpflug optical system as an initial condition. The ophthalmic apparatus of claim 3.

7. the nonlinear projection transformation function is determined based on paths of virtual rays in the eyeball model obtained by actual ray tracing using an output ray vector from the Scheimpflug optical system as an initial condition, terminate the actual ray tracing for the virtual ray in any one of the following cases: when the virtual ray reaches the object surface before reaching the target surface of the eye structure model; when the distance from the intersection point of the virtual ray and the target surface to the object surface is equal to or greater than a threshold; and when the virtual ray does not intersect with the target surface. The ophthalmic device of claim 5.

8. the nonlinear projection transformation function is determined based on paths of virtual rays in the eyeball model obtained by actual ray tracing using an output ray vector from the Scheimpflug optical system as an initial condition, the eyeball model includes one or more eye structure models in which a plurality of target planes are set, the path of the virtual ray is determined by sequentially calculating an exit ray vector from each of the target surfaces when the virtual ray, which has the exit ray vector from the Scheimpflug optical system as the initial condition, passes through the multiple target surfaces in sequence. The ophthalmic device of claim 5.

9. The ray tracing is ray tracing based on the optical system conditions; Ray tracing based on the eyeball conditions; Including, The ophthalmic apparatus of claim 3.

10. The ray tracing based on the optical system conditions is a first ray tracing to determine a first transformation function for the keystone distortion caused by the Scheimpflug optics; determining a second transformation function for image distortion due to distortion of the Scheimpflug optical system; Including, The ophthalmic device of claim 9.

11. the first ray tracing includes paraxial ray tracing based on the optical system condition, with an output ray vector from the Scheimpflug optical system as an initial condition. The ophthalmic device of claim 10.

12. the second transformation function includes a predetermined mathematical expression that represents a relationship between a coordinate system that does not take image distortion into consideration and a coordinate system that takes image distortion into consideration. The ophthalmic device of claim 10.

13. the process for determining the second transformation function includes a second ray tracing that is an actual ray tracing based on the optical system conditions, with an output ray vector from the Scheimpflug optical system as an initial condition. The ophthalmic device of claim 10.

14. the ray tracing based on the optical system conditions determines a composite function of the first conversion function and the second conversion function; The ophthalmic device of claim 10.

15. the ray tracing based on the ocular condition includes a third ray tracing for determining a third transformation function related to image distortion due to refraction by the eye; the nonlinear projection transformation function is a composite function of the composite function of the first transformation function and the second transformation function, and the third transformation function; The ophthalmic device of claim 14.

16. the ray tracing based on the ocular condition includes a third ray tracing for determining a third transformation function related to image distortion caused by refraction by the eye. The ophthalmic device of claim 9.

17. the third ray tracing includes actual ray tracing based on the eyeball condition, with an output ray vector from the Scheimpflug optical system as an initial condition. The ophthalmic device of claim 16.

18. the eyeball model includes a plurality of eye structure models, the eyeball condition is determined based on at least conditions of the plurality of eye structure models, the third ray tracing further includes a specification process of specifying, from the plurality of eye structure models, an eye structure model through which a virtual ray that reaches a predetermined point on the eyeball model passes, When one or more eye structure models are identified by the identification process, the actual ray tracing is A process of determining an exit ray vector corresponding to an incident ray vector on each target surface of each identified eye structure model; A process of determining an intersection point between a virtual ray based on the exit ray vector and the object surface; Including, 18. The ophthalmic device of claim 17.

19. a moving mechanism for moving the Scheimpflug optical system to a plurality of positions; the Scheimpflug optics collects a plurality of anterior segment images corresponding to the plurality of positions; the storage unit stores a plurality of coordinate transformation functions corresponding to the plurality of positions; the image processing unit applies coordinate transformation using a corresponding coordinate transformation function to each of the plurality of anterior eye segment images to generate a plurality of processed images corresponding to the plurality of positions. The ophthalmic device of claim 1.

20. A method for controlling an ophthalmic apparatus for photographing an anterior segment of a subject's eye, comprising: The ophthalmic device includes: a Scheimpflug optical system including an illumination system that projects illumination light onto an anterior segment of the subject's eye, and an imaging system that includes an image sensor and images the anterior segment, the illumination system and the imaging system being configured to satisfy the Scheimpflug condition; A memory unit; A display unit; Processor and Including, the processor, a process of storing a coordinate transformation function representing a relationship between a first coordinate system on an imaging plane of the image sensor and a second coordinate system on an object plane of the Scheimpflug optical system in the storage unit; a process of applying a coordinate transformation using the coordinate transformation function to the anterior eye image generated by the Scheimpflug optical system to generate a processed image; a process of displaying the processed image on the display unit; Execute method.

21. A program that causes a computer to execute the method of claim 20.

22. A computer-readable non-transitory recording medium on which the program of claim 21 is recorded.

Images