Determining ophthalmic-related biometric features of at least one eye from images of eye regions

Abstract

The invention relates to a computer-implemented method for determining individual biometric parameters of at least one eye and to a corresponding method of manufacturing an ophthalmic lens taking into account the determined biometric data. Furthermore, the invention relates to a corresponding computer program product and device. The method for determining biometric data comprises the steps of: providing image data comprising: at least one image of at least a portion of an eye region of the user; and / or geometric information about at least a portion of the eye region of the user, directly resulting from at least one image of at least a portion of the eye region of the user; determining individual additional data comprising at least one individual biometric parameter of the at least one eye of the user, based on the provided image data, using a statistical model describing a relationship between the image data and the additional data.

Description

Technical Field

[0001] This invention relates to a computer-implemented method for determining biometric data of the eye and a corresponding method for manufacturing eyeglass lenses taking into account the determined biometric data. Furthermore, this invention relates to corresponding computer program products and devices. Background Technology

[0002] When calculating spectacle lenses, especially graduated lenses, the biometrics of the eyewearer can be considered, such as those described in document US9,910,294B2. These biometric spectacle lenses offer significant advantages because the quality of the image is no longer evaluated at the vertex sphere (a virtual point in a simple model), but rather where the image actually occurs: on the retina of the eye. Therefore, the interactions between individual aberrations that occur during refraction and propagation through the eye's medium are also taken into account.

[0003] However, a drawback of this method is the need for extensive measurements using sophisticated equipment and facilities. This results in tremendous effort and high costs. Therefore, the advantages of high-quality biometric lenses are only applicable to a relatively small percentage of visually impaired individuals. Summary of the Invention

[0004] The purpose of this invention is to make full use of the advantages of biometric eyeglass lenses without the disadvantages of complex measurements.

[0005] This objective is achieved through a computer-based method, corresponding apparatus, and corresponding computer program product for determining biometric data of the eye, as well as a method and corresponding apparatus for manufacturing spectacle lenses having the features specified in the corresponding aspects of the invention.

[0006] This invention is based on the surprising discovery that, using data generated from one or more images of at least a portion of a user's eye region, individual biometric parameters of at least one eye of the user can be determined or derived. These parameters are used to improve the individual calculation, manufacturing, or adjustment of spectacle lenses. Therefore, individual biometric spectacle lenses with high image quality and wearing comfort can be calculated and manufactured without the need for complex and costly measurements of additional biometric data.

[0007] A first aspect of the present invention relates to a computer-implemented method for determining individual biometric parameters of at least one eye of a user, comprising: Provide image data, which includes or contains: At least one image of at least a portion of the user's eye region; and / or Geometric information about at least a portion of the user's eye region is obtained directly from at least one image of at least a portion of the user's eye region; Based on the provided image data, individual supplementary data is determined using a statistical model describing the relationship between the image data and supplementary data, including at least one individual biometric parameter of at least one eye of the user.

[0008] A particular advantage of this invention is that the image data can be, or has already been, recorded using common and very simple devices (cameras), and in particular, need not be designed so that additional data can be directly determined from the image data. For example, this method offers an advantage if individual additional data cannot be directly determined from the individual image data of the person being measured, because such relationships cannot be derived from the measurement method used without additional information from others regarding the corresponding additional data, for example, because the device providing the image data cannot directly measure additional information.

[0009] A statistical model can be derived, or may have already been derived, using statistical analysis of a training dataset with multiple reference datasets, each of which includes image data and additional data assigned to the image data. The method may include providing a statistical model describing the relationship between the image data and the additional data. Alternatively or additionally, the method may include providing a training dataset and using statistical analysis of the training dataset to derive a statistical model. Deriving the statistical model may include, for example, training an original (untrained) model using the training dataset. For example, the model may include several model parameters that were changed or adjusted during training with the training dataset.

[0010] In this application, the term "provide" includes "designate," "transmit," "obtain," "read out," "retrieve from memory, database, and / or table," "receive," etc. The term "determine" in this application also includes "designate," "calculate," "identify," "predict," etc.

[0011] Therefore, the value of at least one parameter of a person is estimated in a dataset associated with a person, which is related to the biometrics of one or preferably both eyes of that person. This dataset includes data associated with or derived from images of at least a portion of the eye region. The determination is performed using a statistical model that maps the relationship between the parameter and data associated with at least one image of a portion of the eye region applicable to at least several individuals.

[0012] Such determined values ​​can be taken into account when calculating spectacle lenses or other ophthalmic lenses such as contact lenses or intraocular lenses. Image data

[0013] Image data as defined in this specification refers specifically to data associated with images of at least a portion of a user's eye region. This involves, for example, direct images of at least a portion of the eye region, particularly camera images, and / or data / information (particularly geometric) about at least a portion of the user's eye region, which is directly derived from or can be obtained from at least one image of at least a portion of the user's eye region, or such data / information is directly derived from at least one image of at least a portion of the user's eye region. For example, in this method, the camera image itself can be used to determine the pupillary distance, for example, based on which a statistical model is then used to determine additional data, or the individual pupillary distance determined from the camera image can be provided as image data so that additional data can then be determined using a statistical model. Additional data can also be determined directly from individual camera images without first explicitly determining the individual value of the pupillary distance.

[0014] These statements about image data apply to both individual image data used to determine individual biometric data, such as in the case of individual orders, and statistical image data (such as image data that can be part of the corresponding reference data in the case of a training dataset), based on which a statistical model has been or can be created or trained.

[0015] In a preferred embodiment, the image data includes one or more of the following parameters, or the image data consists of one or more of the following parameters:

[0016] In particular, one or more camera images that are visible from the front, showing at least one eye (preferably two eyes);

[0017] In particular, one or more camera images that show at least one eye, especially from the side;

[0018] Pupil distance;

[0019] The distance from the apex of the cornea to the plane containing the iris;

[0020] The geometric path of the corneal surface cross-section (e.g., the vertical cross-section of the cornea in a camera image taken from the side).

[0021] The distance from the center of eye rotation (especially as the optical eye rotation center);

[0022] Corneal diameter (white to white, especially horizontal measurement);

[0023] The shape and / or location of the pupil or a portion thereof (in a given representation, e.g., as a closed or open polygon, as a spline coefficient, etc.).

[0024] The shape and / or location of the outer edge of the iris or a portion thereof (in a given representation, e.g., as a closed or open polygon, as a spline coefficient, etc.).

[0025] 3D model of the eye region (e.g., 3D point cloud, 3D edge network that can be determined using a depth camera, etc.);

[0026] Current line of sight;

[0027] The presence of latent strabismus and / or the preferred form / degree of latent strabismus. Additional data

[0028] Additional data is or includes biometric data of at least one of the user's eyes, particularly biometric data that cannot be directly read or measured from image data (e.g., geometric distances between markers), but which is, for example, routinely determined or already determined, particularly optionally (e.g., by an optician) related to an order for eyeglass lenses and especially considered in cases of individual selection and / or optimization and / or adjustment of at least one eyeglass lens in eyeglasses or eyeglasses. For example, additional data may include data that can (typically) be recorded or has been recorded using an aberrometer, topographic mapper, SAM camera, OCT, biometer, fundus camera, (low-coherence) laser reflectometer and / or another measuring device or another objective optometric method.

[0029] In a preferred embodiment, at least one individual biometric parameter includes one or more of the following parameters:

[0030] Eye length (especially the total geometric length of the eye);

[0031] The optical length of the eye (the integral of the refractive index along a given wavelength (such as 550 nm) of light passing through the point of sharpest vision on the retina and the center of the pupil).

[0032] One or more geometric distances and / or optical path lengths between the vertices of the eye's refractive surfaces and each other or with another common reference point (such as the point on the retina where vision is sharpest);

[0033] The shape of the refractive surface (e.g., the shape and / or tilt of the cornea or one of the lens surfaces) can exist in a given parameterization (e.g., via the radii of curvature of the principal sections and their orientation, the Zernike coefficient of the vertex depth, geometric curvature, or equivalent representation, e.g., as a power vector or polar coordinate representation) and the shape of the refractive surface is understood relative to a coordinate system (e.g., relative to a coordinate system in which the line of sight orientation is aligned with the negative direction of the z-axis and the x-axis lies in the horizontal plane);

[0034] The physical aperture of the eye (i.e., the opening of the iris) or the diameter or radius of the optical entrance pupil (the iris image through the cornea) can be given for a given lighting condition (e.g., given or average indoor lighting) that is particularly different from the primary lighting condition when the image data is recorded.

[0035] The refractive index of the optical medium of the eye;

[0036] The size and / or shape of the retina, especially the size and / or shape of the fovea;

[0037] The orientation of the retina (especially the fovea), for example, as the vector direction perpendicular to the retina in the point of sharpest vision relative to the direction of light incidence (e.g., the point through which light passes through the center of the pupil and the point of sharpest vision);

[0038] The location and orientation of receptors on the retina;

[0039] The size of the retinal receptive field.

[0040] Biometric data of at least one eye of a user can specifically refer to data of a single eye, or data of one or two eyes relative to the head, or data of two eyes relative to each other.

[0041] In particular, the additional data may include one or more of the following parameters: higher-order aberrations of the eye or at least the cornea (such as coma, triceps, secondary astigmatism, spherical aberration, etc.), lower-order and higher-order aberrations of the cornea (spherical (Sph), cylindrical (Cyl), axial (or M, J0, J45), coma, triceps, secondary astigmatism, spherical aberration, etc.), anterior chamber depth, far and near distances and / or pupil size under meso- and photopic vision conditions.

[0042] In addition to image data, supplementary data in the reference dataset may be data that has been recorded or measured for biometric lenses ordered earlier, for example using aberrometers, topographs, SAM cameras, OCT, biometers, fundus cameras, (low-coherence) laser reflectometers and / or another measuring device.

[0043] Additional individual data specified based on the user's standard individual data and additional data determined using a statistical model may, but need not, be the same type of additional data included in the reference dataset and used to derive the statistical model. Statistical Model

[0044] A statistical model can be any statistical model derived from an existing dataset (training dataset) using statistical methods. Exemplary statistical methods include regression (such as linear regression with particularly nonlinear features, nonlinear regression, nonlinear regression with attention mechanisms, nonlinear multitask regression, nonparametric or semiparametric regression, etc.), classification methods, and other machine learning methods. Machine learning algorithms are described, for example, in Jeremy Watt, Reza Borhani, and Aggelos Katsaggelos: Machine Learning Refined: Foundations, Algorithms, and Applications, Cambridge University Press, 2020.

[0045] The statistical model receives at least a portion of individual image data and / or variables derived therefrom as input variables and uses them to calculate at least a portion of additional individual biometric parameters or supplementary data. The relationship between the image data and the image data specified by the statistical model can be linear or non-linear. Furthermore, the relationship can be multiparametric.

[0046] In a preferred embodiment, the statistical model includes at least one or more of the following models, or it is preferably based on one or more of the following models:

[0047] Linear models (which may be associated with nonlinear features, such as nonlinear functions representing parameters that provide geometric information about the eye region);

[0048] Gaussian process;

[0049] Neural networks (e.g., deep neural networks);

[0050] Decision tree or regression tree (regression tree);

[0051] Boltzmann machine (restricted Boltzmann machine)

[0052] Support Vector Machine.

[0053] Exemplary statistical models are linear or nonlinear regression models. For example, neural networks, including deep neural networks, can also be used as nonlinear regression models. Other nonlinear regression models known in the field of machine learning can also be used. Regression models, such as neural networks, can be trained using the provided training dataset.

[0054] Statistical models can also be a combination of several statistical models of different types, such as linear regression models, nonlinear regression models (such as neural networks), classification models, and / or another statistical model.

[0055] Preferably, the statistical model is further improved as new data is added (e.g., in ongoing training of a neural network with new data). To improve the estimation or prediction of the statistical model, in addition to data associated with the image of the eye region, other parameters not associated with the eye region, such as age, sex, race, height, weight, head circumference, etc., can be used. The statistical model may also include constraints (e.g., the predicted eye model is consistent with known refraction).

[0056] The statistical model derived from the training dataset can be stored in a suitable storage device (such as a database, calculator, computing center, or data cloud). At least a portion of the training dataset used for the derivation can be stored along with the statistical model.

[0057] The statistical model derived from the training dataset can also be continuously or periodically checked and / or modified, for example, based on a new reference dataset. Therefore, the method can include modifying the statistical model. Training dataset

[0058] The statistical model describing the relationship between image data and supplementary data is derived using statistical methods based on a training dataset with multiple individual datasets (reference datasets). Each of the reference datasets may include, for example, image data and supplementary data of a specific user determined using appropriate measurement methods. The different reference datasets in the training dataset may preferably include data (image data and supplementary data) from multiple different users (reference users).

[0059] Therefore, existing orders for biometric eyeglass lenses can be used to train a neural network or another statistical model with the dataset. In the case of new individual orders, the trained statistical model can be used, and additional measurement data (additional data) can be calculated or predicted based on the individual's image data included in the new order. Thus, biometric eyeglass lenses can be calculated using neural networks or other statistical models, based on the individual's image data and the additional data calculated therefrom.

[0060] The number of reference datasets can vary. For example, more than 10, 100, 1,000, 10,000, 100,000, or 1,000,000 reference datasets can be used. The reference datasets preferably cover a large range, preferably covering the entire range of spectacle lenses that can subsequently be ordered. For example, the reference datasets can cover a range of refractive values, such as -20 dpt to +20 dpt for a sphere and -8 dpt to +8 dpt for a cylinder.

[0061] In addition, methods for determining individual biometric parameters of at least one eye of a user may include transmitting individual image data and calculated individual additional data to external entities, such as eyeglass lens manufacturers, manufacturing units, manufacturing equipment, etc.

[0062] A second aspect of the invention relates to a method for manufacturing spectacle lenses, comprising: According to one of the inventive methods described herein, particularly one of the preferred embodiments described herein, individual supplementary data, including at least one individual biometric parameter of at least one eye of the user, is determined based on the provided image data; and The eyeglass lenses are calculated based on the determined individual additional data.

[0063] For example, the method described in U.S. Patent No. 9,910,294 B2 or another known method can be used to calculate the spectacle lens, in which individual biometric parameters are taken into account when calculating the spectacle lens. The method may also include manufacturing the calculated spectacle lens. For example, the spectacle lens may be a single-vision lens, a multifocal lens, or a progressive lens.

[0064] In particular, the present invention relates to a method for manufacturing spectacle lenses and / or a method for providing a manufacturing dataset for manufacturing spectacle lenses, comprising:

[0065] Provide image data, including:

[0066] At least one image of at least one eye region of the user; and / or

[0067] Geometric information about at least a portion of the user's eye region is generated directly from at least one image of at least a portion of the user's eye region;

[0068] Based on the provided image data and using a statistical model describing the relationship between the image data and supplementary data, the individual supplementary data is determined to include at least one individual biometric parameter of at least one eye of the user; and

[0069] The eyeglass lenses are calculated based on the determined individual additional data.

[0070] A third aspect of the invention relates to a computer-implemented method for determining a statistical model, the method comprising:

[0071] Multiple reference datasets are provided to the training dataset, each of which includes image data and additional data associated with the image data;

[0072] Statistical analysis of the training dataset was used to derive a statistical model describing the relationship between image data and supplementary data.

[0073] The statistical model is stored on a storage device.

[0074] The fourth aspect of the invention relates to a computer program product that, when loaded into and executed on a computer's memory, causes the computer to perform the method according to one of the above aspects.

[0075] The foregoing preferred embodiments and advantages are similarly applicable to the methods and computer program products described above.

[0076] A device of corresponding design can be used to perform the method according to one of the above aspects. Therefore, the fifth aspect of the invention relates to a device for determining individual biometric parameters of at least one eye of a user, the device comprising a computing device designed to perform the above-described method for determining individual biometric parameters.

[0077] In particular, the present invention provides an apparatus for determining individual biometric parameters of at least one eye of a user, the apparatus comprising a computing device configured to perform the following operations:

[0078] Provide image data, including:

[0079] At least one image of at least one eye region of the user; and / or

[0080] Geometric information about at least a portion of the user's eye region is generated directly from at least one image of at least a portion of the user's eye region;

[0081] Based on the provided image data and using a statistical model describing the relationship between the image data and the supplementary data, individual supplementary data is determined to include at least one individual biometric parameter of at least one eye of the user.

[0082] The computing device may preferably include:

[0083] An image data input interface is used to provide the user's individual image data, which includes:

[0084] At least one image of at least a portion of the user's eye region; and / or

[0085] Geometric information about at least a portion of the user's eye region is generated directly from at least one image of at least a portion of the user's eye region;

[0086] Additional data computing device for calculating individual supplemental data, which includes at least one individual biometric parameter of at least one eye of a user, the calculation being based on individual image data and performed using a statistical model derived from statistical analysis of a training dataset with multiple reference datasets, each of which includes image data and supplemental data associated with the image data.

[0087] Furthermore, the device may include a model input interface for providing a statistical model. For example, the statistical model may be stored in a device such as a database, calculator, and / or a data or calculator cloud. Additionally, the device may provide a training dataset input interface for providing a training dataset; and a model computation device for deriving or computing the statistical model using statistical analysis of the training dataset. For example, a statistical model can be derived or computed by training an original (untrained) model using the training dataset.

[0088] A sixth aspect of the invention relates to an apparatus for manufacturing spectacle lenses and / or for providing spectacle lens manufacturing data, comprising:

[0089] According to the fifth aspect, the device is used to determine the individual biometric parameters of at least one eye of the user;

[0090] Lens computing devices are designed to calculate spectacle lenses based at least on calculated individual biometric parameters.

[0091] Manufacturing equipment may also include equipment for manufacturing calculated spectacle lenses.

[0092] The aforementioned device for providing, determining, specifying, or calculating data (such as (individual) image data, (individual) supplementary data, statistical models, model parameters, weights, etc.) can be implemented by a suitably configured or programmed data processing device (particularly a specialized hardware module, computer, or computer system, such as a computer or data cloud), which has appropriate computing units, electronic interfaces, memory, and data transmission units. The device may also include at least one preferred interactive graphical user interface (GUI) allowing a user to view and / or input and / or modify data.

[0093] The aforementioned device may also have a suitable interface that allows data (such as training datasets, reference datasets, (individual) image data, (individual) supplementary data, etc.) to be transmitted, input, and / or read out. The device may also include at least one storage unit for storing the data used, for example, in the form of a database.

[0094] Manufacturing equipment may include, for example, at least one CNC-controlled machine for directly machining lens blanks according to determined optimized specifications. Alternatively, spectacle lenses can be manufactured using a casting process. The finished spectacle lens may have a first simple spherical or rotationally symmetric aspherical surface and a second individual surface calculated based on individual image data and calculated additional individual data. The simple spherical or rotationally symmetric aspherical surface may be the front surface of the spectacle lens (i.e., the object-side surface). However, it is of course possible to arrange the individual's surface as the front surface of the spectacle lens. Both surfaces of the spectacle lens may also be calculated separately.

[0095] The apparatus for manufacturing spectacle lenses can be designed as a single piece or a standalone machine, meaning that all components of the apparatus (particularly the device for determining individual biometric parameters of at least one eye of a user and the lens computing device) can be part of the same system or the same machine. However, in a preferred embodiment, the apparatus for manufacturing spectacle lenses is not designed as a single unit, but rather implemented through different (particularly separate) systems or machines. For example, the device for determining individual biometric parameters of at least one eye of a user and the lens computing device can be implemented as a first system (particularly including a computer), while the manufacturing apparatus can be implemented as a second system. The various systems can be located in different locations, i.e., they can be physically separated from each other. For example, one or more systems can be located at the front end and one or more other systems can be located at the back end. For example, the various systems can be located in different company locations or operated by different companies. Here, the various systems particularly have communication devices to exchange data with each other. Preferably, the different systems of the apparatus can communicate directly with each other, particularly via a network (e.g., via a local area network and / or via the Internet). The above statements regarding the apparatus for manufacturing spectacle lenses apply not only to this apparatus but generally to all apparatuses described within the scope of the invention. In particular, the apparatus described herein can be designed as a system. In particular, the system may include multiple devices (possibly locally separate) that are designed to execute the various method steps of the corresponding method.

[0096] Another aspect of the invention relates to spectacle lenses manufactured according to the manufacturing method described above. Furthermore, the invention provides the use of spectacle lenses manufactured according to the manufacturing method described above in a predetermined average, ideal, or individual position in front of a particular user's eyes when wearing spectacle lenses, for correcting the user's visual defects. Attached Figure Description

[0097] Preferred embodiments of the invention will be described below by way of example with reference to the accompanying drawings. The corresponding elements of the described embodiments are not limited to the respective embodiments. Rather, elements of the embodiments can be combined with each other as needed, thereby creating new embodiments. The drawings show:

[0098] Figure 1 This is an exemplary method for determining individual biometric data of at least one eye of a user and calculating the lens thickness of eyeglasses;

[0099] Figure 2 This is an example reference dataset;

[0100] Figure 3 This is an example linear regression model;

[0101] Figure 4This is an example of a nonlinear regression model;

[0102] Figure 5. The component c20 of the Zernike coefficient of the cornea as a function of pupillary distance ( Figure 5A ) or as a function of the spherical equivalent ( Figure 5B ). Detailed Implementation

[0103] Figure 1 An exemplary method is shown for determining individual biometric parameters of at least one eye of a user and for calculating spectacle lenses based on the determined individual biometric parameters. The method includes the following steps:

[0104] Step S1: Create or provide a training dataset 1 from multiple datasets (reference datasets) 10, each reference dataset including image data 12 and additional data 14 assigned to this image data.

[0105] Figure 2 An exemplary reference dataset 10 is shown. Image data 12 specifically includes one or more of the following data or parameters:

[0106] In particular, one or more camera images, preferably two eyes, can be seen from the front;

[0107] In particular, images from one or more cameras that show at least one eye when viewed from the side;

[0108] Pupil distance;

[0109] The distance from the center of eye rotation (especially as the optical eye rotation center);

[0110] Corneal diameter (white to white, especially horizontal measurement);

[0111] The shape and / or location of the pupil or a portion thereof (in a given representation, e.g., as a closed or open polygon, as a spline coefficient, etc.).

[0112] The shape and / or location of the outer edge of the iris or a portion thereof (in a given representation, e.g., as a closed or open polygon, as a spline coefficient, etc.).

[0113] 3D model of the eye region (e.g., 3D point cloud, 3D edge network that can be determined using a depth camera, etc.);

[0114] Current line of sight;

[0115] The presence and / or preferred form / range of latent strabismus.

[0116] Additional data 14 specifically includes one or more of the following data or parameters:

[0117] Eye length (especially the total geometric length of the eye);

[0118] The optical length of the eye (the integral of the refractive index of a light beam of a given wavelength (such as 550 nm) passing through the point of sharpest vision on the retina and the center of the pupil).

[0119] The distances from the vertices of the eye's refractive surfaces to each other or to another common reference point (such as the point on the retina where vision is sharpest);

[0120] The shape of the refractive surface (e.g., the shape and / or tilt of the cornea or one of the lens surfaces) can exist in a given parameterization (e.g., via the radii of curvature of the principal sections and their orientation, the Zernike coefficient of the vertex depth, geometric curvature, or equivalent representation, e.g., as a power vector or polar coordinate representation) and be understood relative to a coordinate system (e.g., relative to a coordinate system in which the line of sight orientation is aligned with the negative direction of the z-axis and the x-axis lies in the horizontal plane).

[0121] For a given lighting condition (e.g., given or average indoor lighting), the physical aperture of the eye (i.e., the opening of the iris) or the diameter or radius of the optical entrance pupil (the image of the iris through the cornea) can be given.

[0122] The refractive index of the optical medium of the eye;

[0123] The size and / or shape of the retina, especially the size and / or shape of the fovea;

[0124] The orientation of the retina (especially the fovea), for example, as the vector direction perpendicular to the retina in the point of sharpest vision relative to the direction of light incidence (e.g., the point through which light passes through the center of the pupil and the point of sharpest vision);

[0125] The location and orientation of receptors on the retina;

[0126] The size of the retinal receptive field.

[0127] To form the training dataset, existing orders of biometric eyeglass lenses can be used, where additional data has already been recorded using measurement methods. Exemplary measurement methods include measurements using aberration meters, topographic maps, SAM cameras, OCT, and / or biometers.

[0128] Step S2: Use statistical methods to derive the relationships between image data and supplementary data from multiple reference datasets. In other words, determine or train a statistical model based on a training dataset that describes the relationships (such as correlations) between image data and supplementary data.

[0129] Determining the statistical model may include, for example, training an initially untrained neural network using a training dataset that includes multiple reference datasets. A test dataset can be used to test the trained neural network, and / or a validation dataset can be used to validate the trained neural network. The test and validation datasets may each include multiple datasets (reference datasets) from previous orders, for example... Figure 2 The test dataset includes multiple reference datasets. Preferably, the reference datasets included in the test dataset are neither included in the validation dataset nor in the training dataset. Similarly, the reference datasets included in the validation dataset are preferably neither included in the test dataset nor in the training dataset.

[0130] Step S3: Provide an individual dataset (individual order) that includes at least individual image data. The individual image data may be recorded by the optician as part of the user's individual eyewear order. The individual order may also include further individual information, particularly individual refractive data, especially if this is individual information that can be determined without complex measurements. This may include simple refractive data (especially low-order aberrations) and / or information about the user's wearing and / or ethnicity and / or age and / or height and / or gender and / or weight.

[0131] Step S4: Based on the individual image data included in the individual dataset provided in Step S3, and further based on the relationship between the image data and supplementary data determined in Step S2, calculate the individual supplementary data (supplementary data). For example, the individual's image data can be input into the trained neural network of Step S2. The corresponding output data of the neural network can be directly used as the individual's supplementary data. Alternatively, the output data of the neural network can be further processed first (such as checking for plausibility, smoothing, filtering, classifying, transforming, etc.) instead of being used directly.

[0132] Step S5: Calculate the individual eyeglass lens based on the individual image data included in the individual dataset provided in step S3 and further based on the individual additional data calculated in step S4.

[0133] The calculation of an individual spectacle lens specifically includes calculating at least one surface of the spectacle lens based on individual image data and calculated additional individual data. The surface thus calculated can be the rear or front surface of the spectacle lens. "Calculation of at least one surface of a spectacle lens" includes the calculation of at least a portion or a single surface. In other words, "calculation of at least one surface of a spectacle lens" means the calculation of at least a portion of the surface or the entire surface.

[0134] The surface opposite to the calculated surface can be a simple surface, such as a sphere, rotationally symmetric surface, aspherical surface, complex surface, or arbitrary surface. Two surfaces can also be calculated separately.

[0135] Individual spectacle lenses can be calculated using known methods, such as those known from the literature US 9,910,294 B2.

[0136] Figure 3 and Figure 4 Exemplary statistical models 2 are shown, each trained on training dataset 1. Therefore, such statistical models can be designed at least partially as neural networks.

[0137] If the statistical model is based on or composed of neural networks, then, for example, the input layer of the neural network is occupied by at least a portion of the image data and / or auxiliary variables computed from it. The output layer outputs the value of at least one additional parameter or at least a portion of the supplementary data. In addition to the input and output layers, the neural network may preferably contain one or more hidden layers. During the training of the initial, untrained neural network, an appropriate learning algorithm is used to change the weights. The trained neural network specifies the relationship between the image data and the supplementary data. The structure of the neural network (such as the number and type of layers, the number and type of neurons in different layers, the way layers and neurons are linked to each other, etc.) and the learning algorithm can vary.

[0138] Figure 3 An exemplary linear regression model with an input layer and an output layer is shown. The output variable has K variables. It is calculated from multidimensional input variables of dimension D (e.g., D=26):

[0139] (1)

[0140] in This represents the weight matrix.

[0141] Figure 4 An exemplary nonlinear regression model with an input layer, an output layer, and several hidden layers is shown. Here, the output variable has k variables. It is from multidimensional input variables with dimension D (e.g., D=26). Calculated:

[0142] (2)

[0143] Rectified linear unit (ReLU)

[0144] in , and This represents the weight matrix.

[0145] Figure 5 illustrates an example of a preferred embodiment using an exemplary linear model. The example shown is based on calculating the Zernike coefficient of the cornea of ​​an eye with a diameter of approximately 7 mm around the corneal apex, derived from the pupillary distance. This means that the pupillary distance is used as image data. Specifically, it can be, or has been, determined directly or indirectly from at least one image of at least a portion of the user's eye region. In particular, Figure 5A The training dataset, which includes a large reference dataset, is shown. For a large number of people or multiple pairs of eyes, this training dataset correlates the corresponding pupillary distance with the Zernike coefficient c2 of a cornea with a radius of 3.55 mm. 0 (Average curvature) is associated.

[0146] Preferably, a simple linear regression is used as the statistical model to determine the Zernike coefficient of the cornea formed around the corneal apex with a radius of 3.55 mm, based on the pupillary distance (PD), a parameter associated with the eye region. For individual determination of the Zernike coefficient, model parameters are determined in the statistical model using linear regression:

[0147] c2 0 (PD) = - 304.1 μm + 0.9422 μm / mm PD.

[0148] The predictive power of this model is 0.11 (adjusted R-squared).

[0149] The model can be improved by adding additional terms (such as white-to-white (WTW)).

[0150] c2 0 (PD, WTW) = - 329,7 μm + 0,9841 μm / mm PD + 2,172 μm / mm WTW

[0151] The predictive power is now 0.15 (adjusted R-squared).

[0152] and Figure 5B The comparison also shows that, utilizing the method of the present invention using individual image data, the (statistical) correlation with (optical) relevant parameters used for the optimization and manufacture of individual spectacle lenses becomes achievable, which is conventionally unpredictable or unforeseen. In particular, Figure 5B The mean corneal curvature (c2) is shown. 0The statistical distribution of the mean corneal curvature is considered as a function of the spherical equivalent of refraction in the eye. Even when observed physiologically in isolation, changes in mean corneal curvature have a very large impact on the spherical equivalent of refraction in the eye, statistically, there is no significant correlation between mean corneal curvature and spherical equivalent. In the case of very simple (non-individually optimized) spectacle lenses, knowledge of the spherical equivalent is sufficient to select suitable lenses. However, for improvements, individual manufacturing, or adjustments to spectacle lenses, for example, knowing details about the individual refractive surface of the eye is clearly beneficial. Therefore, improvements in individual adjustments to spectacle lenses can often be achieved using knowledge of mean corneal curvature (as supplementary data)—or at least using statistical estimation. The advantage of knowing (or approximating) this supplementary data is achieved by a very simple means in accordance with the invention.

[0153] Other parameters that are not necessarily associated with the eye region can also be used (such as the user's age and / or gender). Instead of linear functions, nonlinear functions (such as neural networks) and residual distributions other than normal distributions can also be adjusted (e.g., to be able to robustly determine model parameters from data that may include outliers).

[0154] In a further preferred embodiment, image data is used, specifically based again on pupillary distance, taking into account the spherical equivalent of refraction (M), to make a statistical statement about eye length (AL). For individual determination of eye length, model parameters are determined in the statistical model by means of linear regression:

[0155]

[0156] The predictive power of this model is 0.67 (adjusted R-squared).

[0157] Using geometric parameters of the eye region, namely pupil distance (image data), brings advantages over predictions made purely from spherical equivalents, as shown in the comparison of the predictive power of linear models that do not consider pupil distance:

[0158] The predictive power of this model is 0.63 (adjusted R-squared).

[0159] In a further preferred embodiment, image data is used to determine information about corneal topography. A trained deep artificial neural network is preferably used as a statistical model, which is being or is being trained for the relationship between images of the eye region (image data) and corneal topography (additional data). Specifically, to determine the corneal topography, the last few layers of the network, which may have already been pre-trained on the image data, are preferably trained. In particular, eye biometrics can be determined at least partially and / or approximately from images of the head and / or the area around the eyes (image data), wherein at least one of the following features (possibly as intermediate variables) is extracted or learned by the statistical model as a feature formed in the statistical model: ethnic group, (approximate) age, head size (e.g., head circumference), eye color, height, and weight. It is also conceivable that other features learned by the model, or those whose corresponding correlations cannot be excluded a priori, may also be considered.

[0160] In addition to image data, additional parameters (extended data) are preferably used for the creation and / or application of statistical models as input data for training or determining additional data, such as race group, (approximate) age, head size (such as head circumference), eye color, height, and weight, if available.

[0161] In a preferred embodiment, the additional data includes data on corneal topography (e.g., as curvature, as a Zernike polynomial, or using another general description), which, by means of a statistical model, is associated with at least two images of the eye region or eye taken from different directions as image data (e.g., using a dual-camera video centering device with two cameras). Preferably, distinguishing points of the iris imaged by different regions of the cornea are sought in the images. Under assumptions (e.g., the iris is flat and / or the physical aperture is circular, and the refractive indices of the cornea and anterior chamber are known), the shape of the cornea is then determined (e.g., by changing the mathematical model of the anterior surface of the cornea, possibly the mathematical model of the posterior surface of the cornea, and the positional model of different points on the iris), wherein the same physical model of the cornea must explain the positions of different iris points in all images as well as possible. Particularly preferably, images are recorded simultaneously such that the pupil is in the same state.

[0162] In particular, as an example, in a preferred embodiment, the present invention relates to the determination of anterior chamber depth, especially assuming that the iris and the boundary between the cornea and sclera are in the same plane.

[0163] The ellipticity of the pupil shape (image data) can also be used to draw conclusions about corneal astigmatism (additional data).

[0164] Since values ​​may already be derived from images of the eye region (e.g., white-to-white corneal diameter), statistical models can be used to calculate more complex characteristics of eye biometrics (e.g., naturally occurring corneal shape and / or anterior chamber depth and / or central corneal thickness or thickness distribution, appropriately parameterized), without limiting oneself to oversimplified models (e.g., corneal models as part of a spherical or toroidal surface). In this way, the natural form of the cornea can be optimally estimated. Assumptions about the iris or pupil can also rely on statistical models of other parameters, such as age, refraction, race, sex, or other known variables (see Possible Parameters above).

[0165] A further preferred embodiment involves determining a corneal topography (additional data) based on the shape of a line extending between the cornea and sclera (image data), or alternatively or further based on the outer boundary of the iris (image data). Preferably, the shape of the line extending between the cornea and sclera or the shape of the outer boundary of the iris is parameterized, for example:

[0166] Horizontal and vertical measurements of white-to-white

[0167] The flattened elliptical shape of the lines, for example, can be parameterized by their semi-axis, or possibly by their orientation relative to the horizontal plane.

[0168] A flat shape composed of two circular or elliptical segments

[0169] The 3D shape of a line can be determined, for example, by several images from different directions (e.g., using a stereo camera system).

[0170] Parameterization using the Fourier coefficients of the distance from the boundary to a specified point (e.g., the centroid of the pupil) as a function of the angle.

[0171] If the line (between the cornea and sclera, or alternatively or otherwise, the outer boundary of the iris) is not fully visible in the image (e.g., because part of the eyelid covers it), the parameters, in the case of appropriate parameterization, which represents an assumption about the shape of the line, can still be determined specifically using a statistical model. The parameters can be related to a known length (e.g., by using already calibrated data, or by allowing approximate calibration over a known pupillary distance).

[0172] Preferably, the corneal topography is determined sequentially from these parameters using a statistical model by making predictions from an existing dataset, which includes both known corneal topography and appropriate parameters for the boundary between the cornea and sclera.

[0173] Another preferred embodiment involves using a statistical model to determine corneal topography (additional data) from the geometric profile of an individual cross-section of the corneal surface, such as a vertical cross-section of the cornea from a side-view camera image (image data), wherein other variables may also be considered in the statistical model.

[0174] In summary, it is important to note that known conventional video datasets are used to improve lenses individually, but not to estimate parameters associated with the eye's biometrics using statistical models. For example, pupil diameter or pupillary distance can be determined, but this is done without using statistical models. This improvement in determining these parameters is achieved, for instance, by imaging the physical aperture stop of the eye (iris) via the cornea. This physical aperture stop is not known in detail, but can be estimated using statistical models.

[0175] Within the scope of this invention, data at least related to the eye region can be determined from an image in which one or both eyes are visible. Such image data can be determined, in particular, from a greater distance and can be performed more quickly than a complete measurement of eye biometrics.

[0176] Data associated with the biometrics of the eye can be geometric data (e.g., a description of the eye's refractive surfaces, and, if there are several surfaces, their relative positions to each other), and in particular, can include the shape of the cornea (e.g., as one of the corneal surfaces, especially the anterior corneal surface). In this way, complex eye models can be determined, which allow for better individual fitting of spectacle lenses without the need for complex measurements and corresponding measuring equipment required for determining complex eye models individually.

[0177] Data associated with ocular biometrics can also be associated with only one eye (as distinguished from eye rotation points, which are biometric parameters of the head or both eyes, not just a single eye). These parameters, associated with only one eye, can be used to determine the optical properties of complex eye models, which can then be used to calculate improved spectacle lenses without requiring corresponding measurement equipment in the specific cases used to determine the complex eye model. In particular, to determine such eye models in specific cases (i.e., after a statistical model has been provided), specially designed lighting equipment (e.g., in the case of a keratometer or SAM camera) is not required. Simple lighting equipment such as LEDs is sufficient, which may only be turned on briefly (e.g., as a flash to illuminate at least the eye area).

[0178] In particular, in a preferred embodiment, the present invention relates to a method for determining the biometrics of an eye, comprising the following steps:

[0179] Image data of the eye is recorded using a camera system with one or two cameras, which is preferably calibrated relative to each other (stereo camera system).

[0180] Provide and / or apply a statistical model that predicts biometric features (e.g., corneal curvature or anterior chamber depth) based on at least one of the following sets of input parameters:

[0181] Parameter set 1: Image the iris through the cornea from one or two cameras at one or different angles using one or two cameras.

[0182] Parameter set 2: Iris boundary

[0183] Parameter set 3: Human pupillary distance,

[0184] The statistical model provided includes a training process during which, in addition to parameters 1 and / or 2 and / or 3, the associated biometric features of the eye are known and used for training. The iris boundary and pupillary distance can both be determined from the images.

[0185] Reference number list

[0186] 1 Training Dataset

[0187] 2 Statistical Model

[0188] 10 Reference Datasets

[0189] 12 Image Data

[0190] 14 Additional Data S1 to S5: Methods and steps.

Claims

1. A method for calculating and / or manufacturing spectacle lenses, comprising: Provide image data, the image data including: At least one image of at least a portion of the user's eye region; Regarding geometric information about at least a portion of the user's eye region, the geometric information is obtained directly from at least one image of at least a portion of the user's eye region; Based solely on the provided image data and using a statistical model to determine individual supplementary data (14), the individual supplementary data including at least one individual biometric parameter of at least one eye of the user, the statistical model describing the relationship between the image data and the supplementary data (14); and The eyeglass lenses are calculated based on the determined individual additional data.

2. The method according to claim 1, wherein the statistical model (2) is derived or has been derived using statistical analysis of a training dataset (1) having a plurality of reference datasets (10), wherein each of the reference datasets (10) includes image data and additional data assigned to the image data.

3. The method according to claim 2, further comprising: Provide the training dataset (1); as well as The statistical model (2) is derived using statistical analysis of the training dataset (1), wherein deriving the statistical model (2) includes training the original model using the training dataset (1).

4. The method according to any one of the preceding claims, wherein the image data includes one or more of the following parameters: One or more camera images that allow at least one eye to be seen from the front; One or more camera images in which at least one eye can be seen from the side; Pupil distance; The distance from the center of eye rotation; Corneal diameter; The shape and / or position of the pupil or a portion thereof; The shape and / or location of the outer edge of the iris or a portion thereof; The 3D model of the eye area; Current line of sight; The presence of latent strabismus and / or the form / degree of latent strabismus.

5. The method according to any one of claims 4, wherein the 3D model of the eye region comprises a three-dimensional point cloud or a three-dimensional edge network that can be determined using a depth camera.

6. The method according to any one of claims 1 to 3, wherein the at least one individual biometric parameter includes one or more of the following parameters: Eye length; The optical length of the eye; The distances from the vertices of the eye's refractive surfaces to each other or to another common reference point; The shape of the refractive surface, the shape of the refractive surface can exist in a given parameterization, and the shape of the refractive surface is understood relative to the coordinate system; The physical aperture or the diameter or radius of the optical entrance pupil of the eye can be given for a given illumination condition; The refractive index of the optical medium of the eye; Size and / or shape of the retina; The orientation of the retina; The location and orientation of the receptors on the retina; The size of the receptive field on the retina.

7. The method of claim 6, wherein at least one of the following: The eye length is the total geometric length of the eye; The optical length of the eye is the integral of the refractive index along a beam of light of a given wavelength passing through the clearest visual point on the retina and the center of the pupil. The common reference point is the point on the retina where the image is clearest; The shape of the refractive surface is the shape and / or tilt of one of the lens surfaces on the cornea or lens surface; The given parameterization includes the radius of curvature and orientation of the principal section, the Zernike coefficient of the vertex depth, the geometric curvature, or the power vector or polar coordinate representation; The coordinate system is one in which the line of sight is aligned with the negative direction of the z-axis and the negative orientation x-axis lies in the negative orientation horizontal plane. The physical aperture of the eye is the opening of the iris; The optical entrance pupil is the iris through the cornea; The given lighting conditions are given or average indoor lighting; The orientation of the retina is the orientation of the fovea; The orientation of the retina is the vector direction perpendicular to the retina at the clearest visual point relative to the direction of light incidence; The size and / or shape of the retina includes the size and / or shape of the fovea; and The direction of light incidence is the direction in which the light passes through the center of the pupil and the point of clearest vision.

8. The method according to any one of claims 1 to 3, wherein the statistical model comprises one or more of the following models: Linear model; Gaussian process; Neural networks; Decision tree or regression tree; Boltzmann machine; Support Vector Machine.

9. The method of claim 8, wherein the neural network is a deep neural network.

10. A computer-readable storage medium comprising a computer program product, which, when loaded into a computer's memory and executed on the computer, causes the computer to perform the method according to any one of claims 1 to 9.

11. An apparatus for calculating and / or manufacturing spectacle lenses, comprising: A device for determining individual biometric parameters of at least one eye of a user, the device for determining individual biometric parameters including a computing device designed to perform: Provide image data, the image data including: At least one image of at least a portion of the user's eye region; Regarding geometric information about at least a portion of the user's eye region, the geometric information is obtained directly from at least one image of at least a portion of the user's eye region; Based solely on the provided image data and using a statistical model to determine individual supplementary data (14), the individual supplementary data including at least one individual biometric parameter of the user's at least one eye, the statistical model describing the relationship between the image data and the supplementary data (14); and The eyeglass lenses are calculated based on the determined individual additional data.

Images