Method and system for determining an eye-related physiological parameter

Abstract

A method (7000, 7001, 7002) for determining an eye-related physiological parameter (PAR) of a subject comprises generating (7050, 7051, 7052) at least two eye images (Pl, Pr) of the subject, using (7200, 7201, 7202) an eye model (EM) dependent on the eye-related physiological parameter (PAR) of the subject, to determine for each of the at least two eye images (Pl, Pr) a respective measurement for a property (Π) relating to at least one eye of the subject in and / or during generating (7050, 7051, 7052) the at least two eye images (Pl, Pr), using (7400, 7401, 7402) a constraint (C, Cbi, Ct) that applies in and / or during generating (7050, 7051, 7052) the at least two eye images (Pl, Pr) to determine a calibration loss (Δ') for the properties (Π), and using (7600, 7601, 7602) the calibration loss (Δ') to update the eye-related physiological parameter (PAR) of the subject.

Description

METHOD AND SYSTEM FOR DETERMINING AN EYE-RELATEDPHYSIOLOGICAL PARAMETERTECHNICAL FIELD

[0001] Embodiments of the present invention relate to methods and systems for determining an eye-related physiological parameter of a subject. Further embodiments of the present invention relate to using the determined eye-related physiological parameter to calibrate a head-wearable device and calibrated use of the head-wearable device by the subject.BACKGROUND

[0002] Over the last decades a wide variety of camera-based eye-tracking approaches have been proposed. As a result, a number of conceptually different approaches are available for determining eye-related parameters, such as gaze direction, from (streams of) eye images, which may e.g. be captured by near-eye cameras of a head-wearable device. The available approaches can be distinguished along two independent dimensions: 1) the complexity / dimensionality of the input features used (ranging from points, e.g. pupil centers, to full eye images) and 2) the degree to which knowledge about the geometry, optics, and kinematics of the ocular system is explicitly built into and exploited by the respective approach (ranging from only implicitly to model-informed).

[0003] Regression-based approaches are fit on a per-use basis to capture the relationship between extracted low-dimensional image features (points such as pupil centers) and calibration samples. Black-box machine learning (ML) approaches (such as neural networks, NNs) regress eye-related parameters based on full eye images, after suitable training on a large corpus of training data. Both regression and black-box ML approaches are not making explicit assumptions about the ocular system, i.e. are fully implicit.

[0004] Classical model-based approaches for gaze estimation fit a mathematical three- dimensional (3D) eye model, e.g. assuming spherical eyeball and cornea surfaces, to extracted low-dimensional eye-image features (typically a series of pupil contours) and next to 3D gaze direction allow for the estimation of other pertinent parameters characterizing the 3D eye state, as e.g. eyeball-center position and pupil radius. Recently, model-informed ML-approaches have been developed which incorporate suitable mathematical descriptions of ocular geometry, optics, and kinematics into the training paradigm to allow for suitable inductive biases to guidethe training and to effectively force the system to learn to process full eye images into results which are consistent with the employed model of the relevant aspects of the ocular system.

[0005] Model-based approaches, both classical and ML-based, have advantages over implicit approaches such as regression-based and black-box ML-based methods. While regressionbased methods may provide good short-term accuracy e.g. for gaze estimation, they are typically not very robust to changes in the eye tracker's position on the user's head (slippage) because they are overfit to the specific recording conditions during calibration. Black-box ML approaches, on the other hand, typically exhibit robustness to changes in lighting conditions and slippage when the training data is adequately diverse, but often lack effective calibration options and are so far (on average) less accurate than calibrated model-based approaches. In contrast, model-based approaches, by incorporating appropriate geometric biases, can achieve high accuracy, accommodate for slippage, and in particular model-informed ML-based approaches may be robust to lighting conditions.

[0006] In order to account for ocular geometry, optics, and kinematics, model-based approaches rely on the choice of suitable values for pertinent eye-related physiological parameters (e.g. iris radius, refraction index of corneal surfaces, eyeball radius, etc.). Such eye- related physiological parameters critically tune eye-model behavior and thus influence the downstream accuracy of the measurement of eye-related parameters such as gaze direction. Since pertinent eye-related physiological parameters vary across the population and may for a given subject even change over time, for optimal performance model-based approaches in particular necessitate setting and updating such eye-related physiological parameters in a subject-specific manner. For example, taking into account the user-specific inter-eye distance (IED) may be beneficial in many gaze-estimation scenarios with head-mounted eye trackers, in particular with binocular eye trackers.

[0007] In order to further improve the model-based measurement of gaze direction and other eye-related parameters, there is thus a need to develop calibration methods for the robust, accurate, and efficient estimation of eye-related physiological parameters in a subject-specific manner. In particular, gaze-label-free methods, which do not rely on the presentation and subsequent detection of explicit gaze-targets, are desirable.SUMMARY

[0008] According to an embodiment of a method for determining a subject-specific eye-related physiological parameter, the method comprises generating at least two eye images of the subject, using an eye-model-informed approach dependent on the eye-related physiologicalparameter of the subject, to determine for each of the at least two eye images a respective measurement for an eye-related parameter relating to at least one eye of the subject in and / or during generating the at least two eye images, using a constraint that applies in and / or during generating the at least two eye images to determine a calibration loss (quantifying the degree of deviation from the constraint), and using the calibration loss to update the eye-related physiological parameter of the subject.

[0009] In particular, for each of the at least two eye images a measurement of a typically corresponding eye-related parameter of the respective eye of the subject may be determined. For example, for each of the at least two eye images, a 3D eye state (3D state of the eye), or a part thereof may be determined / measured.

[0010] In other words, for each of the at least two eye images, an eye-related parameter relating to at least one eye of the subject in and / or during generating the at least two eye images, such as a respective 3D eye state or a part thereof, is measured (via the eye-model-based approach).

[0011] In the following, we will also refer to an eye-model-informed approach to measuring eye-related parameters, whether classical, ML-based or other, simply as “eye model”. Also, we will refer to eye-related parameters also as “properties of the eye”.

[0012] In other words, the method for determining the subject-specific physiological eye- related parameter, may include generating at least two eye images of the subject, using an eye model dependent on the eye-related physiological parameter of the subject, to determine for each of the at least two eye images a respective measurement for a property relating to at least one eye of the subject in and / or during generating the at least two eye images, using a constraint that applies in and / or during generating the at least two eye images to determine a calibration loss (for the properties determined for the respective eye images), and using the calibration loss to update the eye-related physiological parameter of the subject.

[0013] As the eye model depends on the eye-related physiological parameter(s) and applicable constraint(s) during calibration can be used to determine the (current) calibration loss, the method allows for a robust, accurate, user-specific, and even label-free calibration (resulting in the finally determined / updated eye-related physiological parameter), in particular for gazeestimation applications. The latter is because the constraint(s) do(es) not necessarily rely on the use and detection of dedicated gaze-targets or other independent measurements of eye-related parameters.

[0014] The 3D eye state includes at least one of, typically all of: a 3D center of rotation of an eyeball of the respective eye, and a 3D gaze direction of the eyeball of the respective (left or right) eye.

[0015] The 3D eye state may further include a 3D state of a pupil of the respective eye.

[0016] The 3D state of the pupil typically includes at least one of: a 3D pupil size of the respective eye, a 3D pupil aperture of an iris of the respective eye, a 3D pupil radius of an iris of the respective eye, and a 3D pupil diameter of the iris of the respective eye.

[0017] Instead of or in addition to a 3D eye state, (derived) 2D-quantities such as 2D gazedirection, and a 2D gaze point may be measured as eye-related physiological parameter(s).

[0018] As used herein, the term “eye-related physiological parameter” of the subject intends to describe a quantity which influences the optical, geometric, and / or kinematic properties of the (human) visual system and is at least substantially invariant (constant) during normal use of the visual system, typically at least over a period of hours, days or even weeks or months. The term “eye-related physiological parameter” may be a scalar quantity, but may also include two or more scalar quantities, for example refer to an array or a vectorial quantity.

[0019] Preferably, the eye-related physiological parameter of the subject relates to, includes and / or is selected from: an inter-eye distance of the subject (IED), in particular an interpupillary distance (IPD), a degree of physiologic anisocoria of the eyes of the subject (unequal size of the eyes' pupils), an angle between an optical axis and a visual axis of the left eye of the subject, an angle between an optical axis and a visual axis of the right eye of the subject, a matrix mapping between the optical and the visual axis of the left eye of the subject, a matrix mapping between the optical and the visual axis of the right eye of the subject, an eyeball radius of the left eye of the subject, an eyeball radius of the right eye of the subject, an eyeball aspherity of the left eye of the subject, an eyeball aspherity of the right eye of the subject, a cornea aspherity of the left eye of the subject, a cornea aspherity of the right eye of the subject, an anterior chamber depth of the left eye of the subject, an anterior chamber depth of the right eye of the subject, a rotation radius of the left eye, a rotation radius of the right eye, an iris radius of the left eye of the subject, an iris radius of the right eye of the subject, a refraction coefficient of the cornea of the right eye, a refraction coefficient of the cornea of the left eye, a distance between the medial and lateral canthus of the eyelid of the left eye, a distance between the medial and lateral canthus of the eyelid of the right eye, an angle of the line connecting the medial and lateral canthus of the left eye, an angle of the line connecting the medial and lateral canthus of the right eye.

[0020] In particular for gaze-estimation applications, the IED, IPD, the angles between an optical axis and a visual axis of the respective eye (also known as K (kappa) angle(s)) and the respective mapping matrix are often particularly important for further improving the accuracy and / or reliability of the gaze estimation (2D or 3D gaze direction and / or 2D or 3D gaze point). Taking into account the other listed eye-related physiological parameters may also further improve accuracy and / or reliability, in particular when additionally taken into account, but typically to a lesser extent compared to IED, IPD, and the K angles. However, this may depend on the use scenario and / or the subject, in particular on how far the physiological parameter of the subject deviates from a respective average, e.g. a population wide or subpopulation-wide (e.g. age-group dependent) average of the physiological parameter that may be used when the physiological parameter of the subject is not known.

[0021] As used herein, the terms “loss” and “calibration loss” are to be understood in terms of statistics and mathematical optimization, in particular artificial intelligence and machine learning. The terms “loss” and “calibration loss” embrace a (real) number which quantifies how well (or bad) the (measured) properties fulfill a constraint, which in particular provides a measure of a distance from the constraint (error), and / or is obtained from a respective loss function, also known as error function and cost function, respectively.

[0022] Further, more than one constraint, for example two, three or more (different) constraints that apply in and / or during generating the eye images may be used to determine a corresponding calibration loss for the eye-related parameters or to determine a function of the corresponding calibration losses forming an averaged (or weighted) calibration loss. In particular, the averaged calibration loss may be implemented as a sum or weighted sum of the corresponding calibration losses. In these embodiments, the corresponding calibration losses or the averaged calibration loss is used to update the eye-related physiological parameter of the subject.

[0023] Accordingly, the “calibration loss” may also be determined based on (as a function of) different calibration losses, in particular as an averaged (or weighted) calibration loss, typically obtained as a sum of different calibration losses or even a weighted sum of different calibration losses.

[0024] That a constraint applies in and / or during generating the eye images in particular intends to describe that the constraint applies when the eye images are taken by a camera, in particular a respective eye camera such as a respective eye camera of a head-wearable device worn by the subject. The constraint preferably represents a geometric relationship realized in space and / orover time by the ocular system of the subject and / or the optical system of the visual system of the subject in and / or during generating the at least two eye images.

[0025] The at least two eye images of the subject may be generated in a specific situation ensuring or at least making it likely that the constraint(s) are met such as a dedicated calibration situation, in which the subject performs given (predefined) tasks such as gazing at an object or a point, preferably under defined (controlled or at least recorded) conditions, for example under controlled lighting conditions.

[0026] Accordingly, the constraint s) may be situation-specific constraint s).

[0027] Alternatively or in addition, the at least two eye images of the subject may be generated during normal use of the head-wearable device, i.e. if the subject's gaze behavior is not specified, in particular during using the head-wearable device for eye tracking or a non-eyetracking related device setup. In this embodiment, the method typically includes selecting the at least two eye images based on a heuristic indicating that a gaze behavior of the subject complies with a constraint. For example, by automatic saccade detection, fast gaze sweeps can be detected, during which eyeball position, however, remains constant. Using an eye image from the beginning of a saccade and another eye image from the end of a saccade, a temporal consistency constraint quantifying the apparent change in eyeball position between the two images can be employed.

[0028] Alternatively or in addition, the at least two eye images of the subj ect may be generated during normal use of the head-wearable device and may be randomly selected. Moreover, prior to randomly selecting the at least two eye images, an outlier detection may be performed for the constraint s). For example, it could be assured by inspection of the images that the eye in both images was in its opened state and by an inspection of IMU readings (typically available in head-mounted eye trackers), it could be assured that the eye images were taken during a phase of low head-acceleration, since this limits the risk of measurements being taken during a slippage event of the head-wearable device (movement relative to the head).

[0029] The respective constraint is typically a (an applicable) consistency constraint.

[0030] The respective constraint may in particular be a binocular (consistency) constraint or a temporal (consistency) constraint (monocular or binocular).

[0031] Unlike traditional calibration methods, which assume the subject to look at gaze targets at known locations relative to the eye-tracking setup, the (calibration) methods proposed herein do not necessitate the use of explicit calibration targets. Instead, they are typically based on the exploitation of suitable temporal and / or binocular constraints, which are fulfilled either duringnormal use of the eye-tracking system, in particular a respective head-wearable device or a system comprising the head-wearable device, or which can be assured by the subj ect performing a typically calibration-target-free (also referred to as “label-free”) calibration choreography such as fixating an arbitrary point in the environment while moving their head in a randomized or predefined fashion. The subject may by instructed to perform respective tasks or detected to do so. For example, when the subject wearing the head-wearable device is looking out a side window of a moving vehicle (train) they may perform eye movements according to the optokinetic reflex (OKR) without moving their head (which may be proved with an inertial measurement unit (IMU) of the head-wearable device).

[0032] The calibration methods as proposed herein may be executed from time to time, regularly, in the background (e.g. in parallel ), and / or if a predetermined time of e.g. a day, a few days or a week, has passed since the last execution. Accordingly, a lifelong user-specific calibration may be provided for the head-wearable device. Note that certain eye-related physiological parameters may even change on the time scale of days.

[0033] The binocular constraint may refer to (at least) one of: a relation between an optical axis of the left eye and an optical axis of the right eye, in particular when the subject is looking at a distant object and / or at infinity, a relation between a visual axis of the left eye and a visual axis of the right eye, in particular when the subject is looking at a distant object and / or at infinity, an intersection point of the optical / visual axis of the left eye and the optical / visual axis of the right eye in 2D, an intersection point of the optical / visual axis of the left eye and the optical / visual axis of the right eye in 3D, a minimal distance of the optical / visual axis of the left eye and the optical / visual axis of the right eye in 3D, a relation between the 3D pupil state of the left eye and the 3D pupil state of the right eye, a relation between a 3D position of the left eye and a 3D position of the right eye, a relation between a 3D orientation of the left eye and a 3D orientation of the right eye, and a relation between a rotational speed of the left eye and a rotational speed of the right eye, a relation between the degree of closure of the left eyelid and the degree of closure of the right eyelid, a relation between the speed of closing of the left eyelid and the speed of closing of the right eyelid.

[0034] Furthermore, the binocular constraint typically includes: an angle constraint between the optical axis of the left eye and the optical axis of the right eye, an angle constraint between the visual axis of the left eye and the visual axis of the right eye, a constraint for a difference between a pupil radius of the left eye and a pupil radius of the right eye, and / or a constraint for a positional relation between the left eye and the right eye.

[0035] For example, the binocular constraint may require that the left and right optical and / or visual axes are substantially parallel (near-parallel) when the subject is looking at infinity, that the elevation of left and right optical (or visual) axes is at least substantially the same, that left and right pupil radii are substantially equal, that a positions of left and right eyeball are symmetric with respect to sagittal plane of a head of the subject, or that a rotational speed of the left eye and the right eye is equal.

[0036] The temporal constraint may refer to (at least one of): a 3D pupil state of the left eye as function of time, a 3D pupil state of the right eye as function of time, a relation between the 3D pupil state of the left eye and the 3D pupil state of the right eye as function of time, a 3D position of the left eye as function of time, a 3D position of the right eye as function of time, and a relation between the 3D position of the left eye and the 3D position of the right eye as function of time, a relation between the integrated eye rotation of the left and right eye over a period of time, a correlation between the fluctuations of the optical / visual axis of the left eye and the fluctuations of the optical / visual axis of the right eye.

[0037] Furthermore, the temporal constraint may include: that a vertical / horizontal gaze angle is substantially constant during horizontal / vertical head movements, that a difference between the 3D pupil state of the left eye and the 3D pupil state of the right eye is constant over time, that a 3D position of the left eye is constant over time during a non-slippage gaze sweep, that a 3D position of the right eye is constant over time during a non-slippage gaze sweep, that an angle between the optical axis of the left eye and the optical axis of the right eye is constant over time while rotating the head during a fixation of a typically arbitrary point in 3D, that an angle between the visual axis of the left eye and the visual axis of the right eye is constant over time while rotating the head during a fixation of a typically arbitrary point in 3D, that a pupil radius of the left eye is constant over time or changes over time in accordance with a lighting condition, or that a pupil radius of the right eye is constant over time or changes over time in accordance with the lighting condition and / or with the pupil radius of the left eye.

[0038] As illustrated in Fig. 2B, the eye model EM is typically a computational model (i.e. a model implemented as a computational pipeline), and more typical a computational model at least partly relying on (direct or explicit) physiological, kinematic, geometric, and / or optical findings for the human ocular system (human eye) and / or the optical system of the human visual system.

[0039] Accordingly, the eye model EM as proposed herein may use, similar as classical ML based models, inputs of higher complexity (e.g. full eye images) compared to classic geometricmodels such as the LeGrand eye model and regression based models (e.g. extracted points or ellipses), but also incorporates explicit knowledge about the underlying mechanisms (similar as classic geometric models). Moreover, the advantages of the different methods can be combined in a synergistic way for a user-specific, label-less calibration (choreography). As indicated by the dashed ellipse in Fig. 2B, alternatively or in addition, the eye model EM and the model -based approach, respectively, may also use inputs of lower complexity.

[0040] The eye model EM may in particular refer to the human ocular system and / or the optical system of the human visual system. Typically, the eye model at least refers to the pupil of the human eye (in 3D), such as the pupil position, the pupil radius and the pupil orientation, and / or is configured to determine, based on the eye image(s) and the eye-related physiological param eter(s) of a subject, at least a respective 3D state of the pupil, in particular the pupil contour of the subject’s eye (as a property measurement). More typically, the eye model is configured to determine respective measurements for properties that are derivable from the 3D state of the pupil, in particular the pupil contour such as gaze direction(s) and gaze point(s) in 2D or 3D. The eye model typically reflects the optically relevant anatomical structures of human eyes, in particular of the eyeball. The eye model may be based on an approximation such as a two-sphere eye model (for the left eye and the right eye), with a larger sphere having a center of rotation and representing a 3D eyeball with eyeball center, and a smaller sphere representing the cornea and intersecting with the larger sphere in a plane thereby defining the iris center. In this model, the pupil of the eye may be assumed to be a concentric circle with respect to the iris circle. Further, the center of the eyeball (center of rotation), the center of the iris (iris center) and of the center of the pupil (pupil center) may be co-linear points and define the optical axis of the eye). The visual axis of the eye determining the gaze direction of the eye may be defined by the center of the eye’s fovea (fovea position at the eyeball) and the center of the cornea. The offset between the eye’s visual and optical axes is known as (subject-specific) kappa (K) angle(s) (with pan and tilt angles / components).

[0041] In embodiments in which the eye model is a binocular eye model (a two-eye model describing both (left and right) eyes), the eye model is typically dependent on the (binocular) inter-pupillary distance (IPD) or the inter-eye distance (IED) as eye-related physiological parameters. The IPD may be defined as the distance between the pupil centers (of left and right eyes), the IED may be defined as the distance between the rotation centers (or left and right eyes).

[0042] In embodiments in which the eye model is a monocular eye model, the eye model may be dependent on the left or right eye (monocular) pupillary or eye distance as eye-relatedphysiological parameter. The monocular pupillary distance of the respective eye may be defined as distance of the respective pupil center from the sagittal plane of the subject’s head (distance from nose center) when gazing at an infinitely distant object / point. The monocular eye distance may be defined as the distance of the respective rotation center from the sagittal plane of the subject’s head (distance from nose center). Note that these distances may be different for the left and right eye of a subject.

[0043] As the monocular pupillary / eye distances of the left and right eyes sum up to the (binocular) IPD / IED, the binocular eye model may also be dependent on the monocular pupillary / eye distances of the left and right eyes instead of the IPD / IED.

[0044] The eye model may be based on and / or include a simple two-sphere eye model for the at least one eye such as the so-called LeGrand eye model (see L. Swirski and N. Dodgson: “A fully-automatic, temporal approach to single camera, glint-free 3D eye model fitting”, Proc. PETMEI Lund / Sweden, 13.08.2013) or the so-called Navarro eye model (see R. Navarro, J. Santamaria, J. Bescos: “Accommodation-dependent model of the human eye with aspherics,” J. Opt. Soc. Am. A 2(8), 1273-1281 (1985)) may be used for modelling.

[0045] Further, the eye model may be based on a modification of the binocular algorithm explained in WO 2020 / 244971 Al, which is hereby incorporated in its entirety, in particular in paragraphs

[0130] to

[0147] (“method C”) and

[0149] to

[0152] , see also claims 1 to 5, 10 of WO 2020 / 244971 Al.

[0046] For example, the eye model may, in binocular setup, use the same (LeGrand) eye model for the left and the right eye to determine for the IPD (inter-pupillary distance) / IED (intereyeball distance) as user-specific parameter instead of the respective predetermined value used in WO 2020 / 244971 Al, to calculate all of the 3D eyeball centers, the optical axes (gaze vectors) and the pupil size as eye states of both eyes based on merely a single observation in time of two eyes of a subject.

[0047] Further, a relation between the optical or visual axis of the left eye and the optical or visual axis of the right eye may be used as binocular constraint to determine the (current) calibration loss and to update the eye-related physiological parameter. After user-specific calibration, the determined (user-specifically calibrated) eye-related physiological parameter(s) may be used to determine / measure eye states of both eyes based on eye image pairs of a subject using the eye model.

[0048] Optionally, corneal refraction of the subject’s eyes (also described in WO 2020 / 244971 Al) may be taken into account / used as user-specific physiologicalparameter, e.g. in terms of the effective refraction index of the cornea in a two-sphere eye model.

[0049] The used eye model may be more sophisticated.

[0050] However, the authors of the present application have found out that using simple two- sphere eye models result in sufficiently robust and accurate (calibrated) measurements of eye- related parameters for many applications.

[0051] Similarly, the LeGrand eye model or the Navarro eye model may alternatively or in addition be used to update eye-related physiological parameter(s) of the subject of a single eye based on applicable temporal constraints.

[0052] Alternatively, the eye-model-based approach can be ML-based and may include or even be a trained eye state predictor which is configured to output, for an input of eye-related physiological parameter(s) and an eye image a respective (predicted) 3D eye state of the eye of the subject.

[0053] The trained eye state predictor is preferably implemented as a neural network (NN), in particular as a convolutional neural network (CNN). For example, the eye state predictor (ESP) may be based on an eye state predictor (and trained) as explained in the application PCT / EP2022 / 087323 which is hereby incorporated in its entirety. More particular, in a binocular setup, the ESP may be trained to output for an input of eye images, e.g. a pair of left and right eye images (as eye-related observation in terms of PCTZEP2022 / 087323) taken by respective eye camera(s) and one or more eye-related physiological parameters of the subject (e.g. subject-specific parameter(s) such as interpupillary distance IPD, and angles between an optical and a visual axis, see in particular paragraphs

[0189] -

[0198] referring to Figs. 4A-4C of PCT / EP2022 / 087323) a measurement for a property (called prediction for the eye(s) in PCT / EP2022 / 087323) corresponding to the eyes of the subject in and / or during generating the eye images, such as a 3D eye state of the respective eye or a part thereof, to determine, based on one or more applicable constraints, a calibration loss (called further or third loss in PCT / EP2022 / 087323) for the prediction / measurement for a property, and to update, based on the calibration loss, the one or more eye-related physiological parameters of the subject.

[0054] Accordingly, a model-informed ML-approach can be implemented, which incorporates suitable mathematical / physical descriptions of ocular geometry, optics, and kinematics into a training paradigm (e.g. as differentiable Tenderers in encoder-decoder architectures) to allow for suitable inductive biases to guide the training process and to effectively force the system toleam to process (full) eye images into results which are consistent with the employed model of the relevant aspects of the ocular system.

[0055] While two eye images, e.g. one pair of a left image and a right eye image or two eye images of one eye can be sufficient, the method for determining eye-related physiological parameter(s) typically relies on a plurality of eye images, for example at least ten eye images or pairs of eye images, or at least twenty or even at least 50 eye images or pairs of eye images. However, this may depend on the physiological parameter and the desired accuracy (convergence criterium), respectively.

[0056] Typically, at least the majority of eye images are generated while the subject wearing the head-wearable device follows (is expected to follow) gaze behavior related instruction(s).

[0057] The gaze behavior related instruction(s) may be displayed and / or played via a loudspeaker of the head-wearable device or a typically mobile (companion) computing device that may control the head-wearable device for determining the eye-related physiological parameter(s), and may even be used to determine the eye-related physiological parameter(s).

[0058] Preferably, the gaze behavior related instruction(s) describe a calibration choreography for eye movements, head movements and / or movements of the head-wearable device relative to the head (slippage).

[0059] When wearing the head-wearable device, the subject may be instructed to look into different directions and / or different distances one after the other, in particular without moving their head and without slippage of the head-wearable device. For example, the subject may be instructed to look, after a start signal has been given, into different directions in a specified order, e.g. left, right, up, and down. In this (first) choreography, in particular the same binocular constraints can be assumed for each pair of left and right eye images: the optical / visual axis of the left eye is substantially parallel to the optical / visual axis of the right eye, the minimal distance between the optical / visual axis of the left eye and the optical / visual axis of the right eye is substantially zero, the elevation of the optical / visual axis of the left eye and the elevation of the optical / visual axis of the right eye is substantially equal, the position of the left eye with respect to the sagittal plane and the position of the right eye with respect to the saggital plane is substantially the same, the distance between the position of the left eyeball and the position of the right eye is equal to the IED as measured e.g. with a standard pupillometer, the difference in pupil size between the left eye and the right eye corresponds to a subject-specific degree of anisocoria . Further, in the first choreography, in particular the following temporal constraints can be assumed: an angle difference between the optical axis of the left eye and the optical axisof the right eye is substantially constant, an angle difference between the visual axis of the left eye and the visual axis of the right eye is substantially constant, the minimal distance between the optical / visual axis of the left eye and the optical / visual axis of the right eye is substantially constant, e.g. zero, a difference between a pupil radius of the left eye and a pupil radius of the right eye is substantially constant, e.g. zero, and a difference between an orientation of the left eye and an orientation of the right eye is substantially constant, the distance between the position of the left eye and the position of the right eye is substantially constant, a difference between a 3D pupil state of the left eye and a 3D pupil state of the right eye is constant over time, the pupil radius of the left eye is substantially constant, the pupil radius of the right eye is substantially constant, a pupil radius of the left eye changes over time in accordance with a pupil radius of the right eye, the positions of the left eye and the right eye are strongly correlated, the position of the left eye is constant over time, the position of the right eye is constant over time.

[0060] Alternatively or in addition, the subject may be instructed to look at a fixed point and move the head-wearable device relative to the head (a bit). In this (second) choreography, the same constraints as for the first choreography can be assumed. Moreover, it can be assumed that the intersection point of the visual axis of the left eye and the visual axis of the right eye does not change over time (at least for subjects with normal binocular vision).

[0061] Alternatively or in addition, the subject may be instructed to look at an object displayed on a (separate) screen, for example a screen of a computer, in particular a mobile (companion) computing device, more particular a smartphone or a tablet, and to mimic movements of the displayed object. In particular, the subject may be instructed to look at an avatar displayed on the screen, and to mimic the avatar, for example to mimic movements of eyes of the avatar and / or mimic a slippage between the avatar’s head-wearable device and the avatar’s head. In this (third) choreography, the same constraints as for the first choreography can be assumed.

[0062] Alternatively or in addition, the subject may be instructed to read a text on the (separate) screen or on a display of the head-wearable device (when implemented as XR (Extended Reality), AR (Augmented Reality), VR (Virtual Reality), or MR (Mixed Reality) headwearable device / headset). In this (fourth) choreography, the same constraints as for the first choreography can be assumed.

[0063] Alternatively or in addition, the subject may be instructed to change, typically modulate the illumination (of the eyes), for example switch a lamp on and off while gazing at a fixed point such as infinity, and / or to gaze at the screen or display while the brightness of the screenor display is changed, typically modulated. In this (fifth) choreography, in particular the following temporal constraints can be assumed: a pupil radius of the left eye changes over time in accordance with the lighting condition, a pupil radius of the right eye changes over time in accordance with the lighting condition and / or with the pupil radius of the left eye, a constant position of the left eye, a constant position of the right eye, a constant optical axis of the left eye, a constant optical axis of the right eye, a constant visual axis of the left eye, a constant visual axis of the right eye, a constant intersection point of the optical / visual axis of the left eye with the optical / visual axis of the right eye, a constant gaze point in 3D, a constant gaze direction in 3D, a constant gaze point in 2D, a constant gaze direction in 2D.

[0064] The choreographies may also be mixed. For example, the subject may be instructed to mimic an avatar or read a text on the (separate) screen or on the display which changes its brightness.

[0065] During performing the movements / choreography, eye images may be taken by eye camera(s) of the head-wearable device, e.g. a left and / or a right eye camera, typically by both eye cameras.

[0066] Alternatively, external camera(s) may be used for taking the eye images.

[0067] After taking the eye images, the steps of using an eye model (i.e. an eye-model-informed approach) to determine for the eye images a respective measurement for a property, and using applicable constraint(s) to determine a calibration loss for the properties, and using the calibration loss to update the eye-related physiological parameter of the subject may be performed several time to minimize the calibration loss.

[0068] According to an embodiment of a method for subject-specific calibration of a headwearable device, the method includes storing an (at least one) eye-related physiological parameter of the subject, as determined as explained herein, in a memory of a computing unit of a head-wearable device, in particular a head-wearable device as explained herein.

[0069] According to an embodiment of a method for subject-specific calibrated use of a headwearable device including at least one eye camera configured to generate eye images of an eye of a subject wearing the head-wearable device, the method includes receiving an (at least one) eye-related physiological parameter of the subject, the eye-related physiological parameter being determined as explained herein, generating an eye image of the eye, and using the eye- related physiological parameter and the eye image to determine a (calibrated) measurement for a property for the eye.

[0070] Typically, the head-wearable device includes a respective eye camera for generating eye images of a left eye of the subject wearing the head-wearable device and for generating eye images of a right eye of the subject wearing the head-wearable device.

[0071] In this embodiment, the eye-related physiological parameter, the eye images of the left eye and the eye images of the right eye may be used to determine a (calibrated) measurement for the property (II) for the eyes of the subject.

[0072] The measurement for the property is typically determined in real-time and / or by a computing unit of the head-wearable device and / or a trained neural network (typically implemented by the computing unit), in particular by a trained convolutional neural network.

[0073] The determined / measured property is preferably one of: a 2D gaze direction, a 3D gaze direction, a 2D gaze point, a 3D gaze point (or any other gaze-related parameter), a 3D eye state of the eye and a part of the 3D eye state.

[0074] According to an embodiment a system includes a head-wearable device with at least one eye camera configured to generate eye images of (at least a portion of) an eye of the subject wearing the head-wearable device, and a computing system connectable with the at least one eye camera for receiving the eye images. The computing system is configured to use an eye model dependent on an (at least one) eye-related physiological parameter of the subject, to determine for each of at least two eye images a respective measurement for a property relating to the eye of the subject in and / or during generating the at least two eye images, use a constraint that applies in and / or during generating the at least two eye images to determine a calibration loss for the properties, and use the calibration loss to update the eye-related physiological parameter of the subject.

[0075] The head-wearable device typically has a respective eye camera for each eye of the subject.

[0076] Further, the head-wearable device may be configured to be subject-specifically calibrated as explained herein.

[0077] The computing system may include a first computing unit, which is typically provided by the head-wearable device, and a second computing unit connectable with the first computing unit. The second computing unit may be provided by another device such as a mobile computing device, and is typically being configured to perform the updating of the eye-related physiological parameter(s) as explained herein.

[0078] According to an embodiment a computer program product or a computer-readable storage medium comprising instructions which, when executed by one or more processors of a system comprising a head-wearable device, cause the system to carry out the methods as explained herein.

[0079] The term “3D eye state” as used herein intends to describe a set of quantities or values, in particular a respective vector describing, typically characterizing a (an actual) three- dimensional state of at least one eye of the subject at a given time, i.e. a left eye, in the following also referred to as first eye, a right eye, in the following also referred to as second eye, both eyes, and / or a cyclopean eye of the subject.

[0080] A set (only) consisting of two-dimensional (2D) values that are directly visible in eye images, like for example a 2D pixel location of an eye bounding box within a remotely taken image of the face of a subject or a 2D pupil image (ellipse) measurement for a property is not to be understood as a “3D eye state”.

[0081] The 3D eye state typically refers to and / or includes one or more, typically two or three 3D observables (physical quantities that can be measured) of the at least one eye of the subject.

[0082] The 3D eye state can include any (measured / measurement-based) value that characterizes the physiologically constant or transient parameters describing a respective eye in 3D, in particular the eyeball position / center such as the 3D center of rotation of the eyeball, the 3D gaze direction, e.g. a vector characterizing the optical or visual axis / line-of-sight, and / or a parameter at least related with pupil radius and / or the size of the pupil aperture (“pupil size”) of the eyes in 3D (3D pupil state).

[0083] The 3D eye state typically includes at least one of, typically at least two of, for example three of or even all of: a 3D center (of rotation) of an eyeball of the at least one eye, a 3D gaze direction of the at least one eye, a 3D state of an eyelid of the at least one eye and a 3D state of a pupil of the at least one eye.

[0084] Typically, the 3D state of the pupil includes at least one of a 3D pupil size of the at least one eye, a 3D pupil aperture of an iris of the at least one eye, a 3D pupil radius of an iris of the at least one eye, and a 3D pupil diameter of the iris of the at least one eye.

[0085] The 3D state of the eyelid of the at least one eye may include at least one of an eyelid shape, an eyelid position, and a percentage of an eyelid closure, an angle quantifying the degree of opening of the eyelid, an angle quantifying the state of the upper eyelid, an angle quantifying the state of the lower eyelid.

[0086] Further, the 3D eye state may be a monocular state, a pair of corresponding left and right monocular states or a binocular state.

[0087] The 3D eye state(s) is / are typically determined with respect to a coordinate system that is fixed to the eye camera(s) and / or the head-wearable device.

[0088] For example, a Cartesian coordinate system defined by the image plane(s) of the eye camera(s) may be used.

[0089] Points and directions may also be specified within and / or converted into a device coordinate system, a head coordinate system, a world coordinate system or any other suitable 3D coordinate system.

[0090] The (head-wearable) spectacles device typically includes a spectacles body, which is configured such that it can be worn on a head of a subject, for example in a way usual glasses are worn. Hence, the spectacles device when worn by a subject may in particular be supported at least partially by a nose area of the subject’s face. This state of usage of the head-wearable (spectacles) device being arranged at the subject’s face will be further defined as the “ intended use ” of the spectacles device, wherein direction and position references, for example horizontal and vertical, parallel and perpendicular, left and right, front and back, up and down, etc., refer to this intended use. As a consequence, lateral positions as left and right, an upper and lower position, and a front / forward and back / backward are to be understood from subject’s usual view. Equally, this applies to a horizontal and vertical orientation, wherein the subject’s head during the intended use is in a normal, hence upright, non-tilted, non-declined and non-nodded position.

[0091] The spectacles body (main body) typically includes a left ocular opening and a right ocular opening, which mainly come with the functionality of allowing the subject to look through these ocular openings. Said ocular openings can be embodied, but not limited to, as sunscreens, optical lenses or non-optical, transparent glasses or as a non-material, optical pathway allowing rays of light passing through.

[0092] The spectacles body may, at least partially or completely, form the ocular openings by delimiting these from the surrounding. In this case, the spectacles body functions as a frame for the optical openings. Said frame is not necessarily required to form a complete and closed surrounding of the ocular openings. Furthermore, it is possible that the optical openings themselves have a frame like configuration, for example by providing a supporting structure with the help of transparent glass. In the latter case, the spectacles device has the form similarto frameless glasses, wherein only a nose support / bridge portion and ear-holders are attached to the glass screens, which therefore serve simultaneously as an integrated frame and as optical openings.

[0093] In addition, a middle plane of the spectacles body may be identified. In particular, said middle plane describes a structural center plane of the spectacles body, wherein respective structural components or portions, which are comparable or similar to each other, are placed on each side of the middle plane in a similar manner. When the spectacles device is in intended use and worn correctly, the middle plane coincides with a median plane of the subject.

[0094] Further, the spectacles body typically includes a nose bridge portion, a left lateral portion and a right lateral portion, wherein the middle plane intersects the nose bridge portion, and the respective ocular opening is located between the nose bridge portion and the respective lateral portion.

[0095] For orientation purposes, a plane being perpendicular to the middle plane shall be defined, which in particular is oriented vertically, wherein said perpendicular plane is not necessarily firmly located in a defined forward or backward position of the spectacles device.

[0096] The (near-) eye camera(s) of the head-wearable device is typically of known camera intrinsics and / or has a sensor arranged in or defining an image plane for taking images of a respective eye of the subject, i.e. of a left or a right eye of the subject.

[0097] As used herein, the term “camera intrinsics” shall describe that the optical properties of the camera, in particular the imaging properties (imaging characteristics) of the camera are known and / or can be modelled using a respective camera model including the known intrinsic parameters (known intrinsics) approximating the eye camera producing the eye images. Typically, a pinhole camera model is used for modelling the eye camera. The known intrinsic parameters may include a focal length of the camera, an image sensor format of the camera, a principal point of the camera, a shift of a central image pixel of the camera, a shear parameter of the camera, and / or one or more distortion parameters of the camera.

[0098] In addition, the head-wearable device may have a further eye camera of known camera intrinsics for taking images of a second eye of the subject, i.e. of a right eye or a left eye of the subject. In the following the further eye camera is also referred to as further camera and second eye camera.

[0099] In other words, the head-wearable device may, in a binocular setup, have a left and a right (eye) camera, wherein the left camera serves for taking a (left) image or a stream of images of at least a portion of the left eye of the subject, and wherein the right camera takes a (right) image or a stream of images of at least a portion of a right eye of the subject.

[0100] Typically, the first and second eye cameras have the same or similar camera intrinsics (are of the same type, but may be individually calibrated).

[0101] However, the methods explained herein are also applicable in a monocular setup with one (near) eye camera only.

[0102] The eye camera(s) can be arranged at the spectacles body in inner eye camera placement zones and / or in outer eye camera placement zones, in particular wherein said zones are determined such, that an appropriate picture of at least a portion of the respective eye can be taken for the purpose of determining one or more eye-state-related parameters; in particular, the cameras are arranged in a nose bridge portion and / or in a lateral edge portion of the spectacles frame such, that an optical field of a respective eye is not obstructed by the respective camera. The optical field is defined as being obstructed, if the camera forms an explicitly visible area / portion within the optical field, for example if the camera points out from the boundaries of the visible field into said field, or by protruding from the boundaries into the field. For example, the cameras can be integrated into a frame of the spectacles body and thereby being non-obstructive. In the context of the present invention, a limitation of the visible field caused by the spectacles device itself, in particular by the spectacles body or frame is not considered as an obstruction of the optical field.

[0103] Furthermore, the head-wearable device may have illumination means for illuminating the left and / or right eye of the subject, in particular if the light conditions within an environment of the spectacles device are not optimal.

[0104] Further, the head-wearable device may be provided with a scene camera for taking images of the field of view (FOV) of the subject wearing the head-wearable device, for example an integrated scene camera typically arranged in the middle plane.

[0105] The predicting method may at least in part be controlled and / or performed by a computing (and control) unit of the head-wearable device.

[0106] Alternatively or in addition, a companion computing device functionally connected with the computing and control unit (or only a control unit) of the head-wearable device of the system, e.g. via a wired or wireless network (TCPIP) connection and / or an USBconnection, in particular a mobile companion computing device such as a smart phone, a tablet, or a laptop connected with the head-wearable device, or a desktop computer may supervise, control and / or perform the methods as explained herein or parts thereof.

[0107] The computing system may be configured to perform particular operations or processes by virtue of software, firmware, hardware, or any combination thereof. One or more computer programs can be configured to perform particular operations or processes by virtue of including instructions that, when executed by a one or more processors of the system, cause the system to perform the processes.

[0108] Typically, the computing system performing the methods as explained herein includes one or more processors, in particular one or more CPUs, GPUs and / or DSPs (as hardware) and / or one or more computing units and / or a computing device, typically companion computing device such as a smartphone (of the subject) providing a computing unit, and / or one or more software modules including instructions which, when executed by at least one of the one or more processors cause the system to perform the processes.

[0109] The methods as explained herein may be controlled and / or at least partly be implemented by an app, in particular an app running on the companion computing device (also referred to as companion app). As such the app may implement and / or include the software module(s).

[0110] The term “app” as used herein shall embrace the terms “web application” and “mobile application” (also known as mobile app).

[0111] The (companion) app may control the head-wearable device and / or may send firmware and updates, in particular the eye-related physiological parameter(s) to a computing unit of the head-wearable device.

[0112] In particular, the app may be configured to generate a user profile for the subject, update the user profile, storing the eye-related physiological param eter(s) of the subject in the user profile, and / or uploading the user profile to the computing unit of the head-wearable.

[0113] Further, an optional neural network software module may include instructions which, when executed by at least one of the one or more processors, implement (an instance of) a neural network.

[0114] Other embodiments include (non-volatile) computer-readable storage media or devices, and / or computer programs recorded on one or more computer-readable storage mediaor computer storage devices, each configured to perform the processes of the methods described herein.

[0115] Those skilled in the art will recognize additional features and advantages upon reading the following detailed description, and upon viewing the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0116] The components in the figures are not necessarily to scale, instead emphasis being placed upon illustrating the principles of the invention. Moreover, in the figures, like reference numerals designate corresponding parts. In the drawings:

[0117] Fig. 1A, Fig. IB, and Fig. 1C illustrate respective flows charts of methods for determining an eye-related physiological parameter according to embodiments;

[0118] Fig. ID illustrates a flow chart of a method for subject-specific calibrated use of a head-wearable device according to an embodiment;

[0119] Fig. IE illustrates 3D eye states of a subject according to embodiments;

[0120] Fig. 2A illustrates perspective view of a system including a head-wearable device according to embodiments; and

[0121] Fig. 2B compares different eye-tracking approaches.DETAILED DESCRIPTION

[0122] In the following Detailed Description, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. In this regard, directional terminology, such as “top,” “bottom,” “front,” “back,” “leading,” “trailing,” etc., is used with reference to the orientation of the Figure(s) being described. Because components of embodiments can be positioned in a number of different orientations, the directional terminology is used for purposes of illustration and is in no way limiting. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0123] Reference will now be made in detail to various embodiments, one or more examples of which are illustrated in the figures. Each example is provided by way of explanation, and is not meant as a limitation of the invention. For example, features illustratedor described as part of one embodiment can be used on or in conjunction with other embodiments to yield yet a further embodiment. It is intended that the present invention includes such modifications and variations. The examples are described using specific language which should not be construed as limiting the scope of the appended claims. The drawings are not scaled and are for illustrative purposes only. For clarity, the same elements or manufacturing steps have been designated by the same references in the different drawings if not stated otherwise.

[0124] With respect to FIG. 1 A an exemplary label-free method 7000 for determining an eye-related physiological parameter PAR of a subject is explained.

[0125] In a first block 7050, two eye images Pi, Prof a subject are generated, for example a pair of left and right images are taken at (substantially) the same time or two eye images of the left eye or of the right eye are taken at different times, preferably by a headwearable device 100 of a system 500 as explained above with regard to Fig. 2A.

[0126] Upon inputting each of the eye images together with an eye-related physiological parameter PAR of the subject such as an IPD of the subject to an eye model, the eye model outputs a corresponding measurement for a property II relating to the left eye, the right eye or both eyes of the subject, in a subsequent block 7200.

[0127] In a subsequent block 7400, a calibration loss A' is calculated for the two properties III, II2 based on a constraint C. In particular, the calibration loss A' may be calculated as output of a calibration loss function FPLreceiving the two propertiesA'=FPL(III, II2, C).

[0128] Thereafter, the calibration loss A’ is used to update the eye-related physiological parameter PAR of the subject, in a block 7600.

[0129] As indicated by the dotted arrow in Fig. 1 A between blocks 7400 and 7200, the calibration loss A’ may alternatively be determined in block 7400 as an average calibration loss for several eye images P|, Pr, in particular a batch of eye images P|, Pr.

[0130] The eye-related physiological parameter PAR is typically updated so that the calibration loss A' and the calibration loss function FPLis (excepted to be) reduced (minimized), at least on average and after a plurality of cycles, respectively.

[0131] If, in a block 7700, a termination criterion, in particular a convergence criterium cc for the calibration loss A' and / or the eye-related physiological parameter PAR is met,updating the eye-related physiological parameter PAR of the subject and determining the eye- related physiological parameter PAR may be, respectively, is ended. The present value of the parameter PAR may be output and / or stored for later use.

[0132] In particular, the determined eye-related physiological parameter(s) PAR of the subject may be transferred to / stored in a memory of the computing unit of a head-wearable device with at least one eye camera, typically a left and a right eye camera. Accordingly, the head-wearable device and its computing unit, respectively, is calibrated, and may use the stored subject-specific eye-related physiological parameter(s) (instead of previously stored e.g. average eye-related physiological parameter(s)) to determine, using the same or a different eye model, in particular a two-sphere-based eye model or a model-informed trained NN, measurement(s) for a property(ies) for the user’s eye for later taken eye images more accurately, typically in real time.

[0133] Otherwise, e.g. while the calibration loss A' or a relative change of the eye- related physiological parameter PAR is above a respective predetermined threshold, method 7000 may return to block 7200 for another cycle with the updated eye-related physiological parameter PAR and the same or different eye images (the latter is indicated by the dashed dotted arrow in Fig. 1A).

[0134] Typically, determining the eye-related physiological parameter PAR is finished after performing blocks 7200 to 7600 for a plurality of eye images that may be determined (taken) in advance, for example as video or image sequence(s).

[0135] Preferably, the eye images Pi, Prof the subject are taken while the subject wearing the head-wearable device follows gaze behavior related instruction(s) such as to look into different directions and / or different distances one after the other without moving the head and without slippage of the head-wearable device, to look at a point and move the headwearable device relative to the head, or to look at an object displayed on a screen, in particular to look at an avatar displayed on the screen, and to mimic the avatar.

[0136] Moreover, two or more eye-related physiological parameters PAR may be determined by method 7000, in particular in parallel, i.e. using common iterations cycles.

[0137] Further, several constraints C may apply in and / or during generating the eye images Pi, Prin block 7050, and be used in block 7400 to determine the calibration loss A'.

[0138] With respect to FIG. IB a method 7001 for determining an eye-related physiological parameter PAR of a subject is explained. Method 7001 is typically similar tomethod 7000 explained above with regard to Fig. 1 A and has corresponding blocks 7051, 7201,7401, 7601, 7701.

[0139] However, method 7001 is more specific insofar as pair of eye images Pi, Prwith an image Pi of a left eye of the subject and an image Prof a right eye of the subject, which are taken at (substantially) the same time tk, is used in each iteration cycle block 7201 -7601.

[0140] Accordingly, a binocular constraint Cbimay be used in block 7401 to determine the calibration loss A' for the measurement for a property Hi referring to the left eye such as an optical or visual axis of the left eye at tkand the measurement for a property Hrreferring to the right eye such as an optical or visual axis of the right eye at tk, as determined in block 7201 for the eye images PbPrand eye-related physiological parameter(s) such as IPD and K angle(s). In this embodiment, the binocular constraint Cblmay be a relation between the optical or visual axis of the left eye and the optical or visual axis of the right eye.

[0141] In a first example, the binocular constraint Cbiis that the optical axes of the left and right eyes are at least substantially parallel, when the subject is looking at a point in the distance / at infinity, and the calibration loss (function) A' may be the angular difference between both axes or a cosine similarity. In this example, one or more or even all of the following eye-related physiological parameters of a subject may be determined by minimizing the calibration loss (if the eye model depends on the eye-related physiological param eter(s)): IED, IPD, iris radius (of at least one of the eyes), anterior chamber depth (of at least one of the eyes), rotation radius (of at least one of the eyes), eyeball radius (of at least one of the eyes), anterior chamber depth (of at least one of the eyes), refraction index cornea (of at least one of the eyes), cornea radius (of at least one of the eyes), and cornea asphericity (of at least one of the eyes).

[0142] In a second example, the binocular constraint Cbiis that the visual axes of the left and right eyes are at least substantially parallel, when the subject is looking at a point in the distance / at infinity, and the calibration loss (function) A' may be the angular difference between both axes or a cosine similarity. In this example, one or more or even all of the following eye-related physiological parameters of a subject may be determined by minimizing the calibration loss A' (if the eye model depends on the eye-related physiological param eter(s)): IED, IPD, iris radius, anterior chamber depth, rotation radius, eyeball radius, anterior chamber depth, refraction index cornea, cornea radius, cornea asphericity, kappa angle(s), and a calibration matrix for obtaining visual from optical axes.

[0143] In a third example, the binocular constraint CM is that the visual axes of the left and right eyes intersect in a point. In this example, the calibration loss (function) may be a minimal distance between the visual axes in 3D. By minimizing this calibration loss, the same eye-related physiological parameters of the subject as in the second example may be determined.

[0144] In further examples, the same eye-related physiological parameters of the subject as in the first example may be determined, when the binocular constraint Cbiand the calibration loss (function) A' are chosen according to table 1 :

[0145] Further, as indicated by index k of the (image taking / observation) time tkand the dotted arrow, several or even a plurality of eye image pairs of a subject may be processed batch-wise for determining the average calibration loss A’ in block 7401.

[0146] With respect to FIG. 1C a method 7002 for determining an eye-related physiological parameter PAR of a subject is explained. Method 7002 is typically also similar to method 7000 explained above with regard to Fig. 1A and has corresponding blocks 7052, 7202, 7402, 7602, 7702.

[0147] However, method 7002 is more specific insofar as eye images or even eye image pairs Pi, Prtaken at two or more times tkto tm (with indices m, k, where m-k>l) are used in each iteration cycle from block 7202 -7602.

[0148] Accordingly, a temporal constraint Ctsuch as a constant eyeball position during a non-slippage gaze sweep or a temporal constraint on the pupil radius resulting from a (controlled or measured) lighting condition may be used in block 7402 to determine the calibration loss A’ for the measurement for a property Htkto Htmreferring to left eye, right eye or both eyes as determined in block 7202 for the eye images and the iris radius or radii as respective eye-related physiological parameter PAR.

[0149] In one example, the temporal constraint Ctis that an eyeball center is fixed during non-slippage gaze sweep of the subject, and the calibration loss A' may be determined as distance between the eyeball center at different time points. In this example, one or more or even all of the following eye-related physiological parameters of a subject may be determined by minimizing the calibration loss A': IED, IPD, iris radius, anterior chamber depth, rotation radius of the eye, eyeball radius, anterior chamber depth, refraction index cornea, cornea radius, and cornea asphericity.

[0150] The same eye-related physiological parameters of the subject may be determined by using: a temporal constraint Ctrequiring that the distance between left and right eyeball center is constant over time, and a deviation of the distance between left and right eyeball centers from the constant as calibration loss A’, as well as by using: a temporal constraint Ctrequiring that a pupil radius is constant over time (in particular during non-slippage gaze sweep under constant illumination), and an absolute difference between the pupil radius at different time points as calibration loss A'.

[0151] In another example, the temporal constraint Ctis that a difference between left and right pupil sizes is constant over time, and the calibration loss A’ is provided by the absolute deviation of the difference of the left and right pupil sizes from the constant. In this example, one or more or even all of the following eye-related physiological parameters of a subject may be determined by minimizing the calibration loss A': IED, IPD, iris radius, anterior chamber depth, rotation radius of the eye, eyeball radius, anterior chamber depth, refraction index cornea, cornea radius, cornea asphericity, and a degree of anisocoria.

[0152] The same eye-related physiological parameters of the subject may be determined by using: a temporal constraint Ctrequiring that the left and right pupil sizes changes over time according with a lightening condition, and, the calibration loss A’ may be determined based on derivatives of pupil size and light intensity have the same sign and / or a correlation coefficient between pupil size and light intensity.

[0153] In yet another example, the temporal constraint Ctis that the vergence angle between left and right visual axis is constant over time. By minimizing a corresponding calibration loss A' of an angular deviation of the vergence angle from the constant or a cosine similarity, one or more or even all of the following eye-related physiological parameters of a subject may be determined: IED, IPD, iris radius, anterior chamber depth, rotation radius of the eye, eyeball radius, anterior chamber depth, refraction index cornea, cornea radius, cornea asphericity, kappa angle(s), and a calibration matrix for obtaining visual from optical axes.

[0154] The methods 7000 to 7002 may be performed several times, in real time and / or during normal use of the head-wearable device, in particular during using the head-wearable device for eye tracking.

[0155] With respect to FIG. ID a method 8000 for subject-specific calibrated use of a head-wearable device is explained.

[0156] In a block 8050, one or more eye-related physiological parameters PAR of the subject are received. The eye-related physiological param eter(s) PAR may in particular be determined as explained above with respect to Figs. 1 A-C.

[0157] After, generating eye images Pi, Prof the subject’s eye in a block 8100, the eye- related physiological parameter(s) PAR and the eye image(s) Pi, Prmay be input into an eye model (typically as sole inputs) to determine, in a block 8200, a measurement(s) for a property(s) II for the eye(s) such as a 2D gaze direction, a 3D gaze direction, a 2D gaze point, a 3D gaze point, and a 3D eye state 3DES of the eye(s).

[0158] Blocks 8100, 8200 may be performed by the computing unit of the headwearable device, a trained neural network, in particular by a trained convolutional neural network (implemented by the computing unit), several times, in real time and / or during normal use of the head-wearable device, in particular during using the head-wearable device for eye tracking.

[0159] As illustrated in Fig. IE, the 3D eye state 3DES may be one of: a monocular state 3DESmiof a left eye of the subject (with a 3D center ECiof an eyeball of the left eye of the subject, a 3D gaze direction EGI of the left eye of the subject, and a 3D pupil state EPiof the left eye of the subject), a monocular state 3DESmrof a right eye of the subject (with a 3D center ECrof the eyeball of the right eye of the subject, a 3D gaze direction Ed- of the right eye of the subject, and a 3D pupil state EProf the right eye of the subject), a monocular state 3DESmcof a (virtual) cyclopean eye of the subject (with a 3D center ECcof an eyeball of a cyclopean eye of the subject, a 3D gaze direction EGcof the cyclopean eye of the subject, and a 3D pupil state EPcof the cyclopean eye of the subject), and a binocular state 3DESb(with corresponding monocular 3D states of the left and right eyes of the subject).

[0160] While (the right, left, and cyclopean) monocular states 3DESmi, 3DESmr, 3DESmcmay be represented by a six-dimensional vector, the binocular state 3DESbmay be represented by a ten-dimensional vector or twelve-dimensional vector.

[0161] Fig. 2A illustrates a system 500 for performing the methods explained herein, in particular the calibration methods 7000, 7001, 7002 and 8000.

[0162] In the exemplary embodiment, system 500 includes a head-wearable device 100 implemented as a spectacles device. Accordingly, a frame of head-wearable (spectacles) device 100 has a front portion 114 surrounding a left ocular opening and a right ocular opening. A bridge portion of the front portion 114 is arranged between the ocular openings. Further, a left temple 113 and a right temple 123 are attached to front portion 114.

[0163] An exemplary camera module 140 is accommodated in the bridge portion and arranged on the wearer-side of the bridge portion. A passage opening for a scene camera 160 of module 140 and the field of view (FOV) of scene camera 160, respectively, is formed in the bridge portion.

[0164] The scene camera 160 is typically centrally arranged, i.e. at least close to a central vertical plane between the left and right ocular openings and / or close to (expected) eye midpoint(s) of a human subject wearing the head-wearable device 100 (user). The latter also facilitates a compact design. Furthermore, the influence of parallax error on gaze-related properties may be reduced this way significantly.

[0165] Furthermore, the scene camera 160 may define a Cartesian coordinate system x, y, z and have an optical axis which is at least substantially arranged in the central vertical plane (to reduce parallax error), arranged in the central x, z-plane and / or pointing in x-direction in the exemplary embodiment.

[0166] Leg portions 134, 135 of module 140 may at least substantially complement the frame below the bridge portion so that the ocular openings are at least substantially surrounded by material of frame and module 100.

[0167] A right eye camera 150 for taking right eye images of the user is arranged in right leg portion 135.

[0168] Likewise, a left eye camera (not visible in Fig. 2A ) for taking left eye images of the user may be arranged in left leg portion 134, for example (substantially) mirror- symmetrical to the right eye camera 150 with respect to a middle plane of the spectacles body of head-wearable device 100.

[0169] In the exemplary embodiment, the head-wearable device 500 is additionally provided with an inertial measurement unit 170 for measuring movements and / or orientationsof the head-wearable device 500 and a human subject wearing the head-wearable device 500, respectively.

[0170] Both, the scene camera 160 and the inertial measurement unit 170 may provide additional data for the user.

[0171] As indicated by the dashed-dotted arrows in Fig. 2A, a computing system 200, 300 of system 500 may be connectable with the scene camera 160 for receiving scene images, connectable with the eye camera(s) 150 for receiving eye images, and / or connectable with the inertial measurement unit 170 for receiving data referring to movements and / or orientations of the head-wearable device 100. However, the scene camera 160 and IMU 170 are optional components of system 500 and head-wearable device 100, respectively.

[0172] The computing system 200, 300 is configured to perform the methods explained herein.

[0173] For this purpose, the computing system 200, 300 typically includes one or more processors and a (respective) non-transitory computer-readable storage medium comprising instructions which, when executed by the one or more processors, causes system 500 to carry out the methods as explained herein.

[0174] In the exemplary embodiment, the computing system 200, 300 comprises several interconnectable parts, namely a first computing unit 200 and a second computing unit 300.

[0175] While the first computing unit 200 is typically a local unit or system and / or configured to perform the measurement methods as explained herein, the second computing unit 300 may be a remote unit or system, e.g. even cloud-based, and / or is typically configured to perform the actual training steps (cycles) of the calibration methods as explained herein (after receiving eye-related observational data via the first computing unit 200).

[0176] The second computing unit 300 may be configured to determine eye-related observations from the received eye-related observational data, and even host and / or manage a database for the eye-related observation.

[0177] The first computing unit 200 may be implemented as a controller and may even be arranged within a housing of module 140, at least in part.

[0178] However, the first computing system 200 may also at least in part be provided by a companion computing device connectable (with the second computing unit 300 and) to the controller of the head-wearable device 100, for example via a USB-connection (e.g. one of thetemples may provide a respective plug or socket), in particular a mobile companion computing device such as a smartphone, tablet or laptop.

[0179] The second computing system 300 may be configured to control the first computing unit 200, to instruct the subject (e.g. acoustically and / or using its screen) wearing device 100 to perform a calibration choreography as explained herein, receive eye images of the subject wearing device 100 (during performing the calibration choreography), typically via the first computing unit 200, and / or determining, based on an eye model (such as a two-sphere- based eye model or a trained eye state predictor tESP), an eye-related physiological parameter of the subject as explained herein.

[0180] Further, the second computing unit 300 may be configured to update and / or upload the eye model EM such as the two-sphere-based eye model or the trained eye state predictor tESP and optionally any (subject-specific) eye-related physiological param eter(s) of a subject determined by the second computing unit 300 to the first computing unit 200. After receiving eye-related physiological param eter(s) of the subject, the first computing unit 200 may use the received eye-related physiological param eter(s) of the subject as additional inputs of the eye model EM such as the two-sphere-based eye model or the tESP for determining a (3D) gaze direction and other eye state parameters of the subject, in particular 3D eye states of the subject.

[0181] An app, which may be downloaded from a server may be stored on the second computing unit 300. When running on the second computing unit 300, the (running) app is typically configured to perform the calibration methods as explained herein.

[0182] Furthermore, the (running) app may be configured to generate a user profile for the subject (upon user request), update the user profile, store / update the eye-related physiological param eter(s) of the subject in the user profile.

[0183] The (running) app may also control the calibrated use of the head-wearable device and / or for determining calibrated measurements as explained herein.

[0184] Accordingly, on the second computing unit 300, calibrated measurement s) of one or more properties of the eye of the subject such as 2D / 3D gaze-direction, 2D / 3D gaze point, a 3D eye state of the subject’s eye(s) may be calculated for eye images using the eye- related physiological parameter(s) stored in the user profile.

[0185] The (running) app may decide that a recalibration is to be performed, e.g. a predetermined time since the last calibration has passed, by providing gaze behavior relatedinstruction(s) describing a calibration choreography or even during determining the calibrated measurements and / or as a background task.

[0186] Alternatively, the calibrated measurement(s) may be performed on the first computing unit after the user profile is uploaded to the first computing unit 200.

[0187] Generally, the computing system 200, 300, in particular the second computing unit 300 may implement an eye model EM such as a two-sphere-based eye model or a tESP which outputs for an input of an eye image P|, Prtaken by the respective eye camera 150 and one or more eye-related physiological parameters (PAR) of the subject a measurement for a property corresponding to the eye(s) of the subject in and / or during generating the eye image(s) Pi, Pr, to determine, based on one or more applicable constraints, a calibration loss for the measurement for a property, and to update, based on the calibration loss, the one or more eye- related physiological parameters PAR of the subject.

[0188] Furthermore, the computing system 200, 300 is typically configured to perform the methods explained with respect to Figs. 1 A to ID explained above.

[0189] In other embodiments, the 3D eye state 3DES may in addition (or instead of e.g. the 3D state EPof the pupil) include a 3D state of an eyelid of the at least one eye.

[0190] According to an embodiment of a (computer-implemented) method for determining eye-related physiological param eter(s) of a subject, the method includes feeding an (at least one) eye-related physiological parameter of a subject and eye images of the subject to an eye model (EM) for determining for each eye image a measurement for a property relating to at least one eye of the subject in and / or during generating the eye images, determine, based on a constraint that applies in and / or during generating the eye images, a calibration loss for the properties, and updating, depending on the calibration loss for the properties, the eye-related physiological parameter(s) of the subject, typically so that the calibration loss is expected to decrease, in particular minimized when the method is performed iteratively.

[0191] According to an embodiment of a (computer-implemented) method for determining an (at least one) eye-related physiological parameter PAR of a subject, the method includes generating, using at least one eye camera of a head-wearable device worn by the subject, at least two eye images of the subject, typically a plurality of eye images of one or both eyes of the subject, using an eye model dependent on the (at least one) eye-related physiological parameter PAR of the subject, to determine for each of the at least two eye images a corresponding measurement for a property relating to at least one eye of the subject in and / or during generating the respective eye image, and using a (at least one) constraint that applies inand / or during generating the at least two eye images to determine a (at least one) calibration loss for the properties, and using the (at least one) calibration loss to update the (at least one) eye-related physiological parameter PAR of the subject.

[0192] The methods explained herein may be implemented using a (model-informed) glint-free approach or a (model-informed) glint-based approach.

[0193] In embodiments referring to determining more than one eye-related physiological parameter, for example two or three eye-related physiological parameters such as IPD and the K angles, each eye-related physiological parameter may be determined separately. Alternatively, the two (or more) eye-related physiological parameters may be determined together, in particular in a common iteration using one, two (or more) constraints, for example one constraint per eye-related physiological parameter. In particular, the IPD and the K angles may e.g. be determined for the subject from a sequence of eye image pairs and the following constraint s): 1) the distance between the left eye and right eye is constant over time, 2) the visual axes (obtained from a rotation of the respective optical axes of the left resp. right eye in accordance with Listing's law and using the current value of kappa) need to be intersecting for all eye-image pairs (in a point depending on the respective eye-image pair), 3) the elevation of the visual axes (obtained as above) of the left and of the right eye are substantially the same.

[0194] Although various exemplary embodiments of the invention have been disclosed, it will be apparent to those skilled in the art that various changes and modifications can be made which will achieve some of the advantages of the invention without departing from the spirit and scope of the invention. It will be obvious to those reasonably skilled in the art that other components performing the same functions may be suitably substituted. It should be mentioned that features explained with reference to a specific figure may be combined with features of other figures, even in those cases in which this has not explicitly been mentioned. Such modifications to the inventive concept are intended to be covered by the appended claims.

[0195] While processes may be depicted in the figures in a particular order, this should not be understood as requiring, if not stated otherwise, that such operations have to be performed in the particular order shown or in sequential order to achieve the desirable results. In certain circumstances, multitasking and / or parallel processing may be advantageous.

[0196] Spatially relative terms such as “under”, “below”, “lower”, “over”, “upper” and the like are used for ease of description to explain the positioning of one element relative to a second element. These terms are intended to encompass different orientations of the device in addition to different orientations than those depicted in the figures. Further, terms such as“first”, “second”, and the like, are also used to describe various elements, regions, sections, etc. and are also not intended to be limiting. Like terms refer to like elements throughout the description.

[0197] As used herein, the terms “having”, “containing”, “including”, “comprising” and the like are open ended terms that indicate the presence of stated elements or features, but do not preclude additional elements or features. The articles “a”, “an” and “the” are intended to include the plural as well as the singular, unless the context clearly indicates otherwise.

[0198] With the above range of variations and applications in mind, it should be understood that the present invention is not limited by the foregoing description, nor is it limited by the accompanying drawings. Instead, the present invention is limited only by the following claims and their legal equivalents.Reference numbers100 head-wearable device114 front portion of frame113, 123 temple140 camera module150 (right) eye camera160 scene camera170 inertial measurement unit200, 300 computing system / controller / computing unit / companion device500 system3DES,{EC, EG, Ep} 3D eye stateEM eye modelESP eye state predictor (model)Pi, Preye imagesA’ calibration lossPAR subject specific parameter n,m measurement for a property7000 - 8000 methods, method steps

Claims

Claims1. A method (7000, 7001, 7002) for determining an eye-related physiological parameter (PAR) of a subject, the method comprising:(i) generating (7050, 7051, 7052) at least two eye images (P,, Pr) of the subject;(ii) using (7200, 7201, 7202) an eye model (EM) dependent on the eye-related physiological parameter (PAR) of the subject, to determine for each of the at least two eye images (P|, Pr) a respective measurement of a property (II) relating to at least one eye of the subject in and / or during generating (7050, 7051, 7052) the at least two eye images (Pi, Pr);(iii) using (7400, 7401, 7402) a constraint (C, Cbi, Ct) that applies in and / or during generating (7050, 7051, 7052) the at least two eye images (Pi, Pr) to determine a calibration loss (A'); and(iv) using (7600, 7601, 7602) the calibration loss (A') to update the eye-related physiological parameter (PAR) of the subject.

2. The method (7000, 7001, 7002) of claim 1, wherein several constraints (C, Cbi, Ct) applying in and / or during generating (7050, 7051, 7052) the at least two eye images (Pi, Pr) are used to determine a corresponding calibration loss (A') for the properties (II) and wherein the corresponding calibration loss (A') are used to update the eye-related physiological parameter (PAR) of the subject, wherein the respective constraint represents a physical relationship of the ocular system of the subject and / or the optical system of the visual system of the subject in and / or during generating the at least two eye images, and / or wherein the respective constraint is a consistency constraint and / or a situation-specific constraint, and / or comprises at least one of a binocular constraint and a temporal constraint.

3. The method (7000, 7001, 7002) of claim 2, wherein the binocular constraint refers to at least one of: a relation between an optical axis of the left eye and an optical axis of the right eye, in particular when the subject is looking at a distant object and / or at infinity, a relation between a visual axis of the left eye and a visual axis of the right eye, in particular when the subject is looking at the distant object and / or at infinity, an intersection point of the visual axis of the left eye and the visual axis of the right eye in 2D, an intersection point of the visual axis of the left eye and the visual axis of the right eye in 3D, a relation betweena 3D pupil state (EPi) of the left eye and the 3D pupil state (EPr) of the right eye, a relation between a 3D position of the left eye and a 3D position of the right eye, a relation between a 3D orientation of the left eye and a 3D orientation of the right eye, a relation between a rotational speed of the left eye and a rotational speed of the right eye, and / or wherein the binocular constraint comprises at least one of: an angle constraint between the optical axis of the left eye and the optical axis of the right eye, an angle constraint between the visual axis of the left eye and the visual axis of the right eye, a constraint for the distance between the optical axis of the left eye and the optical axis of the right eye in 3D, a constraint for a distance between the visual axis of the left eye and the visual axis of the right eye, a constraint for a difference between a pupil radius of the left eye and a pupil radius of the right eye, and a constraint for a positional relation between the left eye and the right eye.

4. The method (7000, 7001, 7002) of claim 2 or 3, wherein the temporal constraint refers to at least one of: a 3D pupil state (EPi) of the left eye as function of time, a 3D pupil state (EPi) of the right eye as function of time, a relation between the 3D pupil state (EPi) of the left eye and the 3D pupil state (EPi) of the right eye as function of time, a 3D position of the left eye as function of time, a 3D position of the right eye as function of time, and a relation between the 3D position of the left eye and the 3D position of the right eye as function of time, and / or wherein the temporal constraint comprises to at least one of: a difference between the 3D pupil state (EPi) of the left eye and the 3D pupil state (EPr) of the right eye is constant over time, a 3D position of the left eye is constant over time during a non-slippage gaze sweep, a 3D position of the right eye is constant over time during a non-slippage gaze sweep, the angle between the optical axis of the left eye and the optical axis of the right eye is constant over time while rotating the head during a fixation of a point in 3D, the angle between the visual axis of the left eye and the visual axis of the right eye is constant over time while rotating the head during a fixation of a point in 3D, a pupil radius of the left eye is constant over time or changes over time in accordance with a lighting condition, and pupil radius of the right eye is constant over time or changes over time in accordance with the lighting condition and / or with the pupil radius of the left eye.

5. The method (7000, 7001, 7002) of any preceding claim, wherein the eye model (EM) is a computational model.

6. The method (7000, 7001, 7002) of any preceding claim, wherein the properties (II) comprise a gaze-related parameter such as a 2D or 3D gaze direction, and / or wherein the properties (II) relate to a 3D eye state (3DES, {EC, EG, EP}) of the respective eye of the subject or as a part thereof, and / or are determined as a 3D eye state (3DES, {EC, EG, EP}) of the respective eye of the subject or as a part thereof, and / or are determined based on the 3D eye state (3DES, {EC, EG, EP}).

7. The method (7000, 7001, 7002) of any preceding claim, wherein the eye model (EM) is configured to output, for an input of the eye-related physiological parameter (PAR) and each of the at least two eye images (P,, Pr), a respective 3D eye state (3DES, {Ec, EG, EP}) of the at least one eye.

8. The method (7000, 7001, 7002) of any preceding claim, wherein the eye model (EM) is based on and / or comprises at least one of: physiological, kinematic, geometric, and / or optic findings for the human ocular system and / or the optical system of the human visual system; a two-sphere-based eye model; and a neural network (NN), in particular as a convolutional neural network (CNN), the network typically being trained to output, for an input of the eye-related physiological parameter (PAR) and the at least two eye images (Pi, Pr) the respective measurement for the property (II) in consistency with physiological, kinematic, geometric, and / or optic findings for the human ocular system and / or the optical system of the human visual system.

9. The method (7000, 7001, 7002) of any preceding claim, wherein the eye-related physiological parameter (PAR) of the subject is iteratively determined, and / or further comprising repeating the steps (ii) to (iv) of claim 1 until a termination criterion, in particular a convergence criterium (cc) for the calibration loss (A') and / or the eye-related physiological parameter (PAR) is met, and / or repeating the steps (ii) to (iv) of claim 1 or even the steps (i) to (iv) of claim 1 several times, for example after some time to update the eye-related physiological parameter (PAR).

10. The method (7000, 7001, 7002) of any preceding claim, wherein the at least two eye images (Pi, Pr) of the subject are generated (7050, 7051, 7052) while the subject wearing the headwearable device (100) follows at least one gaze behavior related instruction, the least one gaze behavior related instruction typically describing a calibration choreography for the subject.

11. The method (7000, 7001, 7002) of claim 10, wherein the at least one gaze behavior related instruction is displayed and / or played via a loudspeaker of the head-wearable device or a typically mobile companion computing device, and / or instructs the subject:- to look into different directions and / or different distances one after the other, in particular without moving the head of the subject and without slippage of the headwearable device,- to look at a point and move the head-wearable device relative to the head,- to look at a point and move the head in a random fashion,- to look at an object displayed on a screen, for example a screen of the companion computing device, in particular of a mobile computing device, more particular a smartphone or a tablet, and to mimic movements of the displayed object, in particular to look at an avatar displayed on the screen, and to mimic the avatar,- to read a text on a screen or on a display of the head-wearable device, and / or- to change, typically modulate the illumination of the eyes of the subject, for example while gazing at a fixed point such as infinity, the at least one gaze behavior related instruction typically being initiated and / or provided by an app running on the companion computing device.

12. The method (7000, 7001, 7002) of any preceding claim, wherein the at least two eye images (Pi, Pr) of the subject are generated using at least one eye camera (150) of the head-wearable device (100) worn by the subject.

13. The method (7000, 7001, 7002) of any of the claims 11 to 12, wherein the at least two eye images (Pi, Pr) of the subject are generated during normally using the head-wearable device (100) by the subject, in particular during using the head-wearable device (100) for eye tracking, further comprising selecting the at least two eye images (Pi, Pr) based on a heuristic indicating a gaze behavior of the subject complies with the constraint (C, Cbi, Ct), or wherein the at least two eye images (P,, Pr) of the subject are generated during normally using the head-wearable device (100) by the subject, further comprising selecting the at least two eye images (Pi, Pr).

14. The method (7000, 7001, 7002) of any preceding claim, wherein for each eye image of a plurality of eye images a respective measurement for a property (II) is determined.

15. The method (7000, 7001, 7002) of any preceding claim, wherein the at least two eye images (Pi, Pr) comprises a pair of eye images (Pi, Pr) with an image (Pi) of a left eye of the subject and an image (Pr) of a right eye of the subject which are generated at least substantially at the same time, and wherein the pair of eye images (Pi, Pr) is fed together with the eye- related physiological parameter (PAR) as input to the eye model (EM) to determine respective properties (II) for the left eye and the right eye as output of the eye model (EM), or wherein each of the at least two eye images (Pi, Pr) is separately fed together with the eye-related physiological parameter (PAR) as input to the eye model (EM) to determine the respective measurement for a property (II) as output of the eye model (EM), in particular when the at least two eye images (Pi, Pr) are eye images (Pi, Pr) of the left eye or the right eye and are determined at different times.

16. A method for calibrating a head-wearable device (100) comprising at least one eye camera (150) configured to generate eye images (P,, Pr) of at least one eye of a subject wearing the head-wearable device, and a computing unit (200) functionally connected with the at least one eye camera (150) for receiving the at least one eye image, the method comprising: storing an eye-related physiological parameter (PAR) of the subject in a memory of the computing unit (200), the eye-related physiological parameter (PAR) being determined according to the method (7000, 7001, 7002) of any preceding claim.

17. The method of claim 16, wherein the method is executed from time to time, regularly, in the background, and / or if a predetermined time has passed since the last execution.

18. A method (8000) for subject-specific calibrated use of a head-wearable device (100) comprising at least one eye camera (150) configured to generate eye images (Pi, Pr) of an eye of a subject wearing the head-wearable device, the method comprising: at least one of: receiving (8050) an eye-related physiological parameter (PAR) of the subject, the eye-related physiological parameter (PAR) being determined according to the method of any of the claims 1 to 15, and performing the method of claim 16 or 17; generating (8100) an eye image (Pi, Pr) of the eye; and using (8200) the eye-related physiological parameter (PAR) and the eye image (Pi, Pr) to determine a measurement of a property (II) for the eye.

19. The method (8000) of claim 18, wherein the measurement of a property (II) is determined in real-time and / or by a trained neural network, in particular by a trained convolutional neural network, and / or wherein the head-wearable device (100) comprises a respective eye camera (150) configured to generate eye images (Pi) of a left eye of the subject wearing the head-wearable device and to generate eye images (Pr) of a right eye of the subject wearing the head-wearable device, and / or wherein the eye-related physiological parameter (PAR) and the eye images (Pi, Pr) of the left eye and of the right eye are used to determine a measurement for a property (II) for the eyes, and / or wherein the measurement for a property (II) is selected from a list comprising: a 2D gaze-direction, a 3D gaze direction, a 2D gaze point, a 3D gaze point, a 3D eye state (3DES*, {Ec, EG, EP}) of the at least one eye and a part of the 3D eye state (3DES, {Ec, EG, EP}).

20. The method of any of the claims 6 to 19, wherein the 3D eye state (3DES, {Ec, EG, EP}) comprises a 3D center of rotation of an eyeball of the respective eye, and a 3D gaze direction of the eyeball of the respective eye, and / or wherein the 3D eye state (3DES, {Ec, EG, EP}) comprises a 3D center of rotation of an eyeball of the respective eye, and a 3D gaze direction of the eyeball of the respective one eye.

21. The method of any of the claims 6 to 20, wherein the 3D eye state (3DES, {Ec, EG, EP}) comprises a 3D state of a pupil of the respective eye, and / or wherein the 3D eye state (3DES*, {Ec, EG, EP}) comprises a 3D state of a pupil of the respective eye.

22. The method of claim 21, wherein the 3D state of the pupil comprises at least one of: a 3D pupil size of the respective eye, a 3D pupil aperture of an iris of the respective eye, a 3D pupil radius of the respective eye, and a 3D pupil diameter of the respective eye, and / or wherein the 3D state of the pupil comprises at least one of: a 3D pupil size of the respective eye, a 3D pupil aperture of an iris of the respective eye, a 3D pupil radius of an iris of the respective eye, and a 3D pupil diameter of the iris of the respective eye.

23. The method of any preceding claim, wherein the method refers to at least two eye-related physiological parameters (PAR) of the subject, and / or wherein the respective eye-related physiological parameter of the subject relates to and / or is selected from: an inter-eye distance of the subject, a degree of physiologic anisocoria of the eyes of the subject, an angle between an optical axis and a visual axis of the left eye of the subject, an angle between an optical axis and a visual axis of the right eye of the subject, a matrix mapping between the optical and the visual axis of the left eye of the subject, a matrix mapping between the optical and the visual axis of the right eye of the subject, an eyeball radius of the left eye of the subject, an eyeball radius of the right eye of the subject, an eyeball aspherity of the left eye of the subject, an eyeball aspherity of the right eye of the subject, a cornea aspherity of the left eye of the subject, a cornea aspherity of the right eye of the subject, an anterior chamber depth of the left eye of the subject, an anterior chamber depth of the right eye of the subject, a rotation radius of the left eye, a rotation radius of the right eye, a refraction coefficient of the cornea of the left eye, a refraction coefficient of the cornea of the right eye, an iris radius of the left eye of the subject, and an iris radius of the right eye of the subject.

24. A system (500) comprising:- a head-wearable device (100) comprising at least one eye camera (150) configured to generate eye images (Pi, Pr) of an eye of the subject wearing the head-wearable device; and- a computing system (200, 300) connectable with the at least one eye camera (150) for receiving the eye images, and configured to perform a plurality of operations, the plurality of operations comprising; using (7200, 7201, 7202) an eye model (EM) dependent on an eye-related physiological parameter (PAR) of the subject, to determine for each of atleast two eye images (Pi, Pr) a respective measurement for a property (II) relating to the eye of the subject in and / or during generating (7050, 7051, 7052) the at least two eye images (P|, Pr); using (7400, 7401, 7402) a constraint (C, Cbi, Ct) that applies in and / or during generating (7050, 7051, 7052) the at least two eye images (P|, Pr) to determine a calibration loss (A') for the properties (II); and using (7600, 7601, 7602) the calibration loss (A') to update the eye-related physiological parameter (PAR) of the subject.

25. The system (500) of claim 24, wherein at least two eye-related physiological parameters (PAR) of the subject are updated, and / or wherein the respective eye-related physiological parameter of the subject relates to and / or is selected from: an inter-eye distance of the subject, a degree of physiologic anisocoria of the eyes of the subject, an angle between an optical axis and a visual axis of the left eye of the subject, an angle between an optical axis and a visual axis of the right eye of the subject, a matrix mapping between the optical and the visual axis of the left eye of the subject, a matrix mapping between the optical and the visual axis of the right eye of the subject, an eyeball radius of the left eye of the subject, an eyeball radius of the right eye of the subject, an eyeball aspherity of the left eye of the subject, an eyeball aspherity of the right eye of the subject, a cornea aspherity of the left eye of the subject, a cornea aspherity of the right eye of the subject, an anterior chamber depth of the left eye of the subject, an anterior chamber depth of the right eye of the subject, a rotation radius of the left eye of the subject, a rotation radius of the right eye of the subject, an iris radius of the left eye of the subject, and an iris radius of the right eye of the subject, and / or wherein the system is configured to perform the method according to any of the preceding claims, and / or wherein the head-wearable device (100) comprises a respective eye camera (150) for each eye of the subject, and / or wherein the head-wearable device (100) is configured to perform the method according to any of the claims 18 to 23.

26. The system (500) of claim 25, wherein the computing system (200, 300) comprises a first computing unit (200) typically provided by the head-wearable device (100), and a second computing unit (300) connectable with the first computing unit (200), the second computing unit (300) typically being provided by a companion computing device, in particular a mobile companion computing device, such as a smartphone or a tablet.

1. The system (500) of claim 26, wherein the second computing unit (300) and / or an app, which is typically stored on the second computing unit (300), is, when running on the second computing unit (300), configured to perform the plurality of operations.

28. The system (500) of claim 27, wherein the second computing unit (300) and / or the app, when running on the second computing unit (300), is configured to perform at least one of:- generate a user profile for the subject;- update the user profile;- store the eye-related physiological parameter (PAR) of the subject in the user profile, and- upload the user profile to the first computing unit (200).

29. The system (500) of claim 28, wherein the first computing unit (200) or the second computing unit (300) is configured to:- receive (8100) from the at least one eye camera (150) a further eye image (Pi, Pr) of the eye, for example a pair of further eye images (Pi, Pr); and- use (8200) the further eye image (Pi, Pr) and the eye-related physiological parameter (PAR) stored in the user profile to determine a measurement of a property (II) for the eye of the subject.

30. A computer program product or a computer-readable storage medium comprising instructions which, when executed by a one or more processors of a system comprising a head-wearable device (100), in particular the system of any of claims 24 or 29, cause the system to carry out the method according to any one of the claims 1 to 23.

Images