Acquisition of high-resolution ophthalmometry parameters

Abstract

A system and method for extracting high-resolution eye measurement parameters is disclosed. A video stream having a video of a user's face is processed to obtain a set of eye measurement parameters, such as eyelid data, iris data (e.g., iris translation, iris radius and iris rotation), and pupil data (e.g., pupil center and pupil radius) at a first resolution. To improve the accuracy and resolution of the eye measurement parameters, a deconvolution process is performed on the video stream based on stimulus information of a video stimulus displayed on a client device associated with the user, environmental data of an environment in which the user is located, device data of the client device, etc. The eye measurement parameters are then processed using a predictive model trained based on the high-resolution eye measurement parameters obtained using an eye tracking device to predict the eye measurement parameters at a resolution higher than the first resolution. TIFF2024518427000002.tif170132

Description

[Technical Field]

[0001] Cross-reference to related patent applications This patent application claims the benefit of U.S. Provisional Patent Application No. 63 / 183,388, filed May 3, 2021, entitled "MEASURING HIGH RESOLUTION EYE MOVEMENTS USING BLIND DECONVOLUTION TECHNIQUES," which is incorporated herein by reference in its entirety. [Background technology]

[0002] background Several papers have been published demonstrating the association between micro-eye movements and the progression of neurological disorders (see, for example, the References section below). Typically, these eye movements are measured in a well-controlled laboratory environment (e.g., no movement, controlled ambient light, or other such parameters) using specialized equipment (e.g., infrared eye-tracking devices, pupillometers, or other such devices), which can be difficult to set up, prohibitively expensive, or require significant time and effort to create or maintain a controlled environment. Some prior art uses neural networks to acquire these eye movements, but does not achieve the resolution required for measurement (e.g., about 0.1 mm or higher). Furthermore, prior art focuses on measuring the eye's response to specific, standardized stimuli (e.g., controlled light stimuli, stimuli provided to measure specific parameters, stimuli requiring the patient to be aware, or other such stimuli) in controlled ambient conditions. These and other drawbacks exist. Summary of the Invention

[0003] overview Disclosed embodiments relate to systems and methods for facilitating measurement of micro-ocular parameters or eye movements using video captured by a standard camera (e.g., a smartphone camera, a webcam, or another video capture device) without the need for controlled setup. Embodiments acquire various high-resolution eye measurement parameters. For example, embodiments may acquire eyelid data such as coordinates of the eyelid border. In another example, embodiments acquire iris data such as iris translation or center, iris rotation, iris radius, iris visible ratio, or iris coverage asymmetry. In yet another example, embodiments acquire pupil data such as pupil center, pupil radius, pupil visible ratio, or pupil coverage asymmetry. The eye measurement parameters are acquired at high resolution, e.g., at the sub-pixel level, 0.1 mm, or other higher resolution. In some embodiments, the eye measurement parameters may be used in determining eye movement measurements such as pupil response parameters, gaze, saccades, fixations, etc., which may be used to generate digital biomarkers that can further be used to diagnose or measure the progression of a neurological condition or disorder. Digital biomarkers are objective, sensitive, accurate, correlate with disease progression, and can be performed remotely and even in a distributed manner. Ocular measurement or eye movement parameters can be acquired using non-standard stimuli presented to a user on a user device (e.g., stimuli that do not require the patient to be aware of, stimuli that can be used to measure multiple parameters, stimuli such as a video being played on a display device being viewed by the user, or other such stimuli that do not require controlled ambient lighting conditions).

[0004] In some aspects, disclosed embodiments may use probabilistic methods (e.g., maximum likelihood estimation or other such methods) or signal processing methods (e.g., blind deconvolution or other such methods) to increase resolution and ascertain eye measurement parameters, eye movement parameters at ultra-high resolution (e.g., sub-pixel level, 0.1 mm, or other higher resolution) relative to data layers present in a video frame sequence of eye movements captured by a standard camera in an uncontrolled setting. Disclosed embodiments may use predictive models (e.g., machine learning (ML) models such as neural networks) to obtain one or more eye measurement parameters at high resolution.

[0005] Various other aspects, features, and advantages of the present invention will become apparent through the detailed description of the invention and the accompanying drawings. It is also understood that both the foregoing general description and the following detailed description are exemplary and are not intended to limit the scope of the invention. As used in this specification and the claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Furthermore, as used in this specification and the claims, the term "or" means "and / or" unless the context clearly dictates otherwise. [The present invention 1001] obtaining a video stream from a client device associated with a user, the video stream including video of the user's face; providing the video stream as an input to a first predictive model to obtain a first set of eye measurement parameters for the user's eye, the first set of eye measurement parameters including a first pupil center, first iris data, and first eyelid data, the first set of eye measurement parameters being obtained at a first resolution; deconvolving the video stream based on the first eyelid data, the first pupil center, stimulus data, and environmental data to obtain adjusted eyelid data, an adjusted pupil center, and an adjusted pupil radius as part of the first eye measurement parameter set, wherein the stimulus data includes spatiotemporal and optical information of a video stimulus presented on a display associated with the client device, and the environmental data includes lighting conditions in an environment in which the client device is located; and providing the first ocular measurement parameter set as input to a second predictive model to obtain a second ocular measurement parameter set at a second resolution, the second ocular measurement parameter set including (a) a second pupil center, (b) a second pupil radius, and (c) second iris data including an iris radius, an iris translation, and an iris rotation, the second resolution being higher than the first resolution; A method comprising: [The present invention 1002] 1001. The method of claim 1001, wherein said first resolution is measured in pixel values ​​or other lower resolution values, and said second resolution is measured in sub-pixel values. [The present invention 1003] providing the input to the second predictive model, training the second prediction model using a plurality of datasets to predict the second eye measurement parameter set at the second resolution, each dataset including a video of a specified user's face, a first value set corresponding to the first eye measurement parameter set acquired at the first resolution, and a set of labels having a second value set corresponding to the second eye measurement parameter set acquired at the second resolution; Including, The method of the present invention 1001. [The present invention 1004] The method of the present invention 1003, wherein the second set of values ​​corresponding to the second set of eye measurement parameters is acquired using an eye tracking device configured to acquire the second set of eye measurement parameters at the second resolution for a video of the face of the specified user. [The present invention 1005] deconvolving the video stream, determining a point spread function of a deconvolution process based on the first eyelid data, the first pupil center, the stimulus data, and the environmental data; and deconvolving the video stream with the point spread function to obtain the adjusted eyelid data, the adjusted pupil center, and the adjusted pupil radius. Including, The method of the present invention 1001. [The present invention 1006] The method of the present invention 1005, wherein the step of deconvolving the video stream includes determining the adjusted eyelid data, the adjusted pupil center and the adjusted pupil radius based on a maximum likelihood prediction of the location of the first pupil center and eyelid boundary. [The present invention 1007] determining the adjusted eyelid data; processing the first eyelid data using a maximum likelihood curve fitting operation to obtain a parabolic fit of an eyelid of the eye; and obtaining a plurality of coordinates along the parabolic fit as the adjusted eyelid data; Including, The method of the present invention 1005. [The present invention 1008] determining the adjusted pupil center; determining a plurality of positions for the first pupil center based on the maximum likelihood; generating a pupil shape for each of the plurality of locations based on a set of constraints and a set of assumptions; assigning a score to each of the plurality of shapes of the pupil; selecting a designated shape from the plurality of shapes based on a designated score of the designated shape that satisfies a criterion; and determining a center and a radius of the specified shape as the adjusted pupil center and the adjusted pupil radius, respectively; Including, The method of the present invention 1006. [The present invention 1009] providing the video stream to the first prediction model to obtain the first set of ocular measurement parameters, processing the video stream into a format suitable for the first predictive model to extract the first set of ocular measurement parameters before inputting the video stream into the first predictive model; Including, The method of the present invention 1001. [The present invention 1010] processing the video stream, Reducing noise, adjusting color and brightness of the video, and improving resolution of the video Including, The method of the present invention 1009. [The present invention 1011] providing the video stream to the first prediction model to obtain the first set of ocular measurement parameters, obtaining user data associated with the user, the user data including optometry data for the eye of the user; and applying a correction to the first set of eye measurement parameters based on the user data. further comprising: The method of the present invention 1001. [The present invention 1012] 1001. The method of claim 1001, wherein the first iris data includes (a) a plurality of coordinates representing the shape of the iris, and (b) a first coordinate of an iris translation. [The present invention 1013] providing the first set of ocular measurement parameters to the second predictive model, processing the first iris data using a maximum likelihood based curve fitting operation to obtain an elliptical fit of the shape of the iris; and determining a first iris rotation and a first iris radius based on the ellipse fit; Including, The method of the present invention 1012. [The present invention 1014] 10. The method of claim 10, wherein the first eyelid data includes a plurality of coordinates of a shape representing an eyelid of the eye. [The present invention 1015] A method of the present invention 1001, wherein the first set of eye measurement parameters is acquired as time series data, a first set of values ​​of the first set of eye measurement parameters is acquired for a first time point in the video, and a second set of values ​​of the first set of eye measurement parameters is acquired for a second time point in the video. [The present invention 1016] 10. The method of claim 10, wherein the first set of eye measurement parameters is acquired in multiple coordinate systems. [The present invention 1017] The method of claim 1016, wherein the plurality of coordinate systems includes a device coordinate system having an origin located at the center of a display associated with the client device. [The present invention 1018] The method of the present invention 1016, wherein the plurality of coordinate systems includes a head coordinate system whose origin is located at the center between the eyes projected by a displacement in the z direction onto a plane defining the face of the user. [The present invention 1019] determining an additional set of eye measurement parameters based on the second set of eye measurement parameters, the additional set of eye measurement parameters including at least one of a pupil visible ratio, an iris visible ratio, a pupil coverage asymmetry, or an iris coverage asymmetry; The method of the present invention 1001 further comprising: [The present invention 1020] When executed by a computer, obtaining a video stream from a client device associated with a user, the video stream including video of the user's face; providing the video stream as an input to a first predictive model to obtain a first set of ocular measurement parameters for the user's eye, the first set including a first pupil center, first iris data, and first eyelid data; deconvolving the video stream based on the first eyelid data, the first pupil center, stimulus data, and environmental data to obtain adjusted eyelid data, an adjusted pupil center, and an adjusted pupil radius as part of the first eye measurement parameter set, wherein the stimulus data includes spatiotemporal and optical information of a video stimulus presented on a display associated with the client device, and the environmental data includes lighting conditions in an environment in which the client device is located; and providing the first ocular measurement parameter set as input to a second predictive model to obtain a second ocular measurement parameter set with higher resolution than the first ocular measurement parameter set, the second ocular measurement parameter set including a second pupil center, a second pupil radius, an iris radius, an iris translation, and an iris rotation; instructions for causing the computer to carry out a method including 1. A non-transitory computer-readable medium having: [The present invention 1021] providing the input to the second predictive model, training the second prediction model using a plurality of datasets to predict the second eye measurement parameter set, each dataset including a video of a specified user's face, a set of labels having a first value set corresponding to the first eye measurement parameter set acquired at a specified resolution, and a second value set corresponding to the second eye measurement parameter set acquired at a resolution higher than the specified resolution; Including, The computer-readable medium of the present invention 1020. [The present invention 1022] A computer-readable medium of the present invention 1021, wherein the second set of values ​​corresponding to the second set of eye measurement parameters is acquired using an eye tracking device configured to acquire the second set of eye measurement parameters at a resolution higher than the specified resolution. [The present invention 1023] deconvolving the video stream, determining a point spread function of a deconvolution process based on the first eyelid data, the first pupil center, the stimulus data, and the environmental data; and deconvolving the video stream with the point spread function to obtain the adjusted eyelid data, the adjusted pupil center, and the adjusted pupil radius. Including, The computer-readable medium of the present invention 1020. [The present invention 1024] The computer-readable medium of the present invention 1023, wherein deconvolving the video stream includes determining the adjusted eyelid data, the adjusted pupil center, and the adjusted pupil radius based on a maximum likelihood prediction of the location of the first pupil center and eyelid boundary. [The present invention 1025] determining the adjusted eyelid data; processing the first eyelid data using a maximum likelihood curve fitting operation to obtain a parabolic fit of an eyelid of the eye; and obtaining a plurality of coordinates along the parabolic fit as the adjusted eyelid data; Including, The computer readable medium of the present invention 1024. [The present invention 1026] determining the adjusted pupil center; determining a plurality of positions for the first pupil center based on the maximum likelihood; generating a pupil shape for each of the plurality of locations based on a set of constraints and a set of assumptions; assigning a score to each of the plurality of shapes of the pupil; selecting a designated shape from the plurality of shapes based on a designated score of the designated shape that satisfies a criterion; and determining the center and radius of the specified shape as the adjusted pupil center and the pupil radius, respectively; Including, The computer readable medium of the present invention 1024. [The present invention 1027] A computer-readable medium of the present invention 1020, wherein the first set of eye measurement parameters is acquired as time series data, a first set of values ​​for the first set of eye measurement parameters is acquired for a first time point in the video, and a second set of values ​​for the first set of eye measurement parameters is acquired for a second time point in the video. [The present invention 1028] determining an additional set of eye measurement parameters based on the second set of eye measurement parameters, the additional set of eye measurement parameters including at least one of a pupil visible ratio, an iris visible ratio, a pupil coverage asymmetry, or an iris coverage asymmetry; The computer-readable medium of the present invention 1020 further includes: [The present invention 1029] a memory storing an instruction set; Execute the instruction set, obtaining a video stream from a client device associated with a user, the video stream including video of the user's face; training a first predictive model using a first plurality of datasets to output a first set of eye measurement parameters including a first pupil center, first iris data, and first eyelid data of the user's eye at a first resolution, each dataset of the first plurality of datasets including a video of a specified user's face and the set of eye measurement parameters acquired at the first resolution; deconvolving the video stream based on stimulus data and environmental data to obtain adjusted eyelid data, an adjusted pupil center, and an adjusted pupil radius as part of the first ocular measurement parameter set; and training a second predictive model using a second plurality of datasets to output a second set of eye measurement parameters at a second resolution higher than the first resolution, the second set of eye measurement parameters including a second pupil center, a second pupil radius, an iris radius, an iris translation, and an iris rotation, each dataset including a video of a specified user's face, a set of labels having a first value set corresponding to the first set of eye measurement parameters acquired at the first resolution, and a second value set corresponding to the second set of eye measurement parameters acquired at the second resolution; a processor configured to cause a system to perform the method of The system comprising: [Brief explanation of the drawings]

[0006] [Figure 1A] 1 illustrates a system for facilitating measurement of a user's eye measurement parameters and eye movements, consistent with various aspects. [Figure 1B] 1 illustrates a process for extracting high-resolution (HR) ocular measurement parameters associated with an eye, consistent with various aspects. [Figure 2] 1 illustrates a machine learning model configured to facilitate prediction of an ocular measurement parameter, according to one or more embodiments. [Figure 3] Figure 3A illustrates an example device coordinate system consistent with various embodiments. Figure 3B illustrates an example head coordinate system consistent with various embodiments. [Figure 4] 1 is a flow diagram of a process for obtaining eye measurement parameters at high resolution, consistent with various aspects. [Figure 5] 1 is a flow diagram of a process for deconvolving a video stream to obtain adjusted eye measurement parameters, consistent with various aspects. [Figure 6] FIG. 1 is a block diagram of a computer system that can be used to implement features of the disclosed aspects. DETAILED DESCRIPTION OF THE INVENTION

[0007] Detailed Description In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of aspects of the present invention. However, it will be understood by those skilled in the art that aspects of the present invention may be practiced without these specific details or with equivalent configurations. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring aspects of the present invention.

[0008] FIG. 1A illustrates a system 100 for facilitating measurement of a user's eye measurement parameters and eye movements, consistent with various aspects. FIG. 1B illustrates a process 175 for extracting high-resolution (HR) eye measurement parameters associated with an eye, consistent with various aspects. As shown in FIG. 1A, the system 100 may include a computer system 102, a client device 106, or other components. By way of example, the computer system 102 may include any computing device, such as a server, a personal computer (PC), a laptop computer, a tablet computer, a handheld computer, or other computing equipment. The computer system 102 may include a video management subsystem 112, an eye measurement parameter subsystem (OPS) 114, a marker subsystem 116, or other components. The client device 106 may include any type of mobile terminal, fixed terminal, or other device. By way of example, the client device 106 may include a desktop computer, a notebook computer, a tablet computer, a smartphone, a wearable device, or other client device. A user, such as user 140, may interact with one or more servers or other components of system 100, for example, utilizing client device 106. Camera 120 may capture still or video images of user 140, such as video of the face of user 140. Camera 120 may be integrated with client device 106 (e.g., a smartphone camera) or may be a standalone camera connected to client device 106 (e.g., a webcam or other such camera).

[0009] Components of system 100 may communicate with one or more components of system 100 via a communications network 150 (e.g., the Internet, a cellular network, a mobile voice or data network, a cable network, a public switched telephone network, or other type of communications network, or a combination of communications networks). Communications network 150 may be a wireless or wired network. As an example, client device 106 and computer system 102 may communicate wirelessly.

[0010] It should be noted that while one or more operations are described herein as being performed by particular components of computer system 102, those operations may in some aspects be performed by other components of computer system 102 or other components of system 100. As an example, while one or more operations are described herein as being performed by components of computer system 102, those operations may in some aspects be performed by components of client device 106.

[0011] It should be noted that while some aspects are described herein with respect to machine learning models, in other aspects other predictive models (e.g., statistical models or other analytical models) may be used instead of or in addition to machine learning models (e.g., in one or more aspects a statistical model replaces a machine learning model, and a non-statistical model replaces a non-machine learning model).

[0012] In some embodiments, the system 100 uses a video stream 125 of eye movement (e.g., a sequence of frames of eye images) captured by the camera 120 to facilitate the determination of eye measurement parameters 170-178 of the user 140 at high resolution (e.g., sub-pixel level, 0.1 mm or higher resolution). The video management subsystem 112 obtains the video stream 125 from the client device 106 and stores the video stream 125 in the database 132. The OPS 114 processes the video stream 125 to extract, at high resolution, eye measurement parameters such as (i) pupil center 176, (ii) pupil radius 178, (iii) iris radius 174, (iv) iris translation 172, and (v) iris rotation 170 for each of the eyes, among other parameters. These eye measurement parameters may determine eye movement parameters (e.g., pupil response parameters and gaze parameters) indicative of eye movement in response to non-standard stimuli (e.g., videos or images displayed on the client device 106). The marker subsystem 116 may generate digital biomarkers (eg, parameters) based on eye movement parameters that may serve as indicators of one or more neurological disorders.

[0013] The video management subsystem 112 may store the video stream 125 in the database 132 in any of several formats (e.g., WebM, Windows Media Video, Flash Video, AVI, QuickTime, AAC, MPEG-4, or another file format). The video management subsystem 112 may also provide the video stream 125 as input (e.g., in real time) to the OPS 114 for generating eye measurement parameters. In some aspects, the video management subsystem 112 may perform pre-processing 191 of the video stream 125 to convert the video stream 125 into a format suitable for eye measurement parameter extraction by the OPS 114. For example, the video management subsystem 112 may perform an extract, transform, and load (ETL) process on the video stream 125 to generate a pre-processed video stream 151 for the OPS 114. The ETL process is typically a multi-phase process in which data is first extracted, then transformed (e.g., cleaned, sanitized, and scrubbed), and finally loaded into a target system. Data may be collated from one or more sources and output to one or more destinations. In one example, the ETL process performed by video management subsystem 112 may include adjusting color and brightness (e.g., white balance) in video stream 125. In another example, the ETL process may include reducing noise from video stream 125. In yet another example, the ETL process may include improving the resolution of video stream 125 based on multi-frame data, for example, by implementing one or more multi-frame super-resolution techniques that reconstruct a high-resolution image (or sequence) from a sequence of low-resolution images. In some aspects, super-resolution is a digital image processing technique for obtaining a single high-resolution image (or sequence) from multiple blurry low-resolution images.The basic idea of ​​super-resolution is that low-resolution images of the same scene contain different information due to relative sub-pixel shifts, and thus a high-resolution image with higher spatial information can be reconstructed by image fusion. The video management subsystem 112 may perform the above and other similar pre-processing operations to generate a pre-processed video stream 151 in a format suitable for the OPS 114 to perform ophthalmometry parameter extraction.

[0014] In some aspects, the video management subsystem 112 may obtain a preprocessed video stream 151 (e.g., based on a video stream 125 obtained from a client device 106) via a predictive model. As an example, the video management subsystem 112 may input the video stream 125 obtained from the client device 106 into the predictive model, which then outputs the preprocessed video stream 151. In some aspects, the system 100 may train or configure the predictive model to facilitate the generation of the preprocessed video stream. In some aspects, the system 100 may obtain a video stream from a client device associated with a user (e.g., a video stream 125 having video of the user's 140 face) and provide such information as input to the predictive model to generate a prediction (e.g., a preprocessed video stream 151 with adjusted color or brightness, improved video resolution, etc.). The system 100 may provide reference feedback to the predictive model, and the predictive model may update one or more portions of the predictive model based on the prediction and the reference feedback. As an example, if the predictive model generates predictions based on video streams obtained from a client device, one or more preprocessed video streams associated with such input video streams may be provided as reference feedback to the predictive model. As an example, based on one or more user responses or one or more other user actions via one or more services (e.g., via user confirmation of the preprocessed video, via one or more subsequent actions that substantiate such a goal, etc.), a particular preprocessed video stream may be verified as a suitable video stream for ocular measurement parameter extraction by OPS 114. Such user input information may be provided as input to the predictive model to cause the predictive model to generate predictions for the preprocessed video stream, and the verified preprocessed video stream may be provided as reference feedback to the predictive model to update the predictive model.In this way, for example, a predictive model may be trained or configured to generate more accurate predictions. In some aspects, the training dataset may include several data items, where each data item includes at least a video acquired from a client device and a corresponding preprocessed video as ground truth.

[0015] In some aspects, the aforementioned operations for updating the predictive model may be performed using a training dataset associated with one or more users (e.g., a training dataset associated with a given user to specifically train or configure a predictive model for the given user, a training dataset associated with a given cluster, population, or other group to specifically train or configure a predictive model for a given group, a training dataset associated with any given set of users, or other training dataset). Thus, in some aspects, following updating the predictive model, system 100 may use the predictive model to facilitate generation of a pre-processed video stream to facilitate ocular measurement parameter extraction by OPS 114.

[0016] Video management subsystem 112 may generate preprocessed video stream 151 before inputting video stream 125 to OPS 114 or after storing video stream 125 in database 132. Additionally, video management subsystem 112 may store preprocessed video stream 151 along with video stream 125 in database 132.

[0017] The OPS 114 may process the video stream 125 or the preprocessed video stream 151 to extract, determine, derive, or generate eye measurement parameters (e.g., eye measurement parameters 170-178) that are characteristics of the eye 145 of the user 140. In some embodiments, the eye measurement parameters include parameters such as (i) pupil center 176, (ii) pupil radius 178, (iii) iris radius 174, (iv) iris translation 172, and (v) iris rotation 170. The pupil center 176 is the center of the pupil and may be represented using coordinates (e.g., a three-dimensional (3D) lateral displacement vector (x, y, z)) that identify the location of the pupil center in a coordinate system. The pupil radius 178 may be the radius of the pupil and may be represented as a scalar with distance units. Iris translation 172 (or iris translation 160) is the center of the iris and may be represented using coordinates (e.g., a 3D lateral displacement vector (x, y, z)) that identify the location of the iris center in a coordinate system. Iris radius 174 (or iris radius 162) may be the radius of the iris and may be represented as a scalar with distance units. In some embodiments, iris radius 174 is assumed to have a small variance in the human population (e.g., a 10%-15% variance considering the iris radius ranges from 11 mm to 13 mm across the human population). Iris rotation 170 (or iris rotation 158) represents the rotation of the iris and is taken as an iris normal vector (e.g., perpendicular to the iris circle plane). In some embodiments, iris rotation 170 may be projected onto a 3D vector (e.g., x, y, z) in the device coordinate system or represented by a 2D vector (e.g., azimuth and elevation angles) in the head coordinate system. In some embodiments, OPS 114 may obtain coordinates of the eye measurement parameters in one or more coordinate systems, such as the device coordinate system and the head coordinate system illustrated in Figures 3A and 3B, respectively.

[0018] 3A illustrates a device coordinate system consistent with various aspects. In some aspects, the device coordinate system is a Cartesian coordinate system, and the center of the display 302 of a client device (e.g., client device 106) is considered to be the origin 304 with (x, y, z) coordinates as (0, 0, 0). The device coordinate system defines three orthogonal axes, e.g., an x-axis in the direction of the horizontal edge of the display 302, a y-axis in the direction of the vertical edge of the display 302 and perpendicular to the x-axis, and a z-axis in the direction perpendicular to the plane of the display 302.

[0019] FIG. 3B illustrates a head coordinate system consistent with various embodiments. In some embodiments, the head coordinate system is defined by a plane including the following points within the face bounding polygon: the leftmost, rightmost, and topmost detectable points within the face polygon (points 1, 2, and 3 in FIG. 3B ). For example, topmost point 3 may be defined as the topmost detectable point within the face bounding polygon and may be located within the cross section between the y-axis and the head. When acquiring a face image, the horizontal axis (x) may be defined as the line between the leftmost and rightmost points, and the vertical axis (y) may be defined as the perpendicular vector to that line on the defining plane. The depth axis (z) may be orthogonal to both the x-axis and the y-axis. The origin of the head coordinate system (point 0 in FIG. 3B ) is located centered between the eyes, projected onto the defining plane by displacement in the z-direction. In some embodiments, iris rotation may be represented on the head coordinate system as eye rotation, given two angles (e.g., azimuth and elevation).

[0020] With respect to the OPS 114, the OPS 114 may be configured to represent coordinates of the ocular measurement parameters in either a coordinate system, including a head coordinate system or a device coordinate system. Additionally, the OPS 114 may be configured to transform coordinates from one coordinate system to another, such as from the head coordinate system to the device coordinate system or vice versa.

[0021] The OPS 114 may also acquire eyelid data 164, such as coordinates of the upper and lower eyelid borders of both eyes. The eyelid data 164 may be used in determining or improving the accuracy of eye measurement parameters. In some embodiments, the OPS 114 may acquire the above data (e.g., eye measurement parameters 170-178, eyelid data 164, or other data) as time-series data. For example, the OPS 114 may extract a first set of values ​​for the eye measurement parameters 170-178 at a first time point in the video stream 125, a second set of values ​​for the eye measurement parameters 170-178 at a second time point in the video stream 125, etc. That is, the OPS 114 may continuously extract the eye measurement parameters 170-178 at specified time intervals (e.g., every millisecond, every few milliseconds, or other time resolution). Furthermore, the OPS 114 may acquire the above data for one or both of the eyes 145 of the user 140.

[0022] OPS 114 may perform several processes to extract the above-mentioned eye measurement parameters (e.g., eye measurement parameters 170-178) from preprocessed video stream 151. For example, in extraction process 192, OPS 114 may obtain a first set of eye measurement parameters, such as iris data 153, eyelid data 155, and pupil center 157. In some embodiments, OPS 114 may use computer vision techniques on preprocessed video stream 151 to model the translation and rotation of user 140's face relative to camera 120 and identify the eyes and mouth of the face. After identifying the eye locations, OPS 114 may obtain eyelid data 155 (e.g., coordinates of an array of points describing a polygon representing the eyelids) for each frame (e.g., using the fact that their edges can be approximated by quadratic functions). In some embodiments, OPS 114 may determine iris data 153 (e.g., a shape, such as an ellipsoid, representing the iris) by determining which pixels are within or outside the iris radius at any time based on the position of the eyelid over time. In some embodiments, OPS 114 may be configured to consider the iris and pupil as ellipsoids having a relatively uniform, dark color compared to the sclera (e.g., the white part of the eye) when determining which pixels are within or outside. Iris data 153 may include several coordinates, such as first and second coordinates corresponding to the left and right coordinates of a shape representing the iris (e.g., an ellipse), third and fourth coordinates corresponding to the top and bottom coordinates of the shape, and a fifth coordinate corresponding to the center of the iris. Pupil center 157 may also be determined based on the time series of eyelid data 155 and iris data 153. In some embodiments, pupil center 157 may include a coordinate representing the location of the center of the pupil.

[0023] In some aspects, OPS 114 may implement a predictive model in extraction process 192 and obtain a first set of eye measurement parameters via the predictive model (e.g., based on video stream 125 or preprocessed video stream 151). As an example, OPS 114 may input video stream 125 or preprocessed video stream 151 into the predictive model, which then outputs the first set of eye measurement parameters. In some aspects, system 100 may train or configure the predictive model to facilitate generation of the first set of eye measurement parameters. In some aspects, system 100 may obtain a video stream having video of a user's face (e.g., video stream 125 or preprocessed video stream 151) and provide such information as input to the predictive model to generate predictions (e.g., a first set of eye measurement parameters such as iris data, eyelid data, pupil center, etc.). System 100 may provide reference feedback to the predictive model, and the predictive model may update one or more portions of the predictive model based on the predictions and the reference feedback. As an example, if the predictive model generates predictions based on video stream 125 or preprocessed video stream 151, a first of the eye measurement parameters associated with such input video stream may be provided as reference feedback to the predictive model. As an example, a particular eye measurement parameter set may be verified as an appropriate eye measurement parameter set (e.g., via user confirmation of the eye measurement parameter set, via one or more subsequent actions that substantiate such a goal, etc.). Such user input information may be provided as input to the predictive model to cause the predictive model to generate predictions for the first eye measurement parameter set, and the verified eye measurement parameter set may be provided as reference feedback to the predictive model to update the predictive model. In this manner, for example, the predictive model may be trained or configured to generate more accurate predictions.

[0024] In some embodiments, the aforementioned operations for updating the predictive model may be performed using a training dataset associated with one or more users (e.g., a training dataset associated with a given user to specifically train or configure a predictive model for the given user, a training dataset associated with a given cluster, population, or other group to specifically train or configure a predictive model for a given group, a training dataset associated with any given set of users, or other training dataset). Thus, in some embodiments, following updating the predictive model, system 100 may use the predictive model to facilitate generation of a first set of ocular measurement parameters.

[0025] In some embodiments, OPS 114 may also further adjust, modify, or correct the first set of parameters based on other factors, such as optometric data associated with the user. For example, user 140 may have a particular visual condition, such as myopia, hyperopia, astigmatism, or other visual condition, and the user may wear corrective lenses, such as eyeglasses or contact lenses. OPS 114 may take such optometric conditions of user 140 into account and correct, adjust, or modify one or more values ​​of the first set of eye measurement parameters. In some embodiments, OPS 114 may obtain the optometric data associated with user 140 from user profile data (e.g., stored in database 132).

[0026] In some embodiments, OPS 114 may further process 193 the iris data 153 to extract iris-related parameters, such as iris rotation 158, iris translation 160, and iris radius 162. In some embodiments, the above-mentioned iris-related parameters are obtained using ML techniques. For example, OPS 114 may implement a maximum likelihood estimation (MLE)-based curve-fitting method to estimate the iris-related parameters. The iris data is provided as input to an MLE-based curve-fitting method (e.g., ellipse fitting, based on the assumption that the iris is circular and therefore oval / elliptical when projected onto a 2D plane) that generates the above-mentioned iris-related parameters. In some embodiments, in statistics, MLE is a method for estimating parameters of a probability distribution by maximizing a likelihood function, such that the observed data is most likely under an assumed statistical model. The point in parameter space that maximizes the likelihood function is called the maximum likelihood estimate. In some embodiments, the curve-fitting operation is a process of constructing a curve or mathematical function that has an optimal solution to a set of data points, subject to constraints (e.g., ellipse fitting). Curve fitting can involve either interpolation, where an exact fit to the data is required, or smoothing, where a "smooth" function is constructed that approximately fits the data. Fitted curves can be used as data visualization aids to infer values ​​of functions for which data are not available and to summarize relationships between two or more variables.

[0027] In some embodiments, the accuracy of at least some of the iris-related parameters may be further improved by performing an MLE-based curve-fitting operation. For example, a shape describing the iris boundary may be further refined from the coordinates of the shape in iris data 153, and thus an improved or more accurate iris radius 162 may be determined. In some embodiments, the iris translation 160, also referred to as the iris center, is similar to the coordinates of the iris center in iris data 153. In some embodiments, in process 193, OPS 114 may consider different iris centers, construct an iris shape for each candidate iris center, and assign a confidence score to each of the shapes. Such a method may be repeated for different iris centers, and the shape with a score (e.g., the best score) that meets a score criterion is selected. In some embodiments, the score for each iteration may reflect the log-likelihood of the tested parameter set (e.g., the coordinates of the iris shape, radius, and center) based on known physical constraints and assumptions. In some embodiments, the physical constraints and assumptions may improve the efficiency of the MLE process by introducing better estimated initial conditions before iterating toward convergence. For example, some assumptions may include that the iris is circular (represented as an ellipse / oval when projected onto a plane in 2D, that the iris center is expected to be at the center of gravity of the circle, high contrast between the iris and the sclera, similarity of orientation between the left and right eye, stimulus-based assumptions regarding the expected point of gaze, and brightness of the display of the client device on which the video stimulus is shown to the user. In some aspects, examples of physical constraints may include pupil dilation dependency on overall brightness (light conditions), a limited range of facial orientations when the user views the display, that blinking may cause uncertainty due to readjustment time, or a physically possible range of motion (both face and eyeball).

[0028] The OPS 114 may perform a deconvolution process 194 on the video stream 125 or the preprocessed video stream 151 to adjust (e.g., improve resolution or increase accuracy) some eye measurement parameters, such as eyelid data 155 and pupil center 166, and determine other eye measurement parameters, such as pupil radius. In some embodiments, the OPS 114 may perform a blind deconvolution process to improve the accuracy of the eye measurement parameters. In image processing, blind deconvolution is a deconvolution technique that enables the reconstruction of a target scene from a single “blurred” image or a set of “blurred” images in the presence of an underdetermined or unknown point spread function (PSF). Conventional linear and nonlinear deconvolution techniques may utilize a known PSF. In blind deconvolution, the PSF is estimated from an image or set of images, allowing deconvolution to be performed. Blind deconvolution may be performed iteratively, whereby the PSF and scene estimates are improved with each iteration, or non-iteratively, where a single application of the algorithm extracts the PSF based on external information. After determining the PSF, the PSF may be used in deconvolving the video stream 125 or the preprocessed video stream 151 to obtain adjusted ophthalmometry parameters (e.g., adjusted eyelid data 164, adjusted pupil center 166, or pupil radius 168).

[0029] In some aspects, applying a blind deconvolution algorithm may help reduce or remove blur from the image / video and estimate irradiance (e.g., based on parameters of the estimated convolution vector (e.g., PSF)). In some aspects, the estimated irradiance is the estimated irradiance reflected from the eye of the user 140, which is an indication of the irradiance to which the eye is exposed. After the blur is reduced or removed and the irradiance is obtained, the deconvolution process may be repeated one or more times to obtain adjusted eye measurement parameters (e.g., whose accuracy or resolution is improved compared to the eye measurement parameters before the deconvolution process 194).

[0030] In some embodiments, the blind deconvolution process considers various factors when determining the PSF. For example, the OPS 114 may input (a) a video stream 125 or a preprocessed video stream 151 (e.g., a time-series image sequence) with high temporal resolution (e.g., 30 ms or faster per frame), (b) stimulus data, such as spatiotemporal information about the stimuli presented on the display of the client device 106 and optical property information about the stimuli, including spectral characteristics (e.g., color) and intensity (e.g., brightness), (c) environmental data, such as the lighting in the environment (e.g., room) in which the user is located, which can be measured using information obtained from the camera 120, and (d) device data, such as orientation information of the client device 106 and information from one or more sensors associated with the client device 106, such as an acceleration sensor. Such factors aid in the efficient calculation of the PSF, minimize noise uncertainty, and lead to better overall deconvolution accuracy.

[0031] In some embodiments, the deconvolution process 194 is integrated with an MLE process to further improve the accuracy of the eye measurement parameters. For example, the MLE process may be used to improve the accuracy of the eyelid data 155. As described above, the eyelid data 155 includes coordinates of an array of points that describe the shape (e.g., a polygon) of the eyelid. The MLE process performs a parabolic curve fitting operation on the eyelid data 155 under the constraint that the shape of the eyelid is parabolic to obtain a more accurate representation of the eyelid as adjusted eyelid data 164. In some embodiments, the adjusted eyelid data 164 may include coordinates of a set of points that describe the shape (e.g., a parabola) of the eyelid. Such adjusted eyelid data 164 may be obtained for both eyelids and both eyes. In some embodiments, OPS 114 may use adjusted eyelid data 164 information to predict precise eye measurement parameter values, such as pupil radius center, even when some are physically difficult (e.g., when the upper eyelid covers the pupil center or when only a portion of the iris is exposed to the camera).

[0032] In some embodiments, the MLE process may also be used to obtain or refine pupil-related data, such as pupil radius 168. In some embodiments, the accuracy of pupil-related data may be further improved by performing an MLE-based curve-fitting operation. In some embodiments, in the MLE process, OPS 114 may consider different pupil centers, construct a pupil shape for each candidate pupil center, and assign a confidence score to each of the shapes. Such a method may be repeated for different pupil centers, and the shape with a score (e.g., the best score) that meets a score criterion is selected. Once a shape is selected, the corresponding center may be selected as the adjusted pupil center 166, and pupil radius 168 may be determined based on the selected shape and adjusted pupil center 166. In some embodiments, the score for each iteration may reflect the log-likelihood of the examined parameter set (e.g., pupil radius and center) based on known physical constraints and assumptions. In some embodiments, the physical constraints and assumptions may improve the efficiency of the MLE process by introducing better estimated initial conditions before iterating toward convergence. For example, some assumptions may include that the pupil is circular (represented as an ellipse / oval when projected onto a 2D plane, that the pupil center is expected to be at the center of gravity of the circle, the degree of orientation similarity between the left and right eye, stimulus-based assumptions regarding the expected point of gaze, and the brightness of the display of the client device on which the video stimulus is shown to the user. In some aspects, example physical constraints may include pupil dilation dependency on overall brightness (light conditions), a limited range of facial orientations when the user views the display, that blinking may cause uncertainty due to readjustment time, or a physically possible range of motion (both face and eye). Thus, OPS 114 may obtain adjusted eye measurement parameters such as adjusted eyelid data 164, adjusted pupil center 166, and pupil radius 168 from the deconvolution process 194.In some embodiments, the above-described process 193 for processing iris data and deconvolution process 194 may output the following eye measurement parameters at a first resolution: adjusted eyelid data 164, adjusted pupil center 166 and pupil radius 168, iris rotation 158, iris translation 160, and iris radius 162 (also referred to as a "first set of eye measurement parameters"). For example, the first resolution may be pixel-level or other lower resolution.

[0033] In some embodiments, OPS 114 may further improve the resolution of the first eye measurement parameter set. For example, OPS 114 may improve the resolution from a first resolution to a second resolution (e.g., 0.1 mm, sub-pixel level, or some other resolution higher than the first resolution). In some embodiments, OPS 114 may input the first eye measurement parameter set to resolution improvement process 195 to obtain a second eye measurement parameter set (e.g., eye measurement parameters 170-178) at the second resolution. In some embodiments, resolution improvement process 195 may obtain the second eye measurement parameter set via a predictive model (e.g., based on the first eye measurement parameter set). As an example, OPS 114 may input the first eye measurement parameter set obtained at the first resolution into a predictive model, which then outputs the second eye measurement parameter set at the second resolution. In some embodiments, system 100 may train or configure the predictive model to facilitate generation of the second eye measurement parameter set. In some embodiments, the system 100 may obtain input data such as (a) a video stream having video of the user's face (e.g., video stream 125 or preprocessed video stream 151), (b) a first set of eye measurement parameters (obtained at a first resolution as described above), (c) environmental data such as lighting in the environment (e.g., room) in which the user is located, which can be measured using information obtained from the camera 120, (d) device data such as display size, display resolution, display brightness, or display contrast associated with the display of the client device 106, model and manufacturer information of the camera 120, or (e) user information such as demographics, medical history, optometry data, etc.

[0034] System 100 may provide such input data to the predictive model to generate predictions (e.g., a second set of eye measurement parameters, such as iris rotation 170, iris translation 172, iris radius 174, pupil center 176, and pupil radius 178 at the second resolution). System 100 may provide reference feedback to the predictive model, and the predictive model may update one or more portions of the predictive model based on the predictions and the reference feedback. As an example, if the predictive model generates predictions based on the above-mentioned input data, the second eye measurement parameter set associated with such input data may be provided as reference feedback to the predictive model. As an example, a particular eye measurement parameter set acquired at the second resolution may be verified as an appropriate eye measurement parameter set (e.g., via user confirmation of the eye measurement parameter set, via one or more subsequent actions that substantiate such a goal, etc.). The aforementioned user input information may be provided as input to the predictive model to cause the predictive model to generate predictions of the second eye measurement parameter set, and the verified eye measurement parameter set may be provided as reference feedback to the predictive model to update the predictive model. In this way, for example, the predictive model can be trained or configured to generate more accurate predictions. In some embodiments, the reference feedback having eye measurement parameters at the second resolution can be obtained, determined, or derived from information acquired using one of several eye tracking devices that generate eye measurement parameters at a high resolution (e.g., the second resolution). For example, some tracking devices generate eye measurement parameters such as gaze origin, gaze point, and pupil diameter at the second resolution. OPS 114 can derive a second set of eye measurement parameters, such as iris rotation, iris translation, iris radius, pupil center, and pupil radius, from the eye measurement parameters generated using the eye tracking device and provide the derived second set of eye measurement parameters as reference feedback for training the predictive model. Such reference feedback can be acquired for several videos and provided as a training data set for training the predictive model.

[0035] In some embodiments, the aforementioned operations for updating the predictive model may be performed using a training dataset associated with one or more users (e.g., a training dataset associated with a given user to specifically train or configure a predictive model for the given user, a training dataset associated with a given cluster, population, or other group to specifically train or configure a predictive model for a given group, a training dataset associated with any given set of users, or other training dataset). Thus, in some embodiments, following updating the predictive model, system 100 may use the predictive model to facilitate generation of a first set of ocular measurement parameters.

[0036] In some embodiments, OPS 114 may also extract additional eye measurement parameters, such as pupil visibility ratio, pupil coverage asymmetry, iris visibility ratio, or iris coverage asymmetry. In some embodiments, pupil visibility ratio is calculated as the ratio of the pupil area not covered by the eyelid to the pupil-iris area. Pupil coverage asymmetry may be defined as the average of the pupil-upper eyelid coverage ratio and the pupil-lower eyelid coverage ratio, normalized by the total iris coverage area, with the upper eyelid coverage ratio represented as a positive value and the lower eyelid coverage ratio represented as a negative value. The value of this parameter varies from -1 to 1 and may project the asymmetry of eyelid coverage between the upper and lower eyelids (e.g., "-1" may represent that all covered area is covered by the lower eyelid, "1" may represent that all covered area is covered by the upper eyelid, and "0" may represent that the upper and lower eyelids cover equal areas).

[0037] In some embodiments, the iris visibility ratio is calculated as the ratio of the iris area not covered by the eyelid to the total iris area. In some embodiments, the iris coverage asymmetry can be determined as the average of the iris coverage ratio of the upper eyelid and the iris coverage ratio of the lower eyelid, normalized by the total iris coverage area, where the upper eyelid coverage ratio is represented by a positive value and the lower eyelid coverage ratio is represented by a negative value. The value of this parameter ranges from -1 to 1, projecting the asymmetry of eyelid coverage between the upper and lower eyelids (e.g., "-1" indicates that all the coverage area is covered by the lower eyelid, "1" indicates that all the coverage area is covered by the upper eyelid, and "0" indicates that the upper and lower eyelids cover equal areas).

[0038] The OPS 114 may extract additional eye measurement parameters based on the second eye measurement parameter set 170-178. For example, the OPS 114 may perform geometric projections and calculations using the second eye measurement parameter set 170-178 to determine the additional eye measurement parameters.

[0039] In some embodiments, the OPS 114 may also extract, generate, or derive eye movement parameters (e.g., pupil response parameters or gaze parameters) indicative of eye movement in response to non-standard stimuli (e.g., videos or images displayed on the client device 106). The eye movement parameters may be derived using a second set of eye measurement parameters (e.g., obtained as described above). In some embodiments, the pupil response parameters indicate the eye's response to a particular stimulus, and the gaze parameters indicate where the eye is focusing or looking. The OPS 114 may acquire different types of gaze parameters, such as fixation, saccade, pursuit, or another gaze parameter. In some embodiments, fixation is defined as the distribution of eye movement while inspecting a particular region of a stimulus. In some embodiments, saccade is defined as the distribution of eye movement between inspected regions. In some embodiments, pursuit is defined as the distribution of eye movement during pursuit movements. These eye movement parameters may be used in generating various digital biomarkers.

[0040] In some embodiments, the marker subsystem 116 may generate digital biomarkers (e.g., parameters) that can serve as indicators of one or more neurological disorders. The marker subsystem 116 may generate digital biomarkers based on ocular measurement parameters or eye movement parameters (e.g., as described above). In some embodiments, these digital biomarkers are strongly correlated with (and can serve as surrogates for) disease progression or severity. The digital biomarkers generated by the marker subsystem 116 may be objective, sensitive, accurate (0.1 mm or less), or correlated with clinical disease progression, and may be acquired remotely, outside of a laboratory environment, or even in a distributed manner.

[0041] It should be noted that the eye measurement parameters, eye movement parameters, or digital biomarkers may be acquired in real time (e.g., based on a real-time video stream acquired from the camera 120) or offline (e.g., using video stored in the database 132). Furthermore, the eye measurement parameters (e.g., the second eye measurement parameter sets 170-178 or other eye measurement parameters) may be acquired for one or more eyes of the user as time-series data. That is, the eye measurement parameters 170-178 may be extracted continuously at specified time intervals (e.g., every millisecond, every few milliseconds, or other time resolution).

[0042] In some embodiments, the predictive model described above may include one or more neural networks or other machine learning models. As an example, a neural network may be based on a large collection of neural units (or artificial neurons). A neural network may roughly mimic the way a biological brain works (e.g., via large clusters of biological neurons connected by axons). Each neural unit of a neural network may be connected to many other neural units of the neural network. Such connections can strengthen or suppress their influence on the activation state of the connected neural units. In some embodiments, each individual neural unit may have a summation function that combines the values ​​of all its inputs. In some embodiments, each connection (or the neural unit itself) may have a threshold function such that a signal must exceed a threshold before it propagates to other neural units. These neural network systems may be self-learning and trained rather than explicitly programmed, and can perform significantly better in certain areas of problem solving compared to traditional computer programs. In some embodiments, a neural network may include multiple layers (e.g., where a signal path traverses from a front layer to a back layer). In some embodiments, backpropagation techniques may be utilized by the neural network, with forward stimuli being used to reset the weights of "forward" neural units. In some embodiments, neural network excitations and inhibitions may flow more freely, with connections interacting in more chaotic and complex ways.

[0043] A neural network may be trained (i.e., its parameters are determined) using a training data set (e.g., ground truth). The training data may include a set of training samples. Each sample may be a pair including an input object (typically an image, measurement, tensor, or vector, which may be called a feature tensor or vector) and a desired output value (also called a teacher signal). A training algorithm analyzes the training data and adjusts the behavior of the neural network by adjusting the parameters of the neural network (e.g., the weights of one or more layers) based on the training data. For example, x i is the feature tensor / vector of the ith example, and y i is the teacher signal {(x1,y1),(x2,y2),...(x N ,y N Given a set of N training samples of the form {\displaystyle \mathbb {N}}, the training algorithm computes a neural network g:X → Y, where X is the input space and Y is the output space. A feature tensor / vector is an n-dimensional tensor / vector of numerical features that represent some object (e.g., a complex electric field image). The tensor / vector space associated with these vectors is often called the feature or latent space. After training, the neural network can be used to make predictions using new samples.

[0044] As an example, with reference to FIG. 2 , the machine learning model 202 may receive an input 204 and provide an output 206. In one use case, the output 206 may be fed back to the machine learning model 202 as an input for training the machine learning model 202 (e.g., alone or in combination with a user indication of the accuracy of the output 206, a label associated with the input, or other reference feedback information). In another use case, the machine learning model 202 may update its configuration (e.g., weights, biases, or other parameters) based on an evaluation of its prediction (e.g., the output 206) and reference feedback information (e.g., a user indication of the accuracy, a reference label, or other information). In another use case, if the machine learning model 202 is a neural network, it may adjust connection weights to adjust for the difference between the neural network's prediction and the reference feedback. In a further use case, one or more neurons (or nodes) of the neural network may require that the neuron's respective errors be sent back to them through the neural network to facilitate the update process (e.g., backpropagation of errors). The connection weight updates may reflect, for example, the magnitude of the error propagated backward after a forward pass is completed. In this way, for example, the machine learning model 202 may be trained to generate better predictions.

[0045] 4 is a flow diagram of a process 400 for obtaining eye measurement parameters at high resolution, consistent with various embodiments. In some embodiments, the process 400 may be performed in the system 100 of FIG. 1A.

[0046] At operation 402, a video stream is obtained from a client device associated with a user. The video stream may include video of the user's face. For example, video stream 125 is obtained from client device 106. In some aspects, the video stream may be obtained in real time from the client device, for example, captured by a camera associated with the client device, such as camera 120 of client device 106, or may be recorded video obtained from a data storage device, such as database 132, a data storage device or memory on the client device, or other data storage device.

[0047] In operation 404, a video stream is input to a first predictive model to obtain a first set of eye measurement parameters. In some aspects, the video stream may be preprocessed before being input to the first predictive model. In some aspects, preprocessing the video stream may include performing an ETL process on the video stream to adjust the color and brightness (e.g., white balance) of the video stream, reduce noise from the video stream, improve the resolution of the video stream by performing one or more multi-frame super-resolution techniques, or other such video processing. For example, the video stream input to the first predictive model may be video stream 125 or preprocessed video stream 151.

[0048] In operation 406, a first set of eye measurement parameters is obtained from the first predictive model. The eye measurement parameters correspond to various characteristics of the eye. For example, the first set of eye measurement parameters may include iris data, eyelid data, and a pupil center. The iris data may include coordinates of several points describing the center and shape (e.g., an ellipse) of the iris. The eyelid data may include coordinates of an array of points describing a shape (e.g., a polygon) representing the eyelid. The pupil center may include coordinates of a point representing the center of the pupil. In some aspects, the first set of eye measurement parameters is obtained at a first resolution. For example, the first resolution may be pixel-level or another lower resolution. In some aspects, the iris data is further processed to obtain iris rotation, iris translation, and iris radius from the iris data. For example, iris data 153 is processed to obtain iris rotation 158, iris translation 160, and iris radius 162. The iris rotation may represent the rotation of the iris and may be taken as an iris normal vector (e.g., perpendicular to the iris circle plane). The iris translation may be the center of the iris and may be expressed using coordinates that identify the location of the iris center. The iris radius may be the radius of the iris and may be expressed as a scalar with distance units. In some embodiments, an MLE-based curve fitting method may be performed to estimate the iris-related parameters. The iris data is provided as input to an MLE-based curve fitting method (e.g., ellipse fitting) that generates the above-mentioned iris-related parameters.

[0049] In operation 408, the video stream is deconvolved to adjust (e.g., improve accuracy or resolution) the first set of eye measurement parameters. The video stream input to the deconvolution process may be video stream 125 or preprocessed video stream 151. In some aspects, the video stream is deconvolved using a blind deconvolution process integrated with an MLE process, further details of which are described above with reference to at least FIG. 1A and FIG. 1B and below with reference to at least FIG. 5. In some aspects, applying a blind deconvolution algorithm may help reduce or remove blur in the image / video or any other noise from the video stream to improve the accuracy of the extracted eye measurement parameters. The deconvolution process may be repeated one or more times to obtain adjusted eye measurement parameters (e.g., whose accuracy or resolution is improved compared to the eye measurement parameters before the deconvolution process). The adjusted parameters may include adjusted eyelid data, adjusted pupil center, and adjusted pupil radius.

[0050] At operation 410, the video stream and the first parameter set are input into a second predictive model to obtain high-resolution eye measurement parameters. For example, the first eye measurement parameter set, including iris rotation 158, iris translation 160, iris radius 162, adjusted eyelid data 164, adjusted pupil center 166, and pupil radius 168, is input into the second predictive model. In some embodiments, the second predictive model is trained to predict high-resolution eye measurement parameter sets. For example, the second predictive model is trained with several training datasets, each dataset including a video of a user's face, device data of a client device, environmental data of an environment in which the user is located, and user information of the user as input data, and a corresponding high-resolution eye measurement parameter set as ground truth. In some embodiments, the high-resolution eye measurement parameter set is obtained using one or more eye tracking devices configured to generate high-resolution eye measurement parameters.

[0051] In operation 412, a second set of eye measurement parameters is obtained from the second predictive model at a second resolution, for example, including iris rotation 170, iris translation 172, iris radius 174, pupil center 176, and pupil radius 178, at a second resolution (e.g., 0.1 mm, sub-pixel level, or some other resolution higher than the first resolution).

[0052] Optionally, in operation 414, additional eye measurement parameter sets may be acquired. For example, additional eye measurement parameters such as a pupil visible ratio, pupil coverage asymmetry, iris visible ratio, or iris coverage asymmetry may be acquired based on the second eye measurement parameter set. OPS 114 may acquire the additional eye measurement parameters by performing geometric projections and calculations using the second eye measurement parameter set. In some embodiments, the pupil visible ratio is calculated as the ratio of the pupil area not covered by the eyelid to the pupil-iris area. Pupil coverage asymmetry may be defined as the average of the pupil-upper eyelid coverage ratio and the pupil-lower eyelid coverage ratio, normalized by the total iris coverage area, with the upper eyelid coverage ratio represented as a positive value and the lower eyelid coverage ratio represented as a negative value. The value of this parameter varies from -1 to 1 and may project the asymmetry of eyelid coverage between the upper and lower eyelids (e.g., "-1" may represent that all covered area is covered by the lower eyelid, "1" may represent that all covered area is covered by the upper eyelid, and "0" may represent that the upper and lower eyelids cover equal areas).

[0053] Operations 402-414 may be performed by the same or similar subsystem as OPS 114, according to one or more aspects.

[0054] 5 is a flow diagram of a process 500 for deconvolving a video stream to obtain adjusted eye measurement parameters, consistent with various aspects. In some aspects, process 500 may be performed as part of operation 408 of process 400.

[0055] In operation 502, input data such as a video stream, stimulus data, environmental data, device data, or other data is obtained to deconvolve the video stream. The video stream input to the deconvolution process may be video stream 125 or preprocessed video stream 151. The stimulus data may include spatiotemporal information about the stimulus presented on the display of client device 106, or optical property information of the stimulus, including spectral characteristics (e.g., color) and intensity (e.g., brightness). The environmental data may include information such as lighting in the environment (e.g., room) in which the user is located, which can be measured using information obtained from camera 120. The device data may include information such as orientation information of client device 106, information from one or more sensors associated with client device 106, such as an acceleration sensor. Additionally, the input data may include eyelid data (e.g., eyelid data 155) and pupil center (e.g., pupil center 157).

[0056] In operation 504, a point spread function of a blind deconvolution algorithm is determined based on the input data.

[0057] In operation 506, the video stream is deconvolved based on the point spread function. The deconvolution process removes or reduces any blurring or other noise introduced into the video stream due to user environment-related factors, client device-related factors, camera-related factors, video stimulus-related factors, etc., to improve the video stream and thus improve the accuracy of the extracted eye measurement parameters.

[0058] In operation 508, adjusted eye measurement parameters are derived from the deconvolved video stream as described above. For example, the adjusted eye measurement parameters may include adjusted eyelid data 164, adjusted pupil center 166, or pupil radius 168. In some embodiments, the deconvolution process is integrated with an MLE process to further improve the accuracy of the eye measurement parameters. For example, the MLE process may be used to improve the accuracy of eyelid data 155. The MLE process may be used to perform a parabolic curve fitting operation on eyelid data 155 to obtain a more accurate representation of the eyelid as adjusted eyelid data 164. In another example, the MLE process may be used to obtain or improve the accuracy of pupil-related data, such as pupil radius 168. The MLE process may be used to perform a curve fitting operation (e.g., an ellipse fit) to refine the shape of the pupil. For example, the MLE process may consider different pupil centers, construct a pupil shape for each of the candidate pupil centers, and assign a confidence score to each of the shapes. Such a method may be repeated for different pupil centers, and the shape with the score (e.g., the best score) that meets the score criteria is selected. Once a shape is selected, the corresponding center may be selected as the adjusted pupil center 166, and a pupil radius 168 may be determined based on the selected shape and the adjusted pupil center 166.

[0059] In some embodiments, process 500 may be repeated one or more times to improve the accuracy or resolution of the adjusted eye measurement parameter (e.g., until the accuracy or resolution of the eye measurement parameter meets a criterion (e.g., exceeds a threshold) or the PSF meets a criterion).

[0060] In some embodiments, the various computers and subsystems illustrated in FIG. 1A may include one or more computing devices programmed to perform the functions described herein. A computing device may include one or more electronic storage devices (e.g., training data database(s) 134 for storing training data, model database(s) 136 for storing predictive models, etc., or prediction database(s) 132, which may include other electronic storage devices), one or more physical processors programmed with one or more computer program instructions, and / or other components. A computing device may include communication lines or ports that enable the exchange of information with other computing platforms within a network (e.g., network 150) or via wired or wireless technology (e.g., Ethernet, fiber optic, coaxial cable, Wi-Fi, Bluetooth, near-field communication, or other technology). A computing device may include multiple hardware, software, and / or firmware components operating together. For example, a computing device may be implemented by a cloud of computing platforms operating together as a computing device.

[0061] Electronic storage devices may include non-transitory storage media that electronically store information. The storage media of electronic storage devices may include one or both of: (i) system storage that is integral with (e.g., substantially non-removable from) a server or client device; or (ii) removable storage that is removably connectable to the server or client device via, for example, a port (e.g., a USB port, a FireWire port, etc.) or a drive (e.g., a disk drive, etc.). Electronic storage devices may include one or more of optically readable storage media (e.g., optical disks, etc.), magnetically readable storage media (e.g., magnetic tape, magnetic hard drives, floppy drives, etc.), charge-based storage media (e.g., EEPROM, RAM, etc.), solid-state storage media (e.g., flash drives, etc.), and / or other electronically readable storage media. Electronic storage devices may also include one or more virtual storage resources (e.g., cloud storage, virtual private networks, and / or other virtual storage resources). The electronic storage device may store software algorithms, information determined by a processor, information obtained from a server, information obtained from a client device, or other information that enables the functionality described herein.

[0062] A processor may be programmed to provide information processing capabilities in a computing device. Thus, a processor may include one or more of a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and / or other mechanisms for electronically processing information. In some embodiments, a processor may include multiple processing units. These processing units may be physically located within the same device, or a processor may represent the processing functions of multiple devices operating in concert. A processor may be programmed to execute computer program instructions to perform the functions described herein of subsystems 112-116 or other subsystems. A processor may be programmed to execute computer program instructions by software, hardware, firmware; some combination of software, hardware, or firmware; and / or other mechanisms for configuring processing capabilities on the processor.

[0063] It should be understood that the descriptions of the functionality provided by the different subsystems 112-116 described herein are for purposes of illustration and not limitation, and that any of the subsystems 112-116 may provide more or less functionality than described. For example, one or more of the subsystems 112-116 may be eliminated, and some or all of its functionality may be provided by other of the subsystems 112-116. As another example, additional subsystems may be programmed to perform some or all of the functionality attributed to one of the subsystems 112-116 herein.

[0064] 6 is a block diagram of a computer system that may be used to implement features of embodiments of the disclosure. Computer system 600 may be used to implement any of the entities, subsystems, components, or services shown in the example diagrams above (as well as any other components described herein). Computer system 600 may include one or more central processing units (“processors”) 605, memory 610, input / output devices 625 (e.g., keyboard and pointing devices, display devices), storage devices 620 (e.g., disk drives), and network adapters 630 (e.g., network interfaces), connected to an interconnect 615. Interconnect 615 is illustrated as an abstraction representing any one or more separate physical buses, point-to-point connections, or both connected by appropriate bridges, adapters, or controllers. Thus, the interconnect 615 may include, for example, a system bus, a Peripheral Component Interconnect (PCI) bus or PCI-Express bus, a HyperTransport or Industry Standard Architecture (ISA) bus, a Small Computer System Interface (SCSI) bus, a Universal Serial Bus (USB), an IIC (I2C) bus, or an Institute of Electrical and Electronics Components (IEEE) Standard 1394 bus, also known as "FireWire."

[0065] Memory 610 and storage device 620 are computer-readable storage media that may store instructions that implement at least portions of the described embodiments. Additionally, data structures and message structures may be stored or transmitted via a data transmission medium, such as a signal over a communications link. Various communications links may be used, such as the Internet, a local area network, a wide area network, or a point-to-point dial-up connection. Thus, computer-readable media may include computer-readable storage media (e.g., “non-transitory” media) and computer-readable transmission media. The storage media of the electronic storage device may include one or both of: (i) system storage that is integral with the server or client device (e.g., substantially non-removable), or (ii) removable storage that is removably connectable to the server or client device, for example, via a port (e.g., a USB port, a FireWire port, etc.) or a drive (e.g., a disk drive, etc.). The electronic storage may include one or more of an optically readable storage medium (e.g., optical disk, etc.), a magnetically readable storage medium (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.), a charge-based storage medium (e.g., EEPROM, RAM, etc.), a solid-state storage medium (e.g., flash drive, etc.), and / or other electronically readable storage medium. The electronic storage may also include one or more virtual storage resources (e.g., cloud storage, virtual private networks, and / or other virtual storage resources). The electronic storage may store software algorithms, information determined by a processor, information obtained from a server, information obtained from a client device, or other information that enables the functionality described herein.

[0066] The instructions stored in memory 610 may be implemented as software and / or firmware for programming processor(s) 605 to perform the actions described above. The processor may be programmed to execute computer program instructions by software, hardware, firmware; some combination of software, hardware, or firmware; and / or other mechanisms for configuring processing power on the processor. In some aspects, such software or firmware may be initially provided to computer system 600 by being downloaded by computer system 600 from a remote system (e.g., via network adapter 630).

[0067] The aspects presented herein can be implemented, for example, by programmable circuitry (e.g., one or more microprocessors) programmed with software and / or firmware, or in entirely dedicated hardwired (non-programmable) circuitry, or in a combination of such forms. The dedicated hardwired circuitry may be in the form of, for example, one or more ASICs, PLDs, FPGAs, etc.

[0068] remarks The above description and drawings are illustrative and should not be construed as limiting. Numerous specific details are set forth to provide a thorough understanding of the present disclosure. However, in some cases, well-known details are not described to avoid obscuring the description. Furthermore, various modifications may be made without departing from the scope of the aspects. Accordingly, the aspects are not limited except as by the appended claims.

[0069] References herein to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the present disclosure. Appearances of the phrase "in one embodiment" in various places herein do not necessarily all refer to the same embodiment, nor to separate or alternative embodiments that are mutually exclusive of other embodiments. Furthermore, various features are described that may be exhibited by some embodiments and not by others. Similarly, various requirements are described that may be requirements of some embodiments but not other embodiments.

[0070] The terms used herein generally have their ordinary meanings in the art, within the context of this disclosure and within the specific context in which each term is used. The terms used to describe this disclosure are explained below or elsewhere herein to provide additional guidance to practitioners regarding the description of this disclosure. For convenience, some terms may be highlighted, for example, using italics and / or quotation marks. The use of highlighting does not affect the scope and meaning of a term, which is the same in the same context whether or not it is highlighted. It will be understood that the same thing can be said in multiple ways. It will be understood that "memory" is a form of "storage," and that these terms may be used interchangeably in some cases.

[0071] Therefore, alternative language and synonyms may be used for any one or more of the terms discussed herein, and no special significance should be attached to whether a term is recited or discussed herein. Synonyms for some terms are provided. The description of one or more synonyms does not exclude the use of other synonyms. The use of examples anywhere in this specification, including examples of any term discussed herein, is for illustrative purposes only and is not intended to further limit the scope and meaning of the disclosure or any exemplified term. Similarly, the present disclosure is not limited to the various aspects provided herein.

[0072] Those skilled in the art will appreciate that the logic illustrated in each of the above flowcharts may be modified in various ways, for example, the order of logic may be rearranged, sub-steps may be performed in parallel, illustrated logic may be omitted, or other logic may be included.

[0073] Without intending to further limit the scope of the present disclosure, examples of instruments, devices, methods, and their related results according to the embodiments of the present disclosure are set forth below. Note that in the examples, headings or subheadings may be used for the convenience of the reader and should not limit the scope of the present disclosure in any way. Unless otherwise defined, all scientific and technical terms used herein have the same meaning as commonly understood by those skilled in the art to which this disclosure pertains. In case of conflict, the present specification, including definitions, will control.

[0074] While the present invention has been described in detail for purposes of illustration based on what are presently considered to be the most practical and preferred embodiments, it should be understood that such detail is for that purpose only and that the invention is not limited to the disclosed embodiments, but on the contrary, is intended to cover modifications and equivalent arrangements within the scope of the appended claims. For example, it should be understood that the present invention contemplates that, to the extent possible, one or more features of any embodiment can be combined with one or more features of any other embodiment.

Claims

1. obtaining a video stream from a client device associated with a user, the video stream including video of the user's face; providing the video stream as an input to a first predictive model to obtain a first set of eye measurement parameters for the user's eye, the first set of eye measurement parameters including a first pupil center, first iris data, and first eyelid data, the first set of eye measurement parameters being obtained at a first resolution; deconvolving the video stream based on the first eyelid data, the first pupil center, stimulus data, and environmental data to obtain adjusted eyelid data, an adjusted pupil center, and an adjusted pupil radius as part of an adjusted eye measurement parameter set, wherein the stimulus data includes spatiotemporal and optical information of a video stimulus presented on a display associated with the client device, and the environmental data includes lighting conditions in an environment in which the client device is located; and providing at least the adjusted ocular measurement parameter set as input to a second predictive model to obtain a second ocular measurement parameter set at a second resolution, the second ocular measurement parameter set including (a) a second pupil center, (b) a second pupil radius, and (c) second iris data including an iris radius, an iris translation, and an iris rotation, wherein the second resolution is higher than the first resolution; A method comprising:

2. 10. The method of claim 1, wherein the first resolution is measured in pixel values ​​or other lower resolution values, and the second resolution is measured in sub-pixel values.

3. providing the input to the second predictive model, training the second prediction model using a plurality of datasets to predict the second eye measurement parameter set at the second resolution, each dataset including a video of a specified user's face, a first value set corresponding to the first eye measurement parameter set acquired at the first resolution, and a set of labels having a second value set corresponding to the second eye measurement parameter set acquired at the second resolution; Including, 10. The method of claim 1.

4. 4. The method of claim 3, wherein the second set of values ​​corresponding to the second set of eye measurement parameters is acquired using an eye tracking device configured to acquire the second set of eye measurement parameters at the second resolution for a video of the face of the specified user.

5. deconvolving the video stream, determining a point spread function of a deconvolution process based on the first eyelid data, the first pupil center, the stimulus data, and the environmental data; and deconvolving the video stream with the point spread function to obtain the adjusted eyelid data, the adjusted pupil center, and the adjusted pupil radius. Including, 10. The method of claim 1.

6. 6. The method of claim 5, wherein the step of deconvolving the video stream includes determining the adjusted eyelid data, the adjusted pupil center, and the adjusted pupil radius based on a maximum likelihood prediction of the location of the first pupil center and eyelid boundary.

7. determining the adjusted eyelid data; processing the first eyelid data using a maximum likelihood curve fitting operation to obtain a parabolic fit of an eyelid of the eye; and obtaining a plurality of coordinates along the parabolic fit as the adjusted eyelid data; Including, 7. The method of claim 6.

8. determining the adjusted pupil center; determining a plurality of positions for the first pupil center based on the maximum likelihood; generating a pupil shape for each of the plurality of locations based on a set of constraints and a set of assumptions; assigning a score to each of the plurality of shapes of the pupil; selecting a designated shape from the plurality of shapes based on a designated score of the designated shape that satisfies a criterion; and determining a center and a radius of the specified shape as the adjusted pupil center and the adjusted pupil radius, respectively; Including, 7. The method of claim 6.

9. providing the video stream to the first prediction model to obtain the first set of ocular measurement parameters, processing the video stream into a format suitable for the first predictive model to extract the first set of ocular measurement parameters before inputting the video stream into the first predictive model; Including, 10. The method of claim 1.

10. processing the video stream, Reducing noise, adjusting color and brightness of the video, and improving resolution of the video Including, 10. The method of claim 9.

11. providing the video stream to the first prediction model to obtain the first set of ocular measurement parameters, obtaining user data associated with the user, the user data including optometry data for the eye of the user; and applying a correction to the first set of eye measurement parameters based on the user data. further comprising:

10. The method of claim 1.

12. The method of claim 1 , wherein the first iris data includes (a) a plurality of coordinates representing an iris shape, and (b) a first coordinate of an iris translation.

13. providing the first set of ocular measurement parameters to the second predictive model, processing the first iris data using a maximum likelihood based curve fitting operation to obtain an elliptical fit of the shape of the iris; and determining a first iris rotation and a first iris radius based on the ellipse fit; Including, 13. The method of claim 12.

14. The method of claim 1 , wherein the first eyelid data comprises a plurality of coordinates of a shape representing an eyelid of the eye.

15. The method of claim 1, wherein the first set of eye measurement parameters is acquired as time series data, a first set of values ​​for the first set of eye measurement parameters is acquired for a first time point in the video, and a second set of values ​​for the first set of eye measurement parameters is acquired for a second time point in the video.

16. The method of claim 1 , wherein the first set of eye measurement parameters is acquired in multiple coordinate systems.

17. 17. The method of claim 16, wherein the plurality of coordinate systems includes a device coordinate system having an origin located at the center of a display associated with the client device.

18. 17. The method of claim 16, wherein the plurality of coordinate systems includes a head coordinate system having an origin centered between the eyes projected by a displacement in the z direction onto a plane defining the face of the user.

19. determining an additional set of eye measurement parameters based on the second set of eye measurement parameters, the additional set of eye measurement parameters including at least one of a pupil visible ratio, an iris visible ratio, a pupil coverage asymmetry, or an iris coverage asymmetry; 10. The method of claim 1, further comprising:

20. A non-transitory computer-readable medium, comprising: When executed by a computer, obtaining a video stream from a client device associated with a user, the video stream including video of the user's face; providing the video stream as an input to a first predictive model to obtain a first set of ocular measurement parameters for the user's eye, the first set including a first pupil center, first iris data, and first eyelid data; deconvolving the video stream based on the first eyelid data, the first pupil center, stimulus data, and environmental data to obtain adjusted eyelid data, an adjusted pupil center, and an adjusted pupil radius as part of an adjusted eye measurement parameter set, wherein the stimulus data includes spatiotemporal and optical information of a video stimulus presented on a display associated with the client device, and the environmental data includes lighting conditions in an environment in which the client device is located; and providing at least the adjusted ocular measurement parameter set as input to a second predictive model to obtain a second ocular measurement parameter set with higher resolution than the first ocular measurement parameter set, the second ocular measurement parameter set including a second pupil center, a second pupil radius, an iris radius, an iris translation, and an iris rotation; instructions for causing the computer to carry out a method comprising: The non-transitory computer-readable medium having stored thereon:

21. providing the input to the second predictive model, training the second prediction model using a plurality of datasets to predict the second eye measurement parameter set, each dataset including a video of a specified user's face, a set of labels having a first value set corresponding to the first eye measurement parameter set acquired at a specified resolution, and a second value set corresponding to the second eye measurement parameter set acquired at a resolution higher than the specified resolution; Including, 21. The computer-readable medium of claim 20.

22. 22. The computer-readable medium of claim 21, wherein the second set of values ​​corresponding to the second set of eye measurement parameters is acquired using an eye tracking device configured to acquire the second set of eye measurement parameters at a resolution higher than the specified resolution.

23. deconvolving the video stream, determining a point spread function of a deconvolution process based on the first eyelid data, the first pupil center, the stimulus data, and the environmental data; and deconvolving the video stream with the point spread function to obtain the adjusted eyelid data, the adjusted pupil center, and the adjusted pupil radius. Including, 21. The computer-readable medium of claim 20.

24. 24. The computer-readable medium of claim 23, wherein deconvolving the video stream includes determining the adjusted eyelid data, the adjusted pupil center, and the adjusted pupil radius based on a maximum likelihood prediction of the location of the first pupil center and eyelid boundary.

25. determining the adjusted eyelid data; processing the first eyelid data using a maximum likelihood curve fitting operation to obtain a parabolic fit of an eyelid of the eye; and obtaining a plurality of coordinates along the parabolic fit as the adjusted eyelid data; Including, 25. The computer-readable medium of claim 24.

26. determining the adjusted pupil center; determining a plurality of positions for the first pupil center based on the maximum likelihood; generating a pupil shape for each of the plurality of locations based on a set of constraints and a set of assumptions; assigning a score to each of the plurality of shapes of the pupil; selecting a designated shape from the plurality of shapes based on a designated score of the designated shape that satisfies a criterion; and determining the center and radius of the specified shape as the adjusted pupil center and the pupil radius, respectively; Including, 25. The computer-readable medium of claim 24.

27. 21. The computer-readable medium of claim 20, wherein the first set of eye measurement parameters is acquired as time series data, a first set of values ​​for the first set of eye measurement parameters is acquired for a first time point in the video, and a second set of values ​​for the first set of eye measurement parameters is acquired for a second time point in the video.

28. determining an additional set of eye measurement parameters based on the second set of eye measurement parameters, the additional set of eye measurement parameters including at least one of a pupil visible ratio, an iris visible ratio, a pupil coverage asymmetry, or an iris coverage asymmetry; 21. The computer-readable medium of claim 20, further comprising:

29. a memory storing an instruction set; Execute the instruction set, obtaining a video stream from a client device associated with a user, the video stream including video of the user's face; training a first predictive model using a first plurality of datasets to output a first set of eye measurement parameters including a first pupil center, first iris data, and first eyelid data of the user's eye at a first resolution, each dataset of the first plurality of datasets including a video of a specified user's face and the set of eye measurement parameters acquired at the first resolution; deconvolving the video stream based on the stimulus data and the environmental data to obtain adjusted eyelid data, an adjusted pupil center, and an adjusted pupil radius as part of an adjusted ophthalmometry parameter set; and training a second predictive model using a second plurality of datasets to output a second set of eye measurement parameters at a second resolution higher than the first resolution, the second set of eye measurement parameters including a second pupil center, a second pupil radius, an iris radius, an iris translation, and an iris rotation, each dataset including a video of a specified user's face, a set of labels having a first value set corresponding to the first set of eye measurement parameters acquired at the first resolution, and a second value set corresponding to the second set of eye measurement parameters acquired at the second resolution; a processor configured to cause a system to perform the method of The system comprising:

Images