Estimating the delay from the monitor output to the sensor

Abstract

Disclosed herein are aspects of systems, methods, and computer program products, and / or combinations and subcombinations thereof, for determining a time delay for synchronizing one or more parameters. In one aspect, a plurality of frames are presented on a display device. A brightness level is adjusted in one or more of the plurality of frames. A plurality of image frames of a user's face are received. The plurality of image frames are obtained from a video stream recorded by an image sensor during the presentation of the plurality of frames on the display device. The plurality of image frames are processed to determine a time delay between when the brightness level adjustment is made in one or more frames and when the brightness level adjustment in the one or more frames is reported by the image sensor.

Description

[Technical Field]

[0001] Cross-reference to related patent applications This application claims priority to U.S. Patent Application No. 18 / 404,045, filed January 4, 2024, and U.S. Provisional Patent Application No. 63 / 437,237, filed January 5, 2023, the contents of each of which are incorporated herein by reference in their entirety.

[0002] Field The present disclosure is generally directed to estimating the delay from a monitor output to an image sensor. In particular, the present disclosure relates to estimating the time delay between the presentation of an output on a display device and the detection of a video frame indicative of a user's reaction to the output. [Background technology]

[0003] background The progression of neurological disorders can be determined using minute eye movements. These eye movements are typically measured using specialized devices (e.g., infrared eye trackers, pupillometers, or other such devices) in well-controlled laboratory settings (e.g., no movement, controlled ambient light, or other such parameters). However, specialized devices can be difficult to set up, very costly, or require a significant amount of time and effort to create or maintain a controlled laboratory setup. Such challenges can hinder continuous monitoring of the progression of neurological disorders. Continuous monitoring can aid in the early detection, treatment, and care of individuals suffering from neurological disorders or mental health conditions. Summary of the Invention

[0004] overview Provided herein are aspects of systems, apparatuses, articles of manufacture, methods, and / or computer program products, and / or combinations and subcombinations thereof, for efficiently determining a time delay for synchronizing one or more parameters. Exemplary aspects include presenting a plurality of frames to a display device, adjusting a brightness level in one or more of the plurality of frames, and receiving a plurality of image frames of a user's face. The plurality of image frames are obtained from a video stream recorded by an image sensor while presenting the plurality of frames to the display device. The aspects further process the plurality of image frames to determine a time delay between when adjusting the brightness level in one or more of the plurality of frames occurs and when adjusting the brightness level in the one or more of the plurality of frames is reported by the image sensor.

[0005] Further features of the present disclosure, as well as the structure and operation of various embodiments, are described in detail below with reference to the accompanying drawings. It should be noted that the present disclosure is not limited to the particular embodiments described herein. Such embodiments are presented herein for illustrative purposes only. Further embodiments will be apparent to those skilled in the art based on the teachings contained herein. [Brief explanation of the drawings]

[0006] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate the disclosure and, together with the description, further serve to explain the principles of the disclosure and to enable one skilled in the art to make and use the aspects described herein.

[0007] [Figure 1] FIG. 1 is a block diagram of a system for estimating a time delay from a monitor output to a sensor, according to some aspects. [Figure 2] 2A and 2B are schematic diagrams illustrating gaze-responsive stimuli on a display area, according to some embodiments. [Figure 3A] FIG. 1 is a schematic diagram illustrating adjusting brightness levels over multiple frames, according to some aspects. [Figure 3B] FIG. 1 is a schematic diagram illustrating luminance measurements of multiple image frames, according to some aspects. [Figure 4A] 1 is a schematic diagram illustrating a row of a rolling shutter image sensor at a total exposure time, according to some embodiments. [Figure 4B] 1 is a schematic diagram illustrating a row of a rolling shutter image sensor at half exposure time, according to some embodiments. [Figure 4C] 1 is a schematic diagram illustrating a row of rolling shutter image sensors with short exposure times, according to some embodiments. [Figure 5] 1 is a graph illustrating adapted luminance measurements, according to some aspects. [Figure 6] 1 is an exemplary method for estimating delay from a display device to a sensor, according to some aspects. [Figure 7] 1 illustrates a computer system for implementing various aspects of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0008] Features of the present disclosure will become more apparent from the detailed description set forth below when read in conjunction with the drawings, in which like reference numerals identify corresponding elements throughout. In the drawings, like reference numerals generally indicate identical, functionally similar, or structurally similar elements. Furthermore, the left-most digit(s) of a reference number generally identifies the drawing in which that reference number first appears. Unless otherwise indicated, the drawings provided throughout this disclosure should not be construed as drawings to scale.

[0009] Detailed Description Aspects of the present disclosure relate to a system for estimating a time delay from a display device (e.g., a monitor output) to an image sensor. In particular, the present disclosure relates to estimating the time delay between when an adjustment is presented to a display device and receipt of the adjustment by an image sensor.

[0010] This specification discloses one or more embodiments incorporating features of the present disclosure. The disclosed embodiment(s) are provided as examples. The scope of the present disclosure is not limited to the disclosed embodiment(s). The claimed features are defined by the appended claims.

[0011] References to the described embodiment(s), and to "one embodiment," "one embodiment," "exemplary embodiment," etc. herein, indicate that the described embodiment(s) may include a particular feature, structure, or characteristic, but not all embodiments necessarily include that particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, if a particular feature, structure, or characteristic is described in the context of one embodiment, it should be understood that it is within the knowledge of one of ordinary skill in the art to implement such feature, structure, or characteristic in the context of other embodiments, whether or not explicitly described.

[0012] Spatially relative terms such as "beneath," "below," "lower," "above," "on," "upper," and the like may be used herein for ease of description to describe the relationship of one element or feature shown in the figures to another element(s) or feature(s). Spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation shown in the figures. The device may be otherwise oriented (rotated 90 degrees or facing another direction) and the spatially relative descriptors used herein may be similarly interpreted accordingly.

[0013] Terms such as "about," "approximately," and the like may be used herein to indicate the value of a quantity that may vary within or be found to be within a range of values ​​based on a particular technique. Based on a particular technique, these terms may indicate, for example, the value of a given quantity that is within 1-20% of the value (e.g., ±1%, ±5%, ±10%, ±15%, or ±20% of the value).

[0014] Aspects of the present disclosure may be implemented in hardware, firmware, software, or any combination thereof. Aspects of the present disclosure may also be implemented as instructions stored on a machine-readable medium, which may be read and executed by one or more processors. A machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing device). For example, a machine-readable medium may include read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, electrical, optical, acoustical, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.), among others. Furthermore, firmware, software, routines, and / or instructions may be described herein as performing certain actions. However, it should be understood that such description is merely for convenience and that such actions actually result from a computing device, processor, controller, or other device executing the firmware, software, routines, instructions, etc. In the context of computer storage media, the term "non-transitory" may be used herein to describe all forms of computer-readable media, with the sole exception of transitory propagating signals.

[0015] As mentioned in the background section above, neurological disorders can be assessed using subtle eye movements. These eye movements are typically measured in a well-controlled laboratory setting (e.g., no movement, controlled ambient light, or other such parameters) using a specialized device (e.g., an infrared eye tracker, pupillometer, or other such device). However, setting up and maintaining such a controlled testing environment can be very expensive and require significant time and effort. Furthermore, such well-controlled laboratory settings exist in limited numbers, and making an appointment and / or traveling there can be difficult.

[0016] It would be desirable to have eye movement measurements available at low cost by observing eye behavior in response to visual stimuli, for example, using ubiquitous cameras found in smartphones, tablets, laptop computers, desktop computers, etc. Eye movement measurements can be used to determine digital markers indicative of neurological or mental health conditions.

[0017] The time delay (e.g., time lag) between when a visual stimulus is presented to a user on a display device (e.g., during ongoing testing or monitoring of a subject) and when the user's response is detected by an image sensor can affect the accuracy of eye movement measurements. For example, a user's saccade latency can be determined using eye movement measurements acquired while a gaze-responsive stimulus is shown on a display device. Saccade latency is measured in the range of approximately 150 to approximately 250 milliseconds. In some aspects, the time delay from the display device to the computing system (e.g., application software) that receives and analyzes the image of the user's face to determine the eye movement measurements can be approximately 50 to approximately 100 milliseconds or more, with a variation of + / - tens of milliseconds. Additional delays, such as network delays and server delays, can also affect the accuracy of eye movement measurements. Furthermore, digital markers or computer processors used to determine eye movement measurements may have unpredictable delays (e.g., due to other resource-intensive applications or high demands on available memory). These delays cannot be estimated in advance. It is therefore desirable to adjust eye movement measurements to account for such delays in order to improve the accuracy of the measurements.

[0018] Multiple factors can contribute to time delay. In some cases, the image sensor and display device are not synchronized. This can cause a relatively slow frequency drift that contributes to variations in cumulative processing time. Frames may also be dropped on either the display device or the image sensor. Variations may also result from different types of image sensors and / or display devices used to continuously monitor the progression of neurological disorders. For example, image sensors may include a wide range of cameras used in smartphones, tablets, laptops, and desktop computers. Furthermore, due to the different types of imaging devices used, the software used to measure eye movements may not control the exposure time of the image sensor. Also, the exposure level for each image frame may not be readily available.

[0019] In some aspects, the lighting in a user's environment may vary. For example, the lighting in an indoor environment may vary from a dimly lit room to a brightly lit room. Indoor light levels may vary from approximately 200 lux to approximately 2000 lux. Furthermore, the illuminance of an indoor environment may change rapidly. For example, lights may be turned on or off, and curtains may be opened or closed, rapidly changing the intensity of light coming through a window.

[0020] The embodiments described herein address some or all of the aforementioned problems by determining the time delay between when a pattern is transmitted to a display device and when the effect of that pattern is detected in a video of a user. The embodiments described herein may adjust the brightness level and / or color of a display device and then detect the effect of the adjustment in one or more frames of the user's face. In some aspects, the adjustment of the brightness level may affect the lighting of the user's face. Furthermore, a brightness measurement in one or more image frames of the user is altered due to changes in lighting. The brightness measurement may correspond to the brightness of pixels associated with the user's face. In some aspects, the brightness measurement may correspond to the brightness of all pixels in the image frame when the background behind the subject is close, has sufficient reflectivity, and is stable. In a communications theory framework, this adjustment may be considered a transmission of information over a communications channel to be received and interpreted by programmed application software. Specifically, the software is configured to detect the earliest moment at which a specified change in brightness adjustment reaches the image sensor. The display adjustments are selected to be large enough for good detection and frequent enough to track the varying cumulative processing delay, but in one aspect not visible to the user so as to be annoying or distracting to the user.

[0021] 1 is a block diagram of a system 100 for estimating a time delay from a display device 108 to an image sensor 106, in accordance with some embodiments. The system 100 may include a computing system 102, an image sensor 106, and a display device 108.

[0022] The display device 108 may be a device capable of rendering images generated or acquired by the computing system 102 so that the user 104 can visually perceive the images. The display device 108 may include a display screen integrated with the computing system 102 (e.g., the integrated display of a smartphone, tablet computer, or laptop computer), a monitor separate from the computing system 102 but communicatively coupled to it (e.g., a monitor connected to a desktop computer via a wired connection), or a projector system (e.g., a projector including a projection screen and a light source). The display device 108 may also include a display panel of a standalone or tethered augmented reality headset. In one example, the display device 108 may be a color display monitor with a display rate of 60 frames per second (FPS). In some aspects, the display device 108 may display multiple frames generated by the computing system 102. In some aspects, the multiple frames may be received by the display device 108 over a network.

[0023] A network may be a telecommunications network, such as a wired network or a wireless network. A network can span and represent a variety of networks and network topologies. For example, a network can include wireless communications, wired communications, optical communications, ultrasonic communications, or a combination thereof. For example, satellite communications, cellular communications, Bluetooth, Infrared Data Association standard (IrDA), Wireless Fidelity (WiFi), and Worldwide Interoperability for Microwave Access (WiMAX) are examples of wireless communications that may be included in a network. Cable, Ethernet, Digital Subscriber Line (DSL), fiber optic lines, Fiber to the Home (FTTH), and Plain Old Telephone Service (POTS) are examples of wired communications that may be included in a network. Furthermore, a network can traverse several topologies and distances. For example, a network can include a direct connect, a personal area network (PAN), a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), or a combination thereof.

[0024] In some aspects, the image sensor 106 may be an optical device capable of capturing and storing images and video. The image sensor 106 may include, for example, a digital camera that captures images and video via an electronic image sensor. The image sensor 106 may be integrated with the computing system 102 (e.g., an integrated camera in a smartphone, tablet computer, or laptop computer) or may be part of a device separate from but communicatively coupled to the computing system 102 (e.g., a USB camera or webcam connected to a desktop computer via a wired connection). In one example, the image sensor 106 may be a video camera. The video camera may operate at a rate of 60 FPS. In some aspects, the image sensor 106 may transmit the captured images or video over a network.

[0025] As described previously herein, the computing system 102 may be a mobile device, a laptop computer, or a desktop computer. In some aspects, the computing system 102 may operate on one or more servers and / or databases. The servers may be various centralized or decentralized computing devices. For example, the servers may be grid computing resources, virtualized computing resources, peer-to-peer distributed computing devices, or combinations thereof. The servers may be centralized in a single room, distributed across different rooms, distributed across different geographic locations, or embedded within a network. In some aspects, the computing system 102 may be implemented using the computer system 700 described with reference to FIG. 7. The computing system 102 may provide a cluster or cloud computing platform for performing eye movement measurements based on image frames received or acquired from the image sensor 106. The computing system 102 may determine a time delay and adjust the eye movement measurements based on the time delay.

[0026] The computing system 102 may generate and transmit multiple frames to the display device 108. The computing system 102 may store multiple transmission timestamps. Each transmission timestamp may correspond to when a respective frame was transmitted. In some aspects, the computing system 102 may also transmit display luminance data corresponding to the multiple frames.

[0027] The image sensor 106 may record video of the user 104 while multiple frames are displayed on the display device 108. The computing system 102 may acquire the video from the image sensor 106. The image sensor 106 may include multiple image frames. The computing system 102 may also determine and store multiple reception timestamps. The multiple reception timestamps may correspond to the time each image frame of the video was received by the computing system 102.

[0028] The user 104 may be a person interacting with the computing system 102. In one embodiment, the user 104 may be a person or subject undergoing oculometry testing or monitoring. The testing may include determining the user's 104 eye movement abilities. The oculometry may include a test to determine the user's saccade latency. These are merely examples, and other types of oculometry may be applied to the user 104. In some aspects, the user 104 may be a person interacting with a virtual reality system (e.g., playing a virtual reality game) in which the user's response to stimuli is determined and used to control one or more parameters.

[0029] As described above, the cumulative processing time for generating frames for the display device 108 and receiving image frames from the image sensor 106 can vary significantly. This variation is due to variations in multiple stages of software and hardware processing. For example, the computing system 102 may send frames (display frames) to the display device 108. As described above, the computing system 102 may record a transmission timestamp for each frame. After receiving the frames, a driver (e.g., a monitor driver) for the display device 108 may control the display device 108 to display the frames. The image sensor 106 may integrate light reflected from the face of the user 104. The image sensor 106 may transfer the captured image to the computing system 102. In some aspects, the image sensor 106 may send pixel values ​​to a USB controller via an image sensor interface (e.g., a Mobile Industry Processor Interface (MIPI)). The USB controller may be associated with the computing system 102. The USB controller may send the pixel values ​​to memory in the computing system 102 via the USB interface. The pixels may be transmitted in a tightly packed format to the memory of the computing system 102 via direct memory access (DMA). The computing system 102 may process the pixel values. For example, the computing system 102 may reformat the pixels. In some aspects, the pixel values ​​may be stored in a first-in, first-out (FIFO) buffer in the main memory of the computing system 102. The computing system 102 may read the pixel values ​​of the image frames from the FIFO buffer and associate a corresponding reception timestamp with each frame.

[0030] In some aspects, computing system 102 may include a time delay determination module 110 and a synchronization module 112. The time delay determination module 110 and the synchronization module 112 may be implemented by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode), software (e.g., instructions executed by one or more processors of computing system 102), or a combination thereof.

[0031] The time delay determination module 110 may vary the brightness levels in the frames to determine the time delay. While varying the brightness levels in the frames, the image sensor 106 may capture the face of the user 104. The time delay determination module 110 may analyze the image frames received from the image sensor 106 to determine the time delay between when the brightness level changes and when the change in brightness level is reported by the image sensor 106.

[0032] The time delay determination module 110 may determine luminance measurements of pixels corresponding to the face of the user 104 to determine when the luminance level is reported by the image sensor 106. The time delay determination module 110 may implement one or more facial recognition or detection techniques to identify pixels corresponding to the face of the user 104. The facial recognition or detection techniques include traditional facial detection (e.g., elastic graph matching, singular value decomposition (SVD), Viola-Jones), artificial neural networks (e.g., deep convolutional neural networks), or three-dimensional (3D) facial recognition techniques.

[0033] In some aspects, a driver of display device 108 may control brightness based on brightness data received with or included with multiple frames. For example, in a projection system, a driver of a light source (e.g., a lamp) may adjust lamp power to control the brightness of illumination light emitted from the light source or the brightness of a projected image. In some aspects, the brightness data may include signals for controlling drivers of pixels or sub-pixels (e.g., red, green, and blue) of display device 108.

[0034] In some aspects, the time delay determined by the time delay determination module 110 may include a network delay and / or a server delay. This provides the advantage of improving accuracy and reducing computational inefficiencies (e.g., the network delay and / or the server delay are not calculated separately). The network delay may represent a delay from receiving and transmitting data to a cloud server when the computing system 102 is implemented as a server in a cloud computing network that receives image frames from the image sensor 106 and transmits the frames to the display device 108. The delay may include connection establishment delay, round-trip network delay, data transfer time, etc. The server delay may include connection time, a window exhaustion metric, etc.

[0035] As described previously herein, the computing system 102 may be used to determine the eye movement ability of the user 104. The computing system 102 may transmit one or more frames corresponding to visual stimuli to the display device 108. For example, as described in connection with FIGS. 2A and 2B , the user 104 may be instructed to gaze directly at a fixation point on the display as the display momentarily switches from a first fixation point to a second fixation point (called a gaze response stimulus or saccade test). The image sensor 106 may capture an image of the user 104's face while multiple frames are displayed on the display device 108. Furthermore, the computing system 102 may extract eye regions from the image of the user 104's face. The computing system 102 may analyze the eye regions to determine one or more eye movement measurements or other eye measurement parameters. The computing system 102 may determine saccade latency based on the eye movement measurements. The saccade latency is the time from the presentation of the second fixation point to the initiation of a saccade. The computing system 102 may determine one or more digital markers that may be indicative of the neurological or mental health status of the user 104 based on the saccade latency and / or other oculometry parameters.

[0036] The synchronization module 112 may modify one or more oculometry parameters to account for the time delay. The time delay may be received from the time delay determination module 110. For example, one or more oculometry parameters or other eye-tracking data may be stored as a time series. The time associated with each parameter may be shifted by a time delay (i.e., x milliseconds) to compensate for the time difference or time delay.

[0037] FIG. 2A illustrates a gaze response stimulus displayed on the display device 108 at the beginning of a gaze tracking test, such as a saccade test, according to some embodiments. FIG. 2B illustrates a gaze response stimulus displayed on the display device 108 at the end of a gaze tracking test, according to some embodiments. A saccade is a rapid movement of the eyes between two fixation points. A saccade test measures a subject's ability to move their eyes from one fixation point to another in a single rapid movement. In some aspects, the gaze response test may include displaying a first image on the display device 108. The first image may include a target 202 (e.g., a dot) on a background 204. The background may be a solid background (a solid, uniform color). The target 202 may have different display attributes, such as a different color or intensity, than the background 204. The target 202 may be displayed at a first position. In a second image displayed on the display device 108, the target 202 may be at a second position, as shown in FIG. 2B. The target 202 may move along a vertical and / or horizontal direction. The target 202 may move from left to right, right to left, top to bottom, bottom to top, or diagonally. In some aspects, the target 202 may move in a circle. The target 202 may move in a smooth motion or at suddenly varying speeds. The movement direction, speed, and other attributes of the gaze tracking test may be selected based on the desired oculometry parameters. As described above, the brightness level of the background may be modified. For example, the background may have a solid gray color. The brightness level of the gray background may be varied from a white background (100% brightness) to a black background (0% brightness).

[0038] FIG. 3A is a schematic diagram illustrating the adjustment of brightness levels over multiple frames in accordance with some embodiments. Graph 306 illustrates brightness levels of display device 108. As illustrated by graph 306, the brightness levels may be adjusted over time (e.g., changed for each displayed frame). The brightness levels may be varied around a nominal value N. The change in brightness levels over one or more frames of the multiple frames is referred to herein as adjustment. The nominal value N may represent a solid background (e.g., a grayscale) without adjustment. In some aspects, a first limit 308 may represent a brightness level equal to twice the nominal value (e.g., may be referred to as +100%). A second limit 310 may correspond to a pure black background (e.g., may be referred to as -100%). In some aspects, the brightness level may be varied to a first brightness level (e.g., an upper limit) between first limit 308 and second limit 310. For example, the brightness level may be changed to +50% of the nominal value for a first period (e.g., for three frames) (as shown in graph 306). The brightness level is then changed to a second brightness level that is lower than the nominal value. For example, the brightness level may be changed to -50% of the nominal value for a second period (e.g., for three frames).

[0039] The adjustment rate may be based on the noise level in the received image frame. For example, if the noise level in the image frame is low, the adjustment rate may be decreased (e.g., the brightness level may be adjusted to + / - 20% of the nominal value). In some aspects, if the noise level in the image frame is high, the adjustment rate may be increased (e.g., the brightness level may be adjusted to + / - 70% of the nominal value).

[0040] In some aspects, the adjustment may be performed for a first set number of frames at a first brightness level, followed by a second set number of frames at a second brightness level. In other aspects, the adjustment may be performed for a first set of frames at a second brightness level, followed by a second set of frames at a first brightness limit. In some aspects, the first set of frames and the second set of frames may be contiguous. In some aspects, the first set of frames and the second set of frames may include the same number of frames. Thus, the average value of the adjustment is 0%. In some aspects, the first set of frames may include three frames. In some aspects, the second set of frames may include three frames.

[0041] For a display device operating at a rate of 60 FPS, each frame may have a duration of 16.7 ms, so three frames of a first brightness level, or about 16.7 x 3 = 50.1 ms, are followed by three frames of a second brightness level, or about 50.1 ms.

[0042] In some aspects, the number of frames in each set may be selected so that the adjustment is not perceptible by the user 104. Furthermore, to minimize perception or distraction, additional techniques may be implemented, as described further below.

[0043] 3B is a schematic diagram illustrating brightness measurements of multiple image frames, according to some embodiments. The brightness measurements are taken at time T C3B. As described above, the luminance measurements corresponding to an image frame may vary due to changes in luminance levels in one or more of the frames presented on the display device 108. In some aspects, the luminance measurements may correspond to an average value of pixel values ​​of the image frame. In some aspects, an average value of pixel values ​​of pixels corresponding to the user's face is determined. As shown in graph 302, the luminance measurements vary around a received nominal luminance value 318. The received nominal luminance value 318 may be determined based on luminance measurements corresponding to multiple image frames. For example, an average value of luminance measurements for a specific number of frames may be determined (e.g., 60 frames). The received nominal luminance value 318 may represent a luminance measurement when the displayed frame has a luminance level equal to nominal value N. For convenience of calculation, the vertical axis of FIG. 5 may be selected so that the current received nominal luminance value 318 is aligned with zero. Note that the horizontal axis (time) of FIG. 3B corresponds to the horizontal axis of FIG. 3A.

[0044] The brightness measurements may depend on the position of the user 104 (e.g., distance from the display device 108), the reflectance pattern of the user's face, and the settings (e.g., exposure) of the image sensor 106. The exposure settings of the image sensor 106 may be controlled to keep the brightness scale reasonably stable. Each brightness value 312 in the graph 302 is the calculated face brightness (average brightness of the face pixels) minus the received nominal value 318 of FIG. 3B.

[0045] The received nominal value 318 may be affected by multiple factors (e.g., lighting levels in the user's environment). Thus, the computing system 102 may update the value of the received nominal value 318 (e.g., continuously or periodically). The computing system 102 may calculate an average value of multiple samples of face luminance. For example, a FIFO queue of 60 samples may be collected one second before the first modulation pulse is detected. The received nominal value 318 may correspond to the average value of these 60 samples. For each calculation of an average value to set the received nominal value 318, a corresponding standard deviation S is calculated. An upper noise bound (UB) 314 and a lower noise bound (LB) may be determined as a function of the standard deviation. In some aspects, the UB 314 is set to three times the standard deviation S. The LB 316 may be set to minus three times the standard deviation. In some aspects, the received nominal value 318 measurement may be updated by pushing each subsequent face luminance measurement that is less than UB 314 and greater than LB 316 into a FIFO queue and recalculating. Each luminance value 312 greater than or equal to UB corresponds to the highest luminance shown in graph 306. Each luminance value 312 less than or equal to LB corresponds to the lowest luminance shown in graph 306. The luminance measurements corresponding to the image frames may be used to determine a time delay as further described below.

[0046] In some aspects, the image sensor 106 may employ a rolling shutter mechanism, where the exposure of each row of pixels of the image sensor 106 is slightly delayed from the exposure of the preceding row of pixels. Determining the time delay using image frames acquired from an image sensor employing a rolling shutter mechanism is described below.

[0047] Figure 4A is a schematic diagram illustrating a row of rolling shutter image sensors operating at a full exposure time, Figure 4B is a schematic diagram illustrating a row of rolling shutter image sensors operating at a half exposure time, and Figure 4C is a schematic diagram illustrating a row of rolling shutter image sensors operating at a short exposure time, according to some embodiments.

[0048] 4A-4C show a first image frame 406a and a second image frame 406b. As shown in Figures 4A-4C, readout proceeds from one row to another sequentially from top to bottom, with no overlap in readout times for different rows. Each row 408 is slightly delayed relative to the preceding row.

[0049] As mentioned above, some of the pixels are associated with the face of the user 104. In the example shown in FIGS. 4A-4C, the middle rows (e.g., 25%-75%) may correspond to the face of the user 104. In FIGS. 4A-4C, the rows corresponding to the user's face are indicated by a hatched pattern. Each row 410 indicates a reset time 408a, an exposure time 408b, and a readout time 408c.

[0050] 4A-4C show rows associated with the face of user 104 in a horizontal direction. However, it is understood that the pixels may also be in a vertical direction due to the orientation of the image sensor (e.g., a 90 degree change in the orientation of image sensor 106).

[0051] As described above, a reception timestamp may be associated with each image frame. The reception timestamp is the time when the readout of the corresponding frame is completed. A first reception timestamp RT1 may correspond to the time when the readout of the first image frame 406a is completed. A second reception timestamp RT2 corresponds to the time when the readout of the second image frame 406b is completed.

[0052] For each image frame, the average luminance value of the face pixels (face luminance) is determined. The face luminance sampling occurs during the time indicated by the shaded portion of row 408 in Figures 4A-4C. Each face luminance measurement can be located in time by the time coordinate of the centroid of the shaded region that produced it. The centroid is shown as label C in Figures 4A-4C.

[0053] The time coordinate of the center of gravity precedes the corresponding received timestamp by an offset. The offset may depend on (1) the percentage of the frame period from the end of the first row readout to the end of the last row readout, (2) the exposure rate, and (3) the vertical position of the face (or the horizontal position if the sensor rows are vertical stripes across the subject's face). In some aspects, the user's face may be assumed to be at the center of the image frame. Variations in the offset due to the position of the face may be ignored. In other aspects, the position of the face may be determined using facial recognition techniques. An adjusted offset for each of the image frames may be determined based on the determined position of the face.

[0054] In some cases, the time coordinate of the center of gravity (T C ) can be determined as follows: T C =T RX -(0.5×O×FP)-(0.5×E×FP)=T RX -(0.5×FP)×(O+E), In the formula, T RXwhere is the received timestamp, O is the overlap (i.e., the percentage of the frame period from the end of the first row readout to the end of the last row readout), E is the exposure value, and FP is the frame period (e.g., 16.7 milliseconds when the image sensor 106 is operating at 60 FPS). The overlap O and exposure E may depend on the type and / or mode of operation of the image sensor 106. In some aspects, the overlap O and / or exposure E may not be readily known. For example, the value of overlap O for a particular camera sensor design depends on the speed of the shared readout circuitry. If the readout of the last row completes just before switching to the readout of the first row of the next frame, the maximum overlap may approach 100%. The minimum overlap may correspond to faster readout circuitry. Cost and radio frequency (RF) emission considerations motivate chip designers not to make circuits faster than necessary. For commonly used image sensors (e.g., in laptops, smartphones), an overlap equal to 0.9 may be used. However, the overlap can range from about 0.8 to about 1.0.

[0055] The exposure value E can range from about 0.1 to about 1.0. Good exposure control with typical ambient lighting is near the middle of this range, so exposure E can be assumed to be equal to 0.55. Thus, T C The selected value of can be expressed as: T C =T RX -(0.5×FP)×(0.9+0.55)=T RX -0.725×FP T C The earliest value of is T C =T RX -(0.5×FP)×(1.0+1.0)=T RX -1.0×FP. T C The slowest value of is T C =T RX -(0.5×FP)×(0.8+0.1)=T RX -0.45×FP.

[0056] In some aspects, the image sensor 106 may employ a global shutter mechanism in which all pixels of the image sensor 106 are exposed at once. C ) can also be determined using the techniques described above by using overlap equal to 0.

[0057] 3B, the received nominal value 318 may be used to detect rapid changes in indoor illuminance. If the received nominal value changes too much and / or too quickly, ongoing measurements of eye behavior may be stopped and restarted.

[0058] Instruct the display device 108 to change the brightness level (T TX ) to the earliest detection of a luminance change at the image sensor 106. In particular, the earliest time that the large down transition in the middle of each up / down pulse pair in the modulation begins to affect the image sensor 106, T ED Thus, the time delay between the monitor display pattern transmitted by the software and the appearance of that pattern's effect in the subject's video received by the software is T D =T ED -T TX is.

[0059] The face luminance sample of an up / down pulse pair is determined by looking for two consecutive samples above UB 314. All subsequent samples are added to this sequence of two samples until a sample below LB 316 is followed by a sample above LB 316. The last sample above LB 316 is not used. The remainder of the sequence, usually six samples, is fit to a third-order polynomial.

[0060] In some aspects, T ED is determined by fitting the face luminance samples of each up / down pulse pair to a third order polynomial (e.g., by regression analysis using least squares), as shown in FIG.

[0061] 5 is a diagram 500 illustrating fitted luminance measurements, according to some embodiments. Graph 504 shows the luminance measurements. Graph 502 shows the fitted data. The horizontal axis represents the T C The vertical axis may correspond to the vertical axis of FIG. 3B, but with a specific example of scale. The zero crossing time coordinate of the center of this cubic polynomial is T ED The timestamp T of the first frame of the transmitted down pulse TX From the time delay T D =T ED -T TX is calculated. This is done for each up / down pulse pair.

[0062] In some aspects, the time delay T ED The technique for determining T has the advantage of being robust to errors. For example, the time delay can be determined even if one or more frames may be missed (e.g., a frame is not displayed by the display device 108 and / or an image frame is not transmitted by the image sensor 106). This is because four samples are sufficient to determine a cubic polynomial. Even if one or two frames are missed, T ED A useful value of σ is obtained. Using a larger number of frames (e.g., 5 or 6) has the advantage of being more resistant to fluctuations caused by noise in the image frames.

[0063] To further illustrate, the following is an example of an algorithm by which the time delay determination module 110 determines a time delay, according to some aspects. TIFF2026504009000001.tif242153

[0064] As described in step 1 of the algorithm, the time delay determination module 110 sequentially applies a face detection algorithm to each frame of video captured by the image sensor 106 to find a face in each frame. If the time delay determination module 110 determines that a face has been detected with high confidence, it sets the variable FACE to true for that frame; otherwise, it sets the variable FACE to false for that frame. Any of a wide variety of well-known techniques for performing face detection on images may be used to implement the foregoing aspects of this example algorithm.

[0065] As noted in step 2 of the algorithm, the time delay determination module 110 sets the background luminance to TX Nominal (corresponding to N in FIG. 3A) and begins the test sequence (e.g., presenting multiple frames containing visual stimuli to the user 104), at which point the variable ACTIVE is set to true. At the end of the test sequence, the time delay determination module 110 may set the variable ACTIVE to false.

[0066] As stated in step 3 of the algorithm, the time delay determination module 110 determines the variable T D For example, the time delay determination module 110 may initialize a variable T D may be set to 0.

[0067] As mentioned in step 4, the time delay determination module 110 sets up an empty TX FIFO queue. The time delay determination module 110 utilizes the TX FIFO to determine the time T TX Store.

[0068] Step 5 of the algorithm describes a process for determining initial values ​​for RX Nominal (corresponding to 318 in FIG. 3B), UB, and LB. For example, as shown in the algorithm, if the variables ACTIVE and FACE are true, the time delay determination module 110 may set up a 60-element FIFO queue. The FIFO queue may be used to collect luminance measurements. The time delay determination module 110 determines 60 luminance measurements and stores the measurements in the 60-element FIFO queue. The time delay determination module 110 determines initial values ​​for RX Nominal, UB, and LB. For example, the time delay determination module 110 determines the average value of the 60 luminance measurements and sets the value of RX Nominal to that average value. Furthermore, the time delay determination module 110 determines values ​​for UB and LB as a function of RX Nominal.

[0069] In this embodiment, steps 6 and 7 operate in parallel.

[0070] Step 6 of the algorithm describes the process of controlling the brightness level of the display device. For example, as shown in the algorithm, while ACTIVE and FACE are true, the brightness level of the background of the display device 108 is set to +50% from TX Nominal for a specific duration. In this example, the duration is 3 frames. As further described in step 6, the time delay determination module 110 sets the brightness level of the background of the display device 108 to -50% from TX Nominal for a specific duration (e.g., 3 frames). The time delay determination module 110 determines the brightness level of the background of the display device 108 at time T, which corresponds to the first frame at -50% brightness. TX into the TX FIFO. The time delay determination module 110 then sets the background luminance level to TX Nominal for a specific duration (e.g., 3 frames).

[0071] Step 7 of the algorithm describes the process of collecting luminance measurements and determining a time delay based on the collected luminance measurements. As noted in step 7 of the algorithm, while ACTIVE and FACE are true, the time delay determination module 110 may collect a luminance value B. In this example, the luminance value is the average luminance of the face pixels minus the value of RX Nominal. If the luminance value B is greater than LB but less than UB, the average luminance of the face pixels is pushed into a 60-element FIFO queue. The time delay determination module 110 then calculates new values ​​for RX Nominal, UB, and LB.

[0072] As stated in step 7 of the algorithm, if the brightness value B is greater than or equal to UB, the time delay determination module 110 monitors the brightness value B of the subsequent frame. If the brightness value B of the subsequent frame is greater than or equal to UB, the brightness values ​​B of both frames and the corresponding T C are stored in the list in sequence. As mentioned in step 7, the time delay determination module 110 C 3. Collect each subsequent brightness value B corresponding to a subsequent frame having a T C are stored in the sequential list until one or more criteria are met. For example, the time delay determination module 110 may stop collecting brightness values ​​B if the brightness value of the last sample is less than or equal to LB and the brightness value of the subsequent sample is greater than LB. The time delay determination module 110 may stop collecting brightness values ​​B if the length of the sequential list exceeds a threshold. In this example, the threshold is 6. If the brightness value of the last sample is less than or equal to LB and the brightness value of the subsequent sample is greater than LB, the time delay determination module 110 may stop collecting brightness values ​​B using the sequential list. ED The time delay T is then determined or updated. D is T ED From T TX The time delay determination module 110 determines the time delay from the TX FIFO minus T TX Pull.

[0073] In some aspects, one or more of the steps may be performed in parallel.

[0074] 6 is an exemplary method for estimating a delay from a monitor output to a sensor according to one embodiment of the present disclosure. Method 600 may be performed as a series of steps by a computing unit, such as a processor. For example, method 600 may be implemented by computing system 102 and / or computer system 700 of FIG. 7. It should be understood that not all steps are required to practice the disclosure provided herein. Furthermore, as will be appreciated by those skilled in the art, some of the steps may be performed simultaneously or in a different order than that shown in FIG. 6.

[0075] Although the method 600 is described with reference to FIG. 1, the method 600 is not limited to that exemplary embodiment.

[0076] At 602, the computing system 102 may present multiple frames to a display device.

[0077] At 604, the computing system 102 may adjust the brightness level in one or more frames of the plurality of frames.

[0078] At 606, the computing system 102 may receive multiple image frames of a user's face, the multiple image frames being obtained from a video stream recorded by an image sensor while presenting the multiple frames on a display device.

[0079] At 608, the computing system 102 may process the plurality of image frames to determine a time delay between adjusting the brightness level in one or more frames of the plurality of frames and when adjusting the brightness level in the one or more frames of the plurality of frames is reported by the image sensor.

[0080] In some aspects, the computing system 102 may identify a set of pixels within an image frame of the plurality of image frames. The set of pixels may correspond to a user's face. The computing system 102 may determine a luminance measurement for the set of pixels. In some aspects, the luminance measurement corresponds to an average of the luminance values ​​of each pixel in the set of pixels. Each luminance measurement is determined at a respective luminance measurement time.

[0081] In some aspects, the computing system 102 may identify a first set of image frames from the plurality of image frames. The first set of image frames is identified such that brightness measurements corresponding to each image frame included in the first set exceed an upper threshold. The upper threshold is greater than a nominal brightness level. The computing system 102 may identify a second set of image frames from the plurality of image frames. The second set of image frames may be identified such that brightness measurements corresponding to each image frame in the second set are less than a lower threshold. The lower threshold is less than the nominal brightness level. The image frames of the first set and the second set are consecutive frames. The first set of image frames corresponds to a first set of frames whose brightness levels are changed to a first brightness level. The second set of image frames corresponds to a second set of frames whose brightness levels are changed to a second brightness level that is lower than the lower threshold.

[0082] In some aspects, the computing system 102 may determine a transition detection time based on a luminance measurement time for at least each of the frames of the first image frame set and the second image frame set. The computing system 102 may also determine a time delay as a function of the transition detection time and the adjustment time. The adjustment time corresponds to a time at which adjusting the luminance level in one or more frames of the plurality of frames occurs. In some aspects, the time delay is a difference between the transition detection time and the adjustment time. The transition detection time corresponds to a time at which a change in luminance level from a first luminance level to a second luminance level is detected by the image sensor. In some aspects, the luminance measurement time is determined based on at least one of a reception timestamp of the received frame, a frame period overlap, an exposure value of the image sensor, or a duration of the image frame.

[0083] In some aspects, the computing system 102 may adjust a parameter based on the time delay. The parameter is associated with a user's response to a stimulus presented in multiple frames. For example, the parameter may be an oculometry parameter. The oculometry parameter may be determined from eye data acquired while the stimulus is presented on a display device. For example, a time component of the oculometry parameter may be shifted to compensate for the time delay (e.g., tx, where x is the determined time delay). Thus, the accuracy of the oculometry parameter is improved. Furthermore, one or more digital markers indicative of the user's neurological or mental health status may be obtained from the oculometry parameter.

[0084] In some aspects, the display device 108 may include an array of individually controllable pixels associated with multiple colors. For example, the display device 108 may include a triad (red, green, blue) or other combination of color components. Each color component of each pixel is typically made up of multiple subpixels. As used herein, the term "pixel" is used to describe a triad of subpixels. In some aspects, the display device 108 may include multiple light-emitting diodes (LEDs). Each LED is a light source that can be a white LED or one of various colors, such as red, green, or blue. In some aspects, the brightness of a pixel corresponding to a first color of the multiple colors can be individually controlled. In some aspects, a light source driver coupled to an LED or group of LEDs can control the brightness signaling of the LED or group of LEDs. The brightness can be controlled via analog current or voltage techniques or via digital pulse-width modulation (PWM).

[0085] In some aspects, the red component of a pixel may be changed (i.e., adjusted as described herein) without changing the green or blue components. For example, the luminance of a pixel associated with the red component may be adjusted from −50% to +50%.

[0086] In some aspects, adjusting the brightness of the red component of a display pixel provides advantages. As described in "Quantitative Analysis of Skin using Diffuse Reflectance for Non-invasive Pigments Detection" by Li et al., red light reflects much better from dark, tanned skin than green or blue light. This provides the advantage of increasing the signal-to-noise ratio for detecting brightness adjustments using an image frame (i.e., based on reflections from the user's face). Furthermore, adjusting the brightness of the red pixel component is less noticeable to the user than grayscale adjustments. This is due to the color sensitivity of the human eye. Therefore, larger amplitude adjustments can be used without disturbing the user, increasing the signal-to-noise ratio.

[0087] 7 illustrates a computer system 700, according to some embodiments. Various embodiments and components thereof can be implemented using, for example, the computer system 700 or any other known computer system. For example, the method steps of FIG. 6 may be implemented via the computer system 700.

[0088] In some aspects, computer system 700 may include one or more processors (also referred to as central processing units, or CPUs), such as processor 704. Processor 704 may be connected to a communication infrastructure or bus 706.

[0089] In some aspects, one or more processors 704 may each be a graphics processing unit (GPU). In one aspect, a GPU is a processor that is a dedicated electronic circuit designed to handle mathematically intensive applications. A GPU may have a parallel structure that is efficient for parallel processing of large blocks of data, such as mathematically intensive data common in computer graphics applications, images, video, etc.

[0090] In some aspects, computer system 700 may further include user input / output device(s) 703, such as a monitor, keyboard, pointing device, etc., that communicate with a communications infrastructure 706 via user input / output interface(s) 702. Computer system 700 may further include main or primary memory 708, such as random access memory (RAM). Main memory 708 may include one or more levels of cache. Main memory 708 stores control logic (e.g., computer software) and / or data.

[0091] In some aspects, the computer system 700 may further include one or more secondary storage devices or memories 710. The secondary memory 710 may include, for example, a hard disk drive 712 and / or a removable storage device or drive 714. The removable storage drive 714 may be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, a tape backup device, and / or any other storage device / drive. The removable storage drive 714 may interact with a removable storage unit 718. The removable storage unit 718 may include a computer-usable or readable storage device on which computer software (control logic) and / or data is stored. The removable storage unit 718 may be a floppy disk, magnetic tape, a compact disk, a DVD, an optical storage disk, and / or any other computer data storage device. The removable storage drive 714 reads from and / or writes to the removable storage unit 718 in a well-known manner.

[0092] In some aspects, secondary memory 710 may include other means, devices, or other techniques that allow computer programs and / or other instructions and / or data to be accessed by computer system 700. Such means, devices, or other techniques may include, for example, removable storage unit 722 and interface 720. Examples of removable storage unit 722 and interface 720 include program cartridges and cartridge interfaces (such as those found in video game devices), removable memory chips (such as EPROMs and PROMs) and associated sockets, memory sticks and USB ports, memory cards and associated memory card slots, and / or any other removable storage unit and associated interface.

[0093] In some aspects, computer system 700 may further include a communications or network interface 724. Communications interface 724 enables computer system 700 to communicate and interact with any combination of remote devices, remote networks, remote entities, etc. (individually and collectively referred to by reference numeral 728). For example, communications interface 724 may enable computer system 700 to communicate with remote devices 728 via communications path 726, which may be wired and / or wireless and may include any combination of a LAN, a WAN, the Internet, etc. Control logic and / or data may be transmitted to and from computer system 700 via communications path 726.

[0094] In some aspects, a non-transitory, tangible apparatus or article of manufacture comprising a non-transitory, tangible, computer-usable or readable medium having control logic (software) stored thereon is also referred to herein as a computer program product or program storage device. This includes, but is not limited to, computer system 700, main memory 708, secondary memory 710, and removable storage units 718, 722, as well as tangible articles of manufacture embodying any combination thereof. Such control logic, when executed by one or more data processing devices (such as computer system 700), causes such data processing devices to operate as described herein.

[0095] Based on the teachings contained herein, it will be apparent to one skilled in the art how to make and use aspects of the present disclosure using data processing devices, computer systems and / or computer architectures other than those shown in Figure 7. In particular, aspects may operate in conjunction with software, hardware, and / or operating system implementations other than those described herein.

[0096] It is to be understood that the phrases or terms used herein are for the purpose of description rather than limitation, as the terms or terms of the disclosure would be interpreted by those skilled in the art in light of the teachings herein.

[0097] It is understood that the Detailed Description section, and not the Summary and Abstract sections, is intended to be used to interpret the claims. The Summary and Abstract sections may present one or more exemplary aspects of the disclosure, but not all, contemplated by the inventor(s), and thus are not intended to limit the scope of the disclosure and the appended claims in any way.

[0098] This disclosure has been described above with reference to functional building blocks illustrating implementations of specified functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of description. Alternative boundaries may be defined so long as the specified functions and relationships thereof are appropriately performed.

[0099] While specific aspects of the present disclosure have been described above, it will be understood that aspects of the present disclosure may be practiced otherwise than as described. The description is intended to be illustrative, not limiting. Thus, it will be apparent to one skilled in the art that modifications can be made to the present disclosure as described without departing from the scope of the claims set forth below.

[0100] The foregoing description of specific embodiments sufficiently reveals the general nature of the present disclosure so that others can readily modify and / or adapt such specific embodiments for various uses by applying knowledge within the skill of those skilled in the art without departing from the general concept of the disclosure and without undue experimentation. Accordingly, such adaptations and modifications are intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein.

[0101] The breadth and scope of protected subject matter should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Claims

1. A computer-implemented method for estimating a time delay for synchronizing one or more parameters, comprising: presenting, by at least one computer processor, a plurality of frames on a display device; adjusting a brightness level in one or more frames of the plurality of frames; receiving a plurality of image frames of a user's face, the plurality of image frames being obtained from a video stream recorded by an image sensor during the presentation of the plurality of frames on the display device; and Processing the plurality of image frames to determine the time delay between when the adjusting of the brightness level in the one or more frames of the plurality of frames occurs and when the adjusting of the brightness level in the one or more frames of the plurality of frames is reported by the image sensor.

2. The treating step comprises: identifying a set of pixels in an image frame of the plurality of image frames, the set of pixels corresponding to the face of the user; and determining luminance measurements for said set of pixels; 10. The computer-implemented method of claim 1, comprising:

3. 3. The computer-implemented method of claim 2, wherein the luminance measure corresponds to an average luminance value of each pixel in the set of pixels.

4. The treating step comprises: determining a luminance measurement value for each frame of the plurality of image frames, each luminance measurement value being determined at a respective luminance measurement time; identifying a first set of image frames from the plurality of image frames, wherein the brightness measurement corresponding to each image frame of the first set exceeds an upper threshold; identifying a second set of image frames from the plurality of image frames, wherein the brightness measurements corresponding to each image frame of the second set are less than a lower threshold; determining a transition detection time based on at least respective luminance measurement times for frames of the first set of image frames and the second set of image frames; and determining the time delay as a function of the transition detection time and an adjustment time, the adjustment time corresponding to a time at which the adjusting of the brightness level occurs in the one or more frames of the plurality of frames; Including, 10. The computer-implemented method of claim 1.

5. 5. The computer-implemented method of claim 4, wherein the time delay is the difference between the transition detection time and the adjustment time.

6. the first set of image frames corresponds to a first set of frames in which the brightness level is changed to a first brightness level; the second set of image frames corresponds to a second set of frames in which the brightness level is changed to a second brightness level; and the transition detection time corresponds to a time at which the image sensor detects a change in the luminance level from the first luminance level to the second luminance level.

5. The computer-implemented method of claim 4.

7. 5. The computer-implemented method of claim 4, wherein the luminance measurement time is determined based on at least one of a reception timestamp of an image frame, a frame period overlap, an exposure value of the image sensor, or a duration of the image frame.

8. The computer-implemented method of claim 1 , wherein the time delay includes at least a network delay.

9. the display device includes an array of individually controllable pixels associated with a plurality of colors, and the step of adjusting the brightness level comprises: controlling the luminance of a pixel corresponding to a first color of the plurality of colors; 10. The computer-implemented method of claim 1, comprising:

10. adjusting a parameter based on the time delay, the parameter being associated with a response of the user to stimuli presented in the plurality of frames; 10. The computer-implemented method of claim 1, further comprising:

11. one or more memories; at least one processor each coupled to at least one of the memories; Presenting a plurality of frames on a display device; adjusting a brightness level in one or more frames of the plurality of frames; receiving a plurality of image frames of a user's face, wherein the plurality of image frames are obtained from a video stream recorded by an image sensor during the presentation of the plurality of frames on the display device; and processing the plurality of image frames to determine a time delay between when the adjusting of the brightness level in the one or more frames of the plurality of frames occurs and when the adjusting of the brightness level in the one or more frames of the plurality of frames is reported by the image sensor; At least one processor configured to 1. A system for estimating a time delay for synchronizing one or more parameters, comprising:

12. To process the plurality of image frames, the at least one processor: Identifying a set of pixels in an image frame of the plurality of image frames, where the set of pixels corresponds to the face of the user; and determining a luminance measurement for said set of pixels; 12. The system of claim 11, configured to:

13. To process the plurality of image frames, the at least one processor: determining a luminance measurement for each of the plurality of image frames, each luminance measurement being determined at a respective luminance measurement time; identifying a first set of image frames from the plurality of image frames, wherein the brightness measurement corresponding to each image frame of the first set exceeds an upper threshold; identifying a second set of image frames from the plurality of image frames, wherein the brightness measurements corresponding to each image frame of the second set are less than a lower threshold; determining a transition detection time based on at least respective luminance measurement times for frames of the first set of image frames and the second set of image frames; and determining the time delay as a function of the transition detection time and an adjustment time, where the adjustment time corresponds to a time at which the adjusting of the brightness level occurs in the one or more frames of the plurality of frames; 12. The system of claim 11, configured to:

14. 14. The system of claim 13, wherein the time delay is the difference between the transition detection time and the adjustment time.

15. the first set of image frames corresponds to a first set of frames in which the brightness level is changed to a first brightness level; the second set of image frames corresponds to a second set of frames in which the brightness level is changed to a second brightness level; and the transition detection time corresponds to a time at which the image sensor detects a change in the luminance level from the first luminance level to the second luminance level.

14. The system of claim 13.

16. the display device includes an array of individually controllable pixels associated with a plurality of colors, and the at least one processor for adjusting the brightness level: Controlling the brightness of a pixel corresponding to a first color of the plurality of colors 12. The system of claim 11, configured to:

17. the at least one processor: adjusting a parameter based on the time delay, wherein the parameter is associated with the user's response to stimuli presented in the plurality of frames; 12. The system of claim 11, further configured to:

18. When executed by at least one computing device, the at least one computing device presenting multiple frames on a display device; adjusting a brightness level in one or more frames of the plurality of frames; receiving a plurality of image frames of a user's face, the plurality of image frames being obtained from a video stream recorded by an image sensor during the presentation of the plurality of frames on the display device; and processing the plurality of image frames to determine a time delay between when the adjusting of the brightness level in the one or more frames of the plurality of frames occurs and when the adjusting of the brightness level in the one or more frames of the plurality of frames is reported by the image sensor; A non-transitory computer-readable medium having stored thereon instructions for performing operations including:

19. The processing comprises: determining a luminance measurement value for each frame of the plurality of image frames, each luminance measurement value being determined at a respective luminance measurement time; identifying a first set of image frames from the plurality of image frames, wherein the brightness measurement corresponding to each image frame of the first set exceeds an upper threshold; identifying a second set of image frames from the plurality of image frames, wherein the brightness measurements corresponding to each image frame of the second set are less than a lower threshold; determining a transition detection time based on at least respective luminance measurement times for frames of the first set of image frames and the second set of image frames; and determining the time delay as a function of the transition detection time and an adjustment time, the adjustment time corresponding to a time at which the adjusting of the brightness level occurs in the one or more frames of the plurality of frames; 20. The non-transitory computer-readable medium of claim 18, comprising:

20. The operation is adjusting a parameter based on the time delay, the parameter being associated with a response of the user to stimuli presented in the plurality of frames; 20. The non-transitory computer-readable medium of claim 18, further comprising:

Images