Methods and systems for measuring brain reflexes, as well as their involvement and lifestyle modulatory effects.

Abstract

This disclosure relates to a method and system for measuring emotional engagement in response to auditory and / or visual stimuli, for example, using auditory startle response, prepulse suppression, spontaneous blinking, and / or eye movements. The method includes displaying a first video or image content. A second video is captured, including one or more of the individual's eyes, while the individual is viewing the first video or image content, and while the second video is being captured, the individual is exposed to one or more visual and / or auditory stimuli. The method may include determining a value representing eye closure based on the second video. The method may include calculating a metric based on these values.

Description

[Technical Field]

[0001] Cross-references to related applications This application claims priority to U.S. Provisional Patent Application No. 63 / 460,451, filed on 19 April 2023, the contents of which are incorporated herein by reference in their entirety.

[0002] This disclosure relates to technologies for determining emotional engagement and lifestyle as well as physical movements, specifically technologies for determining the effects of these on auditory startle responses, prepulse suppression, and spontaneous and anticipatory blinking and eye movements. [Background technology]

[0003] This section is intended to introduce to the reader various aspects of the technology that may relate to the various aspects of the invention described and / or claimed below. This discussion is intended to help provide the reader with background information to facilitate a better understanding of the various aspects of the invention. Therefore, these descriptions should be read in this context and not as endorsement of the prior art.

[0004] Emotional engagement is a crucial factor in evaluating the effectiveness of visual stimuli such as videos or photographs. Lifestyle and physical activity have repeatedly been reported to have beneficial effects on brain function, including learning. However, objective tools for measuring such effects are often lacking. Blink conditioning is a well-characterized method for studying the neural basis of associative and procedural learning. Therefore, this paradigm holds potential as a tool for evaluating the extent to which movement influences one of the most fundamental forms of learning. However, until recently, using this paradigm to test human subjects in everyday life has been technically challenging. As a result, there has been little research on how movement affects human blink conditioning. [Overview of the project] [Problems that the invention aims to solve]

[0005] Traditional methods for measuring the impact of emotional engagement and lifestyle on brain function have relied on subjective and potentially unreliable self-report measurements or observer assessments. Physiological measurements have been proposed as objective methods for assessing emotional engagement. Among these, auditory startle response, prepulse suppression, spontaneous and anticipatory blinking, and eye movements are widely used. However, there is a need for convenient and accessible systems that enable remote assessment of these measurements. [Means for solving the problem]

[0006] Summary of the Invention Various shortcomings in the prior art are addressed by the technologies and systems disclosed below.

[0007] In various embodiments, methods can be provided for measuring emotional engagement while viewing video or image content. These methods may include displaying first video or image content on a display device. While an individual is viewing the first video or image content on the display device, these methods may include: 1) capturing a second video including one or more of the individual's eyes using a camera; and 2) exposing the individual to one or more visual and / or auditory stimuli while capturing the second video. These methods may include determining a value representing eye closure based on the second video. These methods may include calculating a metric (e.g., cognitive load, training effectiveness) based on these values. In some embodiments, this metric may be emotional engagement (e.g., with the first video), and this metric may be determined based on eyelid startle responses. In some embodiments, this metric may be the effect of physical activity, and this metric may be determined based on blink conditioning.

[0008] All steps of this method can be performed on a single local device (such as a desktop computer, laptop computer, mobile phone, or tablet). Some of these steps can be performed using one or more remote processing devices. For example, the steps of determining a value and calculating a metric can be performed by one or more remote processing devices.

[0009] The method may include receiving first information from one or more remote processing devices, which may include first video or image content. The first information may also include information relating to stimuli to which the individual is exposed (e.g., one or more visual stimuli and / or auditory stimuli, values ​​representing one or more visual stimuli and / or auditory stimuli, or both). The method may also include transmitting second information to one or more remote processing devices, which may include a second video.

[0010] The display device and camera may be operably coupled to a headset (such as a virtual reality (VR) headset, an augmented reality (AR) or mixed reality (MR) headset). This headset may be operably coupled to one or more processing units performing this method.

[0011] The metric can be calculated by a machine learning algorithm trained using classified video and / or image content. This metric may be at least partially based on the detected alpha startle response.

[0012] This method may include displaying measurement standards to individuals.

[0013] In various aspects, a system for measuring an emotional engagement while viewing video or image content may be provided. The system may include a display device, a camera, a speaker, a memory, the display device, the camera, the speaker, and one or more processing devices operatively coupled to the memory, and a non-transitory computer-readable storage medium. The storage medium may include instructions that, when executed by the one or more processing devices, cause the one or more processing devices to perform the disclosed method in cooperation.

[0014] The system may be configured as a desktop computer, a laptop computer, a mobile phone, or a tablet. The one or more processing devices may include one or more local processing devices and one or more remote processing devices. The one or more remote processing devices may be configured to perform one or more steps of the method. For example, the one or more processing devices may cooperate to calculate a value and compute a measurement criterion.

[0015] The display device and the camera may be operatively coupled to a headset (such as a virtual reality (VR) headset, an augmented reality (AR) or a mixed reality (MR) headset). The headset may be operatively coupled to one or more processing devices performing the method.

[0016] The accompanying drawings are incorporated herein and form a part of this specification, illustrate embodiments of the present invention, and together with the general description of the present invention described above and the detailed description of the embodiments shown below, serve to explain the principles of the present invention.

Brief Description of the Drawings

[0017] [Figure 1] It is a schematic diagram of the system. [Figure 2] It is a diagram of the headset. [Figure 3] It is a flowchart of the method. [Figure 4] It is an explanatory diagram of a template for tracking facial landmarks, particularly eye landmarks. [Figure 5A]Figures 5A and 5B are graphs showing exemplary normalized closed-eye data determined by AI analysis of facial images when users were exposed only to loud sounds (A) or to loud sounds following quiet sounds (B) while watching videos classified as negative, neutral, or positive. [Figure 5B] Figures 5A and 5B are graphs showing exemplary normalized closed-eye data determined by AI analysis of facial images when users were exposed only to loud sounds (A) or to loud sounds following quiet sounds (B) while watching videos classified as negative, neutral, or positive. [Figure 5C] Figures 5C and 5D are graphs showing the closed-eye state during alpha shock, determined by AI analysis of facial images when users were exposed only to loud noises (C) or to loud noises following quiet noises (D) while watching videos classified as negative, neutral, or positive. [Figure 5D] Figures 5C and 5D are graphs showing the closed-eye state during alpha shock, determined by AI analysis of facial images when users were exposed only to loud noises (C) or to loud noises following quiet noises (D) while watching videos classified as negative, neutral, or positive. [Figure 6A]Figure 6A is a graph of conditioned reflex (CR) amplitude per session, combining paired (CS+US) trials and CS-only trials, for the inactive and active groups, with or without movement before the blink conditioning session. Individuals who performed the activity showed significant conditioning, and the post-activity group showed significantly higher conditioned reflex amplitudes in sessions 1 and 2 compared to the non-activity group. Shading represents the mean standard error. Figures 6B and 6C show graphs of average eyelid traces for the inactive (B) and active (C) groups in three blink conditioning sessions, with or without movement (left panel) and with or without movement (right panel), for paired (CS+US) trials (upper panel) and CS-only trials (lower panel). Lightly shaded blocks indicate CS expression over 450 milliseconds, and darkly shaded blocks indicate actual (with US) or expected (without US) 50 millisecond US expression, which ends simultaneously with CS at 450 milliseconds. In paired trials, note the peak in amplitude after the onset of US, i.e., the unconditioned response (UR) expressed in all groups. Note that the timing of the amplitude increase in paired trials shifts to precede the onset of US in later sessions. This is CR, and is particularly pronounced in the activity group and the post-exercise group. Acquisition of conditioned reflexes over three sessions is also demonstrated by the amplitude increase in trials with CS only, which was also particularly pronounced in the post-exercise activity group. Significance level: *p<0.05, **p<0.01; ns = no significant difference. [Figure 6B]Figure 6A is a graph of conditioned reflex (CR) amplitude per session, combining paired (CS+US) trials and CS-only trials, for the inactive and active groups, with or without movement before the blink conditioning session. Individuals who performed the activity showed significant conditioning, and the post-activity group showed significantly higher conditioned reflex amplitudes in sessions 1 and 2 compared to the non-activity group. Shading represents the mean standard error. Figures 6B and 6C show graphs of average eyelid traces for the inactive (B) and active (C) groups in three blink conditioning sessions, with or without movement (left panel) and with or without movement (right panel), for paired (CS+US) trials (upper panel) and CS-only trials (lower panel). Lightly shaded blocks indicate CS expression over 450 milliseconds, and darkly shaded blocks indicate actual (with US) or expected (without US) 50 millisecond US expression, which ends simultaneously with CS at 450 milliseconds. In paired trials, note the peak in amplitude after the onset of US, i.e., the unconditioned response (UR) expressed in all groups. Note that the timing of the amplitude increase in paired trials shifts to precede the onset of US in later sessions. This is CR, and is particularly pronounced in the activity group and the post-exercise group. Acquisition of conditioned reflexes over three sessions is also demonstrated by the amplitude increase in trials with CS only, which was also particularly pronounced in the post-exercise activity group. Significance level: *p<0.05, **p<0.01; ns = no significant difference. [Figure 6C]Figure 6A is a graph of conditioned reflex (CR) amplitude per session, combining paired (CS+US) trials and CS-only trials, for the inactive and active groups, with or without movement before the blink conditioning session. Individuals who performed the activity showed significant conditioning, and the post-activity group showed significantly higher conditioned reflex amplitudes in sessions 1 and 2 compared to the non-activity group. Shading represents the mean standard error. Figures 6B and 6C show graphs of average eyelid traces for the inactive (B) and active (C) groups in three blink conditioning sessions, with or without movement (left panel) and with or without movement (right panel), for paired (CS+US) trials (upper panel) and CS-only trials (lower panel). Lightly shaded blocks indicate CS expression over 450 milliseconds, and darkly shaded blocks indicate actual (with US) or expected (without US) 50 millisecond US expression, which ends simultaneously with CS at 450 milliseconds. In paired trials, note the peak in amplitude after the onset of US, i.e., the unconditioned response (UR) expressed in all groups. Note that the timing of the amplitude increase in paired trials shifts to precede the onset of US in later sessions. This is CR, and is particularly pronounced in the activity group and the post-exercise group. Acquisition of conditioned reflexes over three sessions is also demonstrated by the amplitude increase in trials with CS only, which was also particularly pronounced in the post-exercise activity group. Significance level: *p<0.05, **p<0.01; ns = no significant difference. [Figure 7A] Figures 7A and 7B are graphs showing the latency distribution to the conditioned reflex peak for the inactive group (A) and active group (B) across all sessions, for trials with or without movement, using only the conditioned stimulus (CS). The dark shaded blocks at 400 milliseconds indicate the expected onset of the unconditioned stimulus (omitted US) that is omitted in these trials. The lightly shaded blocks indicate the presentation of the CS. Note that in all groups, the distribution is approximately centered around the expected onset of the US at 400 milliseconds. [Figure 7B]Figures 7A and 7B are graphs showing the latency distribution to the conditioned reflex peak for the inactive group (A) and active group (B) across all sessions, for trials with or without movement, using only the conditioned stimulus (CS). The dark shaded blocks at 400 milliseconds indicate the expected onset of the unconditioned stimulus (omitted US) that is omitted in these trials. The lightly shaded blocks indicate the presentation of the CS. Note that in all groups, the distribution is approximately centered around the expected onset of the US at 400 milliseconds. [Figure 7C] Figures 7C and 7D are box plots showing the percentage of conditioned reflexes (CRs) occurring at appropriate times in the inactive group (C) and the active group (D), respectively, with and without exercise. The central line represents the median of the group, the ends of the box represent the lower and upper quartiles, the whiskers represent the minimum and maximum values ​​of the group, and the dots represent outliers. ns = no significant difference. [Figure 7D] Figures 7C and 7D are box plots showing the percentage of conditioned reflexes (CRs) occurring at appropriate times in the inactive group (C) and the active group (D), respectively, with and without exercise. The central line represents the median of the group, the ends of the box represent the lower and upper quartiles, the whiskers represent the minimum and maximum values ​​of the group, and the dots represent outliers. ns = no significant difference. [Figure 8A] Figures 8A and 8B show graphs and box plots of the group-mean unconditioned response amplitude for individuals who were inactive (A) with movement (solid line) or inactive (B) with movement (dashed line) before the blink conditioning session. Unconditioned response amplitude was calculated for the first two blocks of Session 1 before the conditioned reflex occurred. The dark shaded blocks from 400–450 milliseconds (upper panel) indicate the presentation of the unconditioned stimulus (US). Note that the unconditioned response amplitude was similar for both the inactive and active groups, regardless of movement condition. In the box plot (lower panel), the central line represents the group median, the ends of the box represent the lower and upper quartiles, and the whiskers represent the group minimum and maximum values. ns = no significant difference. [Figure 8B] Figures 8A and 8B show graphs and box plots of the group-mean unconditioned response amplitude for individuals who were inactive (A) with movement (solid line) or inactive (B) with movement (dashed line) before the blink conditioning session. Unconditioned response amplitude was calculated for the first two blocks of Session 1 before the conditioned reflex occurred. The dark shaded blocks from 400–450 milliseconds (upper panel) indicate the presentation of the unconditioned stimulus (US). Note that the unconditioned response amplitude was similar for both the inactive and active groups, regardless of movement condition. In the box plot (lower panel), the central line represents the group median, the ends of the box represent the lower and upper quartiles, and the whiskers represent the group minimum and maximum values. ns = no significant difference. [Modes for carrying out the invention]

[0018] Please understand that the attached drawings are not necessarily to scale and represent various features illustrating the basic principles of the present invention in a somewhat simplified manner. Specific design features of the series of operations disclosed herein, such as the specific dimensions, orientation, position, and shape of various illustrated components, are determined in part by the specific intended use and operating environment. Certain features of the illustrated embodiments are enlarged or distorted compared to others to facilitate visualization and clear understanding. In particular, thin features may be shown in bold, for example, for clarity and explanatory purposes.

[0019] The following description and drawings are merely illustrative of the principles of the present invention. Those skilled in the art will understand that various modifications can be devised to embody the principles of the present invention and to fall within the scope of the invention, even if they are not expressly described or shown herein. Furthermore, all embodiments referenced herein are expressly intended solely for illustrative purposes to help the reader understand the principles of the present invention and the concepts provided by the inventor(s) to further the art, and should be construed as not being limited to such specifically cited examples and conditions. Furthermore, the term “or,” as used herein, means non-exclusive “or” (e.g., “or, otherwise,” or “or, alternatively”) unless otherwise indicated. Also, since some embodiments can be combined with one or more other embodiments to form new embodiments, the various embodiments described herein are not necessarily mutually exclusive.

[0020] Numerous innovative teachings of this application are described with particular reference to currently preferred exemplary embodiments. However, it should be understood that these embodiments only provide a few examples of the many advantageous applications of the innovative teachings herein. In general, the descriptions made in the specification of this application are not necessarily limited to the various claimed inventions. Furthermore, some descriptions may apply to certain inventive features but not to others. A person skilled in the art who has gained knowledge from the teachings herein will understand that the invention is applicable to a variety of other technical fields or embodiments.

[0021] This disclosure provides methods and systems for measuring the impact or effectiveness of activities based on the measurement of brain reflexes. For example, emotional engagement with visual stimuli or the effectiveness of physical activity can be measured based on brain reflexes. In other words, the disclosed techniques can be used to correlate determined values ​​related to eye movements and blinks to various metrics of interest.

[0022] Brain reflexes are basic, unconscious responses that can be used as indicators of the functional integrity of the nervous system. An important reflex is the acoustically evoked startle reflex of the eyelids, which has been studied for over 50 years. Moreover, this startle reflex can function as an effective unconditioned stimulus (US) in Pavlov's blink conditioning, a well-known method for studying the neural correlation between procedural learning and memory. In blink conditioning, an US that reliably elicits a reflexive blink is repeatedly paired with a conditioned stimulus (CS). Eventually, the CS itself elicits an anticipatory blink, which is called a conditioned reflex (CR).

[0023] Any appropriate CS and / or US may be used. This may include one or more visual stimuli such as specific videos, images, or flashes (such as a front camera flash or a solid white image displayed on a screen). This may include one or more auditory stimuli such as tones or white noise generated at one or more frequencies.

[0024] Previously, evaluating blink conditioning in human participants was relatively difficult. As a result, there have been limited studies using neurobehavioral assays to measure the effects of lifestyle interventions on learning in human subjects.

[0025] In some embodiments, auditory and / or visual stimuli may be configured to produce a startle response. In some embodiments, only unconditioned stimuli may be used, rather than attempting to produce a conditioned reflex.

[0026] Similarly, prepulse inhibition (PPI) can also be used. PPI is a behavioral phenomenon in which the magnitude of the startle response is suppressed when a weaker stimulus (prepulse, e.g., a quieter sound) that does not trigger a startle reflex occurs before a short, large startle stimulus (pulse, e.g., a loud sound). In this way, PPI measures sensorimotor gating, that is, the neural mechanism that protects the brain from overstimulation by filtering out irrelevant sensory information, allowing for an appropriate response to relevant stimuli. PPI has low specificity in brain regions and examines the function of the midbrain and the modulating effects it receives from the limbic system, thalamus, and prefrontal cortex.

[0027] Systems for measuring the impact or effectiveness of activities can be provided in various forms.

[0028] Referring to Figure 1, the system (100) may include one or more devices (110). The system may include a display device (111), a camera (112), a speaker (113), memory (114), one or more processing units (115), and a non-transient computer-readable storage medium (116). In some embodiments, the system may include a microphone (117). The storage medium may contain instructions that, when executed by one or more processing units, cause these processing units to perform a set of steps of a specific method.

[0029] As used herein, the term “processing unit” generally refers to a computing device capable of accepting data and performing mathematical and logical operations according to the instructions of program instructions. This may include any central processing unit (CPU), graphics processing unit (GPU), core, hardware thread, or other processing configurations known or to be developed in the future. The term “thread” is used herein to refer to any software or processing unit or configuration thereof configured to support the simultaneous execution of multiple processes.

[0030] This system can be configured as (or may include) a desktop computer, laptop computer, mobile phone, or tablet.

[0031] In some embodiments, only a device-based processing unit (e.g., a smartphone) is used. In some embodiments, one or more steps may be performed by a remote processing unit. For example, these one or more processing units may include one or more local processing units (e.g., processing unit (115)) and one or more remote processing units (e.g., remote processing unit (120) and / or remote processing unit (141)). Apart from steps that necessarily require interaction with a user (130), various steps may be distributed in any way between the local processing unit and the remote processing unit. For example, in some embodiments, one or more processing units may cooperate to calculate values ​​and calculate metrics. In some embodiments, the remote processing unit (or multiple) (120) may be a cloud-based processing unit. In some embodiments, the remote processing unit (or multiple) (141) may be configured to receive and / or display information to a remote user (140), e.g., a clinician, physician, researcher, etc.

[0032] In some embodiments, the system may include headphones (131) for a user (130) to wear.

[0033] Referring to Figure 2, in some embodiments, a display device (111) and a camera (112) may be operably coupled to a headset (200). The display device and camera may be located within a headset housing (201). This headset may be a virtual reality (VR) headset (e.g., a headset that provides a full virtual experience, where the user can only look at the display devices provided within the headset), an augmented reality (AR) headset (e.g., a headset that overlays computer-generated objects(s) onto live or near-live images of the physical world captured by a camera, so that when these live or near-live images and objects(s) are displayed on the screen, they appear as if they are part of the physical world, and the display screen or other control unit may adjust the augmented reality as changes to the captured images of the physical world indicate an updated viewpoint of the physical world), or a mixed reality (MR) headset (meaning a combination of virtual objects or spaces and physical reality objects; this is closely related to augmented reality, but may include, for example, projecting an actual image of a person in a different physical location, capturing an image of that person using a camera, and then overlaying that person into a different physical environment using augmented reality).

[0034] As shown in Figure 2, the headset may have a strap (202) configured to hold the headset on the user's head. The headset may be operably coupled to one or more processing units performing the method, either wirelessly or via a wire. In Figure 2, a wire (211) is used to couple the headset (200) to a housing (210) which includes memory (114), processing unit(s) (115), and a non-transient computer-readable storage medium (116).

[0035] This processing device(s) may be configured to coordinate various steps of the method. Referring to Figure 3, the method(300) may optionally include receiving first information(310) from one or more remote processing devices. This first information may include information defining or relating to a video or image that may be displayed to the user. In some embodiments, the video or image to be displayed is the received one. For example, the researcher may send the video or image directly to the user's terminal, or the researcher may send a URL to the user's device, which can then process the URL to download the video or image found at that URL and save it for later use. The researcher may also send information describing the length and intensity of a prepulse or pulse used for stimulation. In some embodiments, the video or image to be displayed is determined randomly.

[0036] The first piece of information may also include information relating to the stimuli to which the individual is exposed (e.g., one or more visual stimuli and / or auditory stimuli, values ​​representing one or more visual stimuli and / or auditory stimuli, or both).

[0037] The method may include testing the user's brain reflexes (320). This may be done by displaying a first video or image content (which may be a video or image received in the receiving (310) step, or an image or video already on the device) (322) on a display device (e.g., a display device (111)).

[0038] This examination may include capturing a second video (324) (for example, with a camera (112)) that includes one or more of the individual's eyes while the individual is viewing a first video or image content on the display device.

[0039] This test may include exposing the individual to one or more visual and / or auditory stimuli while capturing a second video (326). Any suitable visual or auditory stimulus may be used. In some embodiments, flashing a camera or display device briefly in bright white may be used as a visual stimulus. In some embodiments, a tone or white noise, such as a beep, may be used as an auditory stimulus. In some embodiments, only a visual or auditory stimulus is used. In some embodiments, both a visual and an auditory stimulus are used.

[0040] The second video may capture video of the time before, during, and after stimulation. The time after stimulation can be up to 500 milliseconds.

[0041] This method may optionally include transmitting a second piece of information to a remote processing device (330), the second piece of information including a second video.

[0042] This method may include determining values ​​representing eye-related movements such as eye closure based on a second video (340). This may also include determining values ​​representing blink amplitude, blink duration, and blink timing based on a second video including one or more eyes of an individual. Blinking triggered by the presentation of blink-inducing stimuli, such as unexpected loud noises or visual stimuli, may be measured. In addition, spontaneous blinking may also be measured.

[0043] In some embodiments, these eye-related movements may include spontaneous blinking. In some embodiments, these eye-related movements may include reflexive blinking. Generally, spontaneous blinking occurs without external stimulus and / or internal effort, while reflexive blinking typically occurs in response to an external stimulus. A type of reflexive blink is anticipatory blinking, which may occur during blink conditioning.

[0044] In some embodiments, eye-related movements may include tracking of eye position. This tracking of eye position may include tracking (i) rapid eye movements (saccadic and microsaccadic movements), (ii) smooth tracking movements, and / or (iii) vestibular eye movements. In preferred embodiments, when eye position tracking is utilized, the device is configured to utilize a VR-type viewer as described herein.

[0045] In some embodiments, eye-related movements may include tracking pupil size to measure the user's level of alertness. As is known in the art, pupil size decreases as alertness decreases. By analyzing captured images to measure pupil diameter and, by choice, normalizing them, pupil size can be tracked over time to determine whether the user is sufficiently alert. In some embodiments, alertness is determined by comparing pupil size to other pupil size measurements collected during the user's examination. In some embodiments, the degree of alertness is determined by comparing the measured pupil size to a threshold.

[0046] In some embodiments, conditioned pupillary response can be measured using tracking the size of the pupil of the eye. This is similar to blink conditioning, but instead of eyelid position, pupil size is measured. That is, after experiencing conditioned and unconditioned stimuli, an image including the pupil is captured, the pupil diameter is measured, and preferably this is normalized, similar to what is done using FEC in blink conditioning.

[0047] For example, computer vision and image processing techniques can be used to detect landmarks on human faces fully automatically and in real time. More preferably, this algorithm can be optimized to provide fast and accurate tracking of eyelids in both adults and infants. Here, suitable techniques known for training machine learning algorithms can be utilized.

[0048] An algorithm can be used to detect multiple landmarks on the face. Figure 4 shows an example of a template (400) using 68 landmarks. In some embodiments, the template (400) may include or consist of six landmarks for each eyeball captured in the image. These four landmarks are the left corner (401), the upper left eyelid mark (402), the upper right eyelid mark (403), the right corner (404), the lower right eyelid mark (405), and the lower left eyelid mark (406), as seen in Figure 4. As will be readily apparent to those skilled in the art, other templates and / or mesh models can be easily incorporated here, but this is merely a simplified example.

[0049] Once landmarks are identified, calculations become possible. As an example, the closed-eye percentage (FEC) can be calculated for each image. Using six preferred landmarks as examples, conceptually, this calculation is performed by looking at the differences in the positions of these six points, in particular,

number

[0050] As is understood in the art, while the use of FEC is described, other known techniques for determining values representing eye-related movements may be used as appropriate. For example, the blend shape coefficients of Apple ARKit and MediaPipe can provide coefficients (generally values between 0.0 and 1.0) for detected expressions, including eye blinks of the right and left eyes (eyeBlinkRight and eyeBlinkLeft respectively).

[0051] In some embodiments, when two eyes are detected, various techniques may be used. FEC can be calculated for each eye, and the results can be, for example, averaged together (or otherwise statistically combined). FEC can be calculated for each eye, and the minimum value can be utilized. FEC can be calculated for each eye, and the maximum value can be utilized. FEC can be calculated for each eye, and the difference between those two FEC values can be determined. If that difference exceeds a threshold, the value of a flag can be set to 1, or a variable can be incremented, indicating that an abnormal response has occurred.

[0052] In some embodiments, if no eyes are detected in a given image, or if more than two eyes are detected, that image can be skipped.

[0053] A calibration sequence may be performed before these steps, and the FEC MIN value and the FEC MAX value can be determined based on images or videos captured during calibration. In some embodiments, the FEC MIN value and the FEC MAX value can be determined based only on images or videos captured as part of the above-described inspection.

[0054] Referring briefly to FIG. 5A, when a user is exposed to a stimulus (such as an unexpected loud noise), that individual may close their eyes to some extent. An alpha startle (501) can occur in response to the loud noise. Also, a beta startle (502) response can occur some time after the alpha startle.

[0055] This method may include calculating a measurement standard based on these values ​​(350).

[0056] The calculation of metrics (such as cognitive load) can be performed by a machine learning algorithm trained using classified video and / or image content. Thus, in some embodiments, the method may include training a machine learning algorithm (360).

[0057] In some embodiments, this criterion can be calculated by comparing this value, which represents eye-related movement, to a calibration curve or a predetermined threshold range. These calibration curves or threshold ranges may be individual-specific or general calibration curves or threshold ranges applicable to multiple users.

[0058] As an example, in the case of emotional involvement, in some embodiments, the calibration curve or threshold range may be determined by showing a user (or multiple users) multiple randomized videos or images. These multiple randomized videos or images may include at least one video or image known to have a positive valence (e.g., a calming video or a cute image) and at least one video or image known to have a negative valence (e.g., an angering image or a frightening video). During the examination of each video or image in calibration, the user is exposed to auditory and / or visual stimuli (preferably the same stimuli intended to be used during a normal examination), and eye-related movements in response to these stimuli are detected and measured. After sufficient values ​​representing eye-related movements have been determined for multiple videos in the calibration sequence, the calibration curve or threshold range can be determined. The calibration curve or threshold range can then be used to correlate the eye-related movements with the degree to which the examination video or examination image elicits an emotional (positive or negative) response in the user.

[0059] In some embodiments, this criterion may be at least partially based on the detected alpha-shock response. In some embodiments, this criterion may be at least partially based on the detected beta-shock response.

[0060] In some embodiments, this criterion may be emotional engagement (e.g., with the first video), and this criterion may be determined based on the startle response of the eyelids. In some embodiments, this criterion may be the effect of physical activity, and this criterion may be determined based on blink conditioning.

[0061] Figures 5A–5D are graphs showing examples where participants were shown short video clips with neutral, positive, or negative valencies, for example, on a smartphone, and then exposed to a single pulse and optionally a prepulse stimulus, which were bright light (camera flash) and loud noise (white noise). The video clips were obtained from the Database of Emotion Videos from Ottawa (DEVO). Statistically significant differences can be detected between the three valencies regarding the degree of eye closure experienced when exposed to loud noise with or without a prepulse (Figures 5B, 5D). Furthermore, it was noted that when watching videos that evoked feelings of happiness, users who were emotionally engaged responded less to the stimulus than those who were not emotionally engaged. When users were less emotionally engaged, they responded more. In addition, when the video was frightening, emotionally engaged users evoked anxiety responses.

[0062] Therefore, by using the amount of eye closure (and / or blinking speed, auditory startle response, etc.), metrics (e.g., emotional engagement, motor sufficiency) can be determined. In some embodiments, additional metrics can be determined using these determined metrics. For example, a score for a video may be determined as the average of emotional engagement determined by the amount of eye closure across multiple individuals who watched the video. Thus, by exposing a group of individuals to multiple videos or images and exposing them to stimuli while they are watching these videos or images, it is possible to determine which video or image generated the most emotional engagement.

[0063] All steps of this method can be performed on a single local device (such as a desktop computer, laptop computer, mobile phone, or tablet). Some of these steps can be performed using one or more remote processing devices. For example, the steps of determining a value and calculating a metric can be performed by one or more remote processing devices.

[0064] In some embodiments, instructions on a storage medium may cause the processing unit(s) to include two parts or modules: an inspection module and an analysis module.

[0065] The testing module presents a visual stimulus to the participant and records the physiological response (see Figure 3, step (320) of the testing). As disclosed herein, this visual stimulus may be, for example, a video, a photograph, or any other type of visual content. These recordable physiological responses include, for example, auditory startle response, prepulse suppression, and spontaneous blinking and eye movements. All of these responses may be measured by the analysis module during the execution of the method.

[0066] This disclosure provides a convenient and user-friendly system for measuring emotional engagement with visual stimuli. The system enables remote testing and analysis, making it suitable for use in a variety of settings, including research, marketing, and clinical applications.

[0067] By using auditory startle response, prepulse suppression, and spontaneous blinking and eye movements, a variety of objective and reliable metrics can be provided, serving the benefit of the individual being tested, for example, for self-improvement or diagnostic purposes. The effects of physical activity can be measured to determine, for example, whether the physical activity was effective or whether the activity level was sufficient to produce a detectable benefit. Alternatively, this may include enabling the quantification of emotional engagement, which can be used to optimize the effectiveness of visual stimuli. [Examples]

[0068] participants Forty neurologically normal participants aged 18 to 40 were recruited via social media invitation and participated in the study. This sample size is consistent with other blink conditioning studies in humans. Participants were divided into active or inactive groups based on their weekly physical activity time. The split point for group classification was determined using the lower limit of the WHO guidelines for physical activity in adults aged 18–64 years. Participants engaging in less than 2.5 hours of moderate-intensity exercise or less than 75 minutes of vigorous-intensity exercise were classified as the inactive group, while all other participants were classified as the active group. Moderate intensity was defined as "exercise that raises heart rate but still allows for conversation," and vigorous intensity was defined as "exercise that raises heart rate to the point where conversation is not possible." Educational levels were nearly the same across groups, as all participants were either college graduates or college students. Furthermore, mean age and average nightly sleep duration were also nearly the same across groups (see Table 1).

[0069] [Table 1]

[0070] experiment The experiment was conducted via a smartphone application. During the experiment, participants watched either a standardized audio nature documentary (n=109 views) or television programs such as The Office (n=6 views), Modern Family (n=3 views), and Coco (n=2 views). This study used the delayed blink conditioning paradigm, a form of cerebellar associative learning. The blink conditioning experiment consisted of pairings of CS and US (here, a loud white noise and the activation of a camera's selfie flash). CS (here, a white dot) was presented in the center of the phone screen for 450 milliseconds. In paired trials, US was presented 400 milliseconds after the start of CS and ended simultaneously with CS. In US-only trials, the stimulus was presented for 50 milliseconds within 400 milliseconds from the start of the trial. Each blink conditioning session consisted of 10 blocks and a preblock provided at the start of each session. This preblock consisted of three CS-only trials and two US-only trials. Within each block, blocks 1 through 10 consisted of eight paired trials, one CS-only trial, and one US-only trial, all distributed semi-randomly across the entire block.

[0071] Experimental procedure Participants were instructed to use headphones and complete the experiment in a quiet, well-lit room. All participants completed four sessions of the experiment over a one-week period, and no sessions were conducted on the same day. Session 1 was an introductory session in which participants received remote guidance and became familiar with the procedure of the experiment. This session did not include the blink conditioning paradigm. For the sake of brevity, this example will focus solely on the blink conditioning paradigm, and from here on, sessions 2-4 will be referred to as sessions 1-3, and session 1 as the first blink conditioning session.

[0072] group with exercise Participants were randomly assigned to either the exercise group or the non-exercise group, based on their activity level. Participants in the exercise group were instructed to perform at least 30 minutes of moderate-intensity running or cycling, followed by all blink conditioning sessions as soon as possible. Participants in the non-exercise group were instructed to refrain from exercise for at least 8 hours prior to the test. Before starting the blink training sessions, participants were asked to rate the intensity of their exercise on a 5-point Likert scale within the app (see Table 1).

[0073] Data processing Data processing was performed using R 4.3.1. Trials were baseline-corrected using a 500-millisecond no-stimulus baseline, and the minimum-maximum values ​​were normalized using spontaneous blinking as the reference. For individual eyelid tracks, each track was normalized by dividing it by the maximum signal amplitude of the relevant session. Thus, a value of 1 corresponded to the closed eye state, and a value of 0 corresponded to the open eye state.

[0074] Trials with extreme outliers, and trials in which spontaneous blinking occurred within a time window of 150 milliseconds before to 35 milliseconds after stimulus presentation, were excluded from further analysis. The trials were then re-corrected for baseline using the same time window used to remove spontaneous blinking.

[0075] In Session 0, the mean baseline CR amplitude was measured for each subject. Session 0 was defined as the CS-only trials prior to the block from Session 1. CR amplitude was measured as the maximum signal amplitude at 430 milliseconds for both paired trials and CS-only trials. This time value was chosen to allow for a 30-millisecond delay 400 milliseconds after the expected US presentation. There was a delay in the response to US (Supplementary Figure 2), which is thought to be due to retinal processing of Flash 20.

[0076] To compare the delay to the CR peak between groups, we analyzed trials with only CS. Here, CR was defined as a trial where the maximum signal amplitude exceeded 0.10 within a time window of 60 to 750 milliseconds. In addition, the average percentage of well-timed CRs was calculated for each group. A well-timed CR was defined as a trial where the maximum signal amplitude exceeded 0.10 within a time window of 400 to 500 milliseconds.

[0077] statistical analysis All statistical analyses and visualizations were performed using R 4.3.1. One-way ANOVA was used to examine the possibility of group differences in age, average weekly exercise time, and sleep duration. A t-test was used for unequal variances to compare self-reported exercise intensity levels between the active and inactive groups who completed blink conditioning after exercise.

[0078] For all other analyses, multilevel linear mixed-effects (LME) models were used. These models are robust to deviations from normality and are better suited to the nested data structure of this study. 21、22 In all models, "subjects" were used as the random effect. For the CR amplitude model, random gradients were used for the session effect between subjects. Data for the CR amplitude model were normalized using ordered quantile normalization as proposed by the bestNormalize package in R to enable optimal model fit. Fixed effects included "session" (CR amplitude model), "exercise" (intergroup comparison), and "exercise*session" (intergroup comparison). Model parameters were estimated using restricted maximum likelihood estimation. Model fit was evaluated using the log-likelihood ratio, as well as the AIC and BIC indices. Significance was determined using the alpha value with p<0.05 (two-sided). For multiple comparisons, Hochberg p-value correction was applied to account for the number of comparisons.

[0079] result physical activity Participants in the post-inactive exercise group completed all three sessions in an average of 11 minutes after at least 20 minutes of running or cycling. Two participants did not follow the exercise protocol for one session; one completed session 1 after a 40-minute gym and rowing machine session, and the other completed session 2 after a 1-hour golf session. Participants in the post-active exercise group completed all three sessions in an average of 14 minutes after at least 30 minutes of running or cycling. Two participants in this group did not follow the protocol; one completed session 3 after a 40-minute gym aerobic exercise session, and the other completed session 2 after a 25-minute swim and session 3 after a 30-minute gym strength session. In the no-exercise group, both active and inactive individuals did not perform any aerobic exercise for at least 8 hours prior to the test and completed all three sessions.

[0080] Conditioning - Acquisition In some participants, achieving complete response (CR) began as early as Session 1, and the amplitude and timing of these responses improved throughout the three sessions. On the other hand, some participants did not achieve CR.

[0081] First, we determined whether exercise had an effect on blink conditioning. CR amplitude at 430 milliseconds was measured for each group (see Figures 6A-6C), and a comparison was made between the group that had a session without exercise and the group that had a session immediately after exercise. "With exercise" (F 1,34 =6.27, p=0.017) and “Session” (F 3,7180 The effect of session * with exercise (F = 5.13, p = 0.0015) was significant, but the effect of session * with exercise (F 3,7180 The effect of session 2 (t) was not significant (=2.07, p=0.10). Post-hoc tests showed that session 2 (t) 34 A significant difference was observed between the group without exercise and the group after exercise (at -2.87, p=0.028).

[0082] Next, we investigated whether the effect of rapid exercise on blink conditioning differed between active and inactive individuals. Comparing the active and inactive groups who completed the blink conditioning session after exercise, we found that "lifestyle" (F 1,16 =7.61, p=0.014) and “Session” (F 3,3643 The effect of the lifestyle* session (F = 4.40, p = 0.0043) was statistically significant, but the effect of the lifestyle* session (F 3,3643 The effect of session 1 (t) was not significant (=0.56, p=0.64). In the post-hoc examination, session 1 (t) 16 (t = 2.81, p = 0.037) A significant difference in CR amplitude was already observed, and this difference was maintained in session 3 as well (t 16 (=2.62, p=0.037).

[0083] Next, CR amplitude was compared between the no-exercise group and the post-exercise group, further divided into inactive and active groups. Within the inactive group, there was no significant difference in CR amplitude between conditions in any of the three sessions (see Figures 6A and 6B). In contrast, CR amplitude differed between the no-exercise group and the post-exercise group, but in the exercise group (see Figures 6A and 6C). "With exercise" (F 1,16 =9.40, p=0.0074) and “Session” (F 3,3296 The effect of (=5.49, p=0.00092) was significant. The interaction between "exercise" and "session" was not significant (F 3,3296 =0.95, p=0.42). Post-hoc testing was conducted in session 1 (t 16 =-2.70, p=0.048) and session 2 (t 16 At -3.23 (p=0.021), a significant difference was observed between the group that performed the activity after exercise and the group that did not exercise.

[0084] When the effect of sessions on CR amplitude was examined separately for each group, a significant effect was observed only in the activity group (see Table 2). In the inactive, no-exercise group, the CR amplitude in session 3 (mean = 0.05, ±0.18) was close to the baseline amplitude (mean = 0.03, ±0.09). In the inactive, post-exercise group, the CR amplitude increased slightly from a mean of 0.02 (±0.15) in session 0 to 0.10 (±0.23) in session 3. The effect of "sessions" was not significant (F 3,1907 (=1.71, p=0.16).

[0085] In contrast, in the activity-with-exercise group, CR amplitude increased from -0.05 (±0.06) at baseline to 0.05 (±0.15) in session 1 and to 0.14 (±0.22) in session 3. The effect of "session" was significant (F 3,1560 (=3.99, p=0.0087), post-hoc tests showed a significant difference between session 0 and all other sessions. Ultimately, the group that performed active exercise showed an increase in CR amplitude from 0.01 (±0.03) at baseline to 0.18 (±0.27) in session 1, and further to 0.28 (±0.32) in session 3. The effect of "session" was significant (F 3,1736 (=2.86, p=0.036). Post-hoc testing showed a significant difference between session 0 and all other sessions.

[0086] [Table 2]

[0087] Conditioning - Timing Next, we determined whether aerobic exercise affects the delay to the CR peak. For this purpose, we analyzed trials with CS only. In all groups, the CR peak time was distributed around the expected start of the US (see Figures 7A and 7B). The delay to the CR peak was also observed in the inactive group (F 1,16 =0.001, p=0.97), and in the active group as well (F 1,16(=1.46, p=0.25, see Table 2), no significant difference was observed between the no-exercise group and the post-exercise group.

[0088] For each group, the mean percentage of cases in which a complete response (CR) occurred at the right time was also calculated (see Table 2, Figures 7C and 7D). In the inactive, no-exercise group, the percentage of cases in which a CR occurred at the right time was considerably low (Figure 7C, mean = 11.27% ± 12.13), but the effect of exercise on the percentage of cases in which a CR occurred at the right time was higher in the inactive group (F 1,16 =4.25, p=0.056) or activity group (F 1,16 Neither of the following was statistically significant (=0.19, p=0.67).

[0089] Unconditional reaction To determine whether the effect of aerobic exercise is specific to CR or more general, unconditioned response amplitudes were compared between groups. Unconditioned response amplitudes were determined using pre-block and block 1 trials of Session 1 because these data were obtained before the start of CR, which in principle could influence the unconditioned response amplitude. Non-active group (F 1,14 =0.15, p=0.71) and the activity group (F 1,13 In both cases (=0.11, p=0.75), there was no significant difference in unconditioned response amplitude between the groups that completed the session with or without exercise (see Figures 8A and 8B, Table 2).

[0090] Aerobic exercise facilitated CR acquisition in the blink conditioning paradigm via smartphone. However, this effect of exercise was only observed in individuals with an active lifestyle. This finding is similar to previous research that rapid exercise improved cognitive memory in individuals who underwent a four-week exercise training program, but not in those who did not. Exercise did not significantly affect the amplitude of the unconditioned response or the timing of the CR peak.

[0091] Both the group that performed activity and the group that did not perform activity showed a significant complete response (CR) in Session 1 compared to baseline. Activity may have a priming effect, potentially reducing the number of practice sessions required for implicit learning. Since there was no difference in the amplitude of unconditioned responses between the groups, this enhancing effect of activity was specific to CR. It has been proposed that spontaneous movement directly affects blink conditioning within the cerebellar cortex. Spontaneous movement signaling via cerebellar mossy fibers (MF) may facilitate learning by converging with MF signaling in the cerebellar cortex (CS). While it is possible that activity directly affected the cerebellar cortex to enhance learning, it remains unclear why such an effect differed between individuals who performed activity and those who did not.

[0092] The finding that rapid movement promotes blink conditioning in active individuals but not in inactive individuals may suggest a mechanistic role for neuropeptidergic neurotransmitters and / or neurotrophic factors. Indeed, studies of neuropeptidergic neurotransmitters and neurotrophic factors in both humans and animals have shown a differential effect of rapid movement on active subjects compared to inactive subjects. Similarly, dopaminergic, adrenergic, and norepinephrine pathways are all catecholamine-mediated systems that significantly co-release neuropeptides, and their expression increases after exercise in both humans and animals. While the present role of these neurotransmitters in exercise-induced cognitive benefits has been frequently studied, their potential impact on associative procedural learning has received less attention. Nevertheless, there is evidence suggesting a role for neurotransmitters in cerebellar learning. In rabbits, pharmacological monoamine depletion resulted in a dose-dependent decrease in CR on a blink conditioning task. In addition, in rats, cerebellar norepinephrine was shown to be involved in achieving complete cycling (CR). These findings may extend to humans as well; elevated norepinephrine levels after exercise were associated with improved motor skill acquisition compared to a resting control group, and chronic training increased plasma catecholamine responses after cycling tasks compared to a control group that did not train.

[0093] Similarly, the neurotrophic factor BDNF may promote exercise-induced brain plasticity and memory formation. Notably, BDNF mutant mice exhibited impaired blink conditioning, suggesting that the effect of tDCS on blink conditioning in humans may depend on BDNF mutations. Furthermore, there is evidence that active and inactive individuals have different BDNF responses to exercise. A meta-analysis of the effects of exercise on BDNF in humans reported that active individuals had a more enhanced BDNF response to rapid exercise compared to inactive individuals. Therefore, in this study, rapid exercise in the inactive group may not have been sufficient to induce the BDNF levels necessary to enhance blink conditioning. Taken together, these findings provide a tentative molecular clue as to why aerobic exercise promoted learning in active individuals but not in inactive individuals.

[0094] Unlike the acquisition of complete response (CR), there were no significant differences in the timing of these responses between the groups. Interestingly, although not statistically significant, the proportion of well-timed CRs was slightly higher in the inactive post-exercise group compared to the inactive no-exercise group.

[0095] As disclosed in this embodiment, rapid aerobic exercise facilitated learning acquisition in the blink conditioning paradigm. Furthermore, the effects of exercise differed between active and inactive individuals. These results confirm in humans what has been shown in animals regarding the facilitating effects of exercise on blink conditioning. This study contributes to a more objective understanding of how exercise affects the brain by focusing on a well-characterized learning paradigm.

[0096] Therefore, for example, by determining whether a user has experienced a statistically significant improvement in their baseline CR rate in achieving complete response (CR), this system can determine whether a user is engaged in an effective exercise regimen.

[0097] As described above, the disclosed technology allows the application to correlate calculated values ​​related to eye movements and blinks with various metrics of interest, such as emotional engagement with a given video or the effectiveness of physical activity. In some embodiments, the method may include calculating a cognitive load based on eye movement and blink values, where this cognitive load may be a metric of interest or one or more proxies for such a metric.

[0098] In another example, the effects of physical activity can be observed using these systems. Naturally, non-limiting examples of physical activity that can be considered by this system include exercise, meditation, breathing exercises, and sleep. Individuals engaged in aerobic exercise or fitness programs have distinct phenotypes that can be detected using the disclosed technology. For example, individuals who engage in such activities may respond faster and have fewer "disappointing" results later in their fitness programs compared to individuals who do not. Examples of the effects of physical activity on various metrics are shown in Figures 6A–6C.

[0099] In some embodiments, the user may receive a user interface that displays the results of the test, for example, for the purpose of self-improvement or determining the state of arousal when engaging with certain content.

[0100] In some embodiments, users may receive results indicating their level of engagement with a particular fitness activity, for example, within a user interface such as a watch or phone.

[0101] With respect to various figures, the systems, methods, apparatus, mechanisms, techniques, and parts thereof described herein may be modified in various ways, and such modifications are intended to be within the scope of the invention. For example, while the various embodiments described herein show a specific sequence of steps or arrangement of functional elements, other various sequences / arrangements of steps or functional elements may be used within the context of the various embodiments. Furthermore, modifications to embodiments may be considered individually, and the various embodiments may use multiple modifications simultaneously or sequentially, or use a combination of modifications, etc.

[0102] Although various embodiments incorporating the teachings of the present invention have been shown and described in detail herein, those skilled in the art can easily devise many other various embodiments still incorporating these teachings. Therefore, while the above covers various embodiments of the present invention, other embodiments and further embodiments of the present invention can be devised without departing from its basic scope. Thus, the appropriate scope of the present invention is determined by the claims.

Claims

1. A method for measuring emotional engagement while viewing video or image content, Displaying a first video or image content on a display device, While an individual is viewing the first video or image content on the display device, Using a camera, capture a second video including one or more of the individual's eyes, While capturing the second video, the individual is exposed to one or more visual and / or auditory stimuli, Based on the second video mentioned above, determine the value representing closed eyes, Calculate the measurement standard based on the aforementioned value, Methods that include...

2. The method according to claim 1, wherein the method is performed on a single local device.

3. The method according to claim 2, wherein the single local device is a desktop computer, a laptop computer, a mobile phone, or a tablet.

4. The method according to claim 1, wherein determining the value and calculating the measurement standard are performed by one or more remote processing devices.

5. The method according to claim 1, wherein the display device and camera are operably coupled to a headset, and the headset is operably coupled to one or more processing devices that perform the method.

6. The method according to claim 5, wherein the headset is a virtual reality (VR) headset.

7. The method according to claim 5, wherein the headset is an augmented reality (AR) or mixed reality (MR) headset.

8. The method according to claim 1, wherein the calculation of the measurement criteria is performed by a machine learning algorithm trained using classified video and / or image content.

9. The method according to claim 1, wherein the measurement criterion is at least partially based on the detected alpha startle reaction.

10. The method according to claim 1, further comprising receiving first information from one or more remote processing devices, wherein the first information includes first video or image content.

11. The method according to claim 10, wherein the first information further comprises one or more visual and / or auditory stimuli, a value representing one or more visual and / or auditory stimuli, or both.

12. The method according to claim 1, further comprising transmitting a second piece of information to one or more remote processing devices, wherein the second piece of information includes the second piece of video.

13. The method according to claim 1, further comprising displaying the measurement criteria to the individual.

14. The method according to claim 1, wherein the measurement criterion is emotional engagement, and the measurement criterion is determined based on a shock response.

15. The method according to claim 1, wherein the measurement criterion is the effect of physical activity, and the measurement criterion is determined based on blink conditioning.

16. A system for measuring emotional engagement while viewing video or image content, Display device and Camera and, Speakers and, Memory and One or more processing units operably coupled to the display device, camera, speaker, and memory, A non-transient computer-readable storage medium containing instructions, wherein, when the instructions are executed by one or more processing units, the instructions are coordinately transmitted to the one or more processing units. The first video or image content is displayed on the aforementioned display device. While an individual is viewing the first video or image content on the display device, The camera is made to capture a second video including one or more of the individual's eyes. While capturing the second video, expose the individual to one or more visual and / or auditory stimuli. Based on the second video including one or more of the individual's eyes, a value representing closed eyes is determined. The measurement standard is calculated based on the aforementioned value. Non-transient computer-readable storage media, A system that includes this.

17. The system according to claim 16, wherein the system is configured as a desktop computer, a laptop computer, a mobile phone, or a tablet.

18. The system according to claim 16, wherein the one or more processing units include one or more local processing units and one or more remote processing units, and the one or more remote processing units are configured to cooperate in calculating values ​​and calculating the measurement criteria.

19. The system according to claim 16, wherein the display device and camera are operably coupled to a headset, and the headset is operably coupled to one or more processing devices.

20. The system according to claim 19, wherein the headset is a virtual reality (VR) headset.

21. The system according to claim 19, wherein the headset is an augmented reality (AR) or mixed reality (MR) headset.

Images