System and method for assessing pupillary response

Abstract

An example system has a display and a camera on the same side of a device. In some instances, the system can utilize a user's eyelids to dark adapt the pupil and use ambient light and / or light from the display to adjust stimuli, rather than using a flash to provide the stimuli. The use of a front-facing display and front-facing camera further allows the disclosed system to control ambient light conditions during image capture to ensure that no other pupil stimuli are generated when measuring the raw pupil response.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and benefit to U.S. Provisional Patent Application No. 62 / 892,977, filed August 28, 2019, entitled “System and Method for Evaluating Pupillary Response,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to systems and methods for measuring and analyzing pupillary responses and their characteristics and indicators. Background Technology

[0004] In response to various external (e.g., light) and internal (e.g., cognitive / emotional) stimuli, the pupil constricts and dilates. Pupil responses, such as the pupillary light reflex (“PLR”), are assessed to evaluate many aspects of physiological and behavioral health. A routine measurement is performed using a pupillometer. Pupilometers are expensive, costing up to $4,500, and are primarily used in medical settings and must be used by trained clinicians. Other routine measurements are performed using a penlight, which the clinician holds to the patient's eye and observes the pupillary response. Summary of the Invention

[0005] The above checks are easy to perform, but have significant quality deficiencies, including a lack of standardization, the need for professional training, discrepancies in measurements taken by different operators at different times, and poor reliability or reproducibility among observers. Pen-light checks are typically used in emergency situations where speed, rough assessment, ease of use, and convenience are prioritized over accuracy. Furthermore, even routine semi-automatic methods for measuring pupillary responses require new or external physical hardware to meet any or all of the following conditions: (1) appropriate ambient lighting conditions, (2) proper alignment of the face / eyes guided by the front of the mobile device display, (3) sufficient pupillary response stimulation, and / or (4) sufficient processing capabilities for external image processing / feature extraction.

[0006] In addition to the aforementioned drawbacks, conventional pupil measurement systems use visible light as a stimulus source and then as an illumination source for image capture. In some instances, using the visible spectrum to measure the pupil after stimulation can lead to unexpected pupillary responses, similar to the "observer effect" in physics, where observation of a phenomenon inevitably alters that phenomenon, often due to the instrument itself, as the instrument will inevitably change the state of the phenomenon it is measuring. Furthermore, conventional systems require: (1) sufficient light stimulation to produce the high contrast needed for pupil-iris segmentation, and (2) moderate to sufficient illumination to illuminate the face and achieve adequate image capture.

[0007] Finally, using these conventional methods, signs of disease are often only discovered after acute symptoms appear or the disease has progressed, at which point the most treatable stage of the disease may have already been missed.

[0008] The purpose of this disclosure is to provide a system for assessing pupillary light reflex, including a system that requires a user to close and open their eyelids to deliver a light stimulus. The system includes a mobile device, a camera, a display, a processor, and memory. The mobile device includes a front and a back. The camera and display are located on the front of the mobile device. The memory includes multiple code segments executable by the processor or one or more processors or servers. The multiple code segments include a series of instructions. In some instances, the instructions are used to emit at least one visible light stimulus from the display. Then, the instructions are used to receive image data corresponding to at least one eye of the user from the camera. Next, the instructions are used to process the image data to identify at least one pupil feature. Then, the instructions are used to determine a health condition based on the at least one pupil feature.

[0009] In some instances, the instructions are also used to output the health status on the display.

[0010] In some instances, processing the image data to identify at least one pupil feature includes preprocessing the received image data.

[0011] In some instances, identifying at least one pupil feature based on received image data includes: segmenting the received image data to determine a first data portion corresponding to the pupil and a second data portion corresponding to the iris.

[0012] In some instances, the at least one pupillary feature includes at least one of the following: pupillary response latency, constriction latency, maximum constriction rate, average constriction rate, minimum pupil diameter, dilation rate, 75% recovery time, average pupil diameter, maximum pupil diameter, constriction amplitude, constriction ratio, pupillary escape, baseline pupil amplitude, pupillary response after illumination, and any combination thereof.

[0013] In some instances, determining the health status based on at least one pupil feature further includes: (1) determining the difference between the at least one pupil feature and a corresponding healthy pupil measurement, and (2) determining the health status based on the determined difference between the at least one pupil feature. For example, the corresponding healthy pupil measurement retrieved by the processor comes from an external measurement database.

[0014] In some instances, emitting at least one visible light stimulus from the display includes: (1) receiving first image data of the eye when the display does not provide light stimulation; (2) determining an amount of luminous flux to be provided based on the first image data; (3) determining a region of the display to output the determined amount of luminous flux; and (4) outputting the determined amount of luminous flux over the determined region of the display. In some instances, second image data of the eye is received after the luminous flux is output. In some instances, the output luminous flux is adjusted based on the second image data.

[0015] In some instances, the instructions are also used to label a first pupillary response based on received image data. Then, second image data is received. Then, the instructions are used to determine changes in lighting conditions based on the second image data. Then, a second pupillary response is labeled.

[0016] In some instances, the instructions are used to display on the monitor a directive that the user should close their eyes. This may include an instruction to close the eyes within a predetermined time period. In other instances, it may include an instruction to wait for a prompt sound or vibration to prompt the user to open their eyes. The system can then receive image data corresponding to at least one of the user's eyes from the camera. In some instances, the system can determine whether or when the user has opened their eyes by processing the image data (e.g., by identifying the pupil or iris in the image). The system can then determine the user's health condition based on the at least one pupil feature and display it on the monitor.

[0017] In some instances, instructions given to the user can be text-based messages displayed on the screen. In other instances, the system may issue an audio command to the user to close their eyes. In still other instances, the system may issue other visual instructions to the user that are not based on text messages.

[0018] This disclosure also provides an exemplary method for assessing pupillary light reflex. The method includes emitting at least one visible light stimulus from the display. The method further includes receiving image data corresponding to a user's eye from the camera. The method further includes processing the image data to identify at least one pupil feature. The method then includes determining a health condition based on the at least one pupil feature. Other embodiments of the method are described above with respect to the exemplary system.

[0019] This disclosure also provides a non-transitory machine-readable medium including machine-executable code. When executed by at least one machine, the machine-executable code causes the machine to emit at least one visible light stimulus through the display. The code is then used to receive image data corresponding to a user's eye from the camera. The code is then used to process the image data to identify at least one pupil feature. The code is then used to determine a health condition based on the at least one pupil feature. Other examples of the code are described above with respect to exemplary systems.

[0020] In another exemplary embodiment, this disclosure provides another system for assessing pupillary light reflection. The system includes a hardware device, a camera, a display, a processor, and a memory. The hardware device includes a front and a rear; the camera and display are located on the front of the hardware device. The memory includes a plurality of code segments executable by the processor. The code segments include instructions for causing the display to emit at least one visual stimulus. The instructions are further configured to emit at least one invisible light via an infrared emitting device. The instructions then receive image data corresponding to a user's eye from the camera or an infrared detector. Next, the instructions process the image data to identify at least one pupil feature. Then, the instructions determine a health condition based on the at least one pupil feature.

[0021] In some instances, the wavelength of the invisible light emitted is between 700 and 1000 nanometers. In some instances, the wavelength of the invisible light emitted includes far-infrared wavelengths.

[0022] In some instances, the camera is an infrared camera.

[0023] In some instances, identifying at least one pupil feature based on received image data includes: (1) determining the image contrast of the received image data, (2) determining that the image contrast is below a threshold contrast level, and (3) outputting a prompt on the display to provide the user with second image data in a dimly lit location. For example, the at least one pupil feature is determined based on the second image data.

[0024] In some instances, the at least one pupillary feature includes at least one of the following: pupillary response latency, constriction latency, maximum constriction rate, average constriction rate, minimum pupil diameter, dilation rate, 75% recovery time, average pupil diameter, maximum pupil diameter, constriction amplitude, constriction ratio, pupillary escape, baseline pupil amplitude, pupillary response after illumination, and any combination thereof.

[0025] In some instances, the step of identifying at least one pupil feature based on received image data further includes: segmenting the received image data to determine the data portion corresponding to the pupil and the data portion corresponding to the iris.

[0026] In some instances, the hardware device is headphones.

[0027] In some instances, the hardware device is a smartphone.

[0028] The foregoing is not intended to represent every embodiment or aspect of this disclosure. Rather, it provides only examples of some novel aspects and features set forth herein. The foregoing features and advantages, as well as other features and advantages, will become clear from the following detailed description of representative embodiments and modes of implementation of the invention, taken in conjunction with the accompanying drawings and claims. Attached Figure Description

[0029] The accompanying drawings illustrate embodiments of the invention and, together with the description, serve to explain and elucidate the principles of the invention. The drawings are intended to illustrate the main features of exemplary embodiments in a graphical manner. The drawings are not intended to show the relative dimensions of every feature or element in an actual embodiment and are not drawn to scale.

[0030] Figure 1 An exemplary system 100 according to some embodiments of the present disclosure is shown.

[0031] Figure 2 An exemplary system 200 for measuring pupillary response is shown according to some embodiments of the present disclosure.

[0032] Figure 3 An exemplary method 300 for identifying and analyzing pupil features according to some embodiments of the present disclosure is shown.

[0033] Figure 4A Exemplary pupillary responses according to some embodiments of the present disclosure are shown, wherein the pupillary responses are divided into sub-stages.

[0034] Figure 4B Exemplary pupillary responses compared between healthy and unhealthy subjects are shown according to some embodiments of the present disclosure.

[0035] Figure 5 The average pupillary response measurement results according to some embodiments of this disclosure are shown.

[0036] Figure 6A Exemplary pupillary responses to cognitive load are shown according to some embodiments of the present disclosure.

[0037] Figure 6BExemplary pupillary responses to cognitive load are shown according to some embodiments of the present disclosure.

[0038] Figure 7 Exemplary pupillary responses varying with mild cognitive impairment are shown according to some embodiments of this disclosure.

[0039] Figure 8 An exemplary pupil segmentation method according to some embodiments of the present disclosure is shown.

[0040] Figure 9 An exemplary red-eye reflection is shown according to some embodiments of this disclosure.

[0041] Figure 10 Exemplary corneal light reflection is shown according to some embodiments of the present disclosure.

[0042] Figure 11 Exemplary pupil constriction according to some embodiments of the present disclosure is shown.

[0043] Figure 12 Exemplary software application implementations that automatically detect appropriate lighting and spatial orientation according to some embodiments of the present disclosure are shown.

[0044] Figure 13 Exemplary eye boundary detection according to some embodiments of this disclosure is shown.

[0045] Figure 14 Exemplary methods for determining luminous flux are shown according to some embodiments of the present disclosure.

[0046] Figure 15 Exemplary methods for identifying a second pupil response are shown according to some embodiments of the present disclosure.

[0047] Figure 16 Exemplary methods for measuring pupillary response using invisible light are shown according to some embodiments of the present disclosure.

[0048] Figure 17 Exemplary methods for determining appropriate image contrast according to some embodiments of the present disclosure are shown.

[0049] Figure 18 Exemplary data on pupil-iris segmentation compared under visible and invisible light according to some embodiments of the present disclosure are shown.

[0050] Figure 19 Exemplary iris recognition according to some embodiments of this disclosure is shown.

[0051] Figure 20Exemplary standardized data for identifying the sclera are shown according to some embodiments of the present disclosure.

[0052] Figure 21 Exemplary methods for measuring pupillary response using eyelid-interventional stimulation are shown according to some embodiments of the present disclosure.

[0053] Figure 22A PLR data following alcohol and coffee ingestion, according to some embodiments of this disclosure, are shown, demonstrating the effects on certain indicators of left pupil movement.

[0054] Figure 22B PLR data following alcohol and coffee ingestion, according to some embodiments of this disclosure, are shown, demonstrating the effects on certain indicators of right pupil movement.

[0055] Figure 23A PLR data following the ingestion of alcohol, antihistamines, opioid analgesics, and caffeine, according to some embodiments of this disclosure, are shown, demonstrating the effects on certain indicators of left pupillary movement.

[0056] Figure 23B PLR data following the ingestion of alcohol, antihistamines, opioid analgesics, and caffeine, according to some embodiments of this disclosure, are shown, demonstrating the effects on certain indicators of right pupillary movement.

[0057] Figure 24A PLR data following drinking alcohol and morning stretching, according to some embodiments of this disclosure, are shown, demonstrating the effect on certain indicators of left pupil movement.

[0058] Figure 24B PLR data following drinking alcohol and morning stretching, according to some embodiments of this disclosure, are shown, demonstrating the effect on certain indicators of right pupil movement. Detailed Implementation

[0059] The invention has been described with reference to the accompanying drawings. In all the drawings, the same reference numerals are used to denote similar or identical elements. The drawings are not drawn to scale and are only for illustrating the invention. Several aspects of the invention are described below with reference to examples. It should be understood that numerous specific details, relationships, and methods have been set forth in order to provide a comprehensive understanding of the invention. However, it will be readily apparent to those skilled in the art that the invention may be practiced in the absence of one or more specific details, or by other methods. In other instances, known structures or operations have not been described in detail to avoid obscuring the invention. The invention is not limited to the order of the described operations or events, as some operations may occur in a different order and / or simultaneously with other operations or events. Furthermore, not all described operations or events are necessary for implementing the method according to the invention.

[0060] Overview

[0061] This disclosure relates to systems and methods for measuring pupillary responses. For example, in some instances, the system may utilize the user's eyelids to induce pupillary dark adaptation and use ambient light to modulate the stimulus (here, "eyelid-interventional response" or "EMD"), rather than using a flash or display stimulus. Thus, when the user closes their eyelids, the pupil undergoes a dark adaptation process, acclimatizing to darkness and effectively dilating the pupil. Using this result as a baseline before providing / allowing light stimulation (e.g., the user opening their eyes) facilitates the measurement of latency and other characteristics in some instances (e.g., without the need for a flash on the back of the mobile device) and eliminates the need for a separate light-based stimulus to constrict the pupil, thus allowing the user to use a front-facing camera.

[0062] For example, in this instance, the system can display instructions for the user to close their eyes for a predetermined amount of time, or to open their eyes only upon hearing a prompt or feeling a vibration. This is highly advantageous because the inventors have demonstrated that, after a user has ingested alcohol or other drugs, the contrast between the light entering the user's eyes when the user closes and opens their eyes (thus allowing all ambient light in the room to enter the eyes) is sufficient to trigger a pupillary reflex and detect the difference in pupillary reflexes.

[0063] In another exemplary system, a display and a camera are positioned on the same side of the device. The display provides visible light stimulation to stimulate the user's eye and elicit a pupillary reflex. The camera simultaneously receives image data of the pupillary reflex. Therefore, the exemplary device according to this disclosure can provide a more scalable, usable, affordable, and convenient, and more accurate, objective, and quantitative system than current systems and methods, which can be used by users with or without healthcare professionals or non-health professionals. For example, in previous systems, attempts have been made to measure pupillary light reflex using the rear camera and flash on the back of a smartphone, but users cannot use this system to measure their own PLR, thus requiring a second measurement operator, leading to potentially inconsistent longitudinal measurement results due to multiple measurement operators. However, existing systems have not attempted to use a front-facing camera because the front of the mobile device does not include a flash, and therefore cannot generate stimulation to elicit a pupillary light reflex.

[0064] Therefore, based on the methods and features described herein, it has been found that the display on the front of a smartphone or similar device can be used to provide stimulation. This finding is highly advantageous because using a front-facing camera and display allows users to make more accurate, multiple measurements of pupillary light reflex using a smartphone or other relevant device. This makes the disclosed system more scalable, as it is more affordable and easier to use. For example, since the display is also located on the front of the device, the user can properly align their eyes without assistance from others. The absence of the need for additional personnel to perform measurements allows for more frequent measurements. Consequently, the system allows users to collect data more frequently and obtain longitudinal data on their health status (a single measurement may be insufficient to identify certain situations requiring longitudinal data, such as establishing a baseline and deviation from the baseline). Furthermore, using a display to provide stimulation allows for more precise control and variability of the stimulation within a given range of displayable intensities and colors. Finally, the system is particularly advantageous in some embodiments using infrared detection because infrared detection allows the eye to produce a sufficient pupillary response, while the measurement light does not induce a secondary pupillary response. This is important because the maximum intensity of the display is lower than that of a rear-facing flash, and a secondary response could result in insufficient recording of pupillary light reflex. In some instances, the disclosed systems include smartphones or other handheld computing devices. Such systems enable frequent and accurate data collection, generating important quantitative data about a user's health. In some instances, as described below, this disclosure enables the collection of longitudinal health data and its use to create baseline pupillary index measurements for a user. Therefore, this disclosure provides measurements prior to diagnosis, trauma, and / or disease that can be used to monitor disease and / or trauma progression, and / or establish an individual's longitudinal health baseline.

[0065] In some instances, the visible stimulus produces sufficient photon energy to cause a complete pupillary reflex. Exemplary methods also include collecting data before reaching a light intensity threshold and determining pupillary indicators that vary with other factors influencing pupillary response. By using a front-facing display and a front-facing camera, the disclosed system can control ambient lighting conditions during image capture to ensure that secondary unintended pupillary responses are not triggered when measuring the first intentional pupillary response. In some instances, exemplary methods include detecting ambient light levels to clarify the effect of ambient light levels on detected pupillary indicators. In some instances, data collected before reaching a light intensity threshold can provide a baseline value for the user's pupillary indicators.

[0066] Some examples of this disclosure also propose using visible light stimulation to illuminate the face, followed by image capture using invisible emitted light. Using invisible light avoids interference with the data from unintended stimulus reflections. Furthermore, in some instances, evaluation under dim conditions is more advantageous because a high contrast between the light stimulus intensity and ambient lighting conditions is required to elicit pupillary light reflection. However, in some instances, evaluation in dimly lit areas presents a problem because a dark room can interfere with image capture, resulting in suboptimal eye images. For example, the contrast between the pupil and iris components is often low, especially in individuals with high pigmentation or darker iris colors. Distinguishing between these two features is crucial for accurate feature segmentation for extraction and index calculation. Infrared cameras or other infrared hardware can provide high-resolution pupil images for effective feature segmentation.

[0067] System for measuring pupillary parameters

[0068] Figure 1 An exemplary system 100 according to some embodiments of the present disclosure is shown. In some instances, system 100 is a smartphone, smartwatch, tablet, computing device, head-mounted device, headset, virtual reality device, augmented reality device, or any other device capable of receiving and interpreting physical signals. System 100 includes a housing 110, a display 112, a camera 114, a speaker 118, a vibration motor 120, and a sensor 116. Figure 1 The front of system 100 is shown. The system may also include a camera 114 (not shown) located on the back of housing 110.

[0069] Housing 110 provides a enclosure for display 112, camera 114, speaker 118, vibration motor 120, and sensor 116. Housing 110 also includes any computing components (not shown) of system 100, such as processor, memory, wireless communication elements, and any other components readily conceived by those skilled in the art. The computing components also include software for implementing the processes described below.

[0070] Display 112, for example, is the screen of a smartphone, smartwatch, optical headphones, or any other device. In some instances, display 112 is an LCD screen, OLED screen, LED screen, or any other type of electronic display known in the art, used to display images, text, or other types of graphics. For example, the screen has multiple light-emitting diodes or other means for generating multiple pixels. Each pixel displays a light stimulus.

[0071] Display 112 is used to emit visible light. In some instances, display 112 emits light on a portion of its surface area. In other instances, display 112 emits light on all of its surface areas. The light emitted by display 112 can be controlled to emit light automatically and can increase or decrease visible stimuli. In some instances, display 112 displays image data captured by camera 114. Display 112 can also display text and messages to the user. In some instances, display 112 can display a real-time feed of image data output by camera 114.

[0072] One or more cameras 114 receive image data from their forward field of view. In some instances, camera 114 receives photographic and / or video data. In some instances, camera 114 receives continuous photographic data (e.g., at intervals of seconds, milliseconds, or microseconds). In some instances, camera 114 is a visible light camera. In some instances, camera 114 is an infrared camera and includes an infrared light emitter. In some instances, camera 114 automatically initiates image data capture based on the detection of a specific stimulus (e.g., the user's face, the user's eyes, the user's pupils, and / or the user's iris). In some instances, camera 114 is a multi-camera setup.

[0073] Sensor 116 includes, for example, any one of a light sensor, a proximity sensor, an ambient sensor, and / or an infrared sensor. In some instances, sensor 116 is communicatively connected to camera 114 and is used to initiate and / or terminate camera 114's acquisition of image data. As shown, sensor 116 and camera 114 are located on the same side of system 100. In some instances, sensor 116 is positioned close to camera 114.

[0074] Figure 2 An exemplary system 200 for receiving image data of a user's face is illustrated according to some embodiments of the present disclosure. System 200 includes a system 100, a camera 114, user eyes 202, the head of a user 204, and a camera field of view 206. System 100 and camera 114 are referenced... Figure 1 As mentioned above. Figure 2The system 100 can be positioned so that the camera 114 faces the user 204. For example, the user 204's eyes 202 can be within the camera's field of view 206. Various embodiments of this disclosure can be performed when the user 204 positions the system 100 in front of him.

[0075] Methods for analyzing pupillary response

[0076] The pupillary light reflex (PLR) describes the constriction and subsequent dilation of the pupil in response to light, and serves as an important indicator of autonomic nervous system function. PLR measurements can be used as indicators of abnormalities in various neural pathways within the nervous system (and potentially other systems) and for subsequent disease detection. As described herein, “health status” may include measurements of the pupillary light reflex.

[0077] For example, mental health disorders such as alcoholism, seasonal affective disorder, schizophrenia, and generalized anxiety disorder, Alzheimer's disease and Parkinson's disease, autism spectrum disorders, and diabetes-related glaucoma and autonomic neuropathy can all lead to abnormal pupillary diameter (PLR). The methods described below involve measuring one component of PLR using a smartphone or similar device. In some embodiments, the smartphone can not only capture phenotypic data for measuring PLR but also process the data locally in real time. Similarly, the extraction results of other quantifiable features measured from the eye / face (such as scleral color and deposit density) can also be processed locally. Therefore, user privacy can be better preserved, and measurement time can be reduced. The methods and systems can also be used to calculate dynamically changing pupil diameter. The methods and systems can generate a more stable baseline on which real-time detection statistical biases can be detected. Such biases may be a sign of abnormalities in physiological systems, resulting in variations in measurements.

[0078] The PLR ​​measurements described in this paper can be combined with other measurements in time and space, including but not limited to: measurable monocular or binocular blink changes resulting from the user's autonomic blink rate after projecting the word "blink" onto a screen, reading it, and processing it through neurons in the motor cortex (which can serve as a measure of physiological changes occurring in the autonomic nervous system pathway); sclera (the color gradient of the white of the eye changing to red or yellow); other eye features and iris and corneal rings (e.g., cholesterol deposition and cardiovascular risk); and other measurement features extracted from the face / eyes. These features can be measured by the user within spatial and temporal reach, resulting in a more effective user experience, and can be quantitatively and longitudinally (always) measured in the user's living environment (e.g., home or non-medical setting) to establish a baseline, characterized by convenience, affordability, and usability. As described in this paper, these data can serve as a preliminary understanding of various physiological systems (e.g., the nervous system, the cardiovascular system, etc.) before being used in a medical setting, and are characterized by their large quantity and statistical significance.

[0079] Figure 3 Exemplary methods 300 that can be performed according to various embodiments of the present disclosure are shown. Method 300 can be referenced... Figure 1 and 2 The method is executed on systems 100 and 200. In some instances, method 300 is executed in a dark room, a dimly lit room, a room with natural light, or any other environment. In some instances, method 300 is repeatedly executed by the user at night or before bedtime, for example, when external variables (such as light) are minimal and controllable.

[0080] In some instances, method 300 begins at 310, via a display (e.g., Figure 1 The display 112 or sensor 116 emits visible light stimulation, or provides light stimulation by displaying an instruction on the display indicating that the user should close their eyes for a predetermined amount of time. The light stimulation, for example, causes pupil constriction. In some instances, the degree of pupil constriction increases with increasing contrast between the visible light stimulation and the ambient light level. The amount of visible light stimulation provided can be determined by method 1400 of Figure 4, as described below.

[0081] In some instances of 310, when the user's face is detected (e.g., Figure 2 When user 204 is at an appropriate spatial distance, the camera (e.g., Figure 1 In system 100, the camera 114 automatically emits visible light stimuli. In other instances, when a user's face is detected, the screen can display a message to the user indicating that their eyes are closed. In some instances, the display first issues a notification that a display light stimulus is about to appear. For example, in... Figure 12In this system, the display can show real-time captured image data of the user's face and provide a visual graph showing that the user's features have been correctly detected. In some instances, the display is... Figure 1 The display 112 in the middle. For example, circles 1202 can be displayed on the user's eyes or nose. Figure 13 As shown, the display shows exemplary bounding boxes for the user's eyes, mouth, and nose.

[0082] refer to Figure 3 In some instances, 310 first detects the pupil. If no pupil is detected, the user is notified that the environment settings do not meet the criteria of method 300.

[0083] Then, method 300 receives image data corresponding to the user's eyes at 320. Exemplary image data includes video and / or photographic data. In some instances, the image data is collected over a period of time (e.g., by...). Figure 1 (Collected by camera 114). In some instances, the video recording speed is 30-60 frames per second, or higher. In some instances of 320, a set of still images is generated by the camera. In some instances of 320, the captured image data is a grayscale video / image set, or is converted to grayscale after being received.

[0084] Some instances of 320 include certain visual stimuli such as red-eye reflex, pupillary response, iris and sclera data, eye-tracking data, and skin data.

[0085] Then, method 300 continues to process the image data in 330 to identify pupil features.

[0086] In some instances of 330, the received image data is first preprocessed to filter the data. Exemplary types of data preprocessing will be discussed further below. In a brief exemplary protocol for preprocessing data, the image data of 320 is cropped and filtered to obtain a region of the image. For example, the image is filtered based on set thresholds for brightness, color, and saturation. The image data is then converted to grayscale to improve the contrast between the pupil and iris and to delineate the pupil-iris boundary. In some instances of 330, shape analysis is performed to filter the image data based on a preselected roundness threshold. For example, the contours and convexities of pixels are scanned for shape analysis. In some instances of 330, a baseline image is compared with the received image data in 320 to aid in preprocessing.

[0087] In some instances, 330 is also used to determine the surface area of ​​the pupil and iris regions detected in the image data. For example, by evaluating the elapsed time for each image, imaging analysis software algorithms determine a series of pupil size parameters for the recorded images to determine the rate of change in pupil size over time.

[0088] In some instances, optionally, identification information is removed from the sensor data at step 330. In other words, the most relevant and interesting principal phenotypic features can be extracted from the raw image data. Exemplary features include: pupillary velocity (such as size and orientation), scleral color, measurements of tissue inflammation, and / or other features. These features can be represented as scalar numbers after relevant metrics are extracted from the underlying raw data. Identifiable user images are not used.

[0089] In some instances, 330 determines whether additional data is needed. For example, it displays a warning on the monitor to identify the required type of measurement result, along with user instructions for obtaining the appropriate type of measurement result.

[0090] In some instances of 330, the features include: (1) pupillary response latency, including the measured time for the pupil to respond to light stimulation, for example, in milliseconds; (2) maximum diameter, i.e., the maximum observed pupil diameter; (3) maximum constriction velocity (MCV), i.e., the maximum velocity observed during constriction; (4) average constriction velocity (ACV), i.e., the average velocity observed throughout the constriction period; (5) minimum pupil diameter, i.e., the minimum observed diameter; (6) dilation velocity, i.e., the average velocity observed throughout the dilation period; (7) 75% recovery time, i.e., the time it takes for the pupil to reach 75% of its initial diameter value; (8) average diameter, i.e., the average of all diameters measured at various times; (9) pupillary escape; (10) baseline pupillary amplitude; (11) pupillary response after illumination; (12) maximum pupil diameter; (13) any other pupillary response measurement known in the art; and (14) any combination thereof. Similar indicators for the iris have been identified in some instances of 330.

[0091] For example, the method for reducing the latency period is to reduce (time) 闪光 - Shrink (time) 初始 For example, the constriction rate is a measurement of the pupil's rate of constriction, expressed in millimeters per second. For example, the method for measuring the degree of constriction is (diameter before illumination)... 最大值 (Diameter after illumination) 最小值 For example, by using the diameter 最大值 The ratio is used as the reduction amplitude to measure the reduction ratio. For example, the dilation rate is a measurement of the pupil dilation rate in millimeters per second. Many of the above characteristics can be derived by evaluating the pupil diameter of the first image, the pupil diameter of the second image, and the duration between the two images, as will be readily apparent to those skilled in the art. Furthermore, those skilled in the art will readily understand that the dilation latency, dilation rate, dilation amplitude, and dilation ratio can be calculated in a similar manner based on the data provided in 320.

[0092] Other features include, for example: the speed of spontaneous blinking reflex in response to the screen-projected word "blink" (which can serve as a measure of the autonomic nervous system pathway), scleral (white-to-yellow) color features, iris and corneal ring features (cholesterol deposition and cardiovascular risk), and other measurement features extracted from the face / eyes.

[0093] In some instances of 330, pupil measurements are inferred from the inside or outside based on the observed trajectory of the collected image data.

[0094] Then, method 300 at 340 determines a health status based on the pupillary features identified in 330. In some instances, the health status is a measurement of pupillary light reflex or other clinically relevant pupillary measurements or features. In some instances of 340, the features identified in 330 are compared with corresponding values ​​for healthy individuals to identify abnormalities. In some instances, the features are compared with the user's longitudinal data; the difference between the current measurement and an established longitudinal baseline (for the individual) can serve as an indication of disease status or a measurement of disease manifestation. In some instances of 340, an individual user baseline is established during longitudinal use of system 200, and the user is notified when a pupillary feature identified in 330 deviates from the established individual baseline by 1.5 standard deviations or a predetermined threshold deviation. For example, the threshold deviation varies with disease status. In some instances, 340 utilizes a general or external database of healthy individuals until the individual user provides 20 individual PLR measurements according to method 300.

[0095] In some instances of method 300, the image data includes data from both of the user's eyes. In 330, the reflection of each pupil is analyzed separately. However, in 340, features from both are analyzed simultaneously to determine health status, because the different pupillary light reflexes between the two eyes can indicate a certain disease state (e.g., stroke).

[0096] In some embodiments of method 300, alerts are issued based on the received data. For example, if a digital marker for a disease is detected, system 100 receives a pre-disease detection alert and displays it, for example, on display 112. In some embodiments, audio alerts may be used to supplement or replace graphic alerts. This informs the user of developing diseases, abnormalities, or early signs of disease and allows for timely intervention. Other information, such as recommendations to contact a doctor for a physical examination, may also be received and presented to the system.

[0097] exist Figure 2 System 200 and Figure 3In some instances of Method 300, in an indoor environment with controlled ambient light, the user holds a smartphone and maintains a naturally controlled spatial viewing distance from their face (e.g., 6-24 or 6-12 inches horizontally from the face, 0-6 inches vertically from the eyes, 0-6 inches horizontally (from the user's right to left) from the nose, or other distances). In some embodiments, holding the smartphone in this position for a certain period of time (e.g., at least 5 seconds) activates an application (via sensors and software) that, during recording, records the subject's face (especially the eyes and pupillary reflections) at 60+ or ​​120+ frames per second in high-definition video after the user is stimulated by a brief, intense flash of light from a touchscreen or other light source on the smartphone, or instructs the user on a display to close their eyes for a predetermined amount of time. In some instances, the flash is focused and has a known emission intensity, and the intensity of light reaching the pupil can also be inferred from its known inverse relationship with the square of the distance between the light source and the pupil. Thus, images of the user's face are captured before, during, and after the brief, intense flash. In some embodiments, recording begins at least 1 second and no more than 5 seconds before the flash or instruction for the user to open their eyes, and continues for at least 3 seconds and no more than 8 seconds after the flash or the user opens their eyes. It is noteworthy that the light intensity reaching the pupil can be inferred from its known inverse relationship with the square of the distance between the pupil and the light source.

[0098] Exemplary pupil response curve

[0099] Figure 4A An exemplary pupillary response curve and various features that can be identified at different points on the curve are shown. For example, these features are analyzed with reference to the method 300 described above. Figure 4A The diagram shows that when light stimulation is provided, baseline pupil diameter is first measured, followed by assessment of MCV, MCA, and pupillary escape. When light stimulation is turned off, post-light pupillary response (PIPR) can be assessed.

[0100] Figure 4B Another exemplary PLR curve is shown, including: (1) latency, (2) shrinkage rate, (3) shrinkage magnitude, (4) shrinkage ratio, and (5) expansion rate. Compared with the normal PLR curve shown by the solid line, the dashed line shows the abnormal PLR curve, which has a longer latency, slower speed, and smaller magnitude.

[0101] Data preprocessing and processing

[0102] In some instances of 330, the received image data is preprocessed. Exemplary preprocessing techniques are described herein.

[0103] Frames in the sequence are smoothed to denoise the system, eliminating natural pupil fluctuations, iris color variations, and device-specific variations. A Gaussian smoothing operator can be used to slightly blur the image and reduce noise. The two-dimensional Gaussian equation is as follows:

[0104] Equation 1

[0105] Where sigma is the standard deviation of the distribution, it can be expressed by the following formula:

[0106] Equation 2

[0107] Where x is the i-th PLR measurement, μ is the average PLR, and N is the total number of PLR measurements. In some embodiments, a PLR measurement that is probabilistically significant, such as + / - one standard deviation or + / - 1.5 standard deviations, can trigger an alert indicating the detection of an abnormality in the nervous system. In some such embodiments, the alert may indicate a pre-disease condition. In other embodiments, the alert may simply indicate the detection of an abnormality.

[0108] In some instances of this disclosure, PLR is represented as a smoothed Fourier transform. For example, when using a histogram representation of a smoothed grayscale frame, the image can be binarized using a threshold function. This threshold function can be determined by the difference between dark and bright pixels on the histogram. Based on this, the image can be binarized by marking the white portions of the image as 1 and the black portions as 0, thus distinguishing the sclera and pupil. This effectively generates a black square with a white circle to clearly represent the pupil for analysis. The pupil is typically elliptical in shape but can be represented as circular using an average axis. The diameter can be measured in pixels between the two farthest white pixels. This pixel measurement can be converted to millimeters using a reference with a known size near the eye. For example, a dot projector in a smartphone can be used to determine the depth of the smartphone from the face.

[0109] The differential equation describing the reflection of light by the pupil using the pupil diameter flux that varies with light is as follows:

[0110] Equation 3

[0111] Equation 4

[0112] D is the measured pupil diameter (mm), and Φ(t-τ)r represents the light intensity reaching the retina within time t. Therefore, by using data from the video (e.g., the diameter of the white circle representing the pupil in each frame, the time between frames, and the pixel-to-millimeter conversion), the above differential equation can be used to determine the pupil velocity. The pupil velocity reacting to a flash (diameter decreasing) and the pupil velocity recovering (diameter increasing) can be determined.

[0113] In some instances, preprocessing involves cropping the fragments to include the region of each eye. This can be achieved by applying a simple heuristic to the known structure of the face. The fragments can then be submitted for processing, including, for example, deconstructing the received visual stimulus into a series of images for processing one by one. The images are processed to eliminate aberrations introduced during image capture by glasses, blinking, and minor hand movements. Pupil boundary detection using contour gradient entropy can be used to extract the size of each pupil and create a visual series of data.

[0114] In some embodiments, an eye tracker can be used to capture frames of eyes with varying degrees of dilation. The user can manually label the pupil diameter for each frame. Using the labeled data, a segmentation model can be trained on the labeled pupils. For example, U-Net or a similar service can be used to output a shape that can be used to infer the diameter. A pipeline can be implemented to process the recorded video frames and plot pupil dilation over time.

[0115] In some instances of data processing, received image data is filtered using hue, saturation, and brightness values. For example, if a pixel's "V" value (representing brightness) is greater than 60, that pixel might be filtered out. In another instance, pixels can be filtered based on LAB values, where "L" represents the pixel's brightness, and "A" and "B" represent color antagonism values. Since the pupil is the darkest feature of the eye, pixels with "L" values ​​greater than 50 can be filtered out, leaving only relatively dark pixels that are more likely to include the pupil.

[0116] Other exemplary processing steps include: (1) copying the filtered image and removing the filtered content to display only the region of interest (ROI); (2) converting the filtered ROI pixels to grayscale; (3) filtering grayscale pixels based on brightness or intensity values, for example, filtering pixels with L values ​​higher than 45; (4) scanning the contours and convexities of the remaining pixels; (5) scanning the pixels to obtain incremental gradients of pixel grayscale values; (6) constructing shapes based on contours or constructing shapes defined by contours; (7) filtering shapes based on size and roundness; (8) determining the surface areas of the pupil region and the iris region; and (9) determining the relative changes of the two regions over time.

[0117] In some instances of roundness-based filtering, the device filters out roundness values ​​that are not equal to or close to 1.0. For example, a circle has a roundness value equal to or close to 1.0, while a slender ellipse might have a roundness value of approximately 0.25.

[0118] Predicting health status based on pupil characteristics

[0119] Figure 3 Methods 300 and 340 can be used to identify whether a user has certain disease states, disease severity, or other health conditions. (The following text...) Figure 5-7 Example data corresponding to the example health status is displayed.

[0120] Figure 5 The measured mean pupillary response associated with Alzheimer's disease is shown. For example, as... Figure 5 As shown, there were significant differences in latency, MCV, MCA, and amplitude between the cognitively healthy patient group and the Alzheimer's disease patient group.

[0121] Figures 6A-6B Exemplary pupillary responses to cognitive load are shown according to some embodiments of this disclosure. Figures 6A-6B As shown, psychological pupillary response is associated with Alzheimer's disease. Cognitive load is measured by the subject's ability to recall 3, 6, or 9 digits. Figures 6A-6B The results indicate that, with increasing cognitive load, the pupillary dilation was more pronounced in the amnesic single-domain mild cognitive impairment (S-MCI) group compared to the cognitively healthy control group (CN). Furthermore, under certain cognitive loads, the pupillary dilation was significantly lower in the multi-domain mild cognitive impairment (M-MCI) group compared to the cognitively normal group and the S-MCI group. This suggests that the cognitive load far exceeded the group's capacity.

[0122] Figure 7 Exemplary pupillary responses varying with mild cognitive impairment according to some embodiments of the present disclosure are illustrated. For example, data show that pupillary dilation increases with a load ranging from 3 digits to 6 digits, but decreases once a 9-digit load is reached. Therefore, the present disclosure envisions that individuals with lower cognitive abilities exhibit greater pupillary dilation at lower loads and less pupillary dilation at higher loads.

[0123] Pupil Segmentation

[0124] This disclosure provides a pupil segmentation method. Image data of the eye can be divided into three main parts: the pupil, the iris, and the sclera. Image segmentation algorithms can be used to achieve the desired segmentation.

[0125] Figure 8An exemplary pupil segmentation process is illustrated. First, a grayscale image of the eye is received. Then, a balanced histogram is created based on the grayscale of each pixel. For example, balanced histogram thresholding segmentation, K-means clustering, edge detection, or region filling might be used. The exemplary balanced histogram segmentation algorithm sets a threshold grayscale for each pixel to determine which pixels correspond to the pupil. The pixels corresponding to the pupil will be the darkest pixels.

[0126] In one example, K-means clustering selects k (e.g., k is 4 in this example) data values ​​as initial cluster centers. The distance between each cluster center and each data value is determined. Each data value is assigned to its nearest cluster center. The mean of each cluster center is then updated, and this process is repeated until no further clustering is possible. Each cluster center is analyzed to determine which cluster center contains the pupil pixels, yielding the segmentation result. This method can be used to segment regions of interest from the background based on the four main parts of the eye with different colors: the black pupil, the white sclera, the colored iris, and the skin background.

[0127] Figure 8 The method shown is also used for edge detection and region filling, thereby enhancing the image and connecting the main pixels of the pupil. Filling holes of a certain shape and size yields the final segmentation result.

[0128] After segmentation, the pupil area is determined in pixels. Based on the scale of the camera that collected the image data, the pixel measurements are converted into physical dimensions (e.g., in millimeters).

[0129] Red-eye reflex

[0130] Figure 9 Exemplary red-eye reflex data collection according to some embodiments of this disclosure is illustrated. For example, image data of a red reflex highlighting the retina of a user's eye is collected. This disclosure then determines whether the red reflex is dim (potentially a sign of strabismus or retinoblastoma), whether the reflex is yellow (potentially a sign of exudative retinal disease), and / or whether the reflex is white or contains glare (potentially a sign of retinoblastoma, cataracts, retinal detachment, and / or eye infection). Figure 3 Methods 300, 330, and 340, respectively, can provide characteristics for determining health status.

[0131] corneal light reflection

[0132] Figure 10Exemplary collection of light reflectance data of the cornea according to some embodiments of the present disclosure is illustrated. For example, image data capturing the degree of strabismus (eye misalignment) is collected. The present disclosure then determines whether the captured data includes any of the following: (A) a tiny spot of light at the center of the pupil; (B), (C), and (D) a deviation between the location of said spot and the center of the pupil, indicating eye misalignment. Figure 3 Methods 300, 330, and 340, respectively, can provide characteristics for determining health status.

[0133] Measuring pupil diameter

[0134] Figure 11 Exemplary pupil diameter measurements are shown. For example, 1112 and 1122 show the baseline pupil diameters of subjects 1110 and 1120, respectively. Subject 1110 is a healthy subject, and subject 1120 has Alzheimer's disease. MCV and MCA can be calculated according to the methods described herein.

[0135] Determine the amount of visual stimulus

[0136] Figure 14 Method 1400 provides an exemplary method for determining the amount of visual stimulus to be presented on a display. For example, method 1400 can be used as... Figure 3 A portion of step 310 in method 300 is executed. In some instances, method 1400 is executed separately. Figure 1 and 2 This is performed on systems 100 and 200. In some instances, based on a period of time or determining that the user has opened their eyes, display stimulation is combined with eyelid intervention responses by providing light stimulation to the display before or when the user opens their eyes. Thus, combining dark adaptation of the pupil when the eyes are closed, eye opening, and light stimulation can provide stronger light stimulation, which in some embodiments may be necessary to elicit a full pupillary light reflex.

[0137] In method 1400, initially, first image data is received at 1410 without any light stimulation provided. For example, the camera 114 of system 100 receives image data from the user without any light stimulation provided to display 112 or sensor 116.

[0138] Then, based on the first image data received from 1410, method 1400 determines the amount of light flux to be provided at 1420.

[0139] In some instances of the 1420, the type of light output from the display is also determined. For example, the wavelength (or color of light within the visible spectrum) of the light to be displayed is determined. Each user's eyes have black-vision receptors activated by different colors. Therefore, the 1420 controls the wavelength (or color) of the light to activate certain black-vision receptors and receptor pathways in the user's eyes. In some instances, these pathways can be used to distinguish diseases mediated by certain receptor pathways. This can also be determined based on ambient light. Therefore, the system can adjust the display's output as a stimulus based on the amount and wavelength of ambient light.

[0140] Then, method 1400 determines an area of ​​the display at 1430 to output luminous flux. In some instances, the entire surface area of ​​the display is used. In other instances, only a portion of the display's surface area is used.

[0141] In some instances of method 1400, the amount of luminous flux and the area of ​​the display used to output the luminous flux can be determined simultaneously or in any order (e.g., 1420 and 1430).

[0142] Then, at 1440, method 1400 outputs a defined amount of luminous flux on a defined area of ​​the display.

[0143] In some instances of method 1400, additional image data from the eye is received after the luminous flux is output. In some instances, the luminous flux is adjusted based on the received image data.

[0144] Recognizing responses to multiple pupils

[0145] In some instances of this disclosure, a method for identifying multiple pupillary responses is provided. For example, this method can identify whether an image dataset has been confounded by unexpected pupillary stimulation (e.g., in...). Figure 3 (Among the 300 methods). Figure 15 Exemplary method 1500 for identifying and marking unexpected pupillary responses according to some embodiments of the present disclosure is shown. For example, method 1500 may be used in... Figure 3 Method 300 is executed before, during, and / or after.

[0146] first, Figure 15 Method 1500 in 1510 labels a first pupillary response based on the received image data. For example, the first pupillary response includes any changes in pupillary features described herein.

[0147] Then, at 1520, method 1500 receives second image data after the initially received image data.

[0148] Then, method 1500 determines the change in illumination conditions at 1530. For example, the change in illumination conditions can be determined based on the brightness difference between the image data received at 1510 and the second image data received at 1520.

[0149] Then, method 1500 identifies a second pupillary response in the second image data at 1540. For example, if the second image data is a series of images, one or more images that appear simultaneously or at close intervals after a change in lighting conditions are identified at 1540. In some instances, the identified second pupillary response is any of the pupillary features described herein.

[0150] Infrared measurement implementation method

[0151] This disclosure further provides image capture using invisible light stimulation and / or an infrared camera. For example, Figure 1 The sensor 116, infrared emitter, and / or display 112 can provide invisible light emission. In some instances, the camera 114 is an infrared camera and includes one or more infrared light emitters. Figure 16 An exemplary method 1600 is shown, which can be implemented in... Figure 1 and Figure 2 This is performed on systems 100 and / or 200. This can be used in various embodiments disclosed herein, including in a dark room utilizing screen-based visible light stimulation, employing an eyelid-interventional response. Therefore, screen-based stimulation in a dark room achieves higher contrast because it blocks any remaining light in a dark or dimly lit room when the user's eyes are closed.

[0152] Method 1600 in 1610 is provided by a display (e.g., Figure 1 The display 112 or sensor 116 emits a visible light stimulus. For example, the wavelength of the visible light stimulus is greater than 1000 nanometers. The visible light stimulus is directed at the user's face. This visible stimulus is used to elicit a pupillary response in the user's eyes.

[0153] Then, method 1600 is displayed at 1620 by a monitor (e.g., Figure 1 The display 112 or sensor 116 (e.g., an infrared emitter) emits invisible light stimuli. These invisible light stimuli are used to illuminate the user's face, thereby producing a sufficiently high image contrast (enough for pupil-iris segmentation). Therefore, 1620 utilizes the high image contrast provided primarily by infrared light. For example, the invisible light stimuli provided by 1620 are light stimuli with wavelengths of 600-1000 nanometers.

[0154] Because the illumination provided by 1620 can produce sufficiently high image contrast, compared to methods that only use visible stimuli (including, for example, ...), Figure 3Compared to method 300, method 1600 requires less visible stimulus in step 1610. Therefore, method 1600 is able to elicit a pupillary response more accurately because the visible stimulus provided in 1610 does not require illuminating the user's face.

[0155] Furthermore, method 1600 receives image data corresponding to the user's eyes at 1630. In some instances, the received image data is a set of images or videos. In some instances, the image set is collected at fixed intervals (e.g., intervals measured in seconds, milliseconds, and / or microseconds) over a period of time (e.g., more than one minute, two minutes, or three minutes). In some instances, the image data received at 1630 comes from an infrared camera.

[0156] In 1640, method 1600 further processes the image data to identify pupil features. For example, according to Figure 3 Method 1600, as described in any of the methods of 330 of 300, processes the received image data. Then, method 1600 determines the health status at 1650 based on the identified pupil features. For example, according to... Figure 3 Method 300 of 340 may be used to determine health status.

[0157] Therefore, Method 1600 avoids confounding pupillary response results with other unexpected stimuli.

[0158] Identify appropriate lighting conditions

[0159] In some examples of this disclosure, the lighting conditions are automatically detected to determine whether they provide sufficient quality image data to identify the various pupil features described herein. Figure 17 An exemplary method 1700 for evaluating illumination conditions is shown according to some embodiments of the present disclosure. Method 1700 may be derived from... Figure 1 and 2 System 100 and / or 200 execute respectively. In some instances, method 1700 is executed in... Figure 3 and 16 Executed before, after, and / or during Method 300 and / or Method 1600.

[0160] Law 1700 determines the image contrast of the received image data in 1710. For example, the image contrast is determined based on brightness, color, saturation, and / or any other visual image analysis means known in the art.

[0161] Then, method 1700 determines at 1720 whether the image contrast is below a threshold contrast level. For example, 1720 determines whether pupil-iris segmentation can be performed based on the provided image data. In some instances, 1720 determines whether pupil-iris segmentation can be performed with a certain accuracy threshold and / or confidence measurement.

[0162] Then, if the stimulus is ambient light intervened by the user's eyelids (e.g., the user's eyes are closed / open), method 1700 outputs a cue at 1730 to provide the user with second image data in a darker or brighter location.

[0163] When used in conjunction with Method 1600, Method 1700 ensures that the user is in a sufficiently dimly lit position to produce high contrast for pupil segmentation.

[0164] Experimental data – infrared light

[0165] Figure 18 Exemplary image data comparing image sets taken under visible light (image sets 1810 and 1830) with image sets taken under infrared light (image sets 1820 and 1840) is shown. In image sets 1820 and 1840, a clearer outline is displayed between the pupil and iris of the subject compared to image sets 1810 and 1830 taken under visible light. In particular, image set 1830 was taken from subjects with dark irises, where pupil segmentation is virtually impossible due to the similarity of pupil and iris colors and the low contrast between them. Therefore, Figure 18 It shows Figure 16 Method 1600 (collecting image data using invisible stimuli) and Figure 17 The effect of method 1700 (ensuring the pupil-iris image has sufficiently high contrast).

[0166] Implementation of eyelid intervention response

[0167] Figure 21 This is a flowchart illustrating a detailed example of how the disclosed system and method are implemented using the user's eyelids to induce pupil dark adaptation and to modulate stimulation with ambient light ("eyelid-interventional response"). Thus, when the user closes their eyelids, the pupil undergoes a dark adaptation process, acclimatizing to darkness and effectively dilating the pupil. The results are used as a baseline prior to the application of light stimulation (e.g., when the user opens their eyes)—to facilitate latency measurements and maximal construction.

[0168] For example, in this instance, the system could display instructions for the user to close their eyes for a predetermined amount of time, or to open them only when they hear a prompt or feel a vibration. The advantage of this is that the contrast between the light entering the user's eyes when their eyes are closed and open (thus allowing all ambient light in the room, or screen-based stimuli in a dark or dimly lit room, to enter the eyes) is sufficient to trigger a pupillary reflex.

[0169] For example, at normal viewing distances, the maximum lux emitted from a display (e.g., 200 lux) may not be sufficient to elicit a adequate pupillary light reflex (e.g., 300 lux or more may be required). However, under normal lighting conditions, the contrast between light entering the eye when it is open and closed is sufficient to elicit a pupillary light reflex. Otherwise, due to the brightness of the ambient light, it is difficult to ensure sufficient contrast between the ambient light and the light stimulus to produce a pupillary light reflex. Therefore, the implementation of eyelid intervention avoids the need for other light stimuli (such as flashing or illuminated displays). In other instances, starting from a period of time with the user's eyes closed, the baseline widens, and the stimulation from eyelid intervention allows the display to provide sufficient additional stimulation to elicit a pupillary response.

[0170] Therefore, in some instances, using this system does not require light-based stimulation from the device. Thus, the user can hold the phone facing the display (since a flash is not needed). Furthermore, the display does not need to provide light stimulation to the user's eyes, and in some instances, a rear-facing camera can be used to assess pupillary responses to eyelid intervention. Moreover, utilizing eyelid intervention may be preferable to a flash sufficient to elicit a pupillary reflex, as it may be more comfortable for the user. In other instances, the user closing their eyes combined with light stimulation from the display may be sufficient to elicit a pupillary light reflex.

[0171] Furthermore, the method allows users to easily implement it in any well-lit or bright room with sufficient ambient light to trigger a reflection after the user opens their eyes from a state of eye-closed and dark adaptation. Figure 21 Examples of implementing this method are provided. In some instances, the system may first provide a real-time feed of image data on display 112 so that the user can properly align their eyes in front of camera 114, as described herein (e.g., displaying circles or arrows on the real-time image data to help the user align their eyes in the image). In other instances, a rear-facing camera may be used, and the feedback to the user may be purely audio or vibrational to inform them when to open and close their eyes and when to properly align their eyes with the rear-facing camera.

[0172] Next, the system can provide an instruction 2110 requesting the user to close their eyes. This instruction may include a text-based message displayed on display 112. For example, display 112 may display the text "Close your eyes for 3, 10, or 15 seconds" or "Close your eyes until you hear a beep [or feel a vibration]." The system can then start a timer that lasts for three seconds (or 4, 10, or 15 seconds, or any other suitable time sufficient to trigger a pupillary light reflex) and begin recording image data output from camera 114 after the set time has elapsed. In other instances, the system may issue a beep or activate a vibration motor after the set time has elapsed to notify the user to open their eyes 2120. In these instances, the system may begin recording image data at or before the start of the beep or vibration.

[0173] In some instances, the system may process image data until it determines that the user has at least one eye open (e.g., by using computer vision to identify pupils, irises, or other features of the eyeballs), and filter frames when it determines that the user's eyes are closed. This is particularly important because it allows the system to recognize the first frame in which the user's eyes are open (by initiating recording from camera 114 while the user's eyes are still closed), and thus capture all or most of the pupil's light reflection.

[0174] In some instances, this may involve determining the pupil diameter based on a partial image of the pupil, either before the user's eyes are fully open or if the user's eyes are not fully open. For example, the system may estimate the full pupil diameter from a partial diameter by extrapolation or other means. For instance, if the visible pupil radius is less than 360 degrees, the diameter of the entire pupil can be estimated using known mathematical functions, such as trigonometric functions. This may involve determining the pupil diameter from a visible small portion of the pupil (e.g., a visible radius of 90 degrees). In some instances, the estimated pupil diameter derived from partial measurements may have sufficient accuracy for calculating health conditions, including, for example, quantitative measurements of pupillary light reflex.

[0175] Furthermore, the system can identify frames where the user's eyes are correctly focused on a point on a camera or screen, allowing for accurate measurement of pupil diameter. The system can display indicators (such as arrows) on the monitor showing where the user should be focusing their gaze. In other instances, the system may be able to determine the direction of the user's gaze and approximate the pupil diameter based on these measurements.

[0176] In addition, the system may continuously monitor frames to determine if enough frames have been captured if the user's eyes have been fully open for a sufficiently long period (e.g., if the user closes their eyes prematurely). If not a sufficient number of usable frames are captured to determine pupillary light reflex or other relevant pupillary features, the process will restart.

[0177] Next, the system can receive visual data (320) corresponding to the user's eyes, and the system can then... Figure 3 The same method is used to process image data. This includes processing image data to identify pupil features (330) and processing pupil features to determine the user's health status (340).

[0178] Experimental data example: Smartphone applications using eyelid intervention

[0179] The inventors tested an example of an eyelid intervention-based smartphone application to determine whether this implementation could induce pupillary light reflex (PLR) and detect the use of certain medications. Therefore, data generated by measuring multiple key indicators of PLR after ingestion of several key medications using an eyelid intervention-based application showed that it was consistent with the expected physiological effects described herein when tested using an eyelid intervention-based application. Thus, the data demonstrate that the eyelid intervention implementation can effectively deliver sufficient stimulation to effectively assess pupillary light reflex, consistent with conventional and established methods for assessing PLR, and can also detect patient ingestion of certain medications.

[0180] For example, Figure 22A PLR data are shown, illustrating the effects of alcohol and caffeine intake on certain parameters of left pupillary movement, measured using an application based on eyelid interventional responses. For example, Figure 22A The results show that, compared to baseline, coffee significantly increased (pupil movement) speed, while alcohol slowed it down. Therefore, Figure 22A It has been confirmed that it is possible to determine whether a patient has consumed alcohol using an app on a smartphone or mobile device based on eyelid interventional responses; Figure 22B The displayed PLR data demonstrates that, through the application of eyelid intervention, the effects of alcohol and caffeine intake on certain indicators of right pupil movement can be detected. Figure 23A The displayed PLR data indicate that, through the use of eyelid intervention, the effects of alcohol, antihistamines, opioid analgesics, and caffeine intake on certain indicators of left pupil movement can be detected. Figure 23B The displayed PLR data indicate that, through the use of eyelid intervention, the effects of alcohol, antihistamines, opioid analgesics, and caffeine intake on certain indicators of right pupil movement can be detected. Figure 24A The displayed PLR data indicate that, through the application of an eyelid intervention, the effects of alcohol consumption and morning stretching on certain indicators of left pupillary movement can be detected; and Figure 24B The displayed PLR data indicate that, through the use of eyelid intervention, the effects of alcohol consumption and morning body stretching on certain indicators of right pupil movement can be detected.

[0181] Experimental data: Reproducibility of PLR data using eyelid interventional applications

[0182] Table 1 shows the reproducibility of processing between the right and left eyes when using eyelid intervention after applying smoothing techniques. The high scores in Table 1 indicate that EMD intervention is very accurate in PLR sessions because the indices for both eyes are highly reproducible.

[0183]

[0184] Table 1: Reproducibility of the processing index between the right and left eyes when using eyelid intervention, showing the accuracy of binocular ACV measurements.

[0185] Table 2 shows the standard deviation of the treatment over time when using eyelid intervention after applying the smoothing technique. High scores indicate the stability and reproducibility of the indicator over time.

[0186]

[0187] Table 2: Standard deviation of treatment indicators over time when using eyelid intervention.

[0188] Therefore, Tables 1 and 2 demonstrate the reproducibility of PLR indices between the two eyes and over time when using eyelid interventional applications. Thus, the system and method disclosed herein can be reliably used to measure the characteristics of PLR.

[0189] Additional Software Implementation Methods

[0190] Exemplary software application

[0191] This disclosure envisions an exemplary health application that presents a template with alignment marks for key facial features on a client device's display. The health application instructs the user to align key facial features with the alignment marks displayed on the smartphone screen. The alignment of facial features is selected to ensure triangulation of depth and angles, provided these features remain fixed in three-dimensional space over time and cannot be intentionally or accidentally altered by the user. When a measurement is imminent, the client device may display an indication, such as a green light. The health application presents a flash on the client device and captures video of the user's eyes using a high-definition camera, which is one of the sensors. Using this video, the health application determines the pupillary diameter reflex velocity—the speed at which the user's pupil diameter constricts in response to light and subsequently dilates to its normal baseline size. Thus, dynamic phenotypic data of pupillary velocity can be captured. Pupil velocity can be used to predict developing diseases, abnormalities, or disease precursors. Furthermore, other phenotypic data may be captured due to the use of a camera. For example, the color of the sclera (white of the eye) is visible. The color of the sclera can be used to determine if the user has various developing diseases, abnormalities, or disease precursors. A yellow sclera may indicate jaundice. Redness of the sclera (whites of the eyes) may indicate cardiovascular problems caused by vasoconstriction in the eyes. Similarly, considering frequency and time of day, redness of the sclera may indicate drug abuse. Other phenotypic features in the area surrounding the pupil may indicate cholesterol deposits, often associated with cardiovascular problems. Changes in facial pigmentation or the growth of moles may indicate skin conditions such as melanoma. Therefore, a single dynamic test can generate data as a quantitative measure of multiple phenotypic features associated with various diseases.

[0192] To measure PLR, the user is instructed to align both eyes in the camera. This provides an appropriate image size for further image processing and pupil measurement. A camera session is initiated to detect the user's face and acquire images of their eyes. The background color and phone brightness (if using the front-facing camera) (or the torch level) are adjusted to produce various levels of brightness. Real-time image processing, including segmentation, can be performed to obtain the pupil diameter and time tracking to measure the pupil constriction rate. Finally, the measurement results, including binocular reaction time, constriction rate, and pupil constriction ratio, can be presented to the user.

[0193] Automatic face detection

[0194] Automatic face detection can be performed using the tip of the nose and two pupils. In some embodiments, the controllable spatial distance described above is achieved by the user aligning their face with three red triangular dots on the viewfinder (two for the pupils and one for the tip of the nose). Machine vision identifies alignment between the pupils and the red dots, and between the tips of the nose and the tips of the nose (based on the RGB color of the nasal skin). An ambient light sensor is then used to check for any ambient light (noise) that might interfere with the measurement results. If the alignment (depth / angle) is correct and the lighting is sufficient, the red dots turn green, notifying the user to take a measurement within a certain time frame. This process is as follows: Figure 12 As shown.

[0195] The system emits a flash and captures video. Face detection can be achieved using one or more frames of the video. Thus, after capturing the video above, the smartphone automatically detects the pixel positions of the tip of the nose and the two pupils (which may also be projected onto the screen) with the aid of machine vision-based algorithms, ensuring that the measurements are consistent in triangular geometry and space. The specific geometry and distances of these three reference points will not be intentionally or unintentionally altered by facial muscles over time, further guaranteeing controllability and consistency.

[0196] The face detection / machine vision portion of this measurement method can be performed using open-source and / or proprietary software. This allows for the detection of both the face and eyes (e.g., eyes). Figure 12-13 (As shown). In some embodiments, the input video / video frame is grayscale. If a face is detected in the video, the system will continue to detect eyes within the face coordinates. If no face is detected, the user will be notified that the recorded video does not meet the criteria for valid detection.

[0197] In the pre-capture phase, a real-time guided facial recognition algorithm can be used. In some embodiments, this can be achieved using OpenCV (an open-source computer vision library), ARKit (an augmented reality kit), or other facial recognition mechanisms. Facial recognition identifies the position of the eyes in the image and guides the user to manipulate the device to place the camera in the desired location. Once the camera is positioned, the image data capture phase can begin. Modern smartphones can have a transmittance exceeding 300 nits (1 candela / m²). Video clips can be as short as 10-20 seconds, sufficient to capture enough data for PLR analysis. One or more cameras on a modern smartphone (e.g., Figure 1 The camera 114 is used to capture video before, during, and after the screen flash.

[0198] In some embodiments, face capture combining face and eye recognition can also be used for PLR measurements. Some face recognition frameworks, such as the Vision Framework, can detect and track faces in real time by creating requests and interpreting the results of these requests. Such tools can be used to find and identify facial features (such as eyes and mouth) in images. A face tagging request first locates all faces in the input image and then analyzes each face to detect facial features. In other embodiments, face tracking can be used, for example, through an augmented reality session. ARKit is an example of such a mechanism. Using this mechanism, a front-facing camera system can be used to detect a user's face. By configuring and running an augmented reality session, camera images and virtual content can be simultaneously presented in the view. This mechanism can generate a coarse 3D mesh geometry that matches the size, shape, layout, and current facial expression and features of the user's face. Such a mechanism, or a combination of mechanisms, can be used to capture and analyze images. For example, one mechanism is used to capture an image, while another is used to analyze it.

[0199] Disclosed computer and hardware implementation methods

[0200] First, it should be understood that this disclosure can be implemented using any type of hardware and / or software, and can also be a pre-programmed general-purpose computing device. For example, the system can be implemented using a server, personal computer, laptop computer, thin client, or any suitable one or more devices. This disclosure and / or its components can be a single device located in one location, or multiple devices located in one or more locations, connected together via any suitable communication protocol and communication medium (e.g., cable, fiber optic cable) or wirelessly.

[0201] It should also be noted that the disclosures shown and described herein have multiple modules that perform specific functions. It should be understood that, for clarity, the modules are illustrated schematically according to their functions and do not necessarily represent specific hardware or software. In this regard, these modules can be hardware and / or software used to fully perform the specific functions. Furthermore, in this disclosure, these modules may be combined together or divided into additional modules based on desired specific functions. Therefore, this disclosure should not be construed as limiting the invention, but should be understood as an illustration of an exemplary embodiment.

[0202] A computing system may include clients and servers. Typically, clients and servers are geographically separated but interact via a communication network. The client and server establish a client-server relationship through computer programs running on their respective computers. In some implementations, the server transmits data (e.g., HTML pages) to the client device (e.g., to display data to a user interacting with the client device and to receive user input from it). The server may also receive data generated by the client device (e.g., the result of user interaction).

[0203] The implementations of the main contents described in this specification can be implemented in a computing system, including: back-end components (e.g., as a data server), or middleware components (e.g., an application server), or front-end components (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with the implementations of the main contents described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected via digital data communication of any form or medium, such as a communication network. Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0204] The implementation of the main contents and operations described in this specification can be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or one or more combinations of the above components. The implementation of the main contents described in this specification can be implemented as one or more computer programs, i.e., one or more computer program instruction modules encoded on a computer storage medium for execution by a data processing device or for controlling the operation of the data processing device. Alternatively, or further, the program instructions can be encoded on artificially generated propagating signals, such as machine-generated electrical, optical, or electromagnetic signals, for encoding information transmitted to a suitable receiver device for execution by the data processing device. The computer storage medium can be or is contained in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of the above devices. Furthermore, although the computer storage medium is not a propagating signal, it can be a source or destination of computer program instructions encoded in artificially generated propagating signals. The computer storage medium can also be or be contained in one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0205] The operations described in this specification can be performed by a data processing device on data stored on one or more computer-readable storage devices or received from other sources.

[0206] The term "data processing apparatus" encompasses all types of devices, apparatuses, and machines used for processing data, such as programmable processors, computers, systems-on-a-chip, or a combination thereof. The apparatus may include special-purpose logic circuitry, such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs). In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, cross-platform runtime environments, virtual machines, or combinations thereof. The apparatus and execution environment can implement various computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0207] Computer programs (also called programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as standalone programs, or as modules, components, subroutines, objects, or other units suitable for a computing environment. A computer program may, but must not, correspond to a file in a file system. A program can be stored in a partial file containing other programs or data (e.g., one or more scripts stored in a markup language document), a single file dedicated to said program, or multiple coordination files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on one or more computers located at a single site or distributed across multiple sites and interconnected via a communication network.

[0208] One or more computer programs can be executed by one or more programmable processors to manipulate input data and generate output, thereby performing the processes and logic flows described in this specification. Alternatively, the processes and logic flows can be executed by dedicated logic circuits, and the device can be implemented as dedicated logic circuits, such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs).

[0209] Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors, and one or more processors in any type of digital computer. Typically, a processor receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a processor for performing operations according to instructions, and one or more memory devices for storing instructions and data. Typically, a computer also includes, or is operatively connected to, one or more mass storage devices (e.g., hard disks, magneto-optical disks, or optical disks) for receiving and / or transferring data to. Of course, a computer does not necessarily have to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, personal digital assistant (PDA), mobile audio or video player, game console, GPS receiver, or portable storage device (e.g., a Universal Serial Bus (USB) flash drive), and so on. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, such as semiconductor storage devices like EPROM, EEPROM, and flash memory devices; hard disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. Processors and memory can be supplemented by dedicated logic circuits or integrated into dedicated logic circuits.

[0210] End

[0211] The various methods and techniques described above provide numerous ways of carrying out the present invention. It should be understood, of course, that not all the objectives or advantages described can be achieved according to any particular embodiment presented herein. Therefore, for example, those skilled in the art will recognize that performing the methods may achieve or optimize one or more advantages as presented or implied herein, without necessarily achieving other objectives or advantages as presented or implied herein. Various alternatives are mentioned herein. It should be understood that some embodiments specifically include one, another, or several features, other embodiments specifically exclude one, another, or several features, and still other embodiments make a particular feature less obvious by including one, another, or several advantageous features.

[0212] Furthermore, those skilled in the art will recognize the applicability of the various features in different embodiments. Similarly, the various elements, features, and steps described above, as well as other known equivalents of each element, feature, or step, can be used in various combinations by those skilled in the art to perform the methods according to the principles described herein. In different embodiments, some elements, features, and steps may be specifically included while others are excluded.

[0213] Although this application has been disclosed in some embodiments and examples, those skilled in the art will understand that the embodiments of this application are not limited to the specific disclosed embodiments, and can be extended to other alternative embodiments, and / or their applications and modifications, and their equivalents.

[0214] In some embodiments, the terms “a,” “the,” and similar references used in describing particular embodiments of this application (especially in certain claims below) may be interpreted to include both singular and plural forms. The descriptions of numerical ranges herein are merely a shorthand for each individual value within that range. Each individual value is included in the specification as if it were individually described herein, unless otherwise stated herein. All methods described herein may be performed in any suitable order unless otherwise stated herein or clearly contradicted by the context. The use of any and all instances or exemplary language (e.g., “for example”) presented in certain embodiments herein is intended only to better illustrate the application and does not constitute a limitation on the scope of the claims made herein. No language in this specification should be construed as indicating any unclaimed element essential to the implementation of this application.

[0215] This document describes certain embodiments of this application. After reading the foregoing, variations of these embodiments will be apparent to those skilled in the art. It is understood that those skilled in the art can employ such variations and practice this application in ways different from those specifically described herein. Therefore, where permitted by applicable law, many embodiments of this application include all modifications and equivalents of the main content recited in the appended claims. Furthermore, unless otherwise stated herein or clearly contradicted by the context, any combination of all possible variations of the foregoing elements is included in this application.

[0216] This document describes some implementations of the main content. Other implementations are covered by the following claims. In some cases, the operations described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes shown in the drawings do not necessarily need to be performed in the specific order or sequential order shown to obtain the desired result.

[0217] All patents, patent applications, patent application publications, and other materials, such as articles, books, specifications, publications, documents, things, and / or similar items cited herein, are incorporated herein by reference in their entirety for all purposes, except for those related to any history of prosecution documents, any materials inconsistent with or conflicting with this document, or any materials that may have a limiting effect on the broadest scope of the claims relating to this document, now or thereafter. For example, if there are any inconsistencies or conflicts in descriptions, definitions, and / or terminology used between the incorporated materials and this document, the descriptions, definitions, and / or terminology used in this document shall prevail.

[0218] Finally, it should be understood that the embodiments disclosed herein are merely illustrative of the principles of the embodiments of this application. Other modifications that may be adopted are covered within the scope of this application. Therefore, as examples and not limitations, other configurations of the embodiments of this application can be used based on the content described herein. Therefore, the embodiments of this application are not limited to all the contents shown and described.

Claims

1. A system for evaluating pupillary light reflex, comprising: Mobile devices, including those with both the front and back; The camera located on the front of the mobile device; The display located on the front of the mobile device; processor; and The memory stores a plurality of code segments executable by the processor, the plurality of code segments including instructions for performing the following operations: The real-time feed of image data output from the camera is displayed on the monitor. The display shows a visual alignment guide and provides prompts instructing the user to align their eyes with the visual alignment guide. Detect ambient light levels; When the ambient light level is below a preset threshold, the display emits a calibrated visible light flash stimulus to at least one of the user's eyes, and then records the image data of the at least one eye at a high frame rate. When the ambient light level reaches or exceeds the preset threshold, an instruction is displayed on the display to prompt the user to close their eyes for dark adaptation, open their eyes after the dark adaptation time is over, and record the image data of at least one eye at a high frame rate when the eyes are open. The recorded image data is processed to identify at least one pupil index; as well as Health status is determined based on at least one pupil index.

2. The system of claim 1, wherein the instructions are further configured to output the health status at the display.

3. The system of claim 1, wherein the health status includes pupillary light reflex, alcohol intake, opioid intake, antihistamine intake, and caffeine intake.

4. The system of claim 1, wherein displaying an instruction to request the user to close their eyes on the display comprises displaying a text-based message requesting the user to close their eyes and open them after a predetermined time.

5. The system of claim 1, wherein displaying the instruction that the user should close his / her eyes on the display comprises displaying a text-based message requesting the user to close his / her eyes until he / she hears an audible instruction to open his / her eyes.

6. The system of claim 5, wherein the instruction is further configured to: after displaying the text-based message requesting the user to close their eyes, output sound through a speaker after a predetermined time has elapsed.

7. The system of claim 6, wherein the image data is received after the sound is output.

8. The system of claim 7, wherein the instructions are further configured to process the image data to determine whether one or both of the user's eyes are open.

9. The system of claim 1, wherein displaying an instruction to request the user to close their eyes on the display comprises displaying a text-based message requesting the user to close their eyes until the mobile device vibrates and then opens their eyes.

10. The system of claim 9, wherein the instruction is further configured to excite the vibration motor after a predetermined time has elapsed following the display of the text-based message requesting the user to close their eyes.

11. The system of claim 1, wherein the instructions are further configured to determine when the user's eye, identified in the real-time feed of the image data, is within the visual alignment guidance.

12. The system of claim 11, wherein after determining that the user's eyes are within the visual alignment guide, an instruction for the user to close their eyes is displayed on the display.

13. The system of claim 1, wherein identifying at least one pupil index based on the recorded image data further comprises segmenting the recorded image data to determine a first data portion corresponding to the pupil of the at least one eye and a second data portion corresponding to the iris of the at least one eye.

14. The system of claim 1, wherein the at least one pupil indicator comprises: Pupil response latency, constriction latency, maximum constriction rate, average constriction rate, minimum pupil diameter, dilation rate, 75% recovery time, average pupil diameter, maximum pupil diameter, constriction amplitude, constriction ratio, pupil escape, baseline pupil amplitude, pupil response after light exposure, or any combination thereof, and compared with the user's baseline data.

15. The system of claim 1, wherein determining the health status based on the at least one pupillary indicator further comprises: Determine the difference between each of the at least one pupil index and a corresponding healthy pupil measurement, wherein the corresponding healthy pupil measurement is retrieved by the processor from an external measurement database; and The health status is determined based on the determined difference between each of the at least one pupil index and the corresponding healthy pupil measurement.

16. The system of claim 1, wherein before displaying a prompt to the user to close their eyes on the display, the ambient light level is detected, and a display flash pupil light reflection mode or an eyelid-mediated dark adaptation mode is selected based on a threshold.

17. The system of claim 1, wherein identifying at least one pupil index based on the recorded image data further comprises: Determine the image contrast of the recorded image data; The image contrast is determined to be below a threshold contrast level; and The display outputs a prompt to the user, indicating that a second image data should be provided in a brighter location.

18. The system of claim 1, wherein the mobile device includes an earphone with a camera, a smartphone, or both.

19. A method for evaluating pupillary light reflex, comprising: Display visual alignment guidance and provide prompts to instruct the user to align their eyes with the visual alignment guidance; Detect ambient light levels; When the ambient light level is below a preset threshold, a first instruction is provided that the user should close their eyes at least for a first predetermined time and then open them. Receive the first set of image data from the camera corresponding to at least one of the user's eyes; The first set of image data is processed to determine whether pupillary light reflection has been triggered. This triggers the pupillary light reflex: Process the first set of image data to identify at least one pupil feature; and Health status is determined based on at least one pupil feature; or In response to failure to elicit a pupillary light reflex: Provide a second instruction that the user should close his eyes at least at a second predetermined time and then open them, wherein the second predetermined time is longer than the first predetermined time; Receive a second set of image data from the camera corresponding to at least one of the user's eyes; Process the second set of image data to identify at least one pupil feature; as well as Health status is determined at least in part based on the at least one pupil feature.

20. The method of claim 19, wherein the first indication includes at least one of a text-based message displayed on a display, a visual message, or an audio message emitted through a speaker.

21. The method of claim 19, wherein the image data is filtered to identify frames in which at least one eye of the user is open.

22. The method of claim 19, wherein processing the image data to identify at least one pupil feature further comprises determining pupil light reflex and determining the patient’s alcohol or caffeine intake.

23. A non-transitory machine-readable medium comprising machine-executable code, which, when executed by at least one machine, causes the machine to: Display visual alignment guidance and prompt the user to close their eyes; Detect ambient light levels; When the ambient light level is below a preset threshold, the display emits a calibrated visible light flash stimulus to at least one of the user's eyes. When the ambient light level reaches or exceeds the preset threshold, an instruction is provided to prompt the user to close their eyes for dark adaptation, and to open their eyes after the dark adaptation time is over. Receive image data from the camera corresponding to at least one of the user's eyes; Identify a set of frames from the image data, the frames having timestamps indicating that they were acquired at a predetermined time after the prompt; The image data is processed using at least one processor to identify at least one pupil feature; as well as The at least one processor is used to determine pupil light reflection based on the at least one pupil feature.

24. The non-transitory machine-readable medium of claim 23, wherein the camera includes an infrared camera.

25. The non-transitory machine-readable medium of claim 23, wherein the machine-executable code further enables the machine to filter the image data to identify frames in which the pupil of the user is sufficiently visible to assess pupil features.

Images