Cylindrical-shaped capacitive sensors and their optimal locations for eye-tracking

Abstract

A capacitive sensor, including a composite substrate formed of a plurality of insulating fibers coated with a plurality of carbon nanotubes (CNTs), a plurality of cross-bar junctions of the plurality of insulating fibers at or near a fracture site in the composite substrate, and a sensor core, wherein the composite substrate is wrapped around the sensor core to form a cylindrical capacitive sensor.

Description

CYLINDRICAL-SHAPED CAPACITIVE SENSORS AND THEIR OPTIMAL LOCATIONS FOR EYE-TRACKING CROSS REFERENCE TO RELATED APPLICATION This application claims the benefit of U.S. Provisional App. No. 63 / 686081 filed August 22, 2024, the entire disclosure of which is hereby incorporated by reference. BACKGROUND

[0001] Fatigue can cause severe consequences. Fatigue monitoring is crucial for daily life as fatigue leads to as much as 1.62 times accidents rates increase, reduction of productivity and cognitive functions, and contribute to a larger likelihood of physical and mental illness such as diabetes, cardiovascular diseases, and suicide attempts. Many methods have been introduced for fatigue monitoring, ranging from self-reporting Karolinska Sleepiness Scale (KSS), reaction-based performance-based Psychomotor Vigilance Test (PVT), to biomarker-based methods such as electroencephalography (EEG) or cortisol.

[0002] Self-reporting methods like KSS require participants to rate their fatigue on a numerical scale, whereas for KSS, the range varies from 1 representing extremely alert, to 10 representing extremely sleepy. While these methods are easy to conduct, they require temporary interruptions and can be subjective. On the other hand, in the PVT test, participants are required to react to random visual events, typically presented on a screen, and their reaction times are measured. An increase in reaction time can indicate reduced alertness and increased fatigue. PVT results are quantitively and easy to conduct, besides, its visual event’s randomness reduces the memory effect. Nevertheless, PVT can be challenging in real-life monitoring due to the need for uninterrupted test segments, which can last somewhere between 3 to 10 minutes.

[0003] The EEG is generated by inhibitory and excitatory postsynaptic potentials of cortical nerve cells and can be a valid biomarker for detecting fatigue. EEG is used to monitor brain wave patterns that correlate with different states of alertness and fatigue.Specific EEG markers, such as the presence and increases of theta wave activity and dip in alpha wave activity, are associated with higher levels of fatigue. While EEG has shown trends with brain arousal, sleepiness, and fatigue, the result varies between studies and individuals. Nevertheless, sampling EEG can be difficult to use and not feasible outside of the lab environment. Further, EEG signals can be affected by many issues other than fatigue such as eye closures, cognitive and memory tasks, and emotions.

[0004] In yet another example, salivary cortisol levels can drop in people suffering fatigue, but biochemistry measurements are not convenient and not suitable for long-term monitoring.

[0005] Accordingly, systems, methods, and devices for monitoring and determining fatigue are needed. SUMMARY

[0006] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0007] In one aspect, disclosed herein is a capacitive sensor, including a composite substrate formed of a plurality of insulating fibers coated with a plurality of carbon nanotubes (CNTs), a plurality of cross-bar junctions of the plurality of conducting fibers at or near a fracture site in the composite substrate, and a sensor core, where the composite substrate is wrapped around the sensor core to form a cylindrical capacitive sensor.

[0008] In some embodiments, wherein the sensor substrate is an electrically conductive film. In some embodiments, the sensor core comprises a non-conductive material such as polyurethane. In some embodiments, the sensor further includes a silver pasting located on one edge of the cylindrical composite substrate. In some embodiments, the sensor further includes a coaxial cable coupled to the silver pasting. In some embodiments, the sensor further includes an insulating tape configured to secure thecomposite substrate. In some embodiments, the insulating tape comprises a polyimide self- adhesive tape. In some embodiments, the sensor further includes a shielding layer located around the insulating tape. In some embodiments, the shielding layer comprises copper foil or other electrically conductive film. In some embodiments, the sensor further includes a shielding mesh coupled to the shielding layer.

[0009] In another aspect, disclosed herein is an eye tracking device including a glasses frame having a first lens, and a second lens adjacent to the first lens, a first capacitive sensor as disclosed herein, where the first capacitive sensor is located at a top center of the first lens, and a microcontroller configured to receive one or more capacitance measurements from the first capacitive sensor and detect one or more blinks, an eye closure duration, a percentage of eyelid closure over the pupil over time (PERCLOS), or a combination thereof based on the one or more capacitance measurements.

[0010] In some embodiments, the microcontroller is further configured to determine fatigue of a user based on the one or more blinks, the eye closure duration, the PERCLOS or a combination thereof. In some embodiments, the eye tracking device has an adjustable holder to modify the sensor orientation and distance to an eye. In some embodiments, the eye tracking device further includes a second capacitive sensor as disclosed herein, where the second capacitive sensor is located at a top center of the second lens, where the first capacitive sensor and the second capacitive sensor are configured to measure vertical position of the eyes, a third capacitive sensor as disclosed herein, located at a temporal location of the first lens, and a fourth capacitive sensor as disclosed herein, located at a temporal location of the second lens, where the third capacitive sensor and the fourth capacitive sensor are configured to measure a horizontal position of the eyes.

[0011] In some embodiments, the first capacitive sensor is offset from the second capacitive sensor. In some embodiments, the first capacitive sensor is about 2 mm higher than the second capacitive sensor. In some embodiments, the third capacitive sensor and the fourth capacitive sensor operate using differential measurement to measure the horizontal position of the cornea of the user’s eye. In some embodiments, the firstcapacitive sensor and the second capacitive sensor operate using differential measurement to measure the vertical position of the cornea of the user’s eye.

[0012] In another aspect, disclosed herein is a method of determining fatigue with the eye tracking device as disclosed herein, the method including receiving one or more capacitance measurements from the first capacitive sensor, detecting one or more blinks, an eye closure duration, a percentage of eyelid closure over the pupil over time (PERCLOS), or a combination thereof based on the one or more capacitance measurements, and determining a presence of fatigue, a severity of fatigue, or a combination thereof using a machine learning based classification and scoring algorithms.

[0013] In some embodiments, the method further includes disabling the second capacitive sensor, the third capacitive sensor, and the fourth capacitive sensor before receiving one or more capacitance measurements from the first capacitive sensor.

[0014] In some embodiments, the method further includes using differential measurement to measure a horizontal position of the cornea of the user’s eye with the third capacitive sensor and the fourth capacitive sensor operate.

[0015] In some embodiments, disclosed herein is a method of eye-tracking with the eye-tracking device disclosed herein, and including measuring capacitance from the first capacitive sensor, the second capacitive sensor, the third capacitive sensor and the fourth capacitive sensor, and determining the position of a cornea of each of the user’s eyes based on the measured capacitance. DESCRIPTION OF THE DRAWINGS

[0016] The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:

[0017] FIGs. 1A-1H are example process steps of fabricating a CPC sensor, in accordance with the present technology.

[0018] FIGs.2A-2C are images of an example CPC sensor and its fabrication, in accordance with the present technology.

[0019] FIGs. 3A-3B are SEM images of an example CPC sensor, in accordance with the present technology.

[0020] FIG. 4A shows a modified 3-D printer used to test fabricated sensors, in accordance with the present technology.

[0021] FIG.4B shows an example sensor interface setup, in accordance with the present technology.

[0022] FIGs. 5A-5B shows the baseline capacitance and baseline noise level for the tested sensors, in accordance with the present technology.

[0023] FIGs. 6A-6F show testing setups and results for proximity sensing distance tests, in accordance with the present technology.

[0024] FIGs. 7A-7B shows testing done to determine optimal image sensor locations, in accordance with the present technology.

[0025] FIGs. 8A-8B show the measurement accuracy of the sensors on five subjects, in accordance with the present technology.

[0026] FIGs. 9A-9F show the scanning results of the participants, in accordance with the present technology.

[0027] FIGs.10A-10B are an example eye tracker device, in accordance with the present technology.

[0028] FIGs. 10C-10D show testing of the eye tracker in comparison to conventional eye-trackers, in accordance with the present technology.

[0029] FIGs. 11A-11D show capacitive responses corresponding to different gaze location during on-axis and diagonal movements, in accordance with the present technology.

[0030] FIGs. 12A-12D show the capacitance signal and Tobii Eye-tracker recording of diagonal eye movement, in accordance with the present technology.

[0031] FIGs. 13A-13C show graphs of the on-Axis test, Diagonal test, and a combined test, in accordance with the present technology.

[0032] FIGs. 14A-14C show an example fatigue monitoring system, in accordance with the present technology.

[0033] FIGs.15A-15C show capacitive eye-tracker based blinks and percentage of eyelid closure over the pupil over time (PERCLOS) digital biomarker detection, benchmarked against manual counting and computer-vision based algorithms, in accordance with the present technology.

[0034] FIGs. 16A-16B are statistics for the whole 3-minutes test for 3 subjects, in accordance with the present technology.

[0035] FIGs.17A-17F show the set up and results of the blink rate and PERCLOS of a 15-minute induced fatigue test, in accordance with the present technology.

[0036] FIG. 18 is a method of determining fatigue with the eye tracking device disclosed herein, in accordance with the present technology. DETAILED DESCRIPTION

[0037] Disclosed herein are eye-tracking devices, systems, and methods for determining and measuring fatigue levels. Eye-tracking can be advantageous for fatigue level measurement due to its non-contact nature and being fast and unobtrusive compared with EEG or biochemical markers-based methods. Eye-tracking-based fatigue monitoring may utilize biomarkers such as percentage of eyelid closure over the pupil over time (PERCLOS), blink frequencies, and gaze directions to estimate alertness or fatigue level and have seen commercial applications in driver alertness monitoring systems in automobiles. Research has shown that an increase in fatigue level can lead to an increase in PERCLOS and blinking frequency.

[0038] In one aspect, disclosed herein is a human wearable eye tracker using an improved carbon-nanotube-paper composite (CPC) sensor design. In some embodiments, the CPC sensors may be placed in particular eye regions. In some embodiments, the system includes an online-processing capable eye tracker.

[0039] Flat CPC capacitive sensors are known, however, when adapting flat CPC sensors for a human eye-tracking, many challenges arise.

[0040] First, despite the high sensitivity, the flat CPC sensors still need to be placed close to the eye for better accuracy. Flat CPC sensors have widths of 5 mm and rely on slide structures to move. On a human-eye tracker, slide structures cannot be integrated due to safety concerns and adverse effects on the vision field. Besides, due to a lack of shielding, flat CPC sensors’ high proximity sensitivity may pick up displacements from the forehead or eyebrow, causing interference with gaze signals.

[0041] To address the above positioning and interference issues, a stick-like long electrode was designed to be installed close to the eye. The electrode had its sensitivity concentrated at the tip closest to the eyes. Such a design goal can be achieved by affixing a flat sensor at the end of a stick or wrapping CPC materials around a stick to create a cylindrical roll and shielding. The first option could be difficult to implement because of the difficulty of creating tiny, fractured CPC structures and maintaining stable electrical connections. Additionally, the tiny, affixed sensor could be easily damaged during adjustment or use. Using a copper electrode could be more desirable in the affixed sensor setting. Nevertheless, copper electrodes do not have enough sensitivity to capture tiny eye movements, and the rigidness and sharpness of copper foil could pose safety concerns. Rolling CPC electrodes, on the other hand, have the soft fractured cellulose fiber end exposed, which is highly sensitive and safe.

[0042] In one aspect, disclosed herein is a capacitive sensor, including a composite substrate formed of a plurality of insulating fibers coated with a plurality of carbon nanotubes (CNTs), a plurality of cross-bar junctions of the plurality of insulating fibers at or near a fracture site in the composite substrate, and a sensor core, where the composite substrate is wrapped around the sensor core to form a cylindrical sensor. In some embodiments, the sensor core comprises polyurethane.

[0043] In some embodiments, the capacitive sensor further includes a silver pasting located on one edge of the cylindrical composite substrate. In some embodiments,the capacitive sensor further includes a coaxial cable coupled to the silver pasting. In some embodiments, the capacitive sensor further includes an insulating tape configured to secure the composite substrate. In some embodiments, the insulating tape is a polyimide self- adhesive tape. In some embodiments, the capacitive sensor further includes a shielding layer located around the insulating tape. In some embodiments, the shielding layer comprises copper foil. In some embodiments, the capacitive sensor further includes a shielding mesh coupled to the shielding layer.

[0044] FIGs. 1A-1H are example process steps of fabricating a CPC sensor, in accordance with the present technology. In FIG. 1A, a CPC strip is provided. In FIG. 1B, silver pasting is applied to the CPC strip. In some embodiments, the silver pasting is Ag- 510 silver ink. In some embodiments, the process begins with the application of conductive Ag-510 silver ink to one end of a long CPC strip to form conductive pads.

[0045] In FIG. 1C, a fracture a tensile fracture is formed in the CPC strip. In some embodiments, water may be applied to the CPC strip to induce the fracture. In some embodiments, around halfway along the strip, a controlled fracture may be induced by drawing a water line with a 0.1-mm capillary pen across the shorter edge of the CPC. This pre-weakening allows for a precise break, leaving, in some embodiments, a 12 mm segment from the conductive pads intact.

[0046] In FIG. 1D, following this, the long CPC strip may be sectioned into several pieces. In some embodiments, the pieces are 15 mm-by-12 mm pieces. In some embodiments, to reduce parasitic capacitance and electromagnetic interference, 1.3 mm thin coaxial shielded cables were used for the sensor connection. In some embodiments, preparation steps involve stripping 5 mm of the cable jacket and setting aside the exposed shielding mesh for later use. To increase electrical contact area and robustness to rolling deformation, a 2 mm-by-1 mm copper foil may be soldered to each cable’s inner conductor and connected to the CPC’s silver pattern using conductive silver epoxy.

[0047] In FIGs.1E-1F the CPC section may be rolled to form the cylindrical CPC sensor. In some embodiments, the sensor may be rolled around a 1.6 mm diameter softpolyurethane core, starting from the end opposite to the cable connection to maximize bending curvature, thus reducing the likelihood of loose connection. In FIG. 1G, a polyimide self-adhesive tape may be wrapped over the rolled CPC structure, securing the structure and providing insulation.

[0048] In some embodiments, as shown in FIG. 1H, a second copper foil is subsequently wrapped over the polyimide tape, exposing a 2 mm section at the top of the sensor. The copper foil may then be connected to the shielding mesh previously set aside to extend the sensor’s electromagnetic shielding. In some embodiments, the completed sensor has a diameter of approximately 3.5 mm. In some embodiments, a length of the sensor can be over 12 mm for a larger active shielding coverage.

[0049] FIGs.2A-2C are images of an example CPC sensor and its fabrication, in accordance with the present technology. FIG. 2A shows an unwrapped CPC sensor showing the cable connection and silver pad. FIG.2B is a CPC cylindrical sensor side view. FIG. 3C shows optical microscopy of an example CPC cylindrical sensor showing the cross-section of the fractured fibrous end.

[0050] FIGs. 3A-3B are SEM images of an example CPC sensor, in accordance with the present technology. FIG. 3A is a SEM images of the CPC sensor showing the fibrous network, while FIG.3B is a zoomed-in view of individual fiber. FIG.3A shows the fiber networks while FIG.3B shows an individual fiber with width about 40 um. The EDS spectrum of the sample showed a high weight concentration of carbon and oxygen, which is consistent with earlier research attempting to characterize cellulose fibers. EXAMPLES

[0051] In one example, CPC cylindrical sensors were benchmarked against similarly constructed and identical-sized copper sensors. Three CPC cylindrical sensors and three reference copper foil sensors were fabricated.

[0052] FIG. 4A shows a modified 3-D printer used to test fabricated sensors, in accordance with the present technology. Both types of sensors were mounted on a custom- designed carriage on a modified 3D printer. The carriage consisted of two parts: a topsection with a single circuit board with two AD7747 capacitance-to-digital converters (CDCs), which were the same CDC as the non-human primate eye-tracker, and an nRF52840 microcontroller and a bottom section containing a L-shaped bridge for holding sensors. The sensor platform was loaded on a 3D printer, which can accurately control the sensor position

[0053] FIG.4B shows an example sensor interface setup, in accordance with the present technology. The computer concurrently interfaced two microcontrollers, sending G-code to the 3D-printer’s microcontroller to move the sensor to a designated location while getting reading from the sensor’s microcontroller. Two AD7747 CDCs were installed on each circuit board on their independent i2C channel to interface CPC and copper sensor at the same time. Sensors were connected to positive input in CDCs through active shielded cables. In the upper section, the microcontroller communicated with both CDCs through independent I2C interfaces, converting and relaying the measurement data to a connected laptop. Measurement data were continuously read by the laptop but only saved during predefined sampling windows. The carriage’s bottom features an L-shaped bridge with slots slightly larger than the sensor diameter, holding the sensors through a friction fit achieved by wrapping them with cut-open silicone tubes. The sensor tips were placed about 1 mm from the target.

[0054] Three characterization tests were performed on the CPC and copper sensors: baseline capacitance and noise level, capacitive proximity sensing distance, and response to different-sized hemisphere targets.

[0055] FIGs. 5A-5B shows the baseline capacitance and baseline noise level for the tested sensors, in accordance with the present technology. In the baseline measurements, all six cylindrical sensors (three CPC, three copper) were sequentially connected to a single AD7747 CDC. Three CPC sensors showed an average baseline capacitance and standard deviation of 229.6 ± 0.41 fF, while the copper reference sensors averaged 184.1 ± 0.47 fF. Adjusted for the CDC’s 122.5 fF baseline, the CPC sensors had an average baseline capacitance of 107.1 ± 0.41 fF compared with copper’s 61.6 ± 0.47 fF.The CPC sensor showed a 73.9% higher baseline capacitive while keeping the noise level 13.8% lower.

[0056] FIGs.6A-6I show testing setups and results for proximity sensing distance tests, in accordance with the present technology. For the proximity sensing distance test, CPC and copper sensors were tested in pairs by lowering them from 120 mm to 1 mm above the printer’s aluminum print bed in 0.1 mm increments (FIG.6A). At each increment, the 3D printer stopped for at least 350 ms to gather around 7 readings and kept the average value. Slots containing two sensors were separated by 12 mm in the x direction. To ensure positional consistency, tests were done in two passes with a 12 mm x direction shift. For the full 120 mm to 1 mm range, the reference copper sensor showed an average capacitance increase of 0.032 pF, while CPC averaged 0.066 pF, a value 106% higher. Using three times the noise level determined in the last baseline test as the detection threshold, the CPC sensors had a proximity sensing distance of 85 mm, compared to 45 mm for the copper sensors, representing an 89 % increase. Proximity response of CPC cylindrical sensors and copper cylindrical sensors to the printer bed (aluminum plate of 240 mm-by-240 mm) when the sensor was descending from 120 mm to 1 mm is shown in FIG.6A. The 3 sensors mean capacitance change and standard deviation are plotted in FIGs. 6B. FIG. 6C is a zoomed- in view. FIGs.6D-6F show the capacitive response of both CPC and copper sensors to the large r = 6.5 mm aluminum hemisphere at different heights. FIG.6E only has CPC plotted for less clutter, and the FIG, 6F has variable capacitance range to show finer details for larger height tests. FIGs. 6G-6I show capacitive response of both CPC and copper sensor to the smaller r = 3 mm aluminum hemisphere at different heights.

[0057] For different-sized hemisphere tests, sensors moved over aluminum hemisphere patterns at multiple passes at varying heights z. The small and large target had a diameter of 3 mm and 6.5 mm, respectively. These two sizes were chosen as the average cornea-to-lens distance is 3.6 mm and the average radius is 5.75 mm, making them suitable for approximating the displacement caused by eye rotation. Sensors moved across the hemisphere centers with an initial 1 mm sensor-to-hemisphere-top distance. Multiplepasses were made at different heights with increasing increments as sensitivity decreased with heights. The average and standard deviation of both CPC and reference copper cylindrical sensor (N = 3) were plotted at heights of 1.2, 3.1, 5, and 9.8 mm.

[0058] The CPC sensor consistently outperformed reference copper sensors in the hemisphere test. The capacitance change, calculated from the maximum capacitance minus the minimum, is shown in Table 1. For the large r = 6.5 mm hemisphere pattern showed about 70 fF of capacitive response when crossing the top of the hemisphere at 1.2 mm. The smaller 3 mm pattern, as seen in FIG. 6H generates a smaller response at 41 fF. On average, the CPC cylindrical sensor was twice as sensitive as the copper sensors at heights less than 5 mm in both hemisphere pattern sizes. Considering all heights, the CPC sensor is 75% more sensitive than copper in smaller hemispheres, and 99% more sensitive than copper sensors in large hemispheres, a lead consistent with the earlier proximity sensing distance characterization (106%). Table 1. Capacitance change from scanning over hemisphere patterns at different height Height CPC CPC (small Copper (small Z(fF)(fF) pattern) (fF)ned through eye displacement profiling at different gazes.

[0060] FIGs. 7A-7B shows testing done to determine optimal image sensor locations, in accordance with the present technology. To study how displacement on the eye region changes when the eye rotates, 3D scanning was to profile the eye region at different gaze directions. As the cornea was transparent and could not be captured by a 3D scanner, the focus was on profiling the adjacent eyelid area. Instead of analyzing the entire mesh, markers made from masking tape squares with blue dots were placed at specific positions around the eye for detailed study. Markers were placed near the edge of the uppereyelid and spread across the temporal, central, and nasal sides. These markers served as measurement markers since they moved with eye movements (labeled A1 – A3, C1 - C3 in FIG.7B). Markers outside the eye region (labeled A, B, C, and O* in FIG.7A), such as those on the forehead or cheek area, were assumed to be stationary across gaze movements and served as reference markers for aligning different 3D scans. Marker O* was not a physical tape square. Instead, it represented the interpolated middle point between A and C, serving as the center for multiple scan alignments.

[0061] Five subjects were scanned following the same protocol. Volunteers were instructed to fixate their gaze in a direction (either center, upper, lower, left, or right) for 5 seconds without blinking. During this time, another staff used a handheld EinScan H 3D scanner (Shining 3D, Hong Kong, China) to scan the participant’s face. This scanner used infrared illumination instead of flash, providing less accurate scanning but safe for human eyes. One scan was created for each gaze direction per participant, and the 3D coordinates of each marker were read from the 3D mesh.

[0062] FIGs. 8A-8B show the measurement accuracy of the sensors on five subjects, in accordance with the present technology. To estimate the measurement accuracy, the median absolute deviation (MAD) was calculated of the L1-norm between reference markers A and marker B as it should be constant for each subject. Among all five subjects, the point A-B L1-norm MAD varied between 0.8 mm to 1.41 mm, indicating accurate scanning accuracy.

[0063] Each scan generated a 3D mesh centered on the scanner body with an axis based on the scanner’s facing direction. As a result, marker coordinates from different scans must be transformed into the same global Cartesian coordinate for comparison. A global coordinate system was defined with an origin at point O, the middle point between A and C, and x, y, and z represented protrusion, width, and elevation. This affine transformation can be represented by Equation 1.^^^^^^^ ^ ^ ^ ^^^^^^^^^^^^^ ^^^ ^^^ ^^^ ^^ ^^^^^^^^^^^^^ ^ ^ ^ ^ ^^^^^^system, Pglobalcontains the location in the global coordinate system, and the T matrix contains an affine transformation needed to convert a coordinate from the scanner coordinate system location to the global coordinate system location. For each participant, the Pglobal for reference markers A, B, C, and O* was first established by manually inspecting a scan with center gaze. Then, for each scan of the same participant, T was estimated, satisfying the transformation for all reference markers A, B, C, and O*. Once T was calculated, all measurement marker coordinates from the same scan could be transformed into the global coordinate system.

[0065] FIGs. 9A-9F show the scanning results of the participants, in accordance with the present technology. While the scanning results varied between participants, some common trends were found. For each measurement marker location, this marker’s elevation and protrusion change was calculated from the central gaze when gazing up, down, left, and right. The five participants’ average and MAD are shown in FIGs.9A-9F. Ideally, one marker is only sensitive to one gaze direction, where complementary gazes produce opposite responses and orthogonal gazes produce small responses (for example, an ideal scenario for the vertical gaze sensor is protrusion increases with upward gaze, decreases with downward gaze, and almost no response to either left or rightward gaze).

[0066] The vertical gaze movements caused large protrusion and elevation changes in both nasal (C1, A3) and center (A2, C2) marker points (see the up and down arrows in row 2, center markers, and row 3, nasal markers). The center marker positions were better than nasal positions as vertical movement produced a larger change in elevation and protrusion, therefore a larger and more prominent proximity capacitance change. The horizontal gaze, however, was not particularly prominent in any of these marker locations.Since central and nasal markers were more sensitive to vertical movements, the two temporal side markers were chosen for horizontal measurements. Both saw an elevation and protrusion drop on opposite side movements (rightward gaze on the left-eye temporal side and leftward gaze on the right-eye temporal side) while having a minimal response to same-side movements. Horizontal movements could be measured by taking the difference between them.

[0067] In summary, the left and right eye center markers were best for vertical gaze sensing due to their large magnitude but opposite responses to upward and downward gaze. Although no marker position was ideal for horizontal gaze sensing, left and right temporal markers had low sensitivity to vertical gaze and opposite sensitivity to left and right horizontal gaze, making it possible to combine sensors at these locations for horizontal measurements.

[0068] FIGs.10A-10B are an example eye tracker device, in accordance with the present technology. Following the scanning result from the last section, a CPC eye-tracker with four sensors into two differential pairs was developed. Sensor pair 1 contained two sensors installed at the temporal side of both eyes to sense the horizontal gaze movement. Sensor pair 2 was above the center of both eyes but had slightly different spacing such that the signal did not cancel. Both positive differential inputs are above the left eye with the pair 1 at temporal side and pair 2 at center. The pair 2 negative is 2 mm higher than positive in this case. While using a single-ended setup for vertical sensing might be good enough from the results achieved, a differential setup provided common signal rejection, making the CPC eye-tracker less vulnerable to blinks, droops, and eye closures, which had large vertical signal amplitude but were synchronized on both eyes. All four sensors were adjustable by users at a sensor-to-eye distance between 5 and 10 mm, ensuring user comfort. The picture of the eyeglass can be seen in FIG.10A.

[0069] All four sensors were connected to an FDC1004 (Texas Instruments, TX, USA) CDC. Compared with the AD7747 CDC used in earlier sensor characterization sessions, the FDC1004 CDC operated on the same switched-capacitor principles butoffered a tenfold increase in sampling rate and supported four sensors. Nevertheless, it had a lower measurement accuracy and signal-to-noise ratio. Despite that, this compromise was acceptable because of the larger capacitance change from gaze due to improved sensor design and the larger adult human eyeball. The nRF52840 microcontroller was still used to interface with the CDC and relayed data to a computer through either Bluetooth Low- Energy protocol or USB protocol. USB communication was used in this section for the transmission bandwidth and latency performance.

[0070] CPC eye-trackers could be integrated into off-the-shelf eyeglass frames, allowing easy and non-obtrusive eye monitoring. This compatibility was demonstrated in a prototype created using a modified off-the-shelf safety visor. During the modification, the protective polycarbonate lens of the safety visor was removed, and two strips with equally spaced slots were attached to the top of the eyeglass frame to securely hold the sensors out of the vision field. The sensing circuitry was mounted on the forward-left side of the frame, just outside the left temple area.

[0071] In another aspect, disclosed herein is an eye tracking device including a glasses frame having a first lens and a second lens adjacent to the first lens, a first capacitive sensor as described herein, wherein the first capacitive sensor is located at a top center of the first lens, and a microcontroller configured to receive one or more capacitance measurements from the first capacitive sensor and detect one or more blinks, an eye closure duration, a percentage of eyelid closure over the pupil over time (PERCLOS), or a combination thereof based on the one or more capacitance measurements, as shown and described herein.

[0072] In some embodiments, the microcontroller is further configured to determine fatigue of a user based on the one or more blinks, the eye closure duration, the PERCLOS or a combination thereof.

[0073] In some embodiments, the eye tracking device further includes a second capacitive sensor described herein, where the second capacitive sensor is located at a top center of the second lens, and where the first capacitive sensor and the second capacitivesensor are configured to measure vertical position of the eyes, a third capacitive sensor according as described herein, located at a temporal location of the first lens, and a fourth capacitive sensor as described herein, located at a temporal location of the second lens, where the third capacitive sensor and the fourth capacitive sensor are configured to measure a horizontal position of the eyes.

[0074] In some embodiments, the first capacitive sensor is offset from the second capacitive sensor. In some embodiments, the first capacitive sensor is about 2 mm higher than the second capacitive sensor.

[0075] In some embodiments, the third capacitive sensor and the fourth capacitive sensor operate using differential measurement to measure the horizontal position of the cornea of the user’s eye. In some embodiments, the first capacitive sensor and the second capacitive sensor operate using differential measurement to measure the vertical position of the cornea of the user’s eye.

[0076] FIGs. 10C-10D show testing of the eye tracker in comparison to conventional eye-trackers, in accordance with the present technology. FIG. 10C is an illustration of the test setup. The screen has a size of (67, 39) cm, and participants sit at 80 cm. The full screen size is equivalent to a horizontal gaze angle range of 45.4°, vertical angle gaze of 27.4°. FIG. 10D is a flowchart of the test. The samplings of the reference Tobii eye-tracker and the CPC eye-tracker are independent, and latest sampled data from the Tobii is appended to every CPC eye-tracker measurement regardless of whether it has been refreshed.

[0077] The accuracy of the capacitive CPC eye tracker was benchmarked against the Tobii Pro Nano (Stockholm, Sweden), a commercial pupil and corneal reflection (PCR)-based eye-tracker. Participants sat approximately 80 cm from a 31-inch computer screen with a dimension of 67 cm by 39 cm. A custom Python script concurrently gathered the CPC eye-tracker’s reading via USB and the Tobii eye-tracker’s left-eye gaze data using the manufacturer-supplied Tobii Pro Python SDK. For each received CPC eye-tracker data, the latest Tobii reading was appended and written to a CSV file for later analysis. The CPCeye-tracker had a sampling of 189 Hz. The CPC eye-tracker and the Tobii eye-tracker updated asynchronously with the Tobii eye-tracker’s data updated approximately every sixth CPC sample, equating to a sampling rate of about 31.5 Hz.

[0078] Participants engaged in one-minute simulated saccade tracking tasks categorized into simpler on-axis cases and more complex diagonal cases. Unlike the non- human primate eye-tracker in Chapter 3, for both cases, there were neither red laser dots nor on-screen visual cues guiding the gaze. Instead, the gaze path was rendered on screen and participants moved to any point on the gaze path at their discretion. For the on-axis tasks, participants were asked to perform the saccade tracking on horizontal and vertical axes intersecting at the center of the monitor. In contrast, the off-axis tasks had participants perform a saccade gaze on the diagonal axis across the screen.

[0079] FIGs. 11A-11D show capacitive responses corresponding to different gaze location during on-axis and diagonal movements, in accordance with the present technology. The graph of FIGs.11A-11B show the apacitance signal and Tobii Eye-tracker recording of on-Axis eye movement. The horizontal and vertical components of the gaze angle and sensor capacitance are plotted in a time series and comparison. Tobii loses gaze track at a larger gaze angle or a saccade spanning a large angle. In right-side comparison charts, the gaze points are color encoded by gaze angle in the orthogonal direction.

[0080] FIGs. 12A-12D show the capacitance signal and Tobii Eye-tracker recording of diagonal eye movement, in accordance with the present technology. The horizontal and vertical component of the gaze angle and sensor capacitance, plotted in a time series and comparison.

[0081] FIGs.13A-13C show graphs of the on-Axis test (FIG.13B), Diagonal test (FIG. 13C), and a combined test (FIG. 13A), in accordance with the present technology. Using an 80-20 train-test split, the test case gaze from Tobii (labelled as Actual), and from custom eye tracker device and machine learning (labelled as Predicted).

[0082] The data processing pipelines can be seen in FIG. 10D. Capacitance data were processed through the Savitzky-Golay filter to smooth out high-frequency noises.Wavelet methods were not used due to the human capacitive eye-tracker’s better signal-to- noise ratio and higher sampling rate. The Savitzky-Golay filter, with a window length of 64 data points and a polynomial order of 5, had fewer parameters to tune and computed much faster, allowing possible real-time processing. Both capacitive pair 1 and pair 2’s readings were plotted with the commercial eye-tracker.

[0083] Tobii eye-tracker’s data, reporting as a pair of numbers between 0 and 1, represented the on-screen gaze location as percentages of screen width and height. Tobii’s gaze location followed the sign convention of computer graphics and originated from the upper-left corner of the display, increasing when gazing rightward or downward. Tobii eye- tracker returned a null value if on-screen gaze location could not be found due to eye closure, gaze out of screen area, or gaze velocity higher than tracking capacity. In this benchmark, the center of the screen, or (50%, 50%) location as reported by Tobii as 0° gaze and upward gaze was treated as positive. Thus, the gaze angles were calculated using the following formulas, with the vertical gaze angle flipped to adhere to the sign convention.The conversion is in Equation 2.!"^#^"$%&'^(&^)^*$+') ^ &%&$,^-.^ / 001 ^ ^23^^445 ^67 ^ 89)^%#:&'^(&^)^*$+') ^ 5&%&$,^;.^ / 001 ^ ^!3^^445 ^67 ^ 8 ^ ^,

[0084] where the lowercase w and h are screen width and height, uppercase W, H represents the percentages given by Tobii, and d is the spacing between participant and display.

[0085] The comparison result is shown in FIGs. 11A-11D and 12A-12D. In the Horizontal component of the “on-axis” test, the sensor pair 1 capacitance was linear with the gaze angle up to around ±10° from the center. Performing a least-squares fitting on acubic function model (Equation 3)< ^ &^^ = >^^ = :^ = 8^ ^?

[0086] where c is the capacitance, x is the gaze angle from the Tobii data, and a, b, c, d were function coefficients to be calculated from data where the squared loss was minimized. The cubic function fitting had an R2-score of 0.892. Calculating the sensitivity(fF / °) from the fitted line, the sensitivities were 0.53 fF / °, 0.34 fF / °, and 0.42 fF / ° for horizontal gaze angles smaller than -10°, between -10° and 10°, and larger than 10°, respectively.

[0087] For the vertical component of the on-axis test, the cubic fitting returned an R2-score of 0.814, and the sensitivity is around 0.57 fF / ° in sensor pair 2. While capacitive sensor reading still showed good agreement with the Tobii gaze data, it was affected by horizontal gaze. Encoding each gaze point with color based on horizontal gaze, an up-to-4 fF capacitance change was observed, caused by horizontal gaze movement.4 fF was approximately equal to 6.6° of vertical gaze angle, considering a horizontal gaze range of ±22°, the ratio is about 30%.

[0088] For the diagonal cases, the time series plot showed that sensor pair 1’s horizontal gaze sensitivity varied based on the eye’s vertical gaze. For example, as seen in FIGs. 12A-12D, from seconds 0 to 10, the gaze was upper-right, and sensor pair 1 was highly sensitive to horizontal gaze. However, in seconds 10 to 15, the gaze was lower-right, and the same magnitude rightward horizontal gaze produced a much less noticeable capacitance change. The only difference was that in the latter time, the vertical gaze was downward. After encoding each gaze point with color based on vertical gaze, it was observed that sensor pair 1 had much higher horizontal gaze sensitivity when also gazing upward. Using the same cubic functions curve fitting, when the vertical gaze was upward, the horizontal sensitivity was higher, with sensitivities of 1.7 fF / °, 0.52 fF / °, and 0.48 fF / ° for horizontal gaze angles smaller than -10°, between -10° and 10°, and larger than 10°, respectively. Conversely, when gazing downward, the horizontal sensitivities were at 0.09 fF / °, 0.19 fF / °, and 0.05 fF / °, respectively. The horizontal gaze sensitivity when gazing upward was about 9.4 times higher than gazing downward, and 4.9 times better than gazing center from the on-axis case. This difference was reasonable as all sensors were located on the upper side of the eye, when gazing downward, the distance between the scleral-corneal junction and the sensor increases. As capacitance sensitivity was inversely proportional to distance, the sensitivity became much smaller as distance increased.

[0089] Finally, the vertical component of the diagonal model was evaluated. When the horizontal gaze was to the right side (positive), the linearized sensitivity was about 0.48 fF / °, and the other one was 0.52 fF / °. Both values were similar, indicating the vertical sensitivity is not that affected by horizontal gaze, as the displacement to the sensor remained relatively unchanged regardless of horizontal gaze.

[0090] Using machine learning, it’s possible to deal with non-linearity and cross- channel interaction and make an accurate gaze prediction. For example, tree regression could yield accurate results. For this task, a regression tree ensemble was used, with detailed hyperparameters listed in Table 2. To prevent the model from getting too complex by overfitting edge cases, the maximum tree depth was limited to 15 while requiring a minimum of four gaze points to make a feature split. The dataset combined results from both on-axis and diagonal test cases. Using an 80%-20% train-test split and a random state of 0, the results are shown in FIGs. 13A-13C. Both diagonal and on-axis cases can be predicted well. However, the effect of vertical gaze on horizontal gaze sensitivity led to uncertainties in horizontal gaze predictions, especially when vertical gaze was close to or below zero. Overall, the device predicted gaze with a mean absolute error (MAE) between 0.97° and 1.75°, with combined cases near 1.37°. Table 2. Regression Tree Ensemble parameters for human gaze prediction Parameters Hyperparameters Model Type Regression Tree Ensemble Number of Trees 150 Max Depth 15 Minimum Sample per Leaf 4

[0091] In conclusion, the wearable CPC capacitive eye-tracker demonstrated high performance in tracking gaze when benchmarking against the commercial Tobii Pro Nano eye-tracker. Due to sensor placements, the CPC eye-tracker’s horizontal gaze sensitivity varied with gaze direction, with sensitivity higher for upward gaze. Withmachine learning, the eye-tracker predicted gaze with a median absolute error (MAE) between 0.97° and 1.75°.

[0092] FIGs. 14A-14C show an example fatigue monitoring system, in accordance with the present technology. A fatigue monitoring system was developed based on the gaze tracker. In the gaze tracker, differential sensing was used to attenuate blinks and eye closures. For fatigue monitoring, blink and eye closures were used as biomarkers. As a result, the vertical channel in the eye tracker was switched to a single-ended setup FIG. 14A. Only one sensor data is collected while others are disabled in the software. Another blue indicator LED, LiPo battery and SD card adapter are installed for storing data during unplugged use. Besides, due to the long running time for fatigue monitoring, the wearable fatigue monitor can generate more data than its internal memory can hold. To address this, a micro-SD card was used to buffer data or store it for later analysis. Since writing data to a micro-SD card can take a long and unpredictable amount of time, a double- buffering technique was implemented. This meant two separate buffers were used to store the data temporarily; while one buffer was being written to the SD card, the other buffer collected new data. This ensured no data loss even when operating at a high sampling rate. A real-time operating system (RTOS) helps manage this process, ensuring no data gets lost.

[0093] FIG. 14B shows capacitance decreases from eye closure as the eyelid retracts from the vertical sensor location. The sensor locations are labeled as black rectangles, and the eyelid boundary is in dashed lines. FIG. 14C shows blinks and eye closures detected by different parameters in the find_peak algorithm. The star marker indicated the detected eye closures. Row 1: Using all default parameters. Row 2: Using the Minimum Peak Prominence in Table 11. Row 3: Using all parameters.

[0094] The capacitive data were sampled at 200 Hz and smoothed with the same Savitzky-Golay filter of 64 window size, 5 polynomial order. Blinks and eye closures can be seen as a sudden dip of the capacitive signal due to the temporary retraction of the eyelid during eye closure (as seen in FIG. 14B). To identify the peaks from normal gaze movement, ‘scipy.signal.find_peaks’ function was used to automatically detect them. In itsdefault configuration, the find_peaks algorithm detects every peak (FIG. 14C, first row). To identify significant signals that stand out against noise, a minimum peak prominence was determined. Given the variability in wearing positions, face geometry, and different blinks / eye closure behaviors, it’s impractical to decide a one-size-fits-all prominence threshold. Instead, a threshold based on signal standard deviation was used. A 1-Hz cut- off, second-order high-pass filter was used to attenuate gaze movements, which usually had a longer timeframe. The threshold was subsequently calculated as twice the standard deviation of the filtered signal. The improved signal detection can be seen in row 2 of FIG. 14C.

[0095] In another aspect, disclosed herein is an eye tracking device including a glasses frame having a first lens and a second lens adjacent to the first lens, a first capacitive sensor as described herein, wherein the first capacitive sensor is located at a top center of the first lens, and a microcontroller configured to receive one or more capacitance measurements from the first capacitive sensor and detect one or more blinks, an eye closure duration, a percentage of eyelid closure over the pupil over time (PERCLOS), or a combination thereof based on the one or more capacitance measurements.

[0096] In some embodiments, the microcontroller is further configured to determine fatigue of a user based on the one or more blinks, the eye closure duration, the PERCLOS or a combination thereof.

[0097] Despite improvements, the detection algorithm still had a few false detections where the find_peak algorithm incorrectly identified the minima of a downward gaze as blinks. For instance, a false blink was detected around the second 48 as the algorithm treated it as a local minimum for a long dip between the second 45 and second 53. To better detect blinks rather than gaze movements, the algorithm’s parameters were tuned to align with typical blink characteristics. Various research across demographics have found the blink duration range between 0.25 to 0.55 seconds. As a result, the width of the peak was determined to be between 0.1 to 0.7 seconds when evaluated at 50% of thepeak’s relative height (RH), and the minimum separation between consecutive blinks was set to 0.3s. The full parameters for the blink detection algorithm are in Table 3. Table 3. Parameters for blink detection algorithms for capacitive eye-tracker Peak Detection Parameter Peak Detection Parameter Value Peak Width [0.1, 0.7] second Relative Height for Eye Closure Time Width at 50% peak height Calculation Minimum Spacing between Consecutive 0.3 s Blinks Minimum Peak Prominence Standard Deviation of high-pass (fs= 1 Hz, order = 2) filtered signal

[0098] The detected blinks and durations using the tuned algorithm can be seen in FIG. 14C, row 3. The accuracy of this algorithm was further validated through a test with manual counting.

[0099] Blink frequencies and PERCLOS were biomarkers detected using the capacitive eye-tracker for fatigue characterization. The capacitive eye-tracker’s biomarker detection was validated against camera footage.

[0100] FIGs.15A-15C show capacitive eye-tracker based blinks and percentage of eyelid closure over the pupil over time (PERCLOS) digital biomarker detection, benchmarked against manual counting and computer-vision based algorithms, in accordance with the present technology. FIG. 15A shows flowcharts of how blinks and PERCLOS are counted with all three methods. The 6 landmark points (p1 – p6) for Eye Aspect Ratio (EAR) calculation are labelled. FIG. 15B shows a comparison between capacitive signal and EAR from computer-vision based method. The automatically identified blinks and change amplitude are labelled by the star symbol and vertical line. FIG.15C is a zoomed view of a closing edge of a blink as well as the corresponding video frames, placed at the capacitance level corresponding in time. This specific blink starts at around 100 ms time mark, took about 150 ms for eye closure, and the reopening took about another 300 ms. Eye closure time is the peak width estimated at half of the peak prominence, represented as the red dashed line.

[0101] FIGs. 16A-16B are statistics for the whole 3-minutes test for 3 subjects, in accordance with the present technology. The statistics are calculated on 30-second segments. FIG.16A is the blink count per 30-second deviation from manual counting from custom eye tracker and camera. FIG.16B is the PERCLOS deviation from manual counting from custom eye tracker and camera.

[0102] The setup can be visualized in FIG. 15A. For the video acquisition, a digital camera was set up on a tripod on the same level as the subject’s eyes. The camera was placed at a sufficiently far distance to capture the entire face of the subject and recorded video at 1080p resolution and 60 frames-per-second (fps). Subjects were asked to sit still on a chair and look at the camera lenses for over 3 minutes. When the eye-tracker started up, the installed blue LED light briefly lit up for 2 seconds before starting any sampling. The video segment prior to the LED indication was trimmed.

[0103] Subsequently, the video was divided into 6 counts of 30-second segments using the FFmpeg software tool. For each video segment, the blink count and PERCLOS were counted manually, as well as automatically with the capacitive eye-tracker and a computer-vision-based eye closure counter for cross-references. As shown in the flowchart FIG. 15A, in the manual detection workflow, every 3 frames (50ms) of frames of video were sampled and compiled into a collage for manual inspection. Each frame resembling eye closure was labeled, and the total number of such frames was counted. Each frame labeled was considered as a full 50 ms of closure, a reasonable assumption given that blink and closure durations are much longer than 50 ms. The capacitive sensor blink detection used the algorithm described in the last section.

[0104] In the computer-vision-based detection, the participant’s face was detected using the frontal_face_detector module from the dlib software package with the pre-trained shape_predictor_68_face_landmarks facial landmark model. For each video frame, 6 landmark points (p1 to p6 in FIG.15A) were identified from the eye region, and the eye aspect ratio (EAR) was calculated as Equation 4.BC^ 5 ^ CB = BC^ 5 ^ C@*A ^ ^ D ^ E B^C C ^ G,^B ^^ 5 ^F B

[0105] where aclosures. Through the reduces when subjects rotate their heads vertically, therefore causing false detection. Thus, the same find_peaks algorithm used in the capacitive eye-tracker was ported to detect EAR with peak width and minimum separation parameters set identically. The EAR dip threshold was set at 0.22, with a prominence higher than 0.04.

[0106] The result is shown in FIG.15A-15C and FIGs.16A-16B. In FIG.15B, a 30-second capacitance signal from the capacitive eye-tracker is plotted along with the EAR calculated by the computer-vision algorithm. It is evident that blinks detected by the capacitive eye tracker correspond one-to-one with EAR dip events. Unlike the EAR-based algorithm, the capacitive eye tracker can also detect gaze movement, as seen in the lower- magnitude, longer time-frame capacitance changes. FIG. 15C shows a zoomed-in view of a specific blink’s capacitance change as well as the corresponding video frame. The 50% relative height (RH) line, whose length was treated as blink duration, was marked red.

[0107] Results from three subjects are shown in FIGs. 16A-16B. The capacitive eye-tracker showed good agreement with the manual counting as well as the computer- vision-based algorithm. The capacitive eye-tracker, as well as the computer-vision-based algorithm, is expected to miss less than 1 blink and less than 1% of PERCLOS per 30- second segment. However, the capacitive eye tracker does over-count blink and PERCLOS at times, as seen in outliers. This over-counting outlier indicates a potential impact from the gaze movement, where a blink conjugated with a downward gaze could appear longer.

[0108] It was found that the capacitive eye-tracker could measure blink frequency and eye closure durations, which were important digital biomarkers for eye-based fatigue detection. Benchmark results against manual counting and computer-vision-based detection methods showed that the capacitive eye tracker could accurately track these two biomarkers. The capacitive eye-tracker was also tested with noise-induced fatigue.

[0109] FIGs.17A-17F show the set up and results of the blink rate and PERCLOS of a 15-minute induced fatigue test, in accordance with the present technology. FIG. 17A shows the test protocol. In the first minute, no specific instruction is given and the participant takes a rest. Between minute 2 to minute 15 the participants were asked to solve a double-digits number multiplication or addition question. Participants can choose between multiplication or addition such that the questions are neither too easy nor overly difficult. Between 5 and 10 minutes, noise is played to induce higher fatigue.

[0110] FIG. 17B shows the blink Rate and PERCLOS for minute 1 (resting baseline) and minute 2. FIGs. 17C-17D are boxplots showing blink rate (FIG. 17C) and PERCLOS (FIG.17D) increases from minute 2. The median of all tests was highlighted by the thick line. FIGs.17E-17F are boxplots showing a blink rate (FIG.17E) and PERCLOS (FIG.17F) percentage increases from minute 2. The median is highlighted by the thick line.

[0111] The fatigue tracker was tested on introduced fatigue conditions. The entire test lasted for 15 minutes. In the first minute, the participant didn’t get any instructions and could take a rest, and the blink and PERCLOS measurements were considered baseline values. After the first minute, participants began solving mathematical problems involving the multiplication or addition of two numbers less than 100. Participants could choose between multiplication or addition, so the difficulty was neither too high nor too low. The correctness of math solving was not considered as a fatigue indicator. To induce a higher level of fatigue, between minutes 5 and 10, a noise clip composed of three 120 beats-per- minute metronome ticks followed by a 0.5-second 1000-Hz alarm noise was played. Participants were asked to continue working on the math questions during and after the noise until the end of tests. Participants were asked to report their fatigue level on the Karolinska Sleepiness Scale (KSS) at the end of the test. The whole protocol can be seen in FIG.17A.

[0112] A preliminary group of 10 participants were included in this test. The data are shown in FIGs.17B-17F. FIG.17B contrasts the blink rate and PERCLOS between the first minute (“Baseline”) and the 2nd minute, where the math solving starts. In the first twominutes, the median blink counts were close to 10 blinks per minute, and PERCLOS was around 0.08. However, the range distribution was much wider during the baseline than in the second minute, indicating that participants with higher blink rates experienced a significant decrease when starting the math-solving task. This decrease in eye closure due to concentration has been observed in studies, where blinks have been regarded as breaks in data processing that occurred less frequently during concentration.

[0113] FIG. 17C and FIG. 17E show an increase in blink count and PERCLOS after the second minute, with FIG.17D representing differences and FIG.17F representing percentage change. Both figures demonstrate large individual differences, increasing even more during the noise period. Each box in the boxplot represented one minute in the test. The whole test data were divided into three time segments: Pre-noise (3 to 5 minutes), At- noise (6 to 10 minutes), and Post-noise (11 to 15 minutes). For each time segment, the median of each box’s median value and the average of each box’s IQR were calculated, as shown in Table 4.

[0114] The results showed an increase in fatigue between minutes 2 and 5, suggesting that the time-on-task (ToT) effect of math solving alone could cause fatigue. Between minutes 6 to 10, the combination of noise and math solving led to an even higher blink count and PERCLOS. After the noise ended, both blink rate and PERCLOS decreased. Results indicated that both math-solving and noise could effectively induce fatigue. Notably, the IQR during the noise segments was significantly higher than other math-only segments, indicating the fatigue caused by noise has more individual variance than fatigue from math-solving.Table 4. The median and inter-quartile range (IQR) of increases from minute 2 at different test segments. Metric Time Ranges Absolute Increase Percentage Increase from Minute 2 from Minute 2 (FIGs. -on- nanotube paper composite (CPC) for human eye tracking and fatigue monitoring. The cylindrical CPC sensor addressed the usability issues of flat CPC sensors and, when compared to identically sized copper sensors, demonstrated higher sensitivity and lower noise. The fabricated CPC cylindrical sensors were integrated into an eyeglass frame for non-intrusive human gaze tracking. Benchmark against the commercial Tobii Pro Nano eye-tracker showed that the CPC sensors could accurately track gaze with a median absolute error (MAE) of 0.97° to 1.75°.

[0116] Additionally, the capacitive eye-tracker could detect fatigue by measuring blink frequency and PERCLOS (Percentage of Eyelid Closure over the Pupil) biomarkers. Compared with manual counting and computer vision-based algorithms, the eye-tracker showed less than 1 missed blink count and 1% PERCLOS deviation per 30-second segment. Real-world testing on noise-introduced trials with 10 participants successfully demonstrated the eye-tracker capability in detecting blink and PERCLOS biomarkers associated with time-on-task and noise.

[0117] In conclusion, the cylindrical CPC sensor-based capacitive eye-tracker could be a sensitive, non-contact method for both gaze tracking and fatigue monitoring. Itseasy integration into wearable devices and existing eyeglasses allows new possibilities for continuous and unobtrusive monitoring of the human gaze and fatigue in numerous applications.

[0118] In summary, an ultrasensitive capacitive sensor was developed using carbon nanotube-paper composite (CPC) materials and demonstrated applications in human-machine interfaces (HMI) and eye tracking. Finite element analysis results showed that capacitive sensitivity could be increased by decreasing the baseline capacitance by using a smaller electrode or by increasing the with-target capacitance using a fibrous electrode. Using a single fibrous electrode achieved both. The sensor was made from a tensile fracture of CPC, which exposed numerous cellulose fibers. CPC capacitive sensors had higher capacitive sensitivity due to a higher electric field and total surface area from the high aspect ratio cellulose fiber microstructures. Through numerical simulation and experiment, the 10 mm-width single electrode CPC capacitive sensor demonstrated a 31.5% higher capacitive sensitivity than an identical-sized silver sensor, with a hand proximity sensitivity up to 300 mm and pressure sensitivity as low as 64 Pa. The sensor’s application on water sensing, hand sensing, and machine-learning enhanced gesture recognition was demonstrated. The CPC capacitive sensor is one of the best in terms of sensitivity-to-size ratio and can have great potential in wearable human-machine interface applications.

[0119] Further, a novel wearable capacitive eye-tracker for non-human primates was developed, utilizing four sensors in two pairs to measure proximity changes from the scleral-corneal junction during gaze movement. The capacitive eye-tracker was benchmarked against the gold standard scleral search coil method, showing a high linear correlation up to 0.97. Various data processing methods were explored, and alongside machine learning, the gaze tracking accuracy was as high as 0.3°, with a median accuracy of 0.8° for a single gaze pattern, comparable to commercial solutions while being portable. This non-invasive capacitive eye-tracker served as an alternative to traditional coil and camera-based systems and had a great impact on oculomotor research and vision science.

[0120] The eye-tracking applications were further expanded to human eye- tracking. To address the sensor adjustment, displacement specificity, and limiting vision field challenges exposed in the non-human primate eye-tracker, a novel cylindrical sensor was fabricated by wrapping fractured CPC around a polyurethane core. Benchmark against identical sized copper sensor showed a nearly 100% higher capacitive sensitivity. The eye displacement during different gaze directions was studied with face scanning, and the results were used to optimize sensor placement to maximize sensitivity while mitigating interference. With optimized sensor placement and machine learning, the human eye- tracker demonstrated 1.37° median absolute error (MAE) in various gazes over 45° gaze angles. The application was further extended to detect fatigue by quantifying blink counts and eye closure duration biomarkers. Compared with manual counting and computer- vision-based algorithms, the eye-tracker showed less than 1 missed blink count and 1% PERCLOS deviation per 30-second segment. A preliminary 10-subject test on noise- induced fatigue demonstrated the eye-tracker capability to detect blink and PERCLOS biomarkers in real-world applications among different people. Due to the light weight and low power parameters, this device offers a unique capability for all-day fatigue monitoring in a compact, wearable form, with wide applications in industrial safety, behavioral research, and medical diagnostics.

[0121] The CPC human eye-tracker is lightweight and low profile and offers high tracking resolution, long running time, and high portability. It can be further developed into applications for eye-controlled HMI, for example, allowing elders or disabled people to communicate or signal with assistors with only gaze inputs, or controlling robotics with eye gaze for better human-machine collaboration.

[0122] The fatigue-tracking capability of the CPC human eye-tracker has significant potential for long-term, unobtrusive fatigue monitoring. Preliminary data indicate high accuracy in biomarker tracking and a strong correlation with fatigue in noise- induced fatigue tests.

[0123] In another aspect, disclosed herein is a method of determining fatigue with the eye tracking device as described herein, the method including receiving one or more capacitance measurements from the first capacitive sensor, detecting one or more blinks, an eye closure duration, a percentage of eyelid closure over the pupil over time (PERCLOS), or a combination thereof based on the one or more capacitance measurements, and determining a presence of fatigue, a severity of fatigue, or a combination thereof.

[0124] In some embodiments, the method further includes disabling the second capacitive sensor, the third capacitive sensor, and the fourth capacitive sensor before receiving the one or more capacitance measurements from the first capacitive sensor.

[0125] In some embodiments, the method further includes using differential measurement to measure a horizontal position of the cornea of the user’s eye with the third capacitive sensor and the fourth capacitive sensor operate.

[0126] In yet another aspect, disclosed herein is a method of eye-tracking with the eye-tracking device as described herein, the method including measuring capacitance from the first capacitive sensor, the second capacitive sensor, the third capacitive sensor and the fourth capacitive sensor, and determining a position of a cornea of each of the user’s eyes based on the measured capacitance.

[0127] FIG. 18 is a method of determining fatigue with the eye tracking device disclosed herein, in accordance with the present technology.

[0128] In block 1805, Optionally, a second, third, and fourth capacitive sensor (such as the capacitive sensors of an eye tracker as shown in FIG.10B are disabled.

[0129] In block 1810, capacitance measurements are received from a first capacitive sensor. In the case where block 1805 is performed, this results in a single sensor measurement as shown and described in FIG.14A.

[0130] In block 1015, eye closure metrics are detected. Eye closure metrics may include blinking and PERCLOS, among other metrics.

[0131] In block 1820, the presence of fatigue is determined with a machine learning (ML) algorithm. In some embodiments, the ML algorithm is any of the ML algorithms described in detail herein.

[0132] Optionally, in block 1825, a differential measurement may be used to measure a horizontal position of a user’s eye. In some embodiments, this may be done by one or more of the at least four sensors of the eye-tracker.

[0133] Optionally, in block 1830, the capacitance may be measured from the first, second, third, and fourth capacitive sensor.

[0134] Optionally, in block 1835, the capacitive measurements of the first, second, third, and fourth capacitive sensor may determine a position of a wearer’s cornea.

[0135] While illustrative embodiments have been illustrated and described, it will be appreciated that various changes can be made therein without departing from the spirit and scope of the invention.

[0136] The present application may reference quantities and numbers. Unless specifically stated, such quantities and numbers are not to be considered restrictive, but representative of the possible quantities or numbers associated with the present application. Also, in this regard, the present application may use the term “plurality” to reference a quantity or number. In this regard, the term “plurality” is meant to be any number that is more than one, for example, two, three, four, five, etc. The terms “about,” “approximately,” “near,” etc., mean plus or minus 5% of the stated value. For the purposes of the present disclosure, the phrase “at least one of A, B, and C,” for example, means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C), including all further possible permutations when greater than three elements are listed.

[0137] Embodiments disclosed herein may utilize circuitry in order to implement technologies and methodologies described herein, operatively connect two or more components, generate information, determine operation conditions, control an appliance, device, or method, and / or the like. Circuitry of any type can be used. In an embodiment, circuitry includes, among other things, one or more computing devices such as a processor(e.g., a microprocessor), a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like, or any combinations thereof, and can include discrete digital or analog circuit elements or electronics, or combinations thereof.

[0138] An embodiment includes one or more data stores that, for example, store instructions or data. Non-limiting examples of one or more data stores include volatile memory (e.g., Random Access memory (RAM), Dynamic Random Access memory (DRAM), or the like), non-volatile memory (e.g., Read-Only memory (ROM), Electrically Erasable Programmable Read-Only memory (EEPROM), Compact Disc Read-Only memory (CD-ROM), or the like), persistent memory, or the like. Further non-limiting examples of one or more data stores include Erasable Programmable Read-Only memory (EPROM), flash memory, or the like. The one or more data stores can be connected to, for example, one or more computing devices by one or more instructions, data, or power buses.

[0139] In an embodiment, circuitry includes a computer-readable media drive or memory slot configured to accept signal-bearing medium (e.g., computer-readable memory media, computer-readable recording media, or the like). In an embodiment, a program for causing a system to execute any of the disclosed methods can be stored on, for example, a computer-readable recording medium (CRMM), a signal-bearing medium, or the like. Non- limiting examples of signal-bearing media include a recordable type medium such as any form of flash memory, magnetic tape, floppy disk, a hard disk drive, a Compact Disc (CD), a Digital Video Disk (DVD), Blu-Ray Disc, a digital tape, a computer memory, or the like, as well as transmission type medium such as a digital and / or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link (e.g., transmitter, receiver, transceiver, transmission logic, reception logic, etc.). Further non-limiting examples of signal-bearing media include, but are not limited to, DVD-ROM, DVD-RAM, DVD+RW, DVD-RW, DVD-R, DVD+R, CD-ROM, Super Audio CD, CD-R, CD+R, CD+RW, CD-RW, Video Compact Discs, Super Video Discs, flash memory, magnetic tape, magneto-optic disk, MINIDISC, non-volatile memorycard, EEPROM, optical disk, optical storage, RAM, ROM, system memory, web server, or the like.

[0140] The detailed description set forth above in connection with the appended drawings, where like numerals reference like elements, are intended as a description of various embodiments of the present disclosure and are not intended to represent the only embodiments. Each embodiment described in this disclosure is provided merely as an example or illustration and should not be construed as preferred or advantageous over other embodiments. The illustrative examples provided herein are not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Similarly, any steps described herein may be interchangeable with other steps, or combinations of steps, in order to achieve the same or substantially similar result. Generally, the embodiments disclosed herein are non-limiting, and the inventors contemplate that other embodiments within the scope of this disclosure may include structures and functionalities from more than one specific embodiment shown in the figures and described in the specification.

[0141] In the foregoing description, specific details are set forth to provide a thorough understanding of exemplary embodiments of the present disclosure. It will be apparent to one skilled in the art, however, that the embodiments disclosed herein may be practiced without embodying all the specific details. In some instances, well-known process steps have not been described in detail in order not to unnecessarily obscure various aspects of the present disclosure. Further, it will be appreciated that embodiments of the present disclosure may employ any combination of features described herein.

[0142] The present application may include references to directions, such as “vertical,” “horizontal,” “front,” “rear,” “left,” “right,” “top,” and “bottom,” etc. These references, and other similar references in the present application, are intended to assist in helping describe and understand the particular embodiment (such as when the embodiment is positioned for use) and are not intended to limit the present disclosure to these directions or locations.

[0143] The present application may also reference quantities and numbers. Unless specifically stated, such quantities and numbers are not to be considered restrictive, but exemplary of the possible quantities or numbers associated with the present application. Also, in this regard, the present application may use the term “plurality” to reference a quantity or number. In this regard, the term “plurality” is meant to be any number that is more than one, for example, two, three, four, five, etc. The term “about,” “approximately,” etc., means plus or minus 5% of the stated value. The term “based upon” means “based at least partially upon.”

[0144] The principles, representative embodiments, and modes of operation of the present disclosure have been described in the foregoing description. However, aspects of the present disclosure, which are intended to be protected, are not to be construed as limited to the particular embodiments disclosed. Further, the embodiments described herein are to be regarded as illustrative rather than restrictive. It will be appreciated that variations and changes may be made by others, and equivalents employed, without departing from the spirit of the present disclosure. Accordingly, it is expressly intended that all such variations, changes, and equivalents fall within the spirit and scope of the present disclosure as claimed.

Claims

CLAIMS We claim:

1. A capacitive sensor, comprising: a composite substrate formed of a plurality of insulating fibers coated with a plurality of carbon nanotubes (CNTs); a plurality of cantilever-shaped conducting fibers at or near a fracture site in the composite substrate; and a sensor core, wherein the composite substrate is wrapped around the sensor core to form a cylindrical capacitive sensor.

2. The capacitive sensor of Claim 1, further comprising: wherein the sensor substrate can be replaced with other electrically conductive films.

3. The capacitive sensor of Claim 1, further comprising: wherein the sensor core comprises a non-conductive material such as polyurethane.

4. The capacitive sensor of Claim 1 or Claim 2, further comprising: a silver pasting located on one edge of the cylindrical composite substrate.

5. The capacitive sensor of Claim 3, further comprising: a coaxial cable coupled to the silver pasting.

6. The capacitive sensor of any one of Claims 1-4, further comprising an insulating tape configured to secure the composite substrate.

7. The capacitive sensor of Claim 5, wherein the insulating tape comprises a polyimide self-adhesive tape.

8. The capacitive sensor of Claim 5 or Claim 6, further comprising: an electical shielding layer located around the insulating tape.

9. The capacitive sensor of Claim 7, wherein the shielding layer comprises copper foil or other electrically conductive film.

10. The capacitive sensor of Claim 7 or Claim 8, wherein the sensor further comprises a shielding mesh coupled to the shielding layer.

11. An eye tracking device comprising: a glasses frame comprising: a first lens, and a second lens adjacent to the first lens; a first capacitive sensor according to any one of Claims 1-9, wherein the first capacitive sensor is located at a top center of the first lens; and a microcontroller configured to receive one or more capacitance measurements from the first capacitive sensor and detect one or more blinks, an eye closure duration, a percentage of eyelid closure over the pupil over time (PERCLOS), or a combination thereof based on the one or more capacitance measurements.

12. The eye-tracking device of Claim 11, wherein the microcontroller is further configured to determine fatigue of a user based on the one or more blinks, the eye closure duration, the PERCLOS or a combination thereof.

13. The eye tracking device of Claim 11 or 12, wherein the eye tracking device has an adjustable holder to modify the sensor orientation and distance to an eye.

14. The eye-tracking device of any one of Claims 11-13, wherein the eye tracking device further comprises: a second capacitive sensor according to any one of Claims 1-9, wherein the second capacitive sensor is located at a top center of the second lens, wherein the first capacitive sensor and the second capacitive sensor are configured to measure vertical position, angle, closure, and blink of the eyes;a third capacitive sensor according to any one of Claims 1-9, located at a temporal location of the first lens; and a fourth capacitive sensor according to any one of Claims 1-9, located at a temporal location of the second lens, wherein the third capacitive sensor and the fourth capacitive sensor are configured to measure a horizontal position, angle, closure, and blink of the eyes.

15. The eye-tracking device of Claim 14, wherein the first capacitive sensor is offset from the second capacitive sensor.

16. The eye-tracking device of Claim 14 or Claim 15, wherein the first capacitive sensor is about 2 mm higher than the second capacitive sensor.

17. The eye-tracking device of any one of Claims 14-16, wherein the third capacitive sensor and the fourth capacitive sensor operate using differential measurement to measure the horizontal position or angle of the cornea of the user’s eye.

18. The eye-tracking device of any one of Claims 14-17, wherein the first capacitive sensor and the second capacitive sensor operate using differential measurement to measure the vertical position or angle of the cornea of the user’s eye.

19. A method of determining fatigue with the eye tracking device of any one of Claims 10-16, the method comprising: receiving one or more capacitance measurements from the first capacitive sensor; detecting one or more blinks, an eye closure duration, a percentage of eyelid closure over the pupil over time (PERCLOS), or a combination thereof based on the one or more capacitance measurements; and determining a presence of fatigue, a severity of fatigue, or a combination thereof using a machine learning based classification and scoring algorithms.

20. The method of Claim 19, further comprising:disabling the second capacitive sensor, the third capacitive sensor, and the fourth capacitive sensor before receiving one or more capacitance measurements from the first capacitive sensor.

21. The method of Claim 19 or Claim 20, further comprising: using differential measurement to measure a horizontal position of the cornea of the user’s eye with the third capacitive sensor and the fourth capacitive sensor operate.

22. A method of eye-tracking with the eye-tracking device of any one of Claims 14-18, the method comprising: measuring capacitance from the first capacitive sensor, the second capacitive sensor, the third capacitive sensor and the fourth capacitive sensor; and determining the position of a cornea of each of the user’s eyes based on the measured capacitance.

23. A method of eye-tracking with the eye-tracking device of any one of Claims 14-18, the method comprising measuring the eye movement of closed eyes.

24. A system of eye-tracking with the eye-tracking device of any one of Claims 14-18, wherein the sensors are configured for human machine interfacing, cognitive monitoring, neurological disorder diagnosis, industrial and safety monitoring, training, gaming, entertainment, market research, or a combination thereof.

Images