Instantaneous stress algorithm for wearable computing device

Abstract

A method of monitoring stress of a user includes receiving a plurality of time-series data inputs from a plurality of biometric sensor electrodes of a wearable computing device. The time series data input includes continuous skin electrical activity data and at least one of: heart rate data, skin temperature data, and heart rate variability data. The method further includes processing the time series data input using a plurality of filtering techniques in sequence. Further, the method includes selecting a model from the plurality of models based on a type of data input received as a time series data input, and applying the selected model to the processed time series data input to calculate an indicator of a physiological response of the user at a time. Thus, the selected model is tailored to use all of the time series data inputs in calculating an indicator of the physiological response. Further, the method includes controlling a function of the device when the indicator of the physiological response exceeds a threshold.

Description

Technical Field

[0001] The present disclosure relates generally to wearable computing devices and, more particularly, to transient stress algorithms for wearable computing devices. Background Art

[0002] Recent technological advances, including those available through consumer devices, have provided corresponding advances in health detection and monitoring. For example, biometric monitoring devices such as fitness trackers and smart watches are able to determine information related to the pulse and movement of the person wearing the device.

[0003] Some biometric monitoring devices include multiple sensors for measuring multiple biological parameters that may be beneficial to the user of the device, such as heart rate sensors, multi-purpose electrical sensors compatible with electrocardiogram (ECG) and electrodermal activity (EDA) applications, infrared sensors, gyroscopes, altimeters, accelerometers, temperature sensors, ambient light sensors, Wi-Fi, GPS, vibration sensors, speakers and microphones, and other sensors. As an example, some biometric monitoring devices use multi-path electrical sensors to measure the EDA responses of the user's skin. These responses are observed as sensitive electrical changes in skin conductance and are typically detected on the palm or fingertips of the user using a wet or dry electrode system.

[0004] A typical EDA response can be measured at the palm or fingertips using at least two electrodes, where the skin conductance is calculated using the measured electrical impedance. The EDA response is expressed as the phasic component of skin conductance, the skin conductance response (SCR), and is detected by identifying transient spikes in skin conductance by comparison with a background tonic measurement, the skin conductance level (SCL). In general, due to the high sweat gland density in the palm or fingertips of a user, the SCR is more accurately observed from data collected from these areas.

[0005] Although SCR detection at the palm or fingertips has been fully reported in the literature for assessing stress, SCL alone can be beneficial for assessing stress in a user. By measuring continuous electrodermal activity (cEDA), SCL can be used to observe certain biological events, such as the body's response to an acute stressful event. However, if electrodes mounted on the top surface of the biometric monitoring device are used, it is challenging to use cEDA measurements to assess acute stressful events because cEDA requires continuous skin contact to provide accurate readings (i.e., requiring the user to position their skin above the electrode surface for the duration of the measurement). Therefore, in order to facilitate continuous contact with the skin and not require frequent user input to facilitate continuous EDA measurements, the electrodes for cEDA are positioned on the bottom (skin-facing) side of the biometric monitoring device.

[0006] More specifically, in terms of timekeeping, the main difference between SCL and SCR is that SCR occurs on a second scale, while SCL is evaluated across seconds, minutes, hours, and / or days. As an example, Figure 1 A graphical representation of EDA amplitude versus time is shown. As shown, the graph provides a comparison of the phasic skin conductance component (SCR) expressed as a peak value with the tonic skin conductance component (SCL). Thus, accurate detection of changes in SCL requires continuous measurement (over a few seconds / minutes / hours / days, etc.) and can be used to assess physiological stress with or without detection of SCR.

[0007] Therefore, a wearable computing device that continuously measures EDA in order to accurately detect transient or acute stress events and displays such events to a user would be welcome in the art. In addition, inputs from other sensors (e.g., photoplethysmography data (such as amplitude), accelerometer data, etc.) can provide additional context for evaluating acute stress events and / or displaying acute stress events to a user. Summary of the invention

[0008] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments.

[0009] In one aspect, the present disclosure relates to a method for monitoring stress of a user using a wearable computing device. The method includes receiving a plurality of time series data inputs from a plurality of biometric sensor electrodes of the wearable computing device via a processor communicatively coupled to the wearable computing device. The plurality of time series data includes continuous electrodermal activity (cEDA) data of the user and at least one of the following: heart rate data of the user, skin temperature data of the user, and heart rate variability (HRV) data of the user. The method also includes sequentially processing the plurality of time series data inputs using a plurality of filtering techniques. Further, the method includes selecting a model from a plurality of models based on the type of data input received as the plurality of time series data inputs. In addition, the method includes applying the selected model to the processed plurality of time series data inputs to calculate an indicator of a physiological response of the user at a certain time, wherein the selected model is customized to use all of the plurality of time series data inputs in calculating the indicator of the physiological response. In addition, the method includes controlling a function of the wearable computing device when the indicator of the physiological response exceeds a threshold.

[0010] In another aspect, the present disclosure relates to a wearable computing device. The wearable computing device includes an electronic display, a plurality of biometric sensor electrodes for sensing a plurality of time series data inputs related to a biometric of a user of the wearable computing device, and at least one processor communicatively coupled to the plurality of biometric sensor electrodes. The processor is configured to perform a plurality of operations, including but not limited to receiving a plurality of time series data inputs. The plurality of time series data includes continuous electrodermal activity (cEDA) data of the user and at least one of the following: heart rate data of the user, skin temperature data of the user, and heart rate variability (HRV) data of the user. The operations further include sequentially processing the plurality of time series data inputs using a plurality of filtering techniques, selecting a model from a plurality of models based on the type of data input received as the plurality of time series data inputs, and applying the selected model to the processed plurality of time series data inputs to calculate an indicator probability of a stressful event for the user at a certain time. Thus, the selected model is customized to use all of the time series data inputs in the plurality of time series data inputs when calculating the indicator probability of a stressful event. In addition, the operations further include controlling a function of the wearable computing device when an indicator of a stressful event exceeds a threshold.

[0011] These and other features, aspects and advantages of various embodiments of the present disclosure will be better understood with reference to the following description and appended claims.The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate example embodiments of the present disclosure and, together with the description, serve to explain the relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] A detailed discussion of the embodiments for those skilled in the art is set forth in this specification with reference to the accompanying drawings, in which:

[0013] Figure 1 A graphical representation of electrodermal activity (EDA) amplitude (y-axis) versus time (x-axis) is provided according to one embodiment of the present disclosure;

[0014] Figure 2 A perspective view of a wearable computing device on a user's wrist is provided according to one embodiment of the present disclosure;

[0015] Figure 3 A front perspective view of a wearable computing device according to one embodiment of the present disclosure is provided;

[0016] Figure 4 Provided Figure 3 A rear perspective view of a wearable computing device;

[0017] Figure 5 Provided Figure 3 an exploded view of a display of a wearable computing device;

[0018] Figure 6 A schematic diagram of an example set of devices capable of communicating according to one embodiment of the present disclosure is provided;

[0019] Figure 7 illustrates various controller components of an example system that may be utilized according to one embodiment of the present disclosure;

[0020] Figure 8 A flow chart of an embodiment of a method for monitoring a user's stress indicators using a wearable computing device according to the present disclosure is shown.

[0021] Fig. 9 A flow chart of an embodiment of a momentary stress algorithm for calculating a stress index of a user of a wearable computing device at a certain time according to the present disclosure is provided.

[0022] Fig.10 A graphical representation of an embodiment of unfiltered cEDA data of a user of a wearable computing device during an exercise event in accordance with the present disclosure is shown.

[0023] Fig.11 A graphical representation of an embodiment of filtered cEDA data of a user of a wearable computing device during an exercise event in accordance with the present disclosure is shown.

[0024] Fig.12 A schematic diagram of a normalization factor applied by a transient stress algorithm transmitted from a backend to a mobile device and to a wearable computing device according to the present disclosure is provided. DETAILED DESCRIPTION

[0025] Reference will now be made in detail to embodiments of the present invention, one or more examples of which are shown in the accompanying drawings. Each example is provided by way of explanation of the present invention rather than by way of limitation. In fact, it will be apparent to those skilled in the art that various modifications and changes may be made in the present invention without departing from the scope or spirit of the present invention. For example, a feature shown or described as part of one embodiment may be used together with another embodiment to produce yet further embodiments. Therefore, the present invention is intended to encompass such modifications and changes within the scope of the appended claims and their equivalents. Overview

[0026] Recent technological advances, including those available through consumer devices, have provided corresponding advances in health detection and monitoring. For example, devices such as fitness trackers and smart watches are able to determine information related to the pulse and movement of the person wearing the device. However, the amount and type of health information that can be determined using such devices is limited due to the capabilities of conventional devices.

[0027] However, recent advances in miniaturization of sensors, electronics, and power supplies have allowed personal health monitoring devices to be provided in extremely small sizes that were previously impractical. For example, some biometric monitoring devices include a wristband having a housing that is approximately 1.6 inches wide by 1.6 inches long by 0.5 inches thick. Such biometric monitoring devices typically include a display, battery, sensor, electronics packaging, wireless communication capabilities, power supply, and interface buttons packaged in a small volume.

[0028] In addition, certain biometric monitoring devices include a variety of sensors for measuring multiple biological parameters that may be beneficial to the user of the device, such as heart rate sensors, multi-purpose electrical sensors compatible with ECG and EDA applications, infrared sensors, gyroscopes, altimeters, accelerometers, temperature sensors, ambient light sensors, Wi-Fi, GPS, vibration sensors, speakers and microphones, and other sensors.

[0029] As an example, some biometric monitoring devices use multipath electrical sensors to measure the EDA responses of a user's skin. These responses are observed as sensitive electrical changes in skin conductance and are typically detected on the palm or fingertips of the user using a wet or dry electrode system. Therefore, the EDA responses can be used to assess changes in the user's physiological stress.

[0030] A typical EDA response can be measured at the palm or fingertips using at least two electrodes, where the skin conductance is calculated using the measured electrical impedance. The EDA response is expressed as the phasic component of skin conductance, the skin conductance response (SCR), and is detected by identifying transient spikes in skin conductance by comparison with a background tonic measurement, the skin conductance level (SCL). In general, due to the high sweat gland density in the palm or fingertips of a user, the SCR is more accurately observed from data collected from these areas.

[0031] Although SCR detection at the palm or fingertips has been fully reported in the literature for assessing stress, SCL alone can be beneficial for assessing stress in a user. By measuring continuous electrodermal activity (cEDA), SCL can be used to observe certain biological events, such as the body's response to acute physiological reactions such as stressful events. However, if electrodes mounted on the top surface of the biometric monitoring device are used, it is challenging to use cEDA measurements to assess acute stressful events because cEDA requires continuous skin contact to provide accurate readings (i.e., requires the user to position their skin above the electrode surface for the duration of the measurement). Therefore, in order to facilitate continuous contact with the skin and not require frequent user input to facilitate continuous EDA measurements, the electrodes for cEDA are positioned on the bottom (skin-facing) side of the biometric monitoring device.

[0032] Therefore, the present disclosure relates to a wearable computing device and a computer-implemented method for determining a stress indicator (such as the probability of a stress event exceeding a threshold, stress intensity, stress type, etc.) of a user at a certain time. For example, the wearable computing device can implement a transient stress algorithm (MSA) programmed therein as an on-device algorithm or mobile in-application for determining an indicator of a stress event. In particular, the MSA can be configured to predict physical (or physiological) signs of stress at a specific time and send a notification to the user about the predicted stress event. More specifically, in an embodiment, the MSA typically receives a combination of raw data inputs (e.g., heart rate data, cEDA data, skin temperature, acceleration, altimeter, and heart rate variability data) as time series data processed using various filtering techniques. Filtering techniques typically involve eliminating (e.g., removing) unwanted data, such as a data set, from the raw data input, and / or modifying (e.g., enhancing) certain data from the raw data input, such as a portion of the data and / or certain data sets. Thus, filtering the raw data input sequentially may include applying different filters one after another, each applied filter being used to eliminate and / or modify the value of the raw data input in an (algorithmically) predefined manner. For example, the filtering technique may include checking the raw data input for various modes of the device (such as sleep, exercise, do not disturb mode, and / or off-wrist mode) so that data input collected during specific modes or all of these modes may be eliminated from consideration. The device modes to be considered for elimination may be selected by the user during use of the wearable computing device and / or automatically associated with the raw input data by at least one processor of the wearable computing device (e.g., based on biometric sensor data).

[0033] For example, a user or processor may switch to a sleep mode, an exercise mode, a do not disturb mode, or an off-wrist mode of the wearable computing device such that any input data collected during the corresponding mode is associated with that mode. The raw input data collected during the corresponding mode may then, for example, be generally considered not to be representative of an indication of a physiological response, such as a stressful event. Additionally, certain confounding factors may be used to filter the data inputs such that data inputs corresponding to excessive movement (or a variety of other variables) may also be eliminated. In an example, if the MSA detects increased movement as well as increased cEDA, the user may be exercising (rather than stressed) and such data may be excluded from stress calculations.

[0034] The final data set may also be normalized using additional filling as needed. In particular, in an embodiment, the mean and standard deviation (or any suitable normalization factor) on a time scale appropriate for the input may be required for each MSA data input. In addition, for each input, various features may be calculated to convert the time series data set into a single value. Once a single value for each input is obtained, the MSA is configured to determine which model to use to estimate the user's stress index. For example, the determination of the model to be used may be based on what data input is available. Therefore, in an embodiment, if all input data in the input data are available (e.g., each of heart rate data, cEDA data, skin temperature, and heart rate variability data), the selected model is a model in which all input data are used when calculating the stress index. In contrast, if only one or two input data sets are available, the selected model is a model in which only these two input data sets are used when calculating the stress index. The selected model (e.g., the selected model may be a logistic regression classifier) ​​may then be applied to the available data to determine the user's stress index. In a further embodiment, the MSA may also include post-processing or smoothing of the user's stress index. Thus, the proposed solution includes (automatically) selecting a model based on unprocessed data input (e.g., based on multiple different input data types available or multiple input data time series available) - for further evaluation of processed data. For example, in some embodiments, the MSA may include requiring that the detected stressful events have a certain length (such as from about 3 minutes to about 5 minutes). In other embodiments, the MSA may group multiple stressful events together if the multiple stressful events occur within a certain proximity of each other (e.g., the stressful events are within 5 minutes of each other). Thus, in such embodiments, the MSA infers that the multiple stressful events represent a common stressful event, rather than multiple immediately subsequent stressful events.

[0035] Typically, when an indicator of a stressful event exceeds a threshold, a function of the wearable computing device may be controlled. As described above, such a function of the wearable computing device may be a function of a display of the wearable computing device, for example, causing the indicator of the stressful event to be displayed at the display when the indicator of the stressful event exceeds the threshold. Alternatively or in addition, the function of the wearable computing device controlled due to the calculated indicator of the stressful event exceeding the threshold may include: generating and, for example, sending a stressful event notification to a user of the wearable computing device, and / or launching one or more (software) applications on the wearable computing device, such as applications for mood logging, diary recording, and / or participation recording, and / or triggering a user interaction process via the wearable device in which the user of the wearable computing device must actively confirm the notification of the stressful event. Thus, a technology and a wearable computing device for making a user more efficiently aware of one or more potentially harmful stressful events and automatically providing, in particular initiating, countermeasures for reducing the user's stress level can be provided.

[0036] Referring now to the drawings, example embodiments of the present disclosure will be discussed in further detail. Example Apparatus and Systems

[0037] Now referring to the accompanying drawings, Figures 2 to 5 1 shows a perspective view of a wearable computing device 100 according to the present disclosure. In particular, as shown in FIG. Figure 2 As shown, the wearable computing device 100 can be worn on the user's forearm 102 like a watch. Therefore, as shown, the wearable computing device 100 can include a wristband 103 for fixing the wearable computing device 100 to the user's forearm 102. In addition, as shown in FIG. Figure 2 , Figure 3 and Figure 5 As shown, the wearable computing device 100 has an outer covering 105 and a housing 104 containing electronic components associated with the wearable computing device 100. For example, in an embodiment, the outer covering 105 can be made of glass, polycarbonate, acrylic, or the like. Figure 2 , Figure 3 and Figure 5 As shown, the wearable computing device 100 includes an electronic display 106 disposed within the housing 104 and viewable through the outer cover 105. In addition, as shown, the wearable computing device 100 may also include one or more buttons 108, which may be implemented to provide a mechanism for activating various sensors of the wearable computing device 100 to collect certain health data of the user. In addition, in an embodiment, the electronic display 106 may cover an electronic device package (not shown), which may also be housed within the housing 104.

[0038] Special References Figure 4 , the housing 104 of the wearable computing device 100 further includes: a wrist dorsal side surface 110 configured to rest against the back of the user's wrist when worn by the user, and a plurality of sensor electrodes 112 positioned on the wrist dorsal side surface 110 of the housing 104 so as to maintain skin contact with the user when worn on the wrist of the user. Therefore, in such embodiments, each of the sensor electrodes 112 continuously measures at least the electrical impedance at the location where the user's skin contacts the back of the wrist. Therefore, in one or more embodiments, one or more (or all) of the plurality of sensor electrodes 112 may be cEDA sensor electrodes. In some embodiments, the wearable computing device 100 may include at least one additional biometric sensor electrode in addition to the cEDA sensor electrode. In such embodiments, the additional biometric sensor electrodes may include one or more temperature sensors (such as an ambient temperature sensor or a skin temperature sensor), a humidity sensor, a light sensor, a pressure sensor, a microphone, an optical sensor, or a photoplethysmography (PPG) sensor.

[0039] Further, the sensor electrodes 112 described herein may be made of any suitable material. For example, in embodiments, the sensor electrodes 112 described herein may be made of stainless steel, graphene, or any other material having suitable conductivity and / or corrosion resistance, and may have an optional PVD coating, which may be 1 micron thick titanium nitride. In such embodiments, the PVD coating may provide the sensor electrodes 112 with a desired color, thereby further preventing oxidation beyond that already provided by stainless steel, and also increasing durability.

[0040] In other embodiments, PVD and surface finish can be used to increase / decrease moisture retention, which affects the cEDA signal and user comfort. In certain embodiments, the sensor electrodes 112 can be formed of a tin-nickel alloy (TiN) with a glossy or mirrored surface finish. Additionally, in embodiments, the sensor electrodes 112 can be made of a hydrophobic material or a transparent material.

[0041] Reference now Figure 6 , illustrates components of an example system 200 of a wearable computing device 100 that may be utilized in accordance with various embodiments. In particular, as shown, the system 200 may also include at least one controller 202 communicatively coupled to the plurality of sensor electrodes 112. Further, in embodiments, the controller 202 may be a central processing unit (CPU) or a graphics processing unit (GPU) for executing instructions that may be stored in a memory device 204 (such as flash memory or DRAM, among other such options).

[0042] For example, in embodiments, the memory device 204 may include RAM, ROM, FLASH memory, or other non-transitory digital data storage, and may include a control program including a sequence of instructions that, when loaded from the memory device 204 and executed using the controller 202, causes the controller 202 to perform the functions described herein. As will be apparent to one of ordinary skill in the art, the system 200 may include many types of memory, data storage, or computer-readable media, such as data storage for program instructions executed by the controller or any suitable processor. The same or separate storage may be used for images or data, removable storage may be used to share information with other devices, and any number of communication methods may be used to share with other devices.

[0043] Additionally, as shown, system 200 includes any suitable display 206, such as a touch screen, organic light emitting diode (OLED), or liquid crystal display (LCD), but the device may communicate information via other means, such as through an audio speaker, a projector, or by projecting a display or streaming data to another device such as a mobile phone, where an application on the mobile phone displays the data.

[0044] The system 200 may also include one or more wireless components 212 that are operable to communicate with one or more electronic devices within the communication range of a particular wireless channel. The wireless channel may be any suitable channel for enabling devices to communicate wirelessly, such as a Bluetooth, cellular, NFC, ultra-wideband (UWB), or Wi-Fi channel. It should be understood that the system 200 may have one or more conventional wired communication connections known in the art.

[0045] The system 200 also includes one or more power supply components 208, such as batteries that can be operated to recharge by conventional plug-in methods or by other methods, such as capacitive charging by approaching a power pad or other such devices. In further embodiments, the system 200 may also include at least one additional I / O device 210 capable of receiving conventional input from a user. The conventional input may include, for example, a button, a touch pad, a touch screen, a scroll wheel, a joystick, a keyboard, a mouse, a keypad, or any other such device or element, through which a user can input commands to the system 200. In another embodiment, the I / O device 210 may also be connected via wireless infrared or Bluetooth or other links in some embodiments. In some embodiments, the system 200 may include a microphone or other audio capture element that accepts voice or other audio commands. For example, in a particular embodiment, the system 200 may not include any buttons at all, but may be controlled only by a combination of visual and audio commands, so that the user can control the wearable computing device 100 without having to contact it. In some embodiments, the I / O element 210 may also include one or more of the sensor electrodes 112, optical sensors, air pressure sensors (e.g., altimeters, etc.) described herein, etc.

[0046] Still reference Figure 6 The system 200 may also include a driver 214 and at least some combination of one or more emitters 216 and one or more detectors 218 (referred to herein as an optical package 215) for measuring data of one or more metrics of a human body, such as a person wearing the wearable computing device 100. In such embodiments, Figure 4 As shown, for example, the optical package 215 can be disposed within the housing 104 and at least partially exposed through the dorsal wrist side 110 of the housing 104. Thus, as shown and further explained herein, the sensor electrodes 112 can be positioned on the wrist side 110 of the housing 104 around the optical package 215. In alternative embodiments, various components of the optical package 215 can be positioned around the sensor electrodes 112 and / or positioned in another other suitable configuration, such as adjacent to the optical package 215, interspersed with the optical package 215, surrounded by the optical package 215, or on top of the optical package 215. In certain embodiments, for example, where the sensor electrodes 112 are transparent, the sensor electrodes 112 can be disposed on top of the optical package 215.

[0047] In some embodiments, the system 200 may include at least one imaging element, such as one or more cameras capable of capturing images of the surrounding environment and capable of imaging users, people, or objects near the device. The imaging element may include any appropriate technology, such as a CCD image capture element with sufficient resolution, focal range, and visual area to capture the image of the user when the user operates the device. Further image capture elements may also include depth sensors. Methods for capturing images using camera elements with computing devices are well known in the art and will not be discussed in detail herein. It should be understood that image capture may be performed using a single image, multiple images, periodic imaging, continuous image capture, image streaming, etc. Further, the system 200 may include the ability to start and / or stop image capture, such as when a command from a user, application, or other device is received.

[0048] In one example, Figure 6 The emitter 216 and detector 218 may also be capable of being used to obtain optical PPG measurements. Some PPG techniques rely on detecting light at a single spatial location, adding signals acquired from two or more spatial locations, or an algorithmic combination thereof. Both approaches produce a single spatial measurement from which a heart rate (HR) estimate (or other physiological metric) can be determined. In some embodiments, the PPG device employs a single light source (i.e., a single light path) coupled to a single detector. Alternatively, the PPG device may employ multiple light sources (i.e., two or more light paths) coupled to a single detector or multiple detectors. In other embodiments, the PPG device employs multiple detectors (i.e., two or more light paths) coupled to a single light source or multiple light sources. In some cases, the light source may be configured to emit one or more of green light, red light, infrared (IR) light, and any other suitable wavelength in the spectrum (such as long IR for metabolic monitoring). For example, the PPG device may employ a single light source and two or more light detectors, each of which is configured to detect a specific wavelength or wavelength range. In some cases, each detector is configured to detect a wavelength or wavelength range that is different from one another. In other cases, two or more detectors are configured to detect the same wavelength or wavelength range. In yet another case, one or more detectors are configured to detect a specific wavelength or wavelength range that is different from one or more other detectors. In embodiments employing multiple light paths, the PPG device may determine an average of the signals generated from the multiple light paths before determining an HR estimate or other physiological metric.

[0049] Additionally, in embodiments, the transmitter 216 and the detector 218 may be coupled to the controller 202 directly or indirectly using a driver circuit system, through which the controller 202 may drive the transmitter 216 and obtain a signal from the detector 218. The host computer 222 may communicate with the wireless networking component 212 via one or more networks 220, which may include one or more local area networks, wide area networks, UWB, and / or the Internet using any terrestrial or satellite links. In some embodiments, the host computer 222 executes a control program and / or application configured to perform some of the functions described herein.

[0050] Reference now Figure 7 , a schematic diagram of an environment 300 in which aspects of various embodiments may be implemented is shown. In particular, as shown, a user may have multiple different devices that can communicate using at least one wireless communication protocol. For example, as shown, a user may have a smart watch 302 or a fitness tracker (such as wearable computing device 100), and the user wants the smart watch or fitness tracker to be able to communicate with a smart phone 304 and a tablet computer 306. The ability to communicate with multiple devices can enable the user to obtain information from the smart watch 302 using an application installed on the smart phone 304 or tablet computer 306, such as data captured using sensors on the smart watch 302. The user may also want the smart watch 302 to be able to communicate with a service provider 308 or other such entity, which can obtain and process data from the smart watch and provide functions that may not otherwise be available on the smart watch or application installed on the individual device. Additionally, as shown, the smartwatch 302 may be able to communicate with a service provider 308 via at least one network 220, such as the Internet or a cellular network, or may communicate with one of the individual devices via a wireless connection, such as Bluetooth®, which may then communicate via at least one network. In various embodiments, there may be a variety of other types or reasons for communication.

[0051] In addition to being able to communicate, users may also want devices to be able to communicate in a variety of ways or in certain aspects. For example, users may want communications between devices to be secure, especially when data may include personal health data or other such communications. In at least some cases, the device or application provider may also be required to protect the information. Users may want devices to be able to communicate with each other concurrently rather than sequentially. This may be especially true in the case of pairing, because users may prefer each device to be paired at most once, so that manual pairing is not required. Users may also expect that communication is as standard-based as possible, so that not only little manual intervention on the part of the user is required, but also that the device can communicate with as many other types of devices as possible, which is usually not possible with various proprietary formats. Therefore, users may expect to be able to carry a device around the room and have such a device automatically communicate with another target device with almost no effort on the part of the user. In various conventional methods, a device will use a communication technology such as Wi-Fi to communicate with other devices using a wireless local area network (WLAN). Smaller or lower capacity devices such as many Internet of Things (IoT) devices instead use communication technologies such as Bluetooth®, especially low-power Bluetooth (BLE) with very low power consumption.

[0052] In a further embodiment, Figure 7 The environment 300 shown in enables data to be captured, processed, and displayed in a variety of different ways. For example, data can be captured using sensors on a smart watch 302, but due to limited resources on the smart watch 302, the data can be transmitted to a smart phone 304 or a service provider 308 (or cloud resources) for processing, and the results of the processing can then be presented back to the user on the smart watch 302, smart phone 304, and / or another such device associated with the user (such as a tablet computer 306). In at least some embodiments, the user may also be able to use interfaces on any of these devices to provide input, such as health data, which can then be considered when making the determination.

[0053] Reference now Figure 8 , a flow chart of one embodiment of a method 400 for monitoring stress of a user using a wearable computing device is provided. In an embodiment, for example, the wearable computing device can be any suitable wearable computing device, such as the one described herein with reference to Figures 1 to 7 Therefore, in general, the wearable computing device 100 described herein is referred to as Figures 1 to 7 The method 400 is described with reference to the wearable computing device 100. However, it should be understood that the disclosed method 400 may be implemented using any other suitable wearable computing device having any other suitable configuration. Figure 8The steps performed in a particular order are depicted for the purpose of illustration and discussion, but the methods discussed herein are not limited to a particular order or arrangement. It will be understood by those skilled in the art using the disclosure provided herein that the various steps of the methods disclosed herein may be omitted, rearranged, combined, added, and / or adjusted in various ways without departing from the scope of the present disclosure.

[0054] As described above and herein, the wearable computing device includes a plurality of biometric sensor electrodes located on a dorsal wrist side of a housing of the wearable computing device. Thus, as shown at (402), method 400 includes receiving a plurality of time series data inputs from a plurality of biometric sensor electrodes of the wearable computing device 100 via a processor communicatively coupled to the wearable computing device 100. In such embodiments, as an example, the plurality of time series data may include cEDA data of a user and heart rate data of the user, skin temperature data of the user, heart rate variability (HRV) data of the user, accelerometer data, altimeter data, and / or combinations thereof. In a particular embodiment, for example, the plurality of time series data inputs may include heart rate data of the user, skin temperature data of the user, heart rate variability (HRV) data of the user, accelerometer data, altimeter data, and / or combinations thereof. In a particular embodiment, for example, the plurality of time series data inputs may include heart rate data of the user, skin temperature of the user, heart rate variability of the user, and cEDA data of the user, wherein a combination of such data inputs provides an improved estimate of stress of the user.

[0055] As shown at (404), method 400 includes sequentially processing multiple time series data inputs using multiple filtering techniques. For example, as will be described in more detail herein, sequentially processing multiple time series data inputs using multiple filtering techniques may include: filtering the user's cEDA data using, for example, a high pass filter for SCR, a low pass filter for SCL, a median filter for removing burrs, and / or any other type of filter as desired.

[0056] Further, in an embodiment, sequentially using multiple filtering techniques to process multiple time series data inputs may include: using multiple data inputs to update a time range (time The method of claim 1 , wherein the plurality of time series data inputs are configured to update a cache of data inputs associated with a time range / time window of a predetermined length, determine whether a time series data input from a plurality of time series data inputs indicates a mode of a plurality of modes of the wearable computing device associated with undesirable motion (e.g., from exercise), and if so, eliminate the time series data input from the plurality of time series data inputs, filter the plurality of time series data inputs based on a plurality of confounding factors, and eliminate or modify time series data inputs from the plurality of time series data inputs that satisfy one or more of the plurality of confounding factors, fill one or more data points into the plurality of time series data inputs if a minimum number of data points are missing from the plurality of time series data inputs, or discard the plurality of time series data inputs if a certain number of data points are missing from the plurality of time series data inputs, normalize the plurality of time series data inputs using one or more normalization factors, the one or more normalization factors comprising at least one of a mean, a median, a mode, or a standard deviation, and / or convert each of the plurality of time series data inputs into a single value.

[0057] Still refer to Figure 8 As shown at (406), method 400 includes selecting a model from a plurality of models based on a type or value of a data input received as a plurality of time series data inputs. As shown at (408), method 400 includes applying the selected model to the processed plurality of time series data inputs to calculate an indicator of a physiological response of a user at a time, such as a stressful event, wherein the selected model is customized to use all of the plurality of time series data inputs in calculating the indicator of the stressful event.

[0058] As shown at (410), method 400 includes providing an indicator of stressful events at a certain time to a user via a display. More specifically, in an embodiment, method 400 may include sending a notification to the user indicating at least one of an indicator of stressful events, a graphical representation of stressful events over time, and / or a summary of stressful events over time (e.g., daily, weekly, monthly, etc.). In addition, in an embodiment, method 400 may also include probing the user to respond to the notification via a display (such as display 206). For example, the user may be reminded to perform mood journaling, diary recording, record participation in a prescribed stress relief activity (such as meditation, walking, listening to music, guided breathing, etc.), and / or any other suitable response.

[0059] Typically, when the indicator of the stressful event exceeds a threshold, a function of the wearable computing device may be controlled. As described above, such a function of the wearable computing device may be a function of the display 206 of the wearable computing device 100, for example, causing the indicator of the stressful event to be displayed at the display 206 when the indicator of the stressful event exceeds the threshold. Alternatively or in addition, the function of the wearable computing device 100 that is controlled due to the calculated indicator of the stressful event exceeding the threshold may include: generating and, for example, sending a stressful event notification to a user of the wearable computing device, and / or launching one or more (software) applications on the wearable computing device 100, such as applications for mood logging, diary recording, and / or participation recording, and / or triggering a user interaction process via the wearable computing device 100 in which the user of the wearable computing device 100 must actively confirm the notification of the stressful event. Thus, a technique and a wearable computing device 100 for making a user more efficiently aware of one or more potentially harmful stressful events and automatically providing or even automatically initiating countermeasures for reducing the user's stress level may be provided.

[0060] refer to Figures 9 to 11 Better understanding Figure 8 The method 400. More specifically, Fig. 9 A flow chart of an embodiment of a transient stress algorithm 500 for calculating an indicator of a stress event for a user of a wearable computing device at a certain time according to the present disclosure is shown. In particular, as shown at (502), the algorithm 500 includes ingesting on-device data from the wearable computing device 100. As described above, such raw data may include, for example, cEDA data of the user as well as heart rate data of the user, skin temperature data of the user, accelerometer data, altimeter data and / or HRV data of the user, and combinations thereof.

[0061] As shown at (504), the algorithm 500 includes calculating a current minute signal for certain of the raw data, such as heart rate data, HRV data, and / or cEDA data. More specifically, in an embodiment, as shown at (506), the algorithm 500 may include applying a filter to the cEDA data, for example, using a high pass filter for SCR, a low pass filter for SCL, a median filter for removing glitches, and / or any other type of filter as desired. For example, as shown at Fig.10 and Fig.11 , graphical representations 600, 700, respectively, of embodiments of cEDA data (e.g., SCL; y-axis) of a user of a wearable computing device during an exercise event versus time (x-axis) according to the present disclosure. Fig.10 shows the original data, while Fig.11 Filtered data, for example, via a high pass filter, is shown. Further, as Fig.10As shown, SCL data is represented by 602, cEDA event data such as stressful events is represented by 604, and exercise detection is represented by 606. Fig.11 , SCL data is represented by 702, cEDA / SCL data is represented by 704, and high pass filtered data is represented by 706 and 708, respectively. In particular, as shown, the difference between lines 706 and 708 is the cutoff frequency used to generate 706 compared to the cutoff frequency used to generate 708, where 708 has a lower cutoff frequency (e.g., 120 minutes), which means that the change is maintained over a longer time interval. In other words, line 706 returns to zero about six times faster than line 708.

[0062] Therefore, if Fig.10 and Fig.11 As shown, cEDA / SCL data signal 602 can decay very slowly (e.g., several hours). Therefore, algorithm 500 can filter out exercise events from cEDA data without losing all predictive power of the "downhill" side of the peak value, which can include useful data. However, if linear interpolation is used between the start of exercise and the "downhill" side of the peak value, algorithm 500 will basically reintroduce exercise data. Therefore, in order to solve this problem, a high-pass filter can be applied to cEDA signal 602 after, for example, data has been passed through a slew rate limiter or similar device to avoid introducing a sharp change in value or slope before filtering the data, because such changes will be retained by filtering, resulting in a sharp spike in the filtered value. In short, the algorithm monitors the change of state so that a sharp spike will lead to many false predictions of stress events. The output represented by 706 and 708 is filtered cEDA data.

[0063] If no raw data is available (or not enough raw data is available), the algorithm 500 ends at (526) because no prediction can be made. However, if raw data is available, the algorithm 500 continues at (508). In particular, as shown at (508), the algorithm 500 updates a time window (e.g., an X minute window) cache of one or more of the data inputs. In addition, in an embodiment, as shown at (510), the algorithm 500 may receive a window length (such as 30 minutes) that is set to update the cache. It should be understood that the window length may be selected to be any suitable time range and is not limited to 30 minutes.

[0064] Further, as shown at (512), the algorithm 500 includes determining whether the time series data input from the raw data indicates a mode related to motion in a plurality of modes of the wearable computing device 100. In such embodiments, for example, the mode of the wearable computing device 100 may include a sleep mode, an exercise mode, or an off-wrist mode. If yes, the algorithm 500 ends at (526) because no prediction can be made. However, if no, the algorithm 500 continues at (514).

[0065] In particular, as shown at (514), if the wearable computing device 100 is not operating in one of the aforementioned modes, the algorithm 500 is configured to filter the raw data to remove certain minutes of raw data based on a plurality of confounding factors and eliminate or modify the raw data that satisfies one or more of the plurality of confounding factors. In such embodiments, for example, the plurality of confounding factors may include: the user's cEDA data increases with increased exercise (e.g., generally due to the increase in the cEDA data of the user as shown at (516) Fig.10 ), a percentage of HRV data above a certain threshold as shown at (520), a certain confidence level in the user's heart rate data as shown at (518), partial or complete immersion of the wearable computing device 100 in liquid that can be detected using a combination of cEDA and altimeter data, skin contact between the wearable computing device 100 and / or the user based on the amount of time per minute that the sensor electrodes 112 are in any contact with the user's skin at the same time, or humidity above a humidity threshold.

[0066] Still refer to Fig. 9 As shown at (522), the algorithm 500 is further configured to fill one or more data points into the original data if a certain number of data points are missing from the original data, or to discard the original data if the number of missing data points exceeds a threshold. Thus, as shown, if the number of missing data points exceeds the threshold, the algorithm 500 ends at (526) because no prediction can be made. However, if the original data is missing sufficiently few values / data points, the algorithm 500 fills in all missing data points, for example using interpolation or extrapolation, and continues at (528).

[0067] In particular, as shown at (528), the algorithm 500 is further configured to normalize the raw data using one or more normalization factors. In such embodiments, for example, the normalization factor may include a mean, a median, a mode, a standard deviation, or any other suitable statistical function appropriate for each input over a period of time. For example, Fig.12As shown, the normalization factors may be transferred via a file transfer, similar to how settings may be transferred to the wearable computing device 100. In the illustrated embodiment, for example, the transfer mechanism 800 requires the backend 802 to expose an HTTP endpoint 804 that may be queried by a companion application 808 (e.g., on a mobile device 806). The companion application 806 may then query these values ​​at predefined intervals, and any changes in the payload cause the companion application 808 to send the normalization factors to the wearable computing device 100. The wearable computing device 100 may decode the file, persist the new normalization weights, and then apply the new normalization weights to the algorithm 500, for example, via the MSA algorithm application 812 in the user space 810.

[0068] Thus, returning to reference 9, as shown at (530), the algorithm 500 is configured to extract certain features from the raw data by applying a plurality of commonly used time series transformation functions. In particular, as shown at (532), the algorithm 500 is configured to receive certain features and / or hyperparameters (or can be programmed with certain features and / or hyperparameters) that can be used to extract certain features from the raw data. In an embodiment, for example, the algorithm 500 is configured to convert each of the time series raw data inputs into a single value. Thus, the algorithm 500 is configured to independently apply certain time series transformation functions to each of the input signals.

[0069] Thus, as shown at (534), the algorithm 500 is then configured to determine / select a model from a plurality of models based on the type and / or value of the data input received in the raw data. For example, as shown at (536), if the raw data includes heart rate data, HRV data, cEDA data, and skin temperature, the selected model is configured to use all of the received raw data to determine an indicator of a stressful event experienced by the user at a certain time. In other embodiments, if less raw data is available (such as only cEDA data and skin temperature), the algorithm 500 is configured to select a model that uses a subset of the available data types (such as cEDA data and skin temperature) to determine an indicator of a stressful event experienced by the user at a certain time. Thus, it should be understood that the algorithm is able to select a model that is customized for the available raw data.

[0070] Thus, as shown at (538), the algorithm 500 is configured to apply the selected model to the data to generate an output that may represent a physical (or physiological) sign of stress for the user. In further embodiments, the selected model may be, by way of example, a machine learning model. For example, as shown at (540), the machine learning model may be a logistic regression model, a deep neural network, or any other suitable machine learning model now known in the art or later developed.

[0071] As shown at (542), the algorithm 500 is also configured to post-process the indicator of the user's stressful event. In such embodiments, for example, post-processing the indicator of the user's stressful event may include ensuring that the stressful event has a duration above a certain threshold. In another embodiment, post-processing the indicator of the user's stressful event may include grouping multiple stressful events together if they occur within a certain time range of each other. The algorithm 500 ends at (544).

[0072] Thus, as described above, the output of the algorithm is an indicator of stressful events experienced by the user at a certain time. Thus, in further embodiments, the wearable computing device 100 may send a notification to the user indicating an indicator of stressful events, a graphical representation of stressful events over time, and / or a summary of stressful events over time. Furthermore, as described above, the wearable computing device 100 may prompt the user to respond to the notification via the display, such as by requesting the user to engage in mood journaling, diary recording, and recording participation in prescribed stress relief activities. Additional public content

[0073] In addition to the above description, controls may be provided to the user, which allow the user to make a choice about whether and when the system, program or feature described herein can enable the collection of user information (e.g., information about the user's social network, social actions or activities, occupation, user's preferences or the user's current location) and whether to send content or communication from the server to the user. In addition, certain data may be disposed of in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, the user's identity may be disposed of so that the user's personally identifiable information cannot be determined, or the user's geographic location in which location information is obtained may be generalized (such as to the city, zip code or state level) so that the user's specific location cannot be determined. Therefore, the user can control what information is collected about the user, how to use the information, and what information is provided to the user. To this end, any information related to the user collected as described herein will be kept confidential and will not be improperly used or published.

[0074] The technology discussed herein relates to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for a variety of possible configurations, combinations, and partitions of tasks and functions between and among components. For example, the processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system, or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0075] Although the present subject matter has been described in detail with respect to various specific example embodiments of the present subject matter, each example is provided by way of explanation rather than limitation of the present disclosure. Those skilled in the art, after understanding the foregoing, may easily produce changes, modifications, and equivalents of such embodiments. Therefore, the present disclosure does not exclude such modifications, changes, and / or additions to the present subject matter that will be readily understood by those of ordinary skill in the art. For example, a feature shown or described as part of an embodiment may be used together with another embodiment to produce yet further embodiments. Therefore, the present disclosure is intended to cover such changes, modifications, and equivalents.

Claims

1. A method for monitoring stress of a user using a wearable computing device, the method comprising: receiving, via a processor communicatively coupled to the wearable computing device, a plurality of time series data inputs from a plurality of biometric sensor electrodes of the wearable computing device, the plurality of time series data comprising continuous electrodermal activity (cEDA) data of the user and at least one of: heart rate data of the user, skin temperature data of the user, and heart rate variability (HRV) data of the user; sequentially using a plurality of filtering techniques to process the plurality of time series data inputs; selecting a model from a plurality of models based on a type of data input received as the plurality of time series data inputs; applying the selected model to the processed plurality of time series data inputs to calculate an indicator of a physiological response of the user at a certain time, wherein the selected model is customized to use all of the plurality of time series data inputs in calculating the indicator of the physiological response; as well as A function of the wearable computing device is controlled when the indicator of the physiological response exceeds a threshold.

2. The method according to claim 1, wherein: The plurality of time-series data inputs include the heart rate of the user, the skin temperature of the user, the heart rate variability of the user, and the cEDA data of the user.

3. The method according to claim 1 or 2, wherein: Using the plurality of filtering techniques in sequence to process the plurality of time series data inputs further comprises: The cEDA data of the user is filtered using a high pass filter, a low pass filter or a median filter.

4. The method according to claims 1 to 3, wherein: Using the plurality of filtering techniques in sequence to process the plurality of time series data inputs further comprises: A time range cache is updated using the plurality of data inputs.

5. A method according to any one of the preceding claims, wherein: Using the plurality of filtering techniques in sequence to process the plurality of time series data inputs further comprises: A determination is made as to whether a time series data input from the plurality of time series data inputs indicates one of a plurality of modes of the wearable computing device, and if so, eliminating or modifying the time series data input from the plurality of time series data inputs.

6. The method according to claim 5, wherein: The plurality of modes of the wearable computing device include one of: a sleep mode, an exercise mode, a do not disturb mode, or an off-wrist mode.

7. A method according to any one of the preceding claims, wherein: Using the plurality of filtering techniques in sequence to process the plurality of time series data inputs further comprises: The plurality of time series data inputs are filtered based on a plurality of confounding factors, and time series data inputs among the plurality of time series data inputs that satisfy one or more confounding factors among the plurality of confounding factors are eliminated or modified.

8. The method according to claim 7, wherein: The multiple confounding factors include one of: the cEDA data of the user increases with increased movement, a percentage of the HRV data is above a certain threshold, a certain confidence level of the heart rate data of the user, a motion classifier based on accelerometer values, the wearable computing device is partially or completely immersed in liquid or exposed to the liquid, skin contact between the wearable computing device and the user is below a contact threshold, or humidity is above a humidity threshold.

9. A method according to any one of the preceding claims, wherein: Processing the plurality of time series data inputs using the plurality of filtering techniques further comprises: If a certain number of data points are missing from the plurality of time series data inputs, one or more data points are filled into the plurality of time series data inputs, or if the number of missing data points exceeds a threshold, the plurality of time series data inputs are discarded.

10. A method according to any one of the preceding claims, wherein: Processing the plurality of time series data inputs using the plurality of filtering techniques further comprises: The plurality of time series data inputs are normalized using one or more normalization factors.

11. A method according to any one of the preceding claims, wherein: The model chosen is a machine learning model.

12. A method according to any one of the preceding claims, wherein: Processing the plurality of time series data inputs using the plurality of filtering techniques further comprises: Each of the plurality of time series data inputs is converted to a single value.

13. The method according to any of the preceding claims, further comprising post-processing the indicator of the physiological response, wherein: Post-processing the indicator of the physiological response further includes at least one of ensuring that the physiological response includes a duration above a certain threshold, and grouping together multiple physiological responses if the multiple physiological responses occur within a certain time range of each other.

14. A method according to any one of the preceding claims, wherein: Controlling the function of the wearable computing device includes at least one of: controlling a display of the wearable computing device, and providing the indicator of the physiological response at the certain time to the user via the display.

15. The method of claim 14, further comprising sending a notification to the user via the display, the notification indicating at least one of: an occurrence of the indicator of the physiological response exceeding a threshold, a graphical representation of a physiological response over time, and a summary of a physiological response over time.

16. The method of claim 14 or 15, further comprising prompting the user to respond to the notification via the display of the wearable computing device, wherein The response to the notification includes at least one of: emotion journaling, diary recording, guided breathing, guided meditation, and recording participation in a prescribed stress reduction activity.

17. A method according to any one of the preceding claims, wherein: The processor is part of one of the wearable computing device or a separate mobile device.

18. A wearable computing device comprising: Electronic displays; a plurality of biometric sensor electrodes for sensing a plurality of time-series data inputs related to biometrics of a user of the wearable computing device; as well as at least one processor, the at least one processor being communicatively coupled to the plurality of biometric sensor electrodes, the at least one processor being configured to perform a plurality of operations, the plurality of operations comprising: receiving the plurality of time series data inputs, the plurality of time series data comprising the user's continuous electrodermal activity (cEDA) data and at least one of: the user's heart rate data, the user's skin temperature data, and the user's heart rate variability (HRV) data; sequentially using a plurality of filtering techniques to process the plurality of time series data inputs; selecting a model from a plurality of models based on a type of data input received as the plurality of time series data inputs; applying the selected model to the processed plurality of time series data inputs to calculate an indicator probability of a stressful event for the user performed by the user at a certain time, wherein the selected model is customized to use all of the plurality of time series data inputs in calculating the indicator probability of the stressful event; and A function of the wearable computing device is controlled when the indicator of the stressful event exceeds a threshold.

19. The wearable computing device of claim 18, wherein: Using the plurality of filtering techniques in sequence to process the plurality of time series data inputs further comprises: Filtering the cEDA data of the user using a high-pass filter, a low-pass filter, or a median filter; updating a time range cache using the plurality of data inputs; determining whether a time series data input from the plurality of time series data inputs indicates one of a plurality of modes of the wearable computing device, and if so, eliminating or modifying the time series data input from the plurality of time series data inputs; Filtering the plurality of time series data inputs based on a plurality of confounding factors, and eliminating or modifying the time series data inputs among the plurality of time series data inputs that satisfy one or more of the plurality of confounding factors; If a certain number of data points are missing from the plurality of time series data inputs, one or more data points are filled into the plurality of time series data inputs, or if the number of missing data points exceeds a threshold, the plurality of time series data inputs are discarded; normalizing the plurality of time series data inputs using one or more normalization factors, the one or more normalization factors comprising at least one of a mean, a median, a mode, or a standard deviation appropriate to a time scale of each of the time series data inputs; and Each time series data input of the plurality of time series data inputs is converted to a single value.

20. The wearable computing device of claim 18 or 19, further comprising post-processing the indicator probability of the stressful event, wherein: Post-processing the indicator probability of the stressful event further includes at least one of ensuring that the stressful event includes a duration above a threshold, and grouping multiple stressful events together if the multiple stressful events occur within a time range of each other.

21. The wearable computing device of one of claims 18 to 20, further comprising: sending a notification to the user via the electronic display, the notification indicating at least one of: an occurrence of the indicator of the stressful event exceeding a threshold, a graphical representation of the stressful event over time, and a summary of the stressful event over time; as well as The user is prompted to respond to the notification via the electronic display, wherein the response to the notification includes at least one of: emotion journaling, diary recording, guided breathing, guided meditation, and recording participation in a prescribed stress relief activity.

22. The method according to one of the preceding claims 18 to 21, wherein: Controlling the function of the wearable computing device includes at least one of: controlling the electronic display of the wearable computing device, and providing the indicator of the stressful event at the certain time to the user via the electronic display.

Images