Motion sickness detection system for autonomous vehicles

Abstract

The techniques described herein include detecting the degree of motion sickness experienced by a user in a vehicle. An appropriate combination of physiological data (heart rate, heart rate variability parameters, blood volume pulses, oxygen values, respiration values, electrodermal response, skin conductivity values, etc.), eye gaze data (e.g., images of the user), and vehicle motion data (e.g., accelerometer and gyroscope data indicating vehicle vibrations) may be utilized to identify the degree of motion sickness experienced by the user. One or more autonomous actions may be performed to prevent an increase in the degree of motion sickness experienced by the user or to improve the degree of motion sickness currently experienced by the user.

Description

Background Technology

[0001] Technologies related to autonomous or partially autonomous vehicles have advanced over the years. Motion sickness is expected to occur more frequently in autonomous vehicles due to the shift in roles from driver to passenger, as passengers may engage in activities that increase their susceptibility to motion sickness (e.g., reading, texting, using devices, etc.). Motion sickness not only causes nausea and physiological discomfort but can also affect cognitive and emotional states (e.g., anxiety, lack of concentration). It can be helpful to determine the severity of motion sickness a person may be experiencing. Early detection can be useful in enabling various strategies to prevent and / or alleviate symptoms.

[0002] The techniques described herein are provided to detect the degree of motion sickness currently being experienced by a person in a vehicle. Depending on the degree of motion sickness detected, a number of treatment measures and / or preventive strategies may be utilized.

[0003] Some embodiments may include a method for detecting motion sickness in a person in a vehicle. The method may include acquiring first physiological data from a plurality of sensor devices by one or more processors. The method may further include acquiring vehicle motion data indicating the frequencies of vibrations of the vehicle by one or more processors. The method may further include calculating a motion sickness score by one or more processors based at least partially on the first physiological data and the vehicle motion data, wherein the motion sickness score indicates the likelihood that a person is experiencing motion sickness. The method may further include detecting that a person is experiencing motion sickness based at least partially on the motion sickness score by one or more processors.

[0004] Some embodiments may include a computing device. The computing device may include memory storing executable commands to prevent or improve the degree of motion sickness experienced by a user in a vehicle, and one or more processors coupled to the memory for communication. In some embodiments, one or more processors are configured to execute commands to cause the computing device to perform actions. Actions may include acquiring physiological data from a plurality of sensor devices. Actions may include acquiring vehicle motion data indicating the frequencies of vibrations of the vehicle. Actions may include calculating a motion sickness score based at least partially on the physiological data and the vehicle motion data. Actions may include detecting the degree of motion sickness experienced by the user based at least partially on the motion sickness score. Actions may include causing the performance of an autonomous vehicle action in response to determining the degree of motion sickness experienced by the user. In some embodiments, causing the autonomous vehicle action to be performed improves the degree of motion sickness experienced by the user and / or prevents an increase in the degree of motion sickness experienced by the user at a subsequent time.

[0005] Some embodiments may include a non-transient computer-readable medium. The non-transient computer-readable medium may include stored commands for preventing or improving the degree of motion sickness experienced by a user in a vehicle. In some embodiments, the commands cause one or more processing units to perform actions when executed by one or more processing units. The actions may include acquiring physiological data from a plurality of sensor devices. The actions may include acquiring vehicle motion data indicating the frequencies of vibrations of the vehicle. The actions may include calculating a motion sickness score based at least partially on the physiological data and the vehicle motion data. The actions may include detecting that the user is experiencing a degree of motion sickness based at least partially on the motion sickness score. The actions may include triggering the performance of an autonomous vehicle action in response to determining that the user is experiencing a degree of motion sickness. In some embodiments, having the autonomous vehicle action performed improves the degree of motion sickness experienced by the user and / or prevents an increase in the degree of motion sickness experienced by the user at a subsequent time. Brief explanation of the drawing

[0006] FIG. 1 is a simplified block diagram illustrating a method for classifying the degree of motion sickness currently experienced by a user of a vehicle according to one embodiment.

[0007] FIG. 2 is a collection of graphs illustrating an exemplary sequence of physiological parameters of a user indicating progression from a baseline state to experiencing a stress response and to experiencing a sick response (e.g., motion sickness), according to one embodiment.

[0008] FIG. 3 is a table illustrating various combinations of parameter values ​​for classifying the degree of motion sickness of a vehicle user according to one embodiment.

[0009] FIGS. 4a to 4c are flowcharts illustrating methods for determining a number of variable values ​​according to some embodiments.

[0010] FIG. 5 is a flowchart illustrating a method for determining whether a user of a vehicle is experiencing motion sickness, according to one embodiment.

[0011] FIG. 6 is a simplified block diagram of a method for calculating various features from collected physiological data according to one embodiment.

[0012] FIG. 7 is a simplified block diagram of a method for determining whether a stress response has occurred, according to one embodiment.

[0013] FIG. 8 is a simplified block diagram of a method for determining whether a nausea reaction has occurred, according to one embodiment.

[0014] FIG. 9 is a flowchart of a method for preventing or improving motion sickness in a user in a vehicle according to one embodiment.

[0015] FIG. 10 includes a functional block diagram of a vehicle according to one embodiment.

[0016] FIG. 11 is a schematic diagram of various hardware and software components of a vehicle according to one embodiment.

[0017] FIG. 12 is a schematic diagram illustrating an exemplary configuration of a plurality of cameras arranged to collect eye gaze data and a plurality of physiological sensors configured to collect physiological data of a user in a vehicle, according to one embodiment.

[0018] Similar reference numerals in various drawings indicate similar elements according to specific exemplary embodiments. Additionally, multiple instances of an element may be indicated by following a first number and a second number for the element with a letter or a hyphen. For example, multiple instances of an element (110) may be indicated as 110-1, 110-2, 110-3, etc., or 110a, 110b, 110c, etc. When referring to such an element using only the first number, any instance of the element must be understood (for example, the element (110) of the previous example would refer to elements (110-1, 110-2, and 110-3) or elements (110a, 110b, and 110c)). Specific details for implementing the invention

[0019] Now, various exemplary embodiments will be described in connection with the accompanying drawings, which form part of this invention. Although specific embodiments in which one or more aspects of the present disclosure may be implemented are described below, other embodiments may be used and various modifications may be made without departing from the spirit of the appended claims or the scope of the present disclosure.

[0020] It is estimated that 37% of occupants of autonomous vehicles in the United States experience motion sickness resulting from engaging in various activities, such as reading, texting, and using smartphones or other devices, at least partially while the vehicle is in motion. Vulnerability to motion sickness may be amplified when riding in autonomous vehicles because a person cannot control and / or anticipate the direction of movement, and due to the conflict between vestibular, proprioceptive, and visual senses (i.e., the perception of body movement as determined by the vestibular system of the inner ear versus the perception of body movement as perceived through vision and / or the perception of body movement as perceived by the body's joints and / or muscles). Motion sickness not only causes nausea and physiological discomfort but can also affect cognitive and emotional states (e.g., anxiety, lack of concentration). Other symptoms may include gastrointestinal sensation / discomfort, dizziness, lightheadedness, disorientation, sweating, clamminess, hot flash / cold flash, drowsiness, irritability, anxiety, etc. Therefore, it may be important for safety to be able to determine the degree of motion sickness experienced by the driver, especially before transferring control of the vehicle to the driver after an autonomous driving session.

[0021] The experience of nausea occurs near the final stage of severe motion sickness. However, physiological data combined with other vehicle data can signal preliminary stages of motion sickness before nausea and / or other symptoms occur. Early detection can enable the activation of various strategies to prevent and / or alleviate symptoms. Such strategies may include, but are not limited to, adjusting driving dynamics (e.g., speed, handling, acceleration, etc.), adjusting the sensory environment (e.g., adjusting lighting, playing music, providing warnings, adjusting volume, adjusting temperature settings and / or airflow, modifying the olfactory environment such as providing scents, etc.), and adjusting the horizon view (e.g., adjusting video displays, increasing the viewing area of ​​windows, etc.).

[0022] The embodiments provided herein utilize a combination of physiological data (heart rate, heart rate variability, blood volume pulses (e.g., amplitude), skin conductivity / electric skin responses, skin temperature, etc.) and vehicle motion data (e.g., any suitable data identifying vehicle motion including the amplitude and frequency of vibrations of the vehicle and / or parts of the vehicle) to assess whether a person in a vehicle is experiencing motion sickness and / or the degree of motion sickness experienced by that person. As used herein, physiological data may include any appropriate data associated with any appropriate physiological properties of a person (e.g., skin temperature, heart rate, heart rate variability, respiratory rate, respiratory volume, blood oxygen level (SpO2), skin conductance level (SCL), and skin conductance response (SCR), galvanic skin response (GSR), electrodermal activity (EDA), electrodermal response (EDR), sympathetic skin response (SSR), peripheral autonomic nervous system surface potential, psychogalvanic reflex (PGR), sympathetic skin response (SSR), etc. as measured by a skin conductance sensor (e.g., galvanic skin response (GSR) sensor).

[0023] In some embodiments, vehicle motion data may be captured by any suitable combination of the vehicle's accelerometers and / or gyroscopes. Low-frequency vibrations of 0.1–0.5 Hz in the lateral (front-back) or longitudinal (left-right) directions are the vibrations most likely to contribute to motion sickness. Frequencies within this range are typically generated by driver behaviors: turning, braking, and acceleration (not by road conditions or vehicle structure).

[0024] In some embodiments, one or more cameras may be utilized to capture one or more images and / or videos (also referred to as “eye gaze data”) of one or more passengers in a vehicle (e.g., the faces of the passengers). By utilizing any suitable image recognition technology and / or eye tracking technology, the passenger’s gaze may be identified (e.g., whether the passenger is looking out the window, whether his eyes are on the horizon, whether he is looking down, whether he is reading a book, whether he is playing with his phone, etc.). To minimize motion sickness symptoms, it is best to have a visual field of the horizon as large as possible. A limited visual field results in increased motion sickness because the movement felt by the vestibular system is greater than the movement seen by the visual system, which will lead to collisions. The identification of the passenger’s gaze may be utilized (e.g., in conjunction with physiological data and / or vehicle movement data) to determine a motion sickness score that indicates in real time the degree of motion sickness likely being experienced by the passenger.

[0025] There is a sequence of physiological responses that occur prior to a person experiencing nausea. In some embodiments, this sequence may be defined by a reference response (e.g., indicating the person's usual physiological data) and a subsequent stress response (specific physiological responses shown herein) prior to the nausea response, including nausea, being experienced by the person. Vehicle data, physiological data, and eye gaze data may be collected over time. In some embodiments, observing this sequence based on data collected over time may increase the likelihood that the person is experiencing motion sickness (e.g., a motion sickness score). Conversely, if the data collected over time does not first indicate a stress response, it may be determined that the user is less likely to be currently experiencing a nausea response (e.g., nausea).

[0026] Utilizing the disclosed techniques can enable motion sickness to be detected more accurately and / or faster than with conventional techniques. Because autonomous driving systems can be configured to maintain control of the vehicle while the driver is experiencing motion sickness by detecting motion sickness more accurately and / or faster, the techniques described herein can enable a safer autonomous driving experience. In some embodiments, having knowledge that a passenger in the vehicle is experiencing motion sickness may cause the autonomous driving system to perform one or more actions aimed at preventing and / or mitigating the effects of motion sickness. For example, the autonomous driving system may reduce speed and take less aggressive and / or fewer turns. In some embodiments, the described techniques may be applied in other contexts. For example, similar techniques may be utilized to evaluate cyber sickness (e.g., nausea) commonly experienced in virtual reality environments, augmented reality (AR) environments, and mixed reality (MR) environments.

[0027] FIG. 1 is a simplified block diagram illustrating a method (100) for classifying the degree of motion sickness currently experienced by a user of a vehicle according to one embodiment. Means for carrying out the method (100) may include one or more hardware and / or software components of a vehicle, such as those exemplified in FIG. 8 through 10, which are described below.

[0028] The method (100) may begin in block (102), where operations for detecting vehicle movement are executed. In some embodiments, these operations may include collecting various sensor data from one or more accelerometers, gyroscopes, or other suitable inertial sensors. The vehicle movement data collected from these sensors may be analyzed to determine whether the vehicle is moving.

[0029] The method may proceed to block (104), where a determination regarding whether the vehicle is moving may be made using vehicle movement data collected in block (102). In some embodiments, if it is determined that the vehicle is stopped (e.g., not moving), the method (100) may be terminated. Alternatively, if it is determined that the vehicle is moving, the method (100) may proceed to block (106), block (108), and / or block (110). The operations of blocks (106, 108, and / or 110) may be executed in any suitable sequence, or the operations of blocks (106, 108, and / or 110) may be executed substantially simultaneously.

[0030] In block (106), a vehicle vibration detection operation may be performed. As part of this operation, vibrations (e.g., tremors) of the vehicle in terms of latitude (e.g., front and back) or longitude (left and right) may be identified. In some embodiments, one or more accelerometers, gyroscopes, and / or inertial sensors may be utilized to identify the vehicle vibrations. For example, a 3-axis accelerometer may collect and identify data indicating the vibration frequency of the vehicle. In some embodiments, vehicle vibration data may be collected as part of a vehicle motion detection operation performed in block (102). Vehicle vibration data may be collected by one or more sensors integrated into the vehicle, or by one or more wearable devices, or any suitable combination of the above.

[0031] Once vibration data is collected, the method may proceed to block (112). In block (112), actions may be performed to determine whether latitude and / or longitude vibrations fall within a predefined threshold range. For example, the threshold range may be defined as 0.1–0.5 Hz. In some embodiments, this range may be modified over time as motion sickness is observed and / or confirmed in the passengers of the vehicle. Further details regarding threshold range modifications will be discussed below.

[0032] In block (108), eye gaze detection operations may be performed. In some embodiments, these operations may include acquiring one or more images and / or videos (referred to as “eye gaze data”) of a user (or users) of the vehicle. The eye gaze data may include images of the user’s face and / or eye region. If there are multiple users of the vehicle, the eye gaze data may include features of some combination of those users. Any reference herein to an operation being performed on a single user may be understood as similarly applicable to use cases where multiple users exist.

[0033] Once eye gaze data is collected in block (108), the method (100) may proceed to block (114). In block (114), actions may be performed to determine whether the user's gaze is on the horizon. In some examples, the user's gaze may be determined using a wearable device (e.g., AR glasses, non-display glasses, etc.). For example, the AR device may include an eye-tracking camera and / or an external camera from which the user's gaze can be identified. In some embodiments, an inertial measurement unit of the wearable device may be utilized to determine the user's gaze. As yet another example, a camera facing the user may be utilized to identify the user's gaze. For example, the camera may be attached to the dashboard of the vehicle, the ceiling of the vehicle, the windshield, or any suitable location where the camera can be oriented inward toward the user(s). Based on the known placement of the camera, the direction of the user's gaze may be determined. For example, if a camera is known to be positioned on the windshield, it can be determined that the user is looking out of the windshield if the user's eyes are fully visible in the image captured by the camera (or if a certain threshold amount of eyes is visible). Image recognition technology can be utilized to identify whether the user's eyes are fully visible (although not directly required by the camera, this indicates that the user is looking in the direction of the camera). Similarly, the user's gaze can be determined to be directed in various directions by analyzing features of the user's head or face visible within the image / video and / or features not visible within the image / video. Any suitable eye-tracking technology can be similarly utilized to identify the direction of the user's gaze.

[0034] In block (110), physiological response detection operations may be performed. In some embodiments, these operations may include collecting physiological data from one or more physiological sensors of the vehicle and / or from one or more wearable devices. As discussed above, the physiological data may include any appropriate data associated with any appropriate physiological attribute of a person captured by one or more corresponding physiological sensors (e.g., skin temperature, heart rate, heart rate variability, respiratory rate, respiratory volume, blood oxygen level (SpO2), skin conductivity level (SCL), and skin conductivity response (SCR), electro-skin response (GSR), skin electrical activity (EDA), skin electrical response (EDR), sympathetic skin response (SSR), peripheral autonomic nervous system surface potential, psychoelectric reflex (PGR), sympathetic skin response (SSR), etc. as measured by a skin conductivity sensor (e.g., electro-skin response (GSR) sensor).

[0035] FIG. 2 is a collection of graphs illustrating an exemplary sequence of physiological parameters of a user indicating progression from a baseline state to experiencing a stress response, or to experiencing a nausea response (e.g., degree of motion sickness), according to one embodiment. In some embodiments, physiological parameters may be collected for a set period of time (e.g., 5 minutes, 1 minute, 30 seconds, etc.) before the vehicle moves or at any appropriate time after the vehicle stops. In the example depicted in FIG. 2, the set of physiological sensors used to capture physiological data may include an EKG sensor (e.g., an electrocardiogram sensor also referred to as an "ECG" sensor), GSR sensors on the user's fingers, a thermometer configured to measure skin temperature on the user's fingers, and a GSR sensor placed on the user's forehead. Graphs corresponding to high frequency (HF) and the root mean square of successive differences (RMSSD) may be derived from the EKG data captured over time. In some embodiments, physiological data (e.g., a set of physiological parameters such as heart rate, skin conductivity, temperature, etc.) may be collected for a certain period of time (e.g., 1 minute, 5 minutes, 30 seconds) after movement is first initiated. In some embodiments, a wearable PPG sensor may be used to collect measurements (e.g., blood volume pulse) and physiological parameters (e.g., heart rate, heart rate variability, etc.) may be derived from the measurements. In the above scenarios, the collected physiological parameters may be stored locally and labeled as the user's reference physiological parameters (e.g., associated with them).

[0036] The graphs (200) in FIG. 2 collectively depict a stress response starting at T1. In some embodiments, the onset symptoms of the stress response may include an increased heart rate and an increase in the electrical skin response in the user's fingers. After approximately one minute (e.g., approximately 10-15 seconds), the temperature captured in the user's fingers may drop. In some embodiments, the stress response may include parasympathetic nervous system withdrawal symptoms, such as heart rate variability / low frequency, or heart rate variability / root mean of consecutive differences, as depicted in FIG. 2.

[0037] If symptoms of a nausea response as depicted in FIG. 2 follow symptoms of a stress response, the user may be identified as having experienced motion sickness at T2. The nausea response may include any appropriate combination of the following: increased electrical skin responses on the forehead and / or fingers, slightly increased heart rate, stable or slightly increased finger temperature and blood volume pulse amplitude, and / or stable or increased HF / RMSSD.

[0038] Returning to FIG. 1, as physiological data is collected in block (110), the method (100) can proceed to block (116) where a reference response can be identified for the user. In some embodiments, the physiological data may be collected near the time when the vehicle's movement begins. Thus, such physiological data (e.g., physiological data corresponding to a 2-minute average when the vehicle is stopped, the first 5 minutes of movement, 10 minutes of movement, 1 minute before the movement begins, 5 minutes before the movement begins, etc.) can be utilized to define a reference response for the user. In some embodiments, a first set of physiological data parameters may be stored in local memory and associated with the user's reference response.

[0039] Over time, more physiological data may be collected, and the method (100) may proceed to a block (118) in which the system can determine whether the user is experiencing a stress response. As discussed above in relation to FIG. 2, determining whether the user is experiencing a stress response may include, first, determining the user's baseline response and, then, identifying that a set of conditions is satisfied. That is, the stress response may be defined as the occurrence of the baseline response and a predefined set of changes in corresponding physiological parameters from the baseline response that exceed a predefined threshold amount.

[0040] More physiological data may be collected and the method (100) may proceed to a block (120) in which the system can determine whether the user is experiencing a nausea response. As discussed above in relation to FIG. 2, determining whether the user is experiencing a nausea response may include, first, determining the user's reference response and, then, identifying that a set of conditions associated with a stress response is satisfied, and subsequently, determining that a different set of conditions associated with a nausea response is satisfied.

[0041] The method (100) may proceed to a block (122) in which operations for calculating a motion sickness score are executed. In some embodiments, the motion sickness score indicates the probability and / or degree that the user is experiencing motion sickness. The motion sickness score may be calculated at least partially based on decisions made in blocks (112, 114, 116, 118, and 120) or any combination of such decisions. In some embodiments, a greater number of positive indications obtained from blocks (112, 114, 116, 118, 120) may generate a higher motion sickness score than would be corresponding for a smaller number of positive indications from blocks (112, 114, 116, 118, and 120). In some embodiments, the motion sickness score may be a numeric value that can be mapped to a classification (e.g., none, low, medium, and high). In other embodiments, the motion sickness score may be a classification rather than a numerical value.

[0042] FIG. 3 is a table (300) illustrating various combinations of parameter values ​​for classifying the degree of motion sickness of a person in a vehicle according to one embodiment.

[0043] A column labeled "nausea" is intended to correspond to a decision made in block (120) of FIG. 1. A value of "0" in the nausea column indicates that the decision in block (120) was that the user is not experiencing a nausea reaction, while a value of "1" indicates that the decision in block (120) is that the user is experiencing a nausea reaction.

[0044] The column labeled "Stress" is intended to correspond to a decision made in block (118) of FIG. 1. A value of "0" in the Stress column indicates that the decision in block (118) was that the user is not experiencing a stress response, while a value of "1" indicates that the decision in block (118) is that the user is experiencing a stress response.

[0045] The column labeled "line of sight" is intended to correspond to a decision made in block (114) of FIG. 1. A value of "0" in the line of sight column indicates that the decision in block (114) was that the user was not looking in the direction of the horizon, while a value of "1" indicates that the decision in block (118) was that the user was looking in the direction of the horizon.

[0046] The column labeled "vibration" is intended to correspond to a decision made in block (112) of FIG. 1. A value of "0" in the vibration column indicates that the decision in block (112) was that the vehicle's vibrations were outside a predetermined range, while a value of "1" indicates that the decision in block (112) was that the vehicle's vibrations were within a predetermined range.

[0047] The column labeled "Classification" describes the classification that can be assigned to the data corresponding to each row of the table (300).

[0048] Returning to FIG. 1, in some embodiments, operations performed in block (122) may include comparing decisions obtained in blocks (112, 114, 118, and 120) with a table (300) of FIG. 3 that may be stored in local memory. Based at least partially on this comparison, a classification may be identified from the table and the user may be identified as experiencing no motion sickness, low degree of motion sickness, moderate degree of motion sickness, or high degree of motion sickness.

[0049] In some embodiments, a machine learning model (also referred to as a classification model) may be trained using supervised learning techniques and a training data set to classify new input data (e.g., physiological data, eye gaze data, vehicle movement data, etc.) as indicating the degree of motion sickness a user is experiencing. In some embodiments, the training data set may include any suitable number of examples comprising any suitable combination of physiological data, eye gaze data, and vehicle movement data, for which the classification (e.g., degree of motion sickness experienced by the user) is known. The training data set may include any suitable number of examples indicating the degree of motion sickness. Although the examples described herein utilize four classification values ​​(e.g., none, stress, nausea, low, medium, high), any suitable number of classifications may be similarly utilized. Exemplary supervised learning algorithms may include regression algorithms (e.g., linear regression, logistic regression, etc.), decision trees, random forests, neural networks, naive Bayes, k-nearest neighbors, etc. As a non-limiting example, regression algorithms may be utilized to generate a mathematical model (e.g., a classification model) that models the relationship between two variables by fitting a linear mathematical expression to observed data (e.g., a training dataset). Once the model is generated, new data (e.g., physiological data, eye gaze data, vehicle movement data) may be provided to the model, and in response, the model may provide a classification for the new data.

[0050] In some embodiments, the machine learning model may be improved over time to provide more accurate classifications. For example, new data may be provided to the model and classification may be determined by the model. In some embodiments, feedback may be elicited from the user to verify whether the classification was accurate. A user interface that enables the user to provide such feedback may be provided (e.g., via a vehicle touchscreen). If the feedback is accurate (or if the feedback is inaccurate), the collected data may be labeled as such and added to the training data set. The model may now be retrained based at least partially on the training data set containing the new example(s) and / or updated incrementally.

[0051] As another example, a predefined set of rules or a machine learning model may be utilized to calculate (or obtain) a numerical value corresponding to the motion sickness score. The motion sickness score can quantify the likelihood of the user experiencing it and / or the degree of motion sickness experienced by the user.

[0052] FIGS. 4a through 4c are flowcharts illustrating methods (400A-400C) for determining a plurality of variable values ​​according to some embodiments. Means for carrying out the methods (400A, 400B, and / or 400C) may include one or more hardware and / or software components of a vehicle, such as those exemplified in FIGS. 8 through 10, described below. Prior to the execution of the methods (400A-400C), various variables holding determinations corresponding to blocks (104, 112, 114, 116, 118, and 120) may be initialized to default values ​​(e.g., false, indicating that the condition corresponding to each block is not satisfied).

[0053] FIG. 4a is a flowchart illustrating a method (400A) for determining whether a vehicle is moving. The method (400A) may begin at 402, where vehicle movement data may be captured over time t. Vehicle movement data may be collected by any suitable sensors including accelerometers, gyroscopes, magnetometers, etc. (e.g., one or more of the vehicle sensor(s) (810) discussed below in relation to FIG. 8). In some embodiments, vehicle movement data may be collected by one or more wearable devices such as a smartwatch, glasses, a vehicle's inertial measurement unit (IMU), etc. Vehicle movement data may be collected according to any suitable periodicity / frequency, according to a predefined schedule, and / or according to any suitable predefined conditions.

[0054] In 404, a determination regarding whether the vehicle is moving can be made from analyzing the vehicle movement data. If the vehicle is not moving, a variable (e.g., a “movement” variable) can be set in 406, and the method (400A) can return to 402, where additional vehicle movement data can be captured. This loop can be continued any appropriate number of times over time t. In some embodiments, time t may correspond to the total time the vehicle is in use (e.g., including time the vehicle is idling and stopped or moving).

[0055] If it is determined that the vehicle is moving at 404, the method (400A) may proceed to 406, where a variable (e.g., "movement" variable) may be updated to indicate that the vehicle is moving. After that, the method (400A) may return to 402, and steps (402-408) may be repeated any appropriate number of times over time t.

[0056] FIG. 4b is a flowchart illustrating a method (400B) for determining whether a vehicle is generating vibrations that fall within a predefined threshold (e.g., vibration frequencies that may be determined to be more likely to induce a user's stress and / or nausea response than other vibration frequencies). The method (400B) may start at 409, where the "movement" variable set at 406 may be checked. If the value of the variable indicates that the vehicle is not moving, the method (400B) may end at 410. Alternatively, if the value of the movement variable indicates that the vehicle is moving, the method (400B) may proceed to 412.

[0057] In 412, vehicle motion data can be captured over time. This vehicle motion data may have a similar type to that captured in 402 of method (400A), but may be at a subsequent time. Once this additional vehicle motion data is captured, method (400B) can proceed to 414, where the power spectral density (PSD) of the vehicle's vibrations can be calculated over time t using the additional vehicle motion data captured in 412. The power spectral density of the vehicle's vibrations is the energy fluctuation occurring within the vehicle motion data (e.g., accelerometer readings), measured as frequency per unit mass. In some embodiments, the energy fluctuations may be decomposed along two vertical axes. The transverse component may represent the amount of energy fluctuation of the vibrations corresponding to the axis extending from the front to the rear of the vehicle, while the longitudinal component may represent the amount of energy fluctuation of the vibrations corresponding to the axis extending from one side of the vehicle to the opposite side.

[0058] In 416, a determination may be made as to whether any of the transverse or longitudinal components calculated in 414 fall within a predefined frequency range. As an example, the predefined frequency range may be 0.1 to 0.5 Hz. If the components of the vehicle vibrations do not fall within the predefined frequency range, the method (400B) may proceed to 418, where a variable (e.g., a vibration variable) may be set to a value indicating the same, and the method (400B) may proceed again to 408. This loop may continue any appropriate number of times over time t, as long as the vibration components of the vehicle's shaking do not fall within the predefined frequency range. In some embodiments, time t may correspond to the total time the vehicle is in use (e.g., time when the vehicle is idling and stopped or moving).

[0059] Alternatively, if the components of the vehicle vibrations are within a predefined range, method (400B) may proceed to 420, where a variable (e.g., oscillation variable; "Osc") may be updated to indicate the same. After that, method (400B) may return to 409, and steps (409-420) may be repeated any appropriate number of times in any appropriate order over time t.

[0060] FIG. 4c is a flowchart illustrating a method (400C) which is a method for determining whether the user's gaze is directed sufficiently toward the horizon (exceeding a threshold amount). The method (400C) may start at 422, where the "movement" variable set at 406 of the method (400A) may be checked. If the value of the variable indicates that the vehicle is not moving, the method (400C) may end at 424. Alternatively, if the value of the movement variable indicates that the vehicle is moving, the method (400C) may proceed to 424.

[0061] In 424, eye gaze data may be captured over time. This eye gaze data may include one or more images and / or videos of the user (e.g., the user's face, eyes, pupils, etc.) captured over time t. The eye gaze data may be captured by any suitable combination of one or more cameras (e.g., camera(s) (812) of FIG. 8). The camera(s) may be facing inward toward the user (and in some cases, may also capture other users in the vehicle).

[0062] Method (400C) may proceed to 426, where the user's gaze may be analyzed to determine the amount by which the user's gaze deviates from the horizon (e.g., over a specified time period ('t'). For example, the image(s) / video(s) captured in 424 may be analyzed using any suitable image recognition and / or eye tracking techniques to identify the direction of the user's gaze. Given a known location of the horizon, the amount by which the user's gaze deviates from the horizon may be calculated. In some embodiments, the user's gaze may deviate for a specified time period 't' prior to the execution of actions to identify the user's gaze.

[0063] In 428, a determination may be made as to whether the amount of the user's gaze deviating from the horizon exceeds a threshold over a specified time period (t). As an example, the threshold may be 10 degrees corresponding to 10 degrees in any direction from looking straight ahead toward the horizon visible through the vehicle's front window over a certain time period (e.g., 1 minute, 30 seconds, 5 minutes, etc.). If the amount of deviation does not exceed the threshold, method (400C) may proceed to 430, where a variable (e.g., gaze variable) may be set to a value indicating the same thing, and method (400C) may proceed back to 422. This loop may continue for any appropriate number of times over time t, as long as the amount of deviation does not exceed the threshold.

[0064] Alternatively, if the amount of deviation exceeds a threshold value, method (400C) may proceed to 432, where a variable (e.g., gaze variable ("gaze")) may be updated to display the same thing. After that, method (400C) may return to 422, and steps (422-432) may be repeated any appropriate number of times in any appropriate order over time t.

[0065] FIG. 5 is a flowchart illustrating a method (500) for determining whether a person in a vehicle is experiencing motion sickness, according to one embodiment. Means for carrying out the method (500) may include one or more hardware and / or software components of a vehicle, such as those exemplified in FIG. 8 through 10, which are described below.

[0066] Method (500) can start at 502, where the “movement” variable set at 406 of Method (400A) can be checked. If the value of the variable indicates that the vehicle is not moving, Method (500) can end at 504. Alternatively, if the value of the movement variable indicates that the vehicle is moving, Method (500) can proceed to 506, where the variables (e.g., the “Osc” variable indicating whether the vehicle’s vibrations fall within a predefined frequency range and the “eye” variable indicating whether the user’s gaze exceeds a threshold amount and moves away from the horizon) are determined by the execution of Methods (400B and 400C). If both variables are set to false (or otherwise indicate a negative determination), Method (500) can return to 502. Alternatively, if one (or both) of the variables is set to true (or otherwise indicates a positive decision), the method (500) can proceed to 508.

[0067] In 508, physiological data may be captured and any appropriate features may be calculated / identified from such data. For example, physiological data may be collected by any appropriate combination of one or more physiological sensor(s) (828) of FIG. 8. Thus, physiological data may include any appropriate data related to heart rate, heart rate variability, blood pressure, blood volume pulse, pulse, oxygen, temperature, skin conductivity, electro-skin response (GSR), electro-skin response (EDR), psychoelectrical reflex (PGR), skin conductivity response (SCR), sympathetic skin response (SSR), skin conductivity level (SCL), etc.

[0068] FIG. 6 is a simplified block diagram of a method (600) for calculating various features from collected physiological data according to one embodiment. In some embodiments, the operations performed in 508 may include the operations of the method (600). The steps (602-620) may be performed in any suitable order and / or any suitable number of operations performed in the steps (602-620) may be performed substantially simultaneously.

[0069] The method (600) may begin at 602, where echocardiography (ECG) data may be captured over time t. The ECG data may be captured by the ECG sensor of the physiological sensor(s) (828) of FIG. 8. At 604, physiological parameters including average heart rate, average (or actual) heart rate variability metrics (e.g., low frequency (LF), high frequency (HF), and root mean square of successive differences (RMSSD)) may be calculated based on the ECG data captured over time t.

[0070] Method (600) may proceed to 606, where electro-skin response (GSR) data may be captured over time t. The GSR data may be captured by the GSR sensor of the physiological sensor(s) (828) of FIG. 8. In some embodiments, the GSR data may correspond to a specific part of the body. In 608, based on the GSR data captured in 606 over time t, physiological parameters including the average skin conductivity level and the skin conductivity response rate may be calculated.

[0071] Method (600) can proceed to 610, where skin temperature data can be captured over time t. Skin temperature data can be captured by the temperature sensor of the physiological sensor(s) (828) of FIG. 8. In some embodiments, the temperature data may correspond to a specific part of the body. In 612, based on the temperatures captured over time t, physiological parameters including an average temperature (e.g., temperature captured at the finger where the temperature sensor is placed, temperature captured at the steering wheel where the sensor is placed, temperature captured by the wearable device) can be calculated.

[0072] Method (600) can proceed to 614, where photoplethysmography (PPG) data can be captured over time t. The PPG can be captured by the PPG sensor of the physiological sensor(s) (828) of FIG. 8. In 616, based on the PPG data captured over time t, physiological parameters including blood volume pulse amplitude can be calculated.

[0073] Method (600) may proceed to 618, where additional GSR data may be captured over time t. Additional GSR data may be captured by another GSR sensor among the physiological sensor(s) (828) of FIG. 8. In some embodiments, the additional GSR sensor may be placed on the user's forehead. In 620, based on the GSR captured in 618 over time t, physiological parameters including an average skin conductivity level and a skin conductivity response rate may be calculated.

[0074] Returning to FIG. 5, the method (500) can proceed from 508 to 510, where not only the captured physiological data but also any features calculated therefrom can be stored as reference data and a variable indicating that reference data exists can be set. Steps (502-510) can correspond to block (116) of FIG. 1.

[0075] At 512, the “movement” variable set at 406 of method (400A) may be checked again. If the value of the variable indicates that the vehicle is not moving, method (500) may be terminated at 514. Alternatively, if the value of the movement variable indicates that the vehicle is moving, method (500) may proceed to 516, where subsequent physiological data may be captured and corresponding features may be calculated. In some embodiments, the operations performed at 516 may include the execution of method (600) of FIG. 6.

[0076] In 518, a determination may be made regarding whether the user has experienced a stress response (e.g., as shown in FIG. 2). In some embodiments, this determination may be made at least partially by comparing physiological data and / or corresponding physiological features (e.g., heart rate, BVP, RMSSD, LF, HF, LF / HF, GSR values, temperatures at specific locations of the body, etc.) with a predefined set of physiological feature values ​​associated with a stress response and / or by determining that each feature has changed in a predefined manner (e.g., increased or decreased) or remained substantially constant in relation to the reference data stored in 510. For example, a predefined set of physiological features may indicate that an increase in GSR at the finger and an increase in heart rate signal the onset of a stress response. Comparison of the physiological data and / or features with the predefined set may indicate a stress response when the user's data / features show a similar increase. In other embodiments, the determination that a user is experiencing a stress response may be based at least partially on providing physiological data and / or corresponding physiological features calculated in 516 as input to a machine learning model trained to identify input data as indicating a stress response or not indicating a stress response.

[0077] In some embodiments, the machine learning model utilized in 518 may be trained using supervised learning techniques and a training dataset to classify new input data (e.g., physiological data and / or features derived from such data) as indicating whether or not a user is experiencing a stress response. In some embodiments, the training dataset may include any appropriate number of examples comprising any appropriate combination of physiological data and / or physiological features derived from such data, for which a classification (e.g., stress response = true, stress response = false) is known. The training dataset may include any appropriate number of examples indicating a stress response and / or any appropriate number of examples not indicating a stress response. The model may be trained using supervised learning algorithms such as regression algorithms (e.g., linear regression, logistic regression, etc.), decision trees, random forests, neural networks, naive Bayes, k-nearest neighbors, etc. Once the model is trained, new data (e.g., physiological data and / or corresponding physiological features derived from such data) may be provided to the model, and in response, the model may provide a value (e.g., true, false) that can be considered as a determination regarding whether the user is experiencing a stress response.

[0078] If it is determined in 518 that the user has not experienced a stress response, the method may proceed to 520, where a stress variable may be set (e.g., to indicate a false or otherwise negative decision), and the method (500) may return to 512. Alternatively, if it is determined in 518 that the user has experienced a stress response, the method (500) may proceed to 522, where stress data (e.g., data collected and / or calculated in 516) may be stored and / or a stress variable may be set (e.g., to indicate a true or otherwise positive decision). Steps (512-522) may correspond to block (118) of FIG. 1.

[0079] Method (500) may proceed to 524, where the "movement" variable set at 406 of Method (400A) may be checked again. If the value of the variable indicates that the vehicle is not moving, Method (500) may end at 526. Alternatively, if the value of the movement variable indicates that the vehicle is moving, Method (500) may proceed to 528, where additional physiological data may be captured and corresponding physiological features may be calculated. In some embodiments, the operations performed at 528 may include the execution of Method (600) of FIG. 6.

[0080] In 530, a determination may be made regarding whether the user has experienced a nausea response (e.g., the nausea response shown in FIG. 2). In some embodiments, this determination may be made at least partially based on comparing physiological data and / or corresponding physiological features (e.g., heart rate, BVP, RMSSD, LF, HF, LF / HF, GSR values, temperatures at specific locations of the body, etc.) with a predefined set of physiological feature values ​​associated with a nausea response and / or by determining that each feature has changed in a predefined manner (e.g., increased or decreased) or remained substantially constant in relation to the stress data stored in 522. For example, the predefined set of physiological features may indicate that an increase in GSR at the forehead (e.g., after an increase in GSR and heart rate at the finger as shown in FIG. 2) signals the onset of a nausea response. Comparison of physiological data and / or features against a predefined set may indicate a nausea response when the user's data / features show a similar increase. In other embodiments, the determination that the user is experiencing a nausea response may be based at least partially on providing physiological data and / or corresponding physiological features calculated in 528 as input to a machine learning model trained to identify input data as indicating a nausea response or not indicating a nausea response.

[0081] In some embodiments, the machine learning model utilized in 530 may be trained using supervised learning techniques and a training dataset to classify new input data (e.g., physiological data and / or features derived from such data) as indicating whether or not a user is experiencing a nausea response. In some embodiments, the training dataset may include any appropriate number of examples comprising any appropriate combination of physiological data and / or physiological features derived from such data, for which a classification (e.g., nausea response = true, nausea response = false) is known. The training dataset may include any appropriate number of examples indicating a nausea response and / or any appropriate number of examples not indicating a nausea response. The model may be trained using supervised learning algorithms such as regression algorithms (e.g., linear regression, logistic regression, etc.), decision trees, random forests, neural networks, naive Bayes, k-nearest neighbors, etc. Once the model is trained, new data (e.g., physiological data and / or corresponding physiological features derived from such data) may be provided to the model, and in response, the model may provide a value (e.g., true, false) that can be considered as a determination regarding whether the user is experiencing a stress response.

[0082] If it is determined in 530 that the user has not experienced a nausea reaction, the method may proceed back to 518. Alternatively, if it is determined in 530 that the user has experienced a nausea reaction, the method (500) may proceed to 534, where a nausea variable may be set (e.g., to indicate true, or otherwise to indicate a positive decision). Steps (524-534) may correspond to block (120) of FIG. 1. In some embodiments, physiological data and / or features captured / calculated in 528 may be stored for subsequent use.

[0083] FIG. 7 is a simplified block diagram of a method (700) for determining whether a stress response has occurred, according to one embodiment. Prior to the execution of the method (700), the method (600) may be executed to calculate various features from collected physiological data corresponding to collecting reference data at a first time. The method (600) may be executed to calculate various features from collected physiological data corresponding to collecting current data at a second time (e.g., a second time later than the first time). The method (700) may be initiated after the reference data and current data have been collected.

[0084] Method (700) may start at 702, where the difference between a reference heart rate and a current heart rate may be calculated. At 704, if the difference in heart rate identified at 702 exceeds a predetermined heart rate threshold, it may be determined that the difference in heart rate indicates one feature of a stress response. If the difference in heart rate does not exceed the predetermined heart rate threshold, it may be determined that no feature of a stress response is indicated.

[0085] In 706, the difference between the current SCL and the reference SCL at the fingers (or other locations) and the difference between the current SCR rate and the reference SCR rate at the fingers (or other locations) can be calculated. In 710, if both the difference in SCL and the difference in SCR rate exceed a predetermined threshold, it can be determined that a specific feature of the stress response is displayed. If one or both of the differences do not exceed the threshold, it can be determined that the specific feature of the stress response is not displayed.

[0086] In 710, the difference between the reference skin temperature at the fingers (or other location) and the current skin temperature at the fingers (or other location) can be calculated. In 712, it can be determined that if the difference identified in 710 is less than a corresponding predetermined temperature threshold, the stress response feature is displayed. If the difference identified in 710 is equal to or exceeds the corresponding predetermined temperature threshold, it can be determined that the stress response feature is not displayed.

[0087] In 714, the difference between the current HF and the reference HF or the difference between the current RMSSD and the reference RMSSD may be calculated. In 716, if either of the difference in the difference HF or the difference in the RMSSD is less than a corresponding predetermined threshold, it may be determined that a specific feature of the stress response is displayed. If it is determined that neither of the differences is less than a corresponding predetermined threshold, it may be determined that the specific feature of the stress response is not displayed.

[0088] In 718, the difference between the reference BVP and the current BVP can be calculated. In 720, if the difference in the BVP identified in 716 is smaller than the corresponding predetermined BVP threshold, it can be determined that the difference in the BVP indicates one feature of the stress response. If the difference in the BVP is equal to or exceeds the predetermined BVP threshold, it can be determined that no feature of the stress response is indicated.

[0089] In 722, the difference between the current SCL and the reference SCL at the forehead and the difference between the current SCR rate and the reference SCR rate at the forehead can be calculated. In 724, if both the difference in SCL and the difference in SCR rate are smaller than or equal to a predetermined threshold, it can be determined that the stress response feature is not displayed. If one or both of the differences exceed the threshold, it can be determined that the specific stress response feature is not displayed.

[0090] In 726, a determination regarding whether a set of conditions is satisfied may be made at least partially based on decisions made in 704, 708, 712, 716, 720, and 724. For example, a set of threshold conditions for determining that a stress response has occurred may include determining that the heart rate has changed (see, e.g., 704), determining that the SCR rate has changed, and at least one other positive determination from decisions made in 708, 712, 716, 720, and / or 724. In some embodiments, the threshold conditions required for the decision in 726 may be configurable by the user. If the threshold condition(s) are satisfied, a determination that a stress response has occurred may be made in 728, and the method (700) may be terminated in 730. If the threshold is not met or is exceeded, a determination that no stress response occurred may be made at 732 and the method (700) may be terminated at 730.

[0091] It should be recognized that more or fewer difference values ​​than the numbers shown in FIG. 7 may be calculated. It should also be recognized that the calculations of 702, 706, 710, 714, 718, and 722 may be performed in any appropriate order. In some embodiments, features calculated from collected physiological data corresponding to current data may be stored for subsequent use. This data may be referred to as "stress data." For example, if a decision is made that a stress response is indicated (e.g., 728), the physiological data corresponding to "current data" and the features calculated therefrom may be stored for subsequent use. This data may be referred to as "stress data" below.

[0092] FIG. 8 is a simplified block diagram of a method (800) for determining whether a nausea response has occurred, according to one embodiment. In some embodiments, the method (800) may be performed after the execution of the method (700) of FIG. 7. The method (600) may be performed once again to collect physiological data and to calculate features from the physiological data. This may now be considered as current data. For exemplary purposes, time (Ts) may refer to the time when the stress response of the figure was determined to have occurred.

[0093] Method (800) may begin at 802, where the difference between the heart rate corresponding to time (Ts) and the heart rate of the current data may be calculated. At 804, if the difference in heart rate identified at 802 is less than a predetermined heart rate threshold, it may be determined that the difference indicates one feature of a nausea response. If the difference in heart rate is equal to or exceeds the predetermined heart rate threshold, it may be determined that the feature of the nausea response is not indicated.

[0094] In 806, the difference between the current SCL in the fingers and the SCL in the fingers at time (Ts), and the difference between the current SCR rate and the SCR rate in the fingers at time (Ts) can be calculated. In 808, if both the difference in SCL and the difference in SCR rate exceed a predetermined threshold, it can be determined that a specific feature of the nausea response is displayed. If one or both of the differences are equal to the threshold or do not exceed the threshold, it can be determined that the specific feature of the nausea response is not displayed.

[0095] In 810, the difference between the skin temperature of the fingers at time (Ts) and the current skin temperature of the fingers can be calculated. In 812, it can be determined that if the difference identified in 810 is equal to or exceeds a corresponding predetermined temperature threshold, a feature of a nausea reaction is displayed. If the difference identified in 710 is less than the corresponding predetermined temperature threshold, it can be determined that the feature of a nausea reaction is not displayed.

[0096] In 814, the difference between the current HF and the HF at time (Ts) or the difference between the current RMSSD and the RMSSD at time (Ts) can be calculated. In 816, it can be determined that if the difference in the HF or RMSSD is equal to or exceeds a corresponding predetermined threshold, a specific feature of the nausea response is displayed. If neither the HF nor the RMSSD is equal to or does not exceed the corresponding predetermined threshold, that specific feature of the nausea response is not displayed.

[0097] In 818, the difference between the current BVP and the BVP at time (Ts) can be calculated. In 820, if the difference in the BVP identified in 818 is greater than or equal to the corresponding predetermined BVP threshold, it can be determined that the difference in the BVP indicates one feature of the nausea response. If the difference in the BVP is less than the predetermined BVP threshold, it can be determined that the feature of the nausea response is not indicated.

[0098] In 822, the difference between the current SCL at the forehead and the SCL at the forehead at time (Ts), and the difference between the current SCR rate at the forehead and the SCR rate at the forehead at time (Ts) can be calculated. In 824, if both the difference in SCL and the difference in SCR rate are greater than a predetermined threshold, it can be determined that a feature of nausea response is displayed. If one or both of the differences are equal to or exceed the threshold, it can be determined that the specific feature of nausea response is not displayed.

[0099] In 826, a determination regarding whether a set of conditions is satisfied may be made based at least partially on the decisions made in 804, 808, 812, 816, 820, and 824. For example, a set of critical conditions for determining whether a nausea reaction has occurred may include determining that the SCR rate at the forehead has changed and at least two other positive determinations from the decisions made in 804, 808, 812, 816, 820, and / or 824. In some embodiments, the critical conditions required for the determination in 826 may be configurable by the user. If the critical condition(s) are satisfied, a determination that a nausea reaction has occurred may be made in 828, and the method (800) may be terminated in 830. If the threshold is not met or is exceeded, a determination that no stress response occurred may be made at 832 and the method (800) may be terminated at 830.

[0100] It should be recognized that more or fewer difference values ​​than the numbers shown in FIG. 8 may be calculated. It should also be recognized that the calculations of 802, 806, 810, 814, 818, and 822 may be performed in any appropriate order. In some embodiments, the method (800) may not be executed unless the execution of the method (700) results in a determination that a stress response has occurred.

[0101] FIG. 9 is a flowchart for a method (900) for preventing or improving motion sickness in a user in a vehicle according to one embodiment. Means for performing the method (900) may include one or more hardware and / or software components of a vehicle, such as those exemplified in FIG. 10 through 12, described below. The method (700) may be performed by a computing device (e.g., a vehicle (1000), processor(s) (1002) of FIG. 10, etc.). In some embodiments, instructions for the method (900) may be stored in a non-transient computer-readable medium (e.g., memory (1004) of FIG. 10) and one or more processors (e.g., processor(s) (1002) of FIG. 10) may execute instructions to perform operations of the method (900). In some embodiments, the operations of the method (900) are performed by one or more processor(s) (1002) of FIG. 10 by executing code associated with the motion sickness detection engine (1030) of FIG. 10.

[0102] The method (900) may start in block (902), where physiological data may be obtained from a plurality of sensor devices (e.g., physiological sensor(s) (1028) of FIG. 10) (e.g., by one or more processor(s) (1002)).

[0103] In block (904), vehicle movement data of a vehicle (e.g., vehicle (1000)) can be obtained (e.g., by one or more processor(s) (1002) from vehicle sensor(s) (1010) of FIG. 10).

[0104] In block (906), a motion sickness score may be calculated based at least partially on physiological data and vehicle movement data (e.g., by processor(s) (902)). Although not depicted in FIG. 9, it may be recognized that a motion sickness score may be calculated based at least partially additionally on eye gaze data. In some embodiments, the motion sickness score may be calculated based at least partially on a mathematical function in which certain data may be weighted more than other data. In some embodiments, the motion sickness score may be calculated by providing physiological data and / or vehicle movement data (and / or features and / or eye gaze data derived therefrom) as input to one or more machine learning models (e.g., one or more machine learning models maintained by and / or accessible by one or more processor(s) (1002)). Commands associated with the motion sickness detection engine (1030) may cause any of the machine learning operations discussed herein to be performed when executed by the processor(s) (1002).

[0105] In block (908), the degree of motion sickness experienced by the user can be detected at least partially based on a motion sickness score (e.g., by one or more processor(s) (1002)).

[0106] In block (910), autonomous vehicle action may be performed (e.g., by one or more processor(s) (1002)) in response to determining that the user is experiencing a degree of motion sickness. In some embodiments, performing autonomous vehicle action improves the degree of motion sickness experienced by the user at a subsequent time or prevents an increase in the degree of motion sickness experienced by the user at a subsequent time. For example, autonomous vehicle action may include slowing down the vehicle speed, modifying the route, performing less aggressive turns, maintaining a more constant direction of travel, or any combination of the above.

[0107] FIGS. 10 to 12 are examples of systems, structural devices, vehicle components, and other devices, components, and systems that can be used to implement the technologies provided herein for detecting the degree of motion sickness that a person may experience while moving in an autonomous vehicle.

[0108] FIG. 10 includes a functional block diagram of a vehicle (1000) according to one embodiment. The methods discussed herein may be performed in relation to a person moving within the vehicle (800), as described in the embodiments above. Additionally, hardware and / or software components for executing the blocks illustrated in FIGS. 1 through 9 are described in more detail below. The vehicle (1000) may include, for example, an automobile, a bus, a truck, a motorcycle and / or other electric vehicle capable of driving at least partially autonomously.

[0109] As illustrated in FIG. 10, the vehicle (1000) may include various software and hardware components connected via a bus (1001). For example, the vehicle (1000) may include one or more processor(s) (1002) and a memory (1004). The memory (1004) may include executable instructions that are executable by the processor(s) (1002) for performing autonomous driving activities, including but not limited to external object detection and classification, prediction and planning, and behavior execution. The vehicle (1000) may include one or more wireless transceivers, such as wireless transceiver(s) (1006), for transmitting and receiving data through various means, protocols, and standards, such as SAE or European Telecommunications Standards Institute (ETSI) CV2X messages and data elements or other wireless and wireless protocols. In some embodiments, the wireless transceiver(s) (1006) may be configured to transmit and receive data messages and elements via short-range wireless communication protocols (e.g., Bluetooth®, Bluetooth Low Energy®, etc.), and / or via local and / or wide-area networks, and / or via cellular networks, and / or via any suitable wireless network. Of course, it should be understood that these are merely examples of networks that may be utilized by the vehicle (1000) via wireless links, and the claimed subject matter is not limited thereto. In one embodiment, the wireless transceiver(s) (1006) may include various combinations of WAN, WLAN, and / or PAN transceivers. In one embodiment, the wireless transceiver(s) (1006) may also include a Bluetooth transceiver, a ZigBee transceiver, or other PAN transceivers.

[0110] In some embodiments, the vehicle (1000) may include a Global Navigation Satellite System (GNSS) receiver (1008). The GNSS receiver (1008) may be configured to receive signals from navigation satellites (and / or other vehicles) and process them digitally to provide the receiver's position, speed, and time. The GNSS receiver (1008) may include hardware and / or software components. In one embodiment, GNSS signals from GNSS satellites received by the GNSS receiver (1008) are utilized by the vehicle (1000) for position determination and / or for determining GNSS signal parameters and demodulated data. In one embodiment, signals received by the radio transceiver(s) (806) are used for position determination, either alone or in combination with the GNSS signals received by the GNSS receiver (1008).

[0111] Examples of network technologies that can support wireless transceiver(s) (1006) are GSM, CDMA, WCDMA, LTE, 5G or New Radio Access Technology (NR), HRPD, and V2X car-to-car communication. As mentioned, V2X communication protocols can be defined in various standards such as SAE and ETS-ITS standards. GSM, WCDMA, and LTE are technologies defined by 3GPP. CDMA and HRPD are technologies defined by 3rd Generation Partnership Project II (3GPP2). WCDMA is also part of the Universal Mobile Telecommunications System (UMTS) and can be supported by an HNB.

[0112] Wireless transceiver(s) (1006) may communicate with communication networks through WAN wireless base stations that may include a placement of devices providing subscriber access to a wireless remote communication network for a service (e.g., under a service agreement). Here, the WAN wireless base station may perform the functions of a WAN or cell base station in servicing subscriber devices within a cell, which is determined at least partially based on the range in which the WAN wireless base station can provide access services. Examples of WAN base stations include GSM, WCDMA, LTE, CDMA, HRPD, Wi-Fi, Bluetooth, WiMAX, and 5G NR base stations. In one embodiment, additional wireless base stations may include WLAN and / or PAN transceivers.

[0113] In one embodiment, the vehicle (1000) may include one or more camera(s) (1012). In one embodiment, the camera(s) (1012) may include a camera sensor and a mounting assembly. Different mounting assemblies may be used for different cameras on the vehicle (1000). For example, forward-facing cameras may be mounted on the front bumper, on the stem of a rear-view mirror assembly, or on other forward-facing areas of the vehicle (1000). Rear-facing cameras may be mounted on the rear bumper / fender, on the rear windshield, on the trunk, or on other rear-facing areas of the vehicle. Side-facing mirrors may be mounted on the sides of the vehicle, such as being integrated into mirror assemblies or door assemblies. The cameras can provide object detection and distance estimation, particularly for objects of known size and / or shape (e.g., both stop signs and license plates have standardized size and shape), and can also provide information regarding rotational motion about the vehicle's axis, for example, during turning. When used in conjunction with other sensors, both cameras can be calibrated through the use of other systems, such as LIDAR, wheel tick / distance sensors, and / or GNSS, to verify the distance and angular orientation being moved. The cameras can similarly be used to verify and calibrate other systems to verify that distance measurements are accurate by calibrating for known distances between known objects (landmarks, roadside markers, road mile markers, etc.), and also to verify that object detection is performed accurately so that objects are mapped to accurate locations relative to the vehicle by LIDAR and other systems.Similarly, for example, when combined with accelerometers, the time to collision with road hazards (e.g., the elapsed time before hitting a pothole) can be estimated, which can be verified against the time of the actual collision and / or against stopping models (e.g., compared with the estimated stopping distance when attempting to stop before hitting an object) and / or maneuvering models (verifying whether current estimates of the turning radius at current speed and / or the measure of maneuverability at current speed are accurate under current conditions and modifying the parameters estimated based on camera and other sensor measurements to update accordingly).

[0114] In some embodiments, the camera(s) (1012) may be oriented inward (e.g., toward one or more passengers of the vehicle). The camera(s) (1012) may be utilized to capture one or more images and / or video(s) (referred herein as “eye gaze data”) of at least some part of one or more of the vehicle occupants. For example, the camera(s) (1012) may be positioned to capture images / videos of some part of the face (e.g., eyes) of one or more of the vehicle occupants (e.g., periodically according to a schedule, every second, etc.).

[0115] The vehicle sensor(s) (1010) may include any suitable number of accelerometers, gyroscopes, and / or magnetometers. In some embodiments, the vehicle sensor(s) (1010) may be part of the inertial measurement unit of the vehicle (1000). The vehicle sensor(s) (1010) may be utilized to provide and / or verify movement and orientation information, to monitor wheel and drivetrain performance, and / or to measure the amplitude and frequency of vibrations of the vehicle (1000) and / or parts of the vehicle (1000). For example, an accelerometer (e.g., a 3-axis accelerometer) may measure vibrations of the vehicle (1000), such as movement or mechanical vibrations relative to the equilibrium position of a component of the vehicle (1000). In one embodiment, accelerometers may also be utilized to verify the actual time of collision with road hazards, such as potholes, against predicted times based on existing stopping and acceleration models as well as steering models. In one embodiment, gyroscopes and magnetometers of the vehicle sensor(s) (1010) may be utilized to measure the turning state of the vehicle as well as the bearing relative to magnetic north, respectively, and, in particular when used in conjunction with measurements from other external and internal sensors, such as speed sensors, wheel tick sensors, and / or odometer measurements, to measure and calibrate estimates and / or models of the turning radius at the current speed and / or measures of maneuverability at the current speed. In some embodiments, the vehicle sensor(s) (1010) may be configured to measure vibrations and / or vibration frequencies corresponding to movements performed by the vehicle (1000).

[0116] The vehicle (1000) may include a LIDAR (1014). The LIDAR (1014) may use pulsed laser light to measure distances to objects. While camera(s) (1012) may provide object detection, the LIDAR (1014) may provide means for detecting distances (and orientations) of objects more reliably, particularly with respect to objects of unknown size and shape. LIDAR (1014) measurements may also be used to estimate the speed of movement, vector directions, relative position, and stopping distance by providing accurate distance measurements and delta distance measurements.

[0117] In one embodiment, power and drive systems and related systems (1016) (generator, battery, transmission, engine) and systems (1018) (brake, actuator, throttle control, steering, and electricity) may be controlled by processor(s) (1002) and / or hardware or software, or by the operator of the vehicle, or by any combination thereof. The systems (1018) and power and drive systems and related systems (1016) may be utilized in conjunction with performance parameters and operation parameters to enable the vehicle (1000) to be driven and operated autonomously (and, in relation to warnings and emergency overrides / braking / stopping) safely and accurately, for example, to safely, effectively, and efficiently join traffic situations, stop, accelerate, and otherwise operate the vehicle (800). In one embodiment, to control power and drive systems and related systems (1016) and systems (1018), inputs from various sensor systems such as camera(s) (1012), vehicle sensor(s) (1010) (including accelerometers, gyroscopes, barometers, etc.), LIDAR (1014), GNSS receiver (1008), RADAR (1020), inputs from wireless transceiver(s) (1006), messaging, and / or measurements, or various combinations thereof, may be utilized by processor(s) (1002) and / or DSP (1003) or other processing systems.

[0118] A GNSS receiver (1008) can be utilized to determine a position (absolute position) relative to the Earth, and, when used in conjunction with other information such as mapping data and / or measurements from other objects, to determine a position relative to other objects, such as other vehicles and / or road surfaces. To determine a position, the GNSS receiver (1008) can receive RF signals from one or more GNSS satellites using one or more antenna(s) (1009). The GNSS receiver (1008) can support one or more GNSS constellations as well as other satellite-based navigation systems. For example, in one embodiment, the GNSS receiver (1008) can support global navigation satellite systems such as GPS, GLONASS, Galileo, and / or BeiDou, or any combination thereof.In one embodiment, the GNSS receiver (1008) may support various augmentation systems (e.g., Satellite Based Augmentation Systems (SBAS) or ground Based Augmentation Systems (GBAS)) as well as local navigation satellite systems such as NavIC or QZSS or a combination thereof, as well as satellite integrated Doppler Orbitography and Radio-positioning Integrated by Satellite (DORIS) or wide area augmentation system (WAAS) or European geostationary navigation overlay service (EGNOS) or multi-functional satellite augmentation system (MSAS) or local area augmentation system (LAAS). In an embodiment, the GNSS receiver (1008) and antenna(s) (1009) may support a number of bands and sub-bands such as GPS L1, L2 and L5 bands, Galileo E1, E5 and E6 bands, Compass (BeiDou) B1, B3 and B2 bands, GLONASS G1, G2 and G3 bands, and QZSS L1C, L2C and L5-Q bands.

[0119] A GNSS receiver (1008) may be used to determine a location, a location and relative location that can be utilized for navigation, and, where appropriate, to determine the distance between two points in time, e.g., under clear sky conditions, and to calibrate other sensors, such as an odometer and / or LIDAR, using distance data to calibrate other sensors. In one embodiment, for example, GNSS-based relative locations based on shared Doppler and / or pseudo-distance measurements between vehicles may be used to determine highly accurate distances between two vehicles, and when combined with vehicle information such as GNSS antenna position and shape and model information, may be used to calibrate, verify, and / or influence the reliability level associated with information from LIDAR, cameras, RADAR, SONAR and other distance estimation technologies. GNSS Doppler measurements can also be utilized to determine the linear and rotational movements of a vehicle or of another vehicle, and can be used in conjunction with gyroscopes and / or magnetometers and other sensor systems to maintain the calibration of those systems based on the measured position data. Relative GNSS position data can also be combined with high-reliability absolute positions from road-side units (RSUs) to determine the high-reliability absolute positions of the vehicle. Furthermore, relative GNSS position data can be used during inclement weather that may obscure LIDAR and / or camera-based data sources in order to avoid other vehicles and to stay within a lane or other assigned road area.For example, GNSS measurement data can be provided to a vehicle using a roadside unit (RSU) equipped with a GNSS receiver and V2X capabilities, and the vehicle can be used to navigate the vehicle based on a map while keeping the vehicle in the lane and / or on the road despite a lack of visibility when the vehicle is provided with the absolute position of the RSU.

[0120] RADAR (1020) uses transmitted radio waves reflected from objects. The reflected radio waves are analyzed based on the time it takes for the reflected waves to arrive and other signal characteristics of the reflected waves to determine the location of surrounding objects. RADAR (1020) can be utilized to detect the location of nearby cars and roadside objects (signs, other vehicles, pedestrians, etc.), and generally enables the detection of objects even in cloudy weather such as snow, railway tracks, or hail. Thus, RADAR (1020) can be used to complement LIDAR (1014) and camera(s) (1012) in providing distance measurement information for other objects by providing distance measurements and distance values ​​and information, in cases where visual-based systems typically fail. Furthermore, the RADAR (1020) may be utilized to calibrate and / or check the sanity of other systems, such as the LIDAR (1014) and camera(s) (1012). Distance measurement measurements from the RADAR (1020) may be utilized to determine / measure stopping distance, acceleration, maneuverability at current speed and / or turning radius at current speed and / or measure of maneuverability at current speed. In some systems, ground-penetrating RADAR may also be used to track road surfaces through terrain features, such as RADAR reflective markers or ditches on the road surface, for example.

[0121] Physiological sensor(s) (1028) may include one or more sensors configured to measure skin electrical activity (EDA). EDA is a property of the human body that causes continuous fluctuations in the electrical properties of the skin. Any suitable combination of physiological sensor(s) (1028) may be configured to measure skin conductivity, electro-skin response (GSR), skin electrical response (EDR), psychoelectrical reflex (PGR), skin conductivity response (SCR), sympathetic skin response (SSR), skin conductivity level (SCL), etc. For example, the physiological sensor(s) (1028) may include one or more of the following: electrical skin response sensors configured to measure GSR, skin conductivity sensors configured to measure skin conductivity, EDR sensors configured to measure skin electrical responses, PGR sensors configured to measure psychoelectrical reflexes, SCR sensors configured to measure skin conductivity responses, SSR sensors configured to measure sympathetic skin responses, SCL sensors configured to measure skin conductivity levels (e.g., localized sweating), etc. In some embodiments, a single physiological sensor may measure both SCL and SCR.

[0122] In some embodiments, the physiological sensor(s) (1028) may include one or more sensors configured to acquire (e.g., capture, measure, calculate) heart rate (HR) and / or heart rate variability (HRV) parameters. Heart rate variability consists of variations in the time intervals between successive heartbeats, referred to as Interbeat Intervals (IBI). A healthy heart is not a metronome. The oscillations of a healthy heart are complex and constantly changing, which allows the cardiovascular system to rapidly adjust to sudden physical and psychological challenges to homeostasis. HRV parameters may include the root mean square (RMSSD) of successive differences between successive heartbeats. The Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology has divided heart rate (HR) oscillations into ultra-low-frequency (ULF), very-low-frequency (VLF), low-frequency (LF), and high-frequency (HF) bands. In some embodiments, the HRV parameters provided by the physiological sensor(s) (1028) may include any suitable combination of ULF, VLF, LF, HF, LF / HF (referred to as the ratio of LF to HF power), or any suitable ratio of the heart rate oscillation parameters mentioned above.Physiological sensor(s) (1028) may include heart rate monitors (HRMs), electrocardiography (ECG) sensors configured to measure electrical heart information (also referred to as EKG sensors), photoplethysmography (PPG) sensors configured to measure blood volume, etc. ECG sensors measure biopotentials generated by electrical signals that control the expansion and contraction of the ventricles, which are typically implemented in medical devices. PPG sensors measure blood volume controlled by the pumping action of the heart using light-based technology. PPG sensors may be configured to measure pulse transit time (PTT), which is defined as the time required for a pulse to reach two different distal parts of the body. PPG sensors may be configured to measure blood volume pulse (BVP). BVP is widely used as a method for measuring heart rate. BVP can be used to estimate heart rate based on the volume of blood passing through tissues of a localized area with each beat (pulse) of the heart. In some embodiments, BVP amplitude may be measured. BVP amplitude may be a measure of vasoconstriction / vasodilation—associated with sympathetic (stress response) and thermoregulatory (nausea) responses.

[0123] The physiological sensor(s) (1028) may include, in some embodiments, one or more breathing sensors configured to measure breaths per minute and / or breath volume, one or more thermometers configured to measure body temperature and / or ambient temperature (e.g., in a localized part of the body), etc.

[0124] In some embodiments, memory (1004) may store instructions for implementing a motion sickness detection engine (1030) when executed by the processor(s) (1002). The motion sickness detection engine (830) may store code for performing any suitable combination of methods, calculations, and / or operations discussed above in relation to FIGS. 1 through 9.

[0125] FIG. 11 is a perspective view of an exemplary vehicle (1100) according to one embodiment (e.g., an example of the vehicle (1000) of FIG. 10). Here, some of the components discussed in relation to FIG. 11 and prior embodiments are illustrated. As illustrated and discussed above, the vehicle (1100) may be equipped with camera(s) (1102) (e.g., an example of the camera(s) (1012) of FIG. 10), such as a rear-view mirror-mounted camera (1104), a passenger-facing camera (not illustrated), a front fender-mounted camera (not illustrated), a side-mirror-mounted camera (not illustrated), and a rear camera (not illustrated, but typically located on the trunk, hatch, or rear bumper). The vehicle (1100) may also be equipped with a LIDAR (1106) for detecting objects and measuring distances to those objects; LIDAR (1106) is often mounted on the roof, but, if there are multiple LIDAR units, they may be oriented around the front, rear, and sides of the vehicle. The vehicle (1100) may be equipped with various other positioning systems such as a receiver (1108), such as a GNSS radio receiver (typically located in a shark fin unit on the rear roof as shown) and / or various radio transceivers (e.g., WAN, WLAN, V2X; typically located in a shark fin, but not required), a RADAR (1110) (typically located on the front bumper), and a SONAR (1112) (typically located on both sides of the vehicle, if present). Sensors (1114) may also be present and may include drivetrain sensors such as wheel sensors and / or tire pressure sensors, accelerometers, gyroscopes, wheel rotation detection and / or counters.In some embodiments, any suitable combination of sensors (1114) may be utilized to determine and / or measure vibrations of the vehicle (1100) and / or one or more parts of the vehicle.

[0126] In one embodiment, distance measurements and relative positions determined through various sensors such as LIDAR, RADAR, cameras, GNSS, and SONAR can be combined with information regarding vehicle size and shape and sensor positions to determine distances and relative positions between the surfaces of different vehicles, and as a result, the distance or vector from a sensor to another vehicle or between two different sensors (e.g., two GNSS receivers) is incrementally increased to account for the position of the sensor on each vehicle. Accordingly, the accurate GNSS distance and vector between two GNSS receivers may need to be modified based on the relative positions of various vehicle surfaces with respect to the GNSS receiver. For example, in determining the distance between the front bumper of a rear vehicle and the rear bumper of a preceding vehicle, the distance may need to be adjusted based on the distance between the GNSS receiver and the front bumper of the following vehicle, and the distance between the GNSS receiver of the preceding vehicle and the rear bumper of the preceding vehicle. For example, the distance between the rear bumper of the front vehicle and the front bumper of the following vehicle is (relative distance between the two GNSS receivers) - (distance from the GNSS receiver of the rear vehicle to the front bumper) - (distance from the GNSS receiver of the front vehicle to the rear bumper). It is recognized that this list is not intended to be limiting and that FIG. 11 is intended to provide exemplary locations of various sensors in an embodiment of the vehicle (1100).

[0127] FIG. 12 is a schematic diagram illustrating an exemplary configuration (1200) of a plurality of camera(s) positioned to collect eye gaze data and a plurality of physiological sensors positioned to collect and / or measure physiological data corresponding to an occupant (e.g., occupant (1202)) of a vehicle (e.g., vehicle (1000) of FIG. 10), according to one embodiment. The configuration (1200) is provided as an example and is not intended to limit the present disclosure. Other configurations are considered that include the same or different number and / or types of sensors that can be utilized to collect and / or measure any appropriate physiological data of the occupant (1202).

[0128] The configuration (1200) includes a camera (1204) (e.g., an example of the camera(s) (1012) of FIG. 10). The camera (1204) may be positioned so as to face the occupant (1202) in a manner that enables the camera (1204) to capture the occupant (1202)'s eye gaze data (e.g., one or more images / videos) over time. In some embodiments, the camera (1204) may be positioned to capture the occupant's face and / or eyes' eye gaze data. These images / videos can be processed (e.g., by the processor(s) (1002) of FIG. 10) and any suitable image recognition and / or eye tracking technology can be utilized to identify the gaze of the occupant (1202) (e.g., the direction the occupant (1202) is looking, the amount the user's gaze deviates from the horizon, whether the occupant (1202) is looking toward the horizon or not, whether the occupant is looking out the window or not, etc.).

[0129] The configuration (1200) may include any suitable combination of the physiological sensor(s) (1028) of FIG. 10. As depicted in FIG. 12, the physiological sensor(s) (1028) includes two electro-skin response (GSR) sensors (e.g., GSR sensor (1006) and GSR sensor (1008)), but more or fewer GSR sensors may be utilized. FIG. 12 depicts the GSR sensor (1206) being placed on the forehead of the occupant (1202). In some embodiments, the GSR sensor (1206) may be an object or component of another device, such as a headband, sunglasses, etc. (not depicted). The GSR sensor (1206) may be placed on the forehead to measure electro-skin responses. FIG. 12 depicts a GSR sensor (1008) positioned to measure electrical skin responses on the fingers of a user (1202). In some embodiments, the GSR sensor (1208) may be integrated into a steering wheel of a car, a wearable device (e.g., a smartwatch), etc. While GSR sensors (1206 and 1208) are depicted, any suitable number of sensors may be similarly utilized at any suitable location on the body to measure skin conductivity, electrical skin response (GSR), skin electrical response (EDR), psychoelectrical reflex (PGR), skin conductivity response (SCR), sympathetic skin response (SSR), skin conductivity level (SCL) (e.g., sweating), etc.

[0130] FIG. 12 depicts a skin temperature sensor (1209) configured to measure skin temperature on the fingers of a user (1202). In some embodiments, the skin temperature sensor (1209) may be integrated into a steering wheel of a car, a wearable device (e.g., a smartwatch), etc. Although only one skin temperature sensor is depicted, any suitable number of temperature sensors may be similarly utilized at any suitable location on the body.

[0131] In some embodiments, the physiological sensor(s) (1028) may include one or more sensors configured to acquire (e.g., capture, measure, calculate) heart rate (HR) and / or heart rate variability (HRV) parameters. As depicted in FIG. 12, an electrocardiogram (ECG) sensor (1210) is positioned to measure the heart rate (HR) and heart rate variability (HRV) of the occupant (1202). HRV parameters may include the root mean square (RMSSD) of successive differences between successive heartbeats. In some embodiments, HRV parameters may include ultra-low frequency (ULF) bands, very low frequency (VLF) bands, low frequency (LF) bands, and high frequency (HF) bands, or any suitable combination and / or ratio of the above. In some embodiments, the ECG sensor (1210) measures electrical signals and standard electrodes by making direct contact with the skin. However, in some embodiments, a capacitive heart rate sensor may be utilized through the material of the car seat.

[0132] As depicted in FIG. 12, the occupant (1202) may utilize a wearable device (e.g., a watch, ring, glasses, etc.) configured with a photoplethysmography (PPG) sensor (e.g., PPG sensor (1212)). The PPG sensor (1212) may be configured to measure blood volume, etc. The PPG sensor (1212) may measure the pulse wave delivery time (PTT) and / or blood volume pulse (BVP) of the occupant (1202) (and / or other occupants of the vehicle). In some embodiments, the BVP may be measured via a camera using image processing algorithms configured to identify the BVP from images. Any suitable number of physiological sensor(s) (1028) may be utilized and configured to communicate with the processor(s) (1002) of FIG. 10 and / or each other via any suitable wireless communication protocol. Other combinations and configurations of physiological sensor(s) (1028) are considered.

[0133] As depicted in FIG. 12, an IMU (1214) (e.g., including any suitable number of vehicle sensor(s) (1010) of FIG. 10) may be located in a vehicle occupied by an occupant (1202). The IMU (1214) may be utilized to measure the movement of the vehicle. The IMU (1214) may include any suitable number of accelerometers, gyroscopes, and / or magnetometers configured to measure the amplitude and frequency of vibrations of the vehicle and / or parts of the vehicle where the occupant (1202) is located. Thus, in some embodiments, the IMU (1214) is utilized to capture the vehicle movement data described above in relation to FIG. 1, FIG. 4, and FIG. 5.

[0134] It will be obvious to those skilled in the field of technology that substantial variations may occur depending on specific requirements. For example, custom hardware may also be used, and / or specific elements may be implemented in hardware, software (including portable software such as applets, etc.), or both. Furthermore, connections to other computing devices, such as network input / output devices, may be utilized.

[0135] Referring to the accompanying drawings, components that may include memory (e.g., memory (1004) of FIG. 10) may include non-transient machine-readable media. As used herein, the terms “machine-readable media” and “computer-readable media” refer to any storage medium that participates in providing data that enables a machine to operate in a particular manner. In the embodiments provided above, various machine-readable media may be involved in providing instructions / code to processing units and / or other device(s) for execution. Additionally or alternatively, machine-readable media may be used to store and / or carry such instructions / code. In many embodiments, the computer-readable media is a physical and / or tangible storage medium. Such media may take various forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Computer-readable media of general forms include, for example, magnetic and / or optical media, any other physical media having patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described below, or any other medium in which a computer can read instructions and / or code.

[0136] The methods, systems, and devices discussed herein are examples. Various embodiments may appropriately omit, substitute, or add various procedures or components. For example, features described in connection with specific embodiments may be combined in various other embodiments. Different aspects and elements of the embodiments may be combined in a similar manner. Various components of the drawings provided herein may be embodied in hardware and / or software. Furthermore, technology evolves, and therefore, many of the elements are examples that do not limit the scope of this disclosure to those specific examples.

[0137] It has sometimes proven convenient, primarily for reasons of general use, to refer to such signals as bits, information, values, elements, codes, characters, variables, terms, numbers, numerical values, etc. However, it should be understood that all of these or similar terms are to be associated with appropriate physical quantities and are merely convenient labels. Unless otherwise specifically stated, as is evident from the discussion above, it is recognized that discussions throughout this specification utilizing terms such as “processing,” “computing,” “calculating,” “determining,” “identifying,” “identifying,” “associated,” “measuring,” “performing,” etc. refer to actions or processes of a specific device, such as a special-purpose computer or a similar special-purpose electronic computing device. Accordingly, in the context of this specification, a special-purpose computer or a similar special-purpose electronic computing device may manipulate or convert signals typically represented as physical, electronic, electrical, or magnetic quantities within memories, registers, or other information storage devices, transmission devices, or display devices of a special-purpose computer or a similar special-purpose electronic computing device.

[0138] Terms such as “and” and “or” as used herein may include various meanings that are also expected to depend at least partially on the context in which such terms are used. Typically, when “or” is used to associate a list such as A, B, or C, it is intended to mean A, B, and C (used here in an inclusive sense), as well as A, B, or C (used here in an exclusive sense). Additionally, the term “one or more” as used herein may be used to describe any feature, structure, or characteristic in the singular, or to describe any combination of features, structures, or characteristics. However, it should be noted that this is merely an exemplary example and the subject matter of the claim is not limited to this example. Furthermore, when the term “at least one of” is used to associate a list such as A, B, or C, it may be interpreted to mean any combination of A, B, and / or C, e.g., A, AB, AA, AAB, AABBCCC, etc.

[0139] Although various embodiments have been described, various modifications, alternative configurations, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may be merely components of a larger system, wherein other rules may take precedence over the application of the various embodiments or otherwise modify their application. Additionally, a number of steps may be taken before, during, or after the above elements are considered. Accordingly, the above description does not limit the scope of the disclosure.

Claims

Claim 1 A method for preventing or alleviating motion sickness in a user within a vehicle, comprising: a step of acquiring a first set of physiological data related to the user within the vehicle from a plurality of sensor devices by one or more processors; a step of acquiring a second set of physiological data related to the user from the plurality of sensor devices by one or more processors, wherein the second set of physiological data is captured by the plurality of sensor devices after the first set of physiological data is captured by the plurality of sensor devices; a step of acquiring a third set of physiological data related to the user from the plurality of sensor devices by one or more processors, wherein the third set of physiological data is captured by the plurality of sensor devices after the first set of physiological data and the second set of physiological data are captured by the plurality of sensor devices; a step of determining a first set of physiological features of the user from the first set of physiological data; a step of determining a second set of physiological features of the user from the second set of physiological data; a step of determining a third set of physiological features of the user from the third set of physiological data; and a second set of physiological features A step of determining that the user has experienced a stress response based at least partially on a first comparison between a set and a first set of physiological features; a step of determining that the user has experienced a nausea response after experiencing the stress response based at least partially on a second comparison between a third set of physiological features and a second set of physiological features, subject to the condition that the user has experienced the stress response in response to determining that the user has experienced the stress response;A method for preventing or improving motion sickness of a user in a vehicle, comprising: acquiring vehicle motion data indicating the frequency of the oscillations of the vehicle by the one or more processors; determining by the one or more processors that the user has experienced the nausea response after experiencing the stress response and calculating a motion sickness score based at least partially on the vehicle motion data; detecting by the one or more processors that the user is experiencing a certain degree of motion sickness based at least partially on the motion sickness score; and performing an autonomous vehicle action by the one or more processors in response to determining that the user is experiencing the certain degree of motion sickness, wherein the performance of the autonomous vehicle action improves the degree of motion sickness experienced by the user at a subsequent time or prevents an increase in the degree of motion sickness experienced by the user at a subsequent time. Claim 2 A method for preventing or improving motion sickness in a user in a vehicle, wherein the method further comprises the step of acquiring eye gaze data from an image capture device by the one or more processors, wherein the eye gaze data comprises one or more images representing the user's gaze, and the motion sickness score is calculated at least partially additionally based on the one or more images representing the user's gaze. Claim 3 A method for preventing or improving motion sickness in a user in a vehicle, wherein, in claim 1, the vehicle movement data includes one or more vibration measurements, and the method further comprises the step of determining that the frequency of the vibration measurements falls within a predefined range. Claim 4 A method for preventing or improving motion sickness in a user in a vehicle, wherein the autonomous vehicle action comprises: slowing down the vehicle, modifying the path, performing less aggressive turns, maintaining a more constant direction of travel, or any combination of the above. Claim 5 A method for preventing or improving motion sickness in a user in a vehicle according to claim 1, wherein the plurality of sensor devices each comprise a first sensor and a second sensor configured to measure the user’s galvanic skin responses, and a third sensor configured to measure the user’s heart rate. Claim 6 A method for preventing or improving motion sickness in a user in a vehicle, wherein, in claim 5, the first sensor is positioned on the user's forehead to measure a first electrical skin response, and the second sensor is positioned on the user's palm surface or wrist to measure a second electrical skin response. Claim 7 A method for preventing or improving motion sickness in a user in a vehicle, wherein the first set of physiological features comprises: the user's heart rate, average skin temperature, one or more skin conductivity levels corresponding to one or more areas of the user's skin, one or more skin conductivity responses corresponding to one or more areas of the user's skin, the root mean square of successive differences between the user's normal heartbeats, and a blood volume pulse amplitude indicating relative blood flow, or any combination of the above. Claim 8 A method for preventing or improving motion sickness in a user in a vehicle, wherein, in claim 7, the method further comprises the step of obtaining a set of reference physiological features associated with the user, and determining that the user has experienced the stress response is based at least partially on comparing a first set of the physiological features with a set of reference physiological features. Claim 9 A method for preventing or improving motion sickness in a vehicle user according to claim 1, wherein the one or more processors detect that the user is experiencing a certain degree of motion sickness based at least partially on the motion sickness score, comprising: determining that the user has experienced the stress response based on a first set of the user's physiological features using a first machine learning model; and determining that the user has experienced the nausea response based on a second set of the user's physiological features using a second machine learning model. Claim 10 A method for preventing or improving motion sickness in a vehicle user according to claim 9, wherein the first machine learning model comprises supervised learning, a first training data set for classifying input data, and a first classification model trained using at least a first classification value corresponding to the stress response, and the second machine learning model comprises supervised learning, a second training data set for classifying input data, and a second classification model trained using at least a second classification value corresponding to the nausea response. Claim 11 A computing device comprising: a memory storing executable commands for preventing or improving the degree of motion sickness experienced by a user in a vehicle; and one or more processors coupled to communicate with said memory, wherein said one or more processors enable said computing device: To obtain a first set of physiological data from a plurality of sensor devices; To obtain a second set of physiological data from the plurality of sensor devices — the second set of physiological data is captured by the plurality of sensor devices after the first set of physiological data is captured by the plurality of sensor devices —; The above one or more processors obtain a third set of physiological data related to the user from the plurality of sensor devices, wherein the third set of physiological data is captured by the plurality of sensor devices after the first set of physiological data and the second set of physiological data have been captured by the plurality of sensor devices. Determining a first set of physiological features of the user from the first set of the above physiological data; Determining a second set of the user's physiological features from the second set of the above physiological data; Determining a third set of the user's physiological features from the third set of the above physiological data; Determining that the user has experienced a stress response based at least partially on a first comparison between a second set of the above physiological features and a first set of the above physiological features; In response to determining that the user has experienced the stress response, and conditioned on determining that the user has experienced the stress response, the determination that the user has experienced the stress response is based at least partially on a second comparison between a third set of physiological features and a second set of physiological features, thereby determining that the user has experienced a nausea response after experiencing the stress response; Acquiring vehicle movement data that indicates the frequencies of the vibrations of the above vehicle; Determining that the user experienced the nausea reaction after experiencing the stress reaction, and calculating a motion sickness score based at least partially on the vehicle movement data; Detecting that the user is experiencing a certain degree of motion sickness based at least partially on the above motion sickness score; and A computing device configured to execute the commands to cause the performance of an autonomous vehicle action in response to determining that the user is experiencing a certain degree of motion sickness, wherein causing the autonomous vehicle action to be performed improves the degree of motion sickness experienced by the user at a subsequent time or prevents an increase in the degree of motion sickness experienced by the user. Claim 12 In claim 11, the computing device further enables the executing of the above commands to: acquire eye gaze data from an image capture device, wherein the eye gaze data comprises one or more images representing the user’s gaze, and the motion sickness score is calculated at least partially additionally based on the one or more images representing the user’s gaze. Claim 13 In claim 11, the vehicle movement data comprises one or more vibration measurements, and the executing of the commands further enables the computing device to determine that the frequency of the vibration measurements falls within a predefined range. Claim 14 In paragraph 11, the autonomous vehicle action comprises: slowing down the vehicle, modifying the path, performing less aggressive turns, maintaining a more constant direction of travel, or any combination of the above, a computing device. Claim 15 In claim 11, the computing device comprising a plurality of sensor devices, each comprising a first sensor and a second sensor configured to measure the electrical skin responses of the user, and a third sensor configured to measure the heart rate of the user. Claim 16 A computing device according to claim 15, wherein the first sensor is positioned on the user's forehead to measure a first electrical skin response, and the second sensor is positioned on the user's palm surface or wrist to measure a second electrical skin response. Claim 17 In claim 16, the first set of physiological features comprises: the user’s heart rate, average skin temperature, one or more skin conductivity levels corresponding to one or more areas of the user’s skin, one or more skin conductivity responses corresponding to one or more areas of the user’s skin, the root mean square of successive differences between the user’s normal heartbeats, and a blood volume pulse amplitude indicating relative blood flow, or any combination of the above, a computing device. Claim 18 In paragraph 17, the computing device further enables executing the above commands to: obtain a set of reference physiological features associated with the user; and determine that the user has experienced the stress response based at least partially on comparing a first set of the physiological features with the set of reference physiological features. Claim 19 A computing device according to claim 11, wherein detecting that the user is experiencing a certain degree of motion sickness based at least partially on the motion sickness score by the one or more processors comprises: determining that the user has experienced the stress response based on a first set of the user's physiological features using a first machine learning model; and determining that the user has experienced the nausea response based on a second set of the user's physiological features using a second machine learning model. Claim 20 A computing device according to claim 19, wherein the first machine learning model comprises supervised learning, a first training data set for classifying input data, and a first classification model trained using at least a first classification value corresponding to the stress response, and the second machine learning model comprises supervised learning, a second training data set for classifying input data, and a second classification model trained using at least a second classification value corresponding to the nausea response. Claim 21 A non-transient computer-readable storage medium storing commands for preventing or improving the degree of motion sickness experienced by a user in a vehicle, wherein, when executed by one or more processing units, the commands cause the one or more processing units to: acquire a first set of physiological data from a plurality of sensor devices; acquire a second set of physiological data from the plurality of sensor devices — said second set of physiological data is captured by the plurality of sensor devices after said first set of physiological data is captured by the plurality of sensor devices —; acquire a third set of physiological data related to the user — said third set of physiological data is captured by the plurality of sensor devices after said first set of physiological data and said second set of physiological data are captured by the plurality of sensor devices —; determine a first set of physiological features of the user from the first set of physiological data; determine a second set of physiological features of the user from the second set of physiological data; and determine a third set of physiological features of the user from the third set of physiological data To determine; to determine that the user has experienced a stress response based at least partially on a first comparison between a second set of physiological features and a first set of physiological features; and to determine that the user has experienced a nausea response based at least partially on a second comparison between a third set of physiological features and a second set of physiological features, subject to the condition that the user has experienced the stress response in response to the determination that the user has experienced the stress response; and to acquire vehicle movement data indicating the frequencies of the vibrations of the vehicle;A non-transient computer-readable storage medium, wherein determining that the user has experienced the nausea response after experiencing the stress response and calculating a motion sickness score based at least partially on the vehicle movement data; detecting that the user is experiencing a certain degree of motion sickness based at least partially on the motion sickness score; and causing the performance of an autonomous vehicle action in response to determining that the user is experiencing the certain degree of motion sickness, and causing the autonomous vehicle action to be performed improves the degree of motion sickness experienced by the user at a subsequent time or prevents an increase in the degree of motion sickness experienced by the user. Claim 22 A non-transient computer-readable storage medium, wherein, in paragraph 21, the above commands, when executed by the one or more processing units, additionally cause the one or more processing units to acquire eye gaze data from an image capture device, the eye gaze data includes one or more images representing the user's gaze, and the motion sickness score is calculated at least partially additionally based on the one or more images representing the user's gaze. Claim 23 In claim 21, the vehicle movement data comprises one or more vibration measurements, and the commands, when executed by the one or more processing units, additionally cause the one or more processing units to determine that the frequency of the vibration measurements falls within a predefined range, a non-transient computer-readable storage medium. Claim 24 In paragraph 21, the autonomous vehicle action comprises: slowing down the vehicle, modifying the path, performing less aggressive turns, maintaining a more constant direction of travel, or any combination of the above, in a non-transient computer-readable storage medium. Claim 25 A non-transient computer-readable storage medium, wherein, in paragraph 21, the above commands, when executed by the one or more processing units, further cause the one or more processing units to: acquire a set of reference physiological features associated with the user; and determine that the user has experienced the stress response based at least partially on comparing a first set of the physiological features with the set of reference physiological features. Claim 26 In claim 21, detecting that the user is experiencing a certain degree of motion sickness based at least partially on the motion sickness score by the one or more processing units comprises: determining that the user has experienced the stress response based on a first set of the user's physiological features using a first machine learning model; and determining that the user has experienced the nausea response based on a second set of the user's physiological features using a second machine learning model. Claim 27 A non-transient computer-readable storage medium, wherein, in claim 26, the first machine learning model comprises supervised learning, a first training data set for classifying input data, and a first classification model trained using at least a first classification value corresponding to the stress response, and the second machine learning model comprises supervised learning, a second training data set for classifying input data, and a second classification model trained using at least a second classification value corresponding to the nausea response.

Images