System and method for providing fatigue mitigation and habituation control

Abstract

In at least one embodiment, a system for providing fatigue mitigation in a vehicle is provide. The system includes at least one sensor and at least one controller. The at least one sensor is configured to transmit a plurality of signals indicative of any number of drowsiness levels for a user in the vehicle. The at least one controller is programmed to receive a first signal indicative of a first drowsiness level for the user in the vehicle from the at least one sensor and to compare the first drowsiness level of the user to a first threshold level. The at least one controller is further programmed to control one or more vehicle systems to provide a first stimulus to the user to mitigate drowsiness being experienced by the user responsive to the first drowsiness level exceeding the first threshold level.

Description

SYSTEM AND METHOD FOR PROVIDING FATIGUE MITIGATION AND HABITUATION CONTROLTECHNICAL FIELD

[0001] Aspects disclosed herein generally relate to a system and method for providing fatigue mitigation and habituation control. These aspects and others will be discussed in more detail below.BACKGROUND

[0002] Automotive Original Equipment Manufacturers (OEMs) implement systems that are aimed at reducing driver fatigue. Such systems are not adaptive over time and their ability to reduce driver fatigues generally reaches a state of habituation. For example, with these systems, drivers develop a diminished physiological or emotional response to a frequently repeated stimulus that is imparted on the driver by the vehicle in moments where the driver is detected to experience fatigue.SUMMARY

[0003] In at least one embodiment, a system for providing fatigue mitigation in a vehicle is provide. The system includes at least one sensor and at least one controller. The at least one sensor is configured to transmit a plurality of signals indicative of any number of drowsiness levels for a user in the vehicle. The at least one controller is programmed to receive a first signal indicative of a first drowsiness level for the user in the vehicle from the at least one sensor and to compare the first drowsiness level of the user to a first threshold level. The at least one controller is further programmed to control one or more vehicle systems to provide a first stimulus to the user to mitigate drowsiness being experienced by the user responsive to the first drowsiness level exceeding the first threshold level. The at least one controller is further programmed to determine whether the first drowsiness level for the user has exhibited a change within a first predetermined time frame and to control the one or more vehicle systems to provide a second stimulus to the user to mitigate thedrowsiness being exhibited by the user based at least on the first drowsiness level for the user exceeding the first predetermined time frame, wherein the first stimulus is different than the second stimulus to mitigate habituation for the user.

[0004] In at least another embodiment, a method for providing fatigue mitigation in a vehicle is provided. The method includes transmitting a plurality of signals indicative of any number of drowsiness levels for a user in the vehicle and receiving, by at least one controller, a first signal indicative of a first drowsiness level for the user in the vehicle from at least one sensor. The method further includes comparing the first drowsiness level of the user to a first threshold level and controlling one or more vehicle systems to provide a first stimulus to the user to mitigate drowsiness being experienced by the user responsive to the first drowsiness level exceeding the first threshold level. The method further includes determining by the at least one controller whether the first drowsiness level for the user has exhibited a change within a first predetermined time frame and controlling the one or more vehicle systems to provide a second stimulus to the user to mitigate the drowsiness being exhibited by the user based at least on the first drowsiness level for the user exceeding the first predetermined time frame, wherein the first stimulus is different than the second stimulus to mitigate habituation for the user.

[0005] In at least another embodiment, a computer-program product embodiment in a non- transitory computer readable medium executable by at least one controller to provide fatigue mitigation in a vehicle is provided. The computer-program product comprising instructions to receive a first signal indicative of a first drowsiness level for a driver in the vehicle from at least one sensor and to compare the first drowsiness level of the driver to a first threshold level. The computer-program product comprising instructions to control one or more vehicle systems to provide a first stimulus to the driver to mitigate drowsiness being experienced by the user responsive to the first drowsiness level exceeding the first threshold level and to determine whether the first drowsiness level for the driver has exhibited a change within a first predetermined time frame. The computer-program product comprising instructions to control the one or more vehicle systems to provide a second stimulus to the driver to mitigate the drowsiness being exhibited by the user based at least on the first drowsiness level for the driver exceeding the first predetermined time frame, wherein the first stimulus is different than the second stimulus to mitigate habituation for the user.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The embodiments of the present disclosure are pointed out with particularity in the appended claims. However, other features of the various embodiments will become more apparent and will be best understood by referring to the following detailed description in conjunction with the accompany drawings in which:

[0007] FIGURE 1 depicts a system for providing fatigue mitigation and habituation control in accordance with one embodiment;

[0008] FIGURE 2 depicts another system for providing fatigue mitigation and habituation control in accordance with one embodiment;

[0009] FIGURE 3 depicts a method for performing habituation mitigation control in accordance with one embodiment;

[0010] FIGURE 4 depicts a first use case that provides fatigue mitigation in accordance with one embodiment;

[0011] FIGURE 5 depicts a second use case that provides fatigue mitigation in accordance with one embodiment; and

[0012] FIGURE 6 depicts a third use case that provide fatigue mitigation in accordance with one embodiment.DETAILED DESCRIPTION

[0013] As required, detailed embodiments of the present invention are disclosed herein; however, it is to be understood that the disclosed embodiments are merely exemplary of the invention that may be embodied in various and alternative forms. The figures are not necessarily toscale; some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present invention.

[0014] Aspects disclosed herein generally provide a system that offers comprehensive and effective solution for combating the issue of habituation in fatigue mitigations scenarios. The disclosed systems apply multisensory-based interventions which includes auditory, visual, haptic, olfactory, proprioception, and interception. The system includes a detection system that detects the fatigue level of the user. The system executes an algorithm that analyzes fatigue level and provides personalized interventions based on a combination of a user’s history and current state. The intervention system includes multiple actuators which can target a user’s auditory, visual, haptic, olfactory, proprioception, and interception sensory modals. The disclosed system may prevent the user from habituating by varying sensory stimulation unpredictable outputs thus mitigating habituation which generally occurs when the body is subjected to predictable and repeating stimuli thereby resulting in ineffectiveness of the intervention. The disclosed system utilizes the actuators in an unpredictable sequence by varying targeted modalities, intensity, and duration to postpone habituation and improve the effectiveness of fatigue mitigation.

[0015] The disclosed system resolves, among other things, the issue of habituation when mitigating fatigue for a user. Current fatigue mitigation systems may not address habituation which leads to an ineffectiveness in mitigating driver fatigue. As evolutionary biological humans adjust quickly to environmental changes, predictable and repeated stimuli exposures lead to habituation. Unpredictable interventions that utilize multisensory modalities. For example, the system utilizes multiple sensory modalities such as auditory, visual, haptic, olfactory, proprioception, and intcroccption to provide interventions that arc varied and effective. These aspects may prevent habituation for the user and keep the user alert and aroused. Two aspects that may mitigate habituation generally involve that the stimulus is varied and unpredictable. The system may provide personalized interventions via a personalized intervention algorithm that is executed via one or more controllers that consider a user’s fatigue level. This personalized approach may be more effective in combating fatigue compared to non-personalized solutions.

[0016] The system may also provide improved safety such as a comprehensive solution for fatigue mitigation, which may be necessary for safety in high-risk setting transportation systems. Similarly, the disclosed system may also be versatile that can be used in a variety of settings, such as, but not limited to, transportation, healthcare, manufacturing, and any other setting where fatigue can pose a risk to safety and performance.

[0017] FIGURE 1 depicts a system 100 for providing fatigue mitigation and habituation control in a vehicle 101 in accordance with one embodiment. The system 100 includes a plurality of sensors 102, one or more controllers 104 (“the controller 104”), a user interface 106, one or more vehicle seats 108 (“the vehicle seat 108”), an audio system 110 including one or more loudspeakers, a heating, ventilation, and air conditioning (HVAC) system 112, and vehicle lighting 114. The controller 104 includes a first controller section 104a, a second controller section 104b, and a third controller section 104c. In general, the sensors 102 provide signals indicative of a drowsiness level of a driver (or user, occupant, etc.) in the vehicle 101 to the controller 104. The controller 104 assess the drowsiness level and determines a intervene level for mitigating the fatigue or drowsiness level for the driver. The controller 104 controls various subsystems 120 in the vehicle 101. The subsystems 120 generally include the vehicle seat 108, the audio system 110, HVAC system 112, and vehicle lighting 114 in the vehicle 101.

[0018] In addition, the controller 104 may selectively control aspects related to the various subsystem 120 in an unpredictable sequence by varying targeted modalities, intensities, and duration associated with one or more aspects of the subsystem 120. This aspect may postpone habituation and may improve effectiveness of fatigue mitigation. Habituation generally involves the diminishing of a physiological or emotional response to a frequently repeated stimulus. Thus, in this regard, the system 100 attempts to prevent the driver from ignoring any one or more of the triggers generated via the subsystem 120 to provide an effective stimulus to eliminate the drowsiness level being experienced by the driver to minimize fatigue and to ensure a safe driving experience for the driver and occupants in the vehicle 101.

[0019] The plurality of sensors 102 may include a wearable device 130, a mobile device 131, and various in-cabin sensors 132. The wearable device 130 may include a smart watch that can monitor aspects related to vital sign sensing such as heart rate for the driver, sleep patterns for thedriver, sleep durations for the driver, etc. The wearable device 130 may generally wirelessly communicate bi-directionally to one or more servers (not shown) positioned exterior to the vehicle 101 via Wi-Fi, LTE, etc. The in-cabin sensors 132 may include one or more cameras 140. The cameras 140 may include any number of in-ward facing cameras, eye gaze sensors, etc. In one example, the in-ward facing camera may be positioned within an interior of the vehicle 101 to capture images of the driver. In addition, the eye gaze sensor may also be positioned within the interior of the vehicle 101 to capture an eye gaze of the driver.

[0020] The first controller section 104a of the controller 104 receives information from the plurality of sensors 102 and analyzes the information provided by the sensors 102. For example, the first controller section 104a receives eye tracking information for the driver, vital signs for the driver, and contextual data. The vital sign sensing information may include the heart rate, heart rate variability, respiratory rate, core temperature, or blood pressure for the driver. The contextual data may include sleeping patterns and / or sleep duration for the driver to determine the level of sleep the driver received, activity levels, or current driving performance or environment. The first controller section 104a includes an adder 160 that combines the contextual data, eye tracking, and vital sign sensing information. The first controller section 104a analyzes the output of the adder 160 to determine a mental state, health state, and visual state of the driver based on the summed information provided by the adder 160. The mental state may include information corresponding to whether the driver is drowsy, distracted, or stressed. The health state may include information related to the driver’s physical and mental functions. The visual state may correspond to information such as whether the driver’s eyes are focused on the road.

[0021] The second controller section 104b of the controller receives information corresponding to the mental state, the health state, and the visual state as provided by the first controller section 104a and provides a use case correlation based on such information. For example, the second controller section 104b receives the one or more-use case correlations based on the information related to the mental state, the health state, and the visual state. The use case correlation may include actions 170 that provide increased wellbeing 170a, improving a mental state for the driver (e.g., “Arrive more relaxed”) 170b, driver safety (e.g., Drive Safely) 170c, and personalize (or personalization) 170e. For example, a drowsy mental state and / or poor sleep patterns may correlatewith actions related to increased wellbeing, or the distracted mental state and the distracted visual state may correlate with actions related to drive safely.

[0022] The third controller section 104c of the controller 104 receives the various actions 170 to develop various interventions 180 (or “intervention actions 180”). The intervention actions 180 may include interventions provided in the cabin environment 180a, interventions by way of music / sound and / or scent 180b, or coaching programs 180c. The interventions related to the cabin environment 180a generally include aspects related to the controller 104 controlling the HVAC system 112 to adjust the temperature of the vehicle 101 with of intent in at least preventing or mitigating fatigue being experienced by the driver (e.g., improve alertness or keep me fit) and / or improving a driver’s mood (e.g., “help me relax” or “help me rejuvenate”). For example, upon detecting that the driver may be experiencing drowsiness or fatigue, the controller 104 may trigger the various intervention actions 180 to activate any one or more of features associated with the speakers 110, the HVAC system 1 1 , and vehicle lighting 1 14. The intervention related to coaching programs 180c generally includes the audio system 110 providing feedback to the driver about recent or upcoming traffic situations or explaining the current state of their drowsiness level and intervention status.

[0023] For example, the controller 140 may trigger the intervention related to the cabin environment 180a that may include the controller 140 controlling any one or more of the audio system 110, the HVAC system 112, and the vehicle lighting 114 to mitigate or remove the drowsiness condition being exhibited by the driver. In one example, the controller 140 may control the audio system 110 to provide an audio output signal at an increased volume level to increase the alertness level of the driver. In another example, the controller 140 may control the HVAC system 112 to one of increase and decrease the temperature within the vehicle cabin in an attempt to increase the alertness level of the driver to mitigate fatigue or a drowsiness level of the driver. The heating and cooling of the seat 108 may mitigate the drowsiness aspect.

[0024] In addition, the HVAC system 112 may also include a diffuser 115 that releases the scent 180b to invigorate and energize the driver as noted above. The controller 140 may control the diffuser 115 to only diffuse or transmit the scented oil in moments where drowsiness is detected. The scent 180b may correspond to eucalyptus, peppermint, etc. that may mitigate drowsiness. If thedriver is detected to be stressed, the diffuser 115 may transmit lavender to reduce stress. The controller 104 may control the HVAC system 112 to provide cold air in the vehicle cabin. In another example, the controller 140 may control aspects of the vehicle lighting system 114 to increase the level of the interior lighting being presented within the cabin of the vehicle 101. In this case, the vehicle lighting system 114 may increase the brightness level within the vehicle. In addition, the controller 140 may control the vehicle lighting system 114 to repeatedly flash (at predetermined frequencies or random frequencies) and / or change ambient colors within the vehicle cabin to increase driver alertness and to mitigate the condition of the driver experiencing fatigue or drowsiness. Additionally, the vehicle lighting system 114 generates a cold interior lighting scheme (e.g., cool white light at more than, for example, 5000 Kelvin) (normal sunlight is around 7 Kelvin)) scheme in an attempt to mitigate the drowsiness level being experienced by the driver. It is recognized that normal sunlight is generally around 7 Kelvin.

[0025] In another example, the controller 104 may control various aspects on the seat 108 to serve as a stimulus to mitigate drowsiness or fatigue. For example, the seat 108 may include any number of haptic feedback mechanisms (e.g., haptic motor) to vibrate for any number of time durations and gain the attention of the driver as the driver exhibits a drowsy or fatigued condition. In addition, the seat 108 may be moved to a straight position to eliminate the drowsiness being experienced by the driver. It is recognized that for all instances of the controller 104 moving the seat 108 to the straight position, the controller 104 controls the seat 108 to move the recline position of the seat to a fully upright position to increase awareness of the road for the user. It is further recognized that the controller 104 may perform any one or more of the foregoing intervention actions 180 either separately or by forming any one or more combinations of the foregoing intcrv ention actions 180.

[0026] The controller 104 may also adjust, in an unpredictable (or random) or prcdicablc (or linear) sequence in which the intensity and / or duration of the stimuli (or interventions 108) provided by any vehicle systems 107 such as one or more of seats systems (or seats) 108, the speakers 110, the HVAC system 112, or the vehicle lighting 114 are applied to the driver to mitigate or eliminate habitation effects to ensure that the stimuli is adhered to minimize the driver’s ability of getting into a traffic accident due to fatigue or drowsiness. As noted above, the controller 140 may control the audio system 110 to provide an audio output signal at an increased volume level to increase thealertness level of the driver. To prevent habituation, the controller 140 may randomly increase or decrease the audio output level from the audio system 110 so that the driver is not accustomed to a single volume level as output by the audio system 110. In one example, the controller 140 may control the audio system 110 to output the audio output signal at a first volume level for a first timing interval and then control the audio system 110 to output the audio output signal at a second volume level (e.g., that is equal to or different from the first volume level) for a second timing interval (e.g., that is equal to or different from the first timing interval) to mitigate habituation.

[0027] As noted above, the controller 140 may control the HVAC system 112 to one of increase and decrease the temperature within the vehicle cabin in an attempt to increase the alertness level of the driver to mitigate fatigue or a drowsiness level of the driver. To prevent habituation, the controller 140 may randomly increase or decrease the temperature within the vehicle cabin from the HVAC system 112 so that the driver is not accustomed to a single temperature as output by the audio system 1 10. In one example, the controller 140 may control the HVAC system 112 to output the temperature at a first temperature level for a first timing interval and then control the HVAC system 112 to output the temperature at a second temperature level (e.g., that is equal to or different from the first temperature level) for a second timing interval (e.g., that is equal to or different from the first timing level) to mitigate habituation.

[0028] As noted above, the controller 140 may control aspects of the vehicle lighting system 114 to increase the level of the interior lighting being presented within the cabin of the vehicle 101. In this case, the vehicle lighting system 114 may increase the brightness level within the vehicle. To prevent habituation, the controller 140 may control the vehicle lighting system 114 to repeatedly flash (at predetermined frequencies or random frequencies) for a first timing interval and change the flashing sequence to a different frequency at a second timing interval that is either the same or different from the first timing interval. It is recognized that the particular flashing sequence as noted above may take on any number of forms and shouldn’t be limited to the sequences noted herein. Further it is recognized that the controller 140 may control the lighting system 114 to illuminate any number of colors in a random fashion to mitigate habituation in response to the driver being detected to exhibit a drowsy or fatigue level.

[0029] As noted above, the controller 104 may control various aspects on the seat 108 to serve as a stimulus to mitigate drowsiness or fatigue. To prevent habituation, the controller 140 may randomly increase or decrease the haptic feedback so that the driver is not accustomed to a single haptic level level as output by seat 108. In one example, the controller 140 may control the seat 108 to provide the haptic feedback at a first level for a first timing interval and then control the seat 108 to provide the haptic feedback at a second level (e.g., that is equal to or different from the first level) for a second timing interval (e.g., that is equal to or different from the first timing interval) to mitigate habituation. It is recognized that the controller 104 may control any one or more of the audio system 110, the HVAC system 112, or the vehicle lighting system 114 to provide any one or more of the stimuli to be varied and / or unpredictable.

[0030] It is also recognized that that that the driver may input via the user interface 106 that type if stimulus (or intervention action 180) that the vehicle 101 can employ in moments in which the driver is detected to be drowsy or fatigued. For example, the driver may select that the stimulus is applied by the seat 108, the speakers 110, the HVAC system 112 and / or the vehicle lighting system 114 in response to the driver being detected to be drowsy or fatigued. In this regard, the driver can establish and / or program various preferences that he / she prefers for the stimulus to be applied. In one example, the driver can select his / her favorite song to be played by the controller 104 which is then played back by the audio system 110. Conversely, the driver can select the corresponding volume level at which the song or other selected audio output should be provided. Similarly, the driver may select various colors and / or lighting sequences in addition to desired temperatures that serve as stimuli when the controller 104 determines that the driver is exhibiting fatigue or is drowsy.

[0031] In general, the vehicle 101 may employ the interventions 180 to target modalities of the driver to mitigate drowsiness being experienced by the driver while driving the vehicle 101 by employing any one or more of the intervention actions 180. The system 100 may trigger the one or more interventions 180 to target any one or more modalities such as, thermoception, audition, vision, tactition, olfaction, proprioception, and all modalities sense of self. Thermoception may involve the controller 104 controlling the HVAC system 112 to generate or change temperature in the vehicle cabin and / or by blowing cold air to target thermoception. Audition may involve the controller 104 controlling the audio system 110 to change a volume level of the audio output or output energeticmusic to target the driver’s auditory sense. Vision may involve the controller 104 controlling the lighting system 114 such that the interior lighting is changed as noted above and / or represents a cold interior lighting to target the driver’s visual sense. Oflacation may involve the controller 104 triggering or emitting a Eucalyptus essential oil to target the driver’s olfactory sense. Proprioception may involve the controller 10 controlling the seat 108 to move in straight position or triggering a haptic feedback mechanism to target a tactile sense. The controller 104 may play back through the audio system 110 guided verbal exercises that target multiple modalities. For example, a voice (audition) may guide the driver through seated stretching exercises (e.g., proprioception) while also asking the driver to note the change in air flow (e.g., thermoception) and lighting (e.g., vision).

[0032] The user interface 106 includes various a first field (or drowsiness level field) 181, a second level field (or manual distraction field) 182, and a third level field (or stress level field) 184. The drowsiness level field 181 depicts the drowsiness level of the driver based on the inputs transmitted by the sensor 102 and processed by the controller 104. The manual distraction field 182 depicts whether the driver is manually distracted or not. The controller 104 may monitor signals transmitted from the eye gaze sensor. The stress level field 184 may depict the stress level for the driver. The controller 104 determines the stress level for the driver based on heart rate metrics provided by either or each of the smartwatch, in-cabin sensor 130, or mobile device 131. In one example, the smartwatch 130 and / or the mobile device 131 may execute Samsung Health ® app to provide exercise patterns exhibited by the driver prior to entering into the vehicle, a sleep pattern for the driver, and / or a heart rate for the driver.

[0033] FIGURE 2 depicts another system 250 for providing fatigue mitigation and habituation control in accordance with one embodiment. The system 250 includes the controller 104. The controller 104 includes an inference engine 251 and an interaction controller 252 that outputs various available sources. The available sources generally include a cabin control 254, trip summary 256, and trip planning 258. The inference engine 251 includes a logger 260 (or memory), a monitor 262, and an analyzer 264. The controller 104 processes the inputs provided by the sensor 202 and the processed inputs correspond to: a drowsiness value, a car control value, external weather information, location information, driving distance information, driving time information, exercise pattern information, sleep pattern information, and heart rate information. The inference engine 251 stores inputs provided by the sensors 102 in the logger 260.

[0034] The analyzer 264 receives the processed inputs from the logger 260. The analyzer 264 processes the inputs to determine the drowsiness, the distraction level, and the stress level for the driver. For example, the analyzer 264 includes an initial model 264a, a preferred model 264b, a cascading model 264c, and / or an adaptive model 264d. The initial model 264a employs default inputs / outputs when the driver is user the vehicle 101 for the first time. The preferred model 264b collects a history of stimuli for any given driver that has driven the vehicle 101 and the impact to the driver when the stimuli is applied. For example, the preferred model 264b stores data corresponding to the stimulus that has been detected to be both successful in mitigating the drowsiness for the and unsuccessful in deterring the drowsiness level for the driver. The cascading model 264c represents the variability of the stimulus that is applied to mitigate drowsiness and habituation. The controller 104 executes aspects related to the cascading model 264c as shown in FIGURE 4.

[0035] The adaptive model 264 may employ machine learning techniques to learn preferences and the effect of stimuli employed on the driver that may have been most effective in mitigating the drowsiness level of the driver. The controller 104 may employ the most effective stimuli as determined by the adaptive model 264 in moments in which the driver exhibits a high drowsiness level. The adaptive model 264 may even take into account what time of day the various stimuli are most effective and correspondingly apply any one of the interventions that are determined to be most effective based on the recorded time. For example, if historically, it has been found that changing the lighting sequence as provided by the lighting system 114 is most effective late in the evening (e.g., 10 pm) in removing a high drowsiness level for the driver than the intervention of controlling the audio system 110 to change a volume of the audio, the controller 104 may then control the lighting system 114 late in the evening to apply the intervention to remove the high drowsiness level as opposed to controlling the audio system 110 to increasing the volume.

[0036] For example, one or more aspects of the controller 104 may collect data on a running time interval basis to determine which stimuli has proven to be most effective in gaining the driver’s attention in mitigating drowsiness. The controller 104 may continue to collect such information for any number of drivers of the vehicle. The controller 104 may then also perform data preprocessing to clean the collected data (e.g., remove duplicates, errors, etc.), handle missing data (e.g., by either removing the misaligned data or filling such data in), and normalize the data (e.g., scaling the data to a standardized format). The controller 104 may employ any one or more of linear regression,decision trees, and neural networks as a machine learning model. The machine learning model may then be trained (e.g., via unsupervised learning or supervised learning) and then evaluated for accuracy. The model may require hyperparameter tuning and optimization to further improve performance. Once the model is training, the model may be able to make predictions on the collected (or historical data) that is captured by the controller 104.

[0037] In another example, the adaptive model 264 may be implemented as a random generator trigger mechanism (or pseudo random generator, pseudo random number generator, deterministic random bit generator, etc.) in which the controller 104 may be configured to randomly select from any one of the interventions (e.g., audio system 110, HVAC system 112, vehicle lighting system 114) to apply whenever a driver is exhibited to have a high drowsiness level. The adaptive model 264 may generally aid in avoiding driver habituation by either applying historically successful stimuli (or interventions) or randomly applied stimuli (or intervention) in moments in which the driver is detected to exhibit a high drowsiness level.

[0038] The inference engine 251 provides processed outputs and / or commands such as drowsiness level, change HVAC command, change light command, change music command, drowsiness pattern, and change driving route command. The interaction controller 252 receives the drowsiness level, the change HVAC command, the change light command, and the change music command controls one or more aspects of the cabin control 254 such as the audio system 110, HVAC system 112, and the vehicle lighting system 114 in the event the drowsiness level exceeds a predetermined threshold. The interaction controller 252 receives the drowsiness pattern and output the trip summary on the user interface 106. Trip summary provides information to the driver at the end of his / her trip about their driver state and affected performance during the drive. The interaction controller 252 receives the change driving route command and outputs a trip plan on the user interface 106. In the case where the driver is detected as being drowsy, the audio system 110, under the control of the controller 104, recommends a change in the driving route that affects the trip plan to the nearest rest stop for the driver to become more alert.

[0039] FIGURE 3 depicts a method 300 for performing habituation mitigation control in accordance with one embodiment. The method 300 may correspond to the cascading model 264c as executed by the controller 104 to mitigate habituation. In operation 302, the controller 104 receivessignals indicative of the drowsiness level for the driver from the sensors 102 (e.g., wearable device 130, in-cabin sensors 132 including the cameras 140 having any number of the in-ward facing cameras, the eye gaze sensors, etc.).

[0040] In operation 304, the controller 104 determines the drowsiness level based on the signals received from the sensors 102. In one example, the drowsiness level may correspond to a range of anywhere between 1 - 9 as set forth in the Karolinska Sleepiness Scale (e.g., levels 1 - 3 generally correspond to an alert and engaged state, levels 4 - 5 correspond to neither an alert or drowsy state, levels 6 - 7 correspond to early signs of drowsiness, level 8 corresponds to the driver being drowsy, and levels 9 - 10 correspond to a very drowsy or sleeping state. For example, if the controller 104 determines that the drowsiness level for the driver is above a first threshold level of, for example, 5, then the method moves to operation 306. If not, then the method 300 proceeds to operation 302.

[0041] In operation 306, the controller 104 provides at least one intervention (or at least one first stimulus) via the cabin environment 180a, and / or interventions by way of music / sound or scents 180b, and / or interventions related to coaching programs 180c as noted above in connection with FIGURE 1. In operation 308, the controller 104 determines whether the intervention is being applied for a period that is longer than a first predetermined time frame (e.g., 2 minutes). If this condition is not true (i.e., the intervention that has been applied is less than the first predetermined time frame), then the method 300 moves to operation 310. If the condition is true (i.e., the intervention that is being applied is greater than the first predetermined time frame), then the method 300 moves to operation 314. Generally, the operations 306, 308, and 310 may correspond to the condition in which an intervention is not successful in mitigating the drowsiness level after, for example, two minutes, then the method 300 moves from operation 310 and back to operation 306 to reapply the intervention.

[0042] In operation 310, the controller 104 receives the signals from the sensors 102 and determines the drowsiness level for the driver. If the controller 104 determines that the drowsiness level for the driver is above the threshold of, for example, 5, then the method moves back to operation 306. If not, then the method 300 proceeds to operation 312 and stops the intervention. Ifso, then the method 300 moves back to operation 306. Generally, if the controller 104 does not detect that an intervention (or stimulus) applied to the driver is not successful.

[0043] In operation 314, the controller 104 continues to receive the signals from the sensors 102 and determines the drowsiness level for the driver. If the controller 104 determines that the drowsiness level for the driver is above the threshold of, for example, 5, then the method 300 moves to operation 318. If not, then the method 300 proceeds to operation 316 and stops the intervention. In operation 318, the controller 104 applies a second level of interventions. In this case, the controller 104 may apply an intervention that is different than the intervention applied in operation 306 or that the controller 104 can vary the intensity (e.g., increase or decrease) of the stimuli applied between the first intervention applied in operation 304 and the second intervention applied in operation 320 (e.g., first intervention plays back audio at a first volume level and second intervention plays back audio a second volume level that is different than the first volume level). For example, the controller 104 may control the seat 108 to provide the haptic feedback or move the seat 107 to the straight position in operation 306. The controller 104 may then trigger the audio system 110 to playback the audio at a higher volume or play energetic music, adjust the interior temperature or speed of the blower in the vehicle 101, and / or control the lighting system 114 to change the lighting scheme interior in the vehicle 101. It is recognized that the controller 104 may control any one or more of the audio system 110, the HVAC system 112, and the vehicle lighting system 114 to provide the first intervention as set forth in operation 306 and then control any one or more of the audio system 110, the HVAC system 112, and the vehicle lighting system to provide a second intervention that is different from the first intervention or the same as the first intervention. By employing different stimuli (or interventions), it is possible to mitigate the effects of habituation that drivers may experience when such drivers are exposed to the same stimuli over and over. This may be effective in mitigating the drowsiness or fatigue that the driver is experiencing.

[0044] In operation 320, the controller 104 determines whether the second intervention is being applied for a period that is longer than the first predetermined time frame (e.g., 2 minutes). If this condition is not true (i.e., the intervention that has been applied is less than the first predetermined time frame), then the method 300 moves to operation 322. If the condition is true (i.e., the intervention that is being applied is greater than the first predetermined time frame), then the method 300 moves to operation 326 and stops executing the second intervention.

[0045] In operation 322, the controller 104 receives the signals from the sensors 102 and determines the drowsiness level for the driver. If the controller 104 determines that the drowsiness level for the driver is above the threshold of, for example, 5, then the method moves back to operation 318. If not, then the method 300 proceeds to operation 324 and stops the intervention. If so, then the method 300 moves back to operation 318 to continue to apply the second intervention. Generally, the operations 318, 320, and 322 may correspond to the condition in which an intervention is not successful in mitigating the drowsiness level after, for example, two minutes, then the method 300 moves from operation 322 and back to operation 318 to reapply the intervention.

[0046] FIGURE 5 depicts a first use case 350 for a method 351 that provides fatigue and habituation mitigation in accordance with one embodiment. In operation 352, the controller 104 receives signals from the sensors 102 to determine the drowsiness level for the driver. In operation 354, the controller 104 determines whether the driver is experiencing any form of drowsiness. Similarly, the controller 104 determines whether the driver has had a poor night of sleep based on signals transmitted by the small watch 130 that provides information related to the level of sleep for the driver for a previous night If the controller 104 determines that the driver is not experiencing any form of drowsiness, but that the driver had a poor night of sleep, then the method 351 to operation 356. In operation 356, the controller 104 controls a scent generator to output a scent (e.g., Eucalyptus) in the vehicle cabin.

[0047] In operation 358, the controller 104 determines whether a high drowsiness level has been detected for the driver. For example, the controller 104 determines whether the drowsiness level for the driver is a value between a threshold of, for example, 5 and 7. If the controller 204 determines that the drowsiness level for the driver is between the threshold of 5 and 7, the method 351 moves to operation 360. If not, then the method 351 moves to operation 362. It should be noted that in the event the drowsiness level is detected to be a value that is, for example, between 1 and 4, then the controller 104 may trigger any one or more of the interventions 180.

[0048] In operation 360, the controller 104 may increase the number of interventions 180 that are being employed. For example, the controller 104 may control interventions 180 being employed by two or more of the seats 108, the audio system 110, the HVAC system 112, and the vehicle lighting system 114. In operation 362, the controller 104 determines whether the drowsinesslevel for the driver is between above the threshold, for example, of 7. If this condition is true, then the method 351 moves to operation 364. If not, then the method 351 moves back to operation 352. In operation 362, the controller 104 may increase an overall intensity and time duration for employing the interventions 180 to immediately alarm / awake the driver.

[0049] FIGURE 5 depicts a second use case 380 for a method 382 that provides fatigue and habituation mitigation in accordance with one embodiment. The method 382 generally corresponds that distance and time can be used to provide an increased intensity of a various intervention to mitigate driver drowsiness so as to not reach a maximum level of applied intensity for a given intervention. The driver may enter information correspond to a distance and / or time threshold for storage in the controller 104. In another embodiment, the controller 104 may store a predetermined distance and / or time threshold. If the drive is long in terms of distance and time, then the controller 104 provides an intensity increase that should be minimal as to not hit a maximum.

[0050] In operation 384, the controller 104 receives signals from the sensors 102 to determine the drowsiness level for the driver. In operation 386, the controller 104 determines whether the driver is experiencing any form of drowsiness. If so, then the method 380 moves to operation 388. If not, the method 380 returns back to operation 384. In operation 388, the controller 104 receives distance and time information from a navigation system (e.g., from the mobile device or from a in-vehicle navigation apparatus (not shown). In operation 390, the controller 104 determines whether the driving distance and / or the driving time exceeds a predetermined driving distance value and / or a predetermined driving time, respectively. If either condition is true, the method 382 moves to operation 392. If not, then the method 380 moves back to operation 384.

[0051] In operation 392, the controller 104 determines whether the driver is experiencing a high degree of drowsiness (e.g., above 6). If the drowsiness level is above 6, then the method 380 moves to operation 394 (e.g., the driver is exhibiting a high drowsiness level). If not, then the method 380 moves to operation 384. In operation 394, the controller 104 may control the HVAC system 112 to output cold air with an increased intensity (e.g., a gradual decrease in temperature while employ a gradual increase in fan speed). Additionally or alternatively, the controller 104 may control the seat 108 and corresponding motors and bladders (not shown) positioned therein to provide an energetic seat massage with an increased frequency. Additionally or alternatively, thecontroller 104 may control the audio system 110 to playback energetic music and may even gradually increase the volume of the music.

[0052] FIGURE 6 depicts a third use case 400 for a method 402 that provides fatigue and habituation mitigation in accordance with one embodiment. In operation 404, the controller 104 receives signal from the smart watch 130 or mobile device 131 that are indicative of the driver’s heart rate, exercise pattern, and or sleep pattern and monitors the level of drowsiness or fatigue that is being exhibited by the driver. The exercise pattern corresponds to the driver’s exercise patterns outside of the vehicle that can inform the driver of his / her drowsiness state inside the vehicle 101. For example, if the driver had an intense exercise earlier in the day, then this might contribute to the driving experiencing more fatigue while driving. The sleep pattern corresponds to the driver’s sleep pattern for example the night before that can be used to determine the drowsiness level of the driver. If the controller 104 determines that the driver is exhibiting a level of drowsiness or fatigue that is less than a threshold of 4, for example, then the method 402 moves to operation 406. If the controller 104 determines that the driver is exhibiting a level of drowsiness or fatigue that is greater than the threshold of 4, for example, then the method 402 moves to operation 408. It is recognized that the level of drowsiness is generally similar to the level of fatigue.

[0053] In operation 406, the controller 104 controls the audio system 110 to provide a guided verbal mediation exercise to decrease the driver’s heart rate. In general, if the low fatigue is detected by the controller 104 and the exercise patterns outside of the vehicle 101 indicate that exercise is low and the heartrate is low, then then the guided mediation exercises could provide a driver with a form of exercise to prevent the driver from becoming fatigued and interjecting an exercise that may be missing from their normal routine.

[0054] In operation 408, the controller 103 controls the HVAC system provide a cold air temperature (and may increase the blower) and the audio system 110 to play energetic music. Additionally or alternatively the controller 104 controls the audio system 110 to provide verbal commands of performing seated exercises (e.g., proprioception) of breathing exercises. The seated exercises include seated physical exercises that can be done in the vehicle 101 by the driver. For example, seated exercises may "seat pushes" to exercise triceps where the driver holds the steeringwheel at a 10 and 2 position thereby gripping tightly with elbows that are loose and bent, and then straightening the elbows and then pushing against the steering wheel with back into the seat.

[0055] It is recognized that the controllers as disclosed herein may include various microprocessors, integrated circuits, memory devices (e.g., FLASH, random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), or other suitable variants thereof), and software which co-act with one another to perform operation(s) disclosed herein. In addition, such controllers as disclosed utilize one or more microprocessors to execute a computer-program that is embodied in a non-transitory computer readable medium that is programmed to perform any number of the functions as disclosed. Further, the controller(s) as provided herein includes a housing and the various number of microprocessors, integrated circuits, and memory devices ((e.g., FLASH, random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM)) positioned within the housing. The controller(s) as disclosed also include hardware-based inputs and outputs for receiving and transmitting data, respectively from and to other hardware-based devices as discussed herein.

[0056] While exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the invention. Additionally, the features of various implementing embodiments may be combined to form further embodiments of the invention.

Claims

WHAT IS CLAIMED IS:

1. A system for providing fatigue mitigation in a vehicle, the system comprising: at least one sensor configured to transmit a plurality of signals indicative of any number of drowsiness levels for a user in the vehicle; and at least one controller programmed to: receive a first signal indicative of a first drowsiness level for the user in the vehicle from the at least one sensor; compare the first drowsiness level of the user to a first threshold level; control one or more vehicle systems to provide a first stimulus to the user to mitigate drowsiness being experienced by the user responsive to the first drowsiness level exceeding the first threshold level; determine whether the first drowsiness level for the user has exhibited a change within a first predetermined time frame; and control the one or more vehicle systems to provide a second stimulus to the user to mitigate the drowsiness being exhibited by the user based at least on the first drowsiness level for the user exceeding the first predetermined time frame, wherein the first stimulus is different than the second stimulus to mitigate habituation for the user.

2. The system of claim 1, wherein the at least one controller is further programmed to receive a second signal indicative of second drowsiness level for the user from the at least one sensor prior to controlling the one or more vehicle systems to provide the second stimulus.

3. The system of claim 2, wherein the at least one controller is further programmed to compare the second drowsiness level for the user to a second threshold level after the first drowsiness level for the user exceeds the first predetermined time frame.

4. The system of claim 3, wherein the at least one controller is further programmed to control the one or more vehicle systems to provide the second stimulus to the user to mitigate the drowsiness being experienced by the user responsive to the second drowsiness level for the exceeding the second threshold level.

5. The system of claim 1, wherein the one or more vehicle systems include an audio system and the at least one controller is further programmed to control the audio system to playback an audio output signal at a first volume as the first stimulus and to control the audio system playback the audio output signal at a second volume as the second stimulus, wherein the first volume is different that the second volume.

6. The system of claim 1, wherein the one or more vehicle systems include a heating, ventilation, air conditions (HVAC) system and the at least one controller is further programmed to control the HVAC system to adjust a temperature of an interior cabin of the vehicle to reach a first temperature level as the first stimulus and to adjust the temperature of the interior cabin of the vehicle to reach a second temperature level as the second stimulus, wherein the first temperature level is different than the second temperature level.

7. The system of claim 1, wherein the one or more vehicle systems include a lighting system and the at least one controller is further programed to control the lighting system to adjust an interior light of the vehicle to illuminate at a first color as the first stimulus and to adjust the interior light of the vehicle to illuminate at a second color as the second stimulus, wherein the first color is different from the second color.

8. The system of claim 1, wherein the one or more vehicle systems include a vehicle seat system and the at least one controller is further programed to control the vehicle seat system to at least one of (i) provide at least one of a first haptic feedback for a first timing interval as the first stimulus or (ii) move a seat of the vehicle seat system to a first position as the first stimulus, and to a least one (i) provide at least one of a second haptic feedback for a second timing interval as the second stimulus or (ii) move the scat of the vehicle scat system to a second position as the second stimulus.

9. The system of claim 1, wherein the at least one controller is further programmed to control a first system of the one or more vehicle systems to provide the first stimulus responsive to the first drowsiness level exceeding the first threshold level, and the at least one controller is furtherprogrammed to control a second system of the one or more vehicle systems to provide the second stimulus responsive to a second drowsiness level exceeding a second threshold level, wherein the first system is different than the second system.

10. The system of claim 1, wherein the one or more vehicle systems include a heating, ventilation, air condition (HVAC) system having a diffuser and the at least one controller is further programmed to control the diffuser to transmit a scented oil within an interior cabin of the vehicle as one of the first stimulus and the second stimulus.

11. The system of claim 1, wherein the at least one controller is further programmed to control the one or more vehicle systems to provide a second stimulus based at least on historical information that corresponds to which stimulus has been most successful in mitigating the drowsiness for the user.

12. The system of claim 1, wherein the at least one controller is further programmed to execute a random generator trigger mechanism to randomly control the one or more vehicle systems to provide at least one of the first stimulus and the second stimulus.

13. A method for providing fatigue mitigation in a vehicle, the method comprising: transmitting a plurality of signals indicative of any number of drowsiness levels for a user in the vehicle; and receiving, by at least one controller, a first signal indicative of a first drowsiness level for the user in the vehicle from at least one sensor; comparing the first drowsiness level of the user to a first threshold level; controlling one or more vehicle systems to provide a first stimulus to the user to mitigate drowsiness being experienced by the user responsive to the first drowsiness level exceeding the first threshold level; determining by the at least one controller whether the first drowsiness level for the user has exhibited a change within a first predetermined time frame; and controlling the one or more vehicle systems to provide a second stimulus to the user to mitigate the drowsiness being exhibited by the user based at least on the first drowsiness level for theuser exceeding the first predetermined time frame, wherein the first stimulus is different than the second stimulus to mitigate habituation for the user.

14. The method of claim 13 further comprising: controlling a first system of the one or more vehicle systems to provide the first stimulus responsive to the first drowsiness level exceeding the first threshold level, and controlling a second system of the one or more vehicle systems to provide the second stimulus responsive to a second drowsiness level exceeding a second threshold level, wherein the first system is different than the second system.

15. The method of claim 13 further comprising controlling a diffuser to transmit a scented oil within an interior cabin of the vehicle as one of the first stimulus and the second stimulus.

16. The method of claim 13 wherein controlling the one or more vehicle systems to provide the second stimulus is based at least on historical information that corresponds to which stimulus has been most successful in mitigating the drowsiness for the user.

17. The method of claim 16 further comprising executing a random generator trigger mechanism to randomly control the one or more vehicle systems to provide at least one of the first stimulus and the second stimulus.

18. A computer-program product embodiment in a non-transitory computer readable medium executable by at least one controller to provide fatigue mitigation in a vehicle, the computer-program product comprising instructions to: receive a first signal indicative of a first drowsiness level for a driver in the vehicle from at least one sensor; compare the first drowsiness level of the driver to a first threshold level; control one or more vehicle systems to provide a first stimulus to the driver to mitigate drowsiness being experienced by the user responsive to the first drowsiness level exceeding the first threshold level;determine whether the first drowsiness level for the driver has exhibited a change within a first predetermined time frame; and control the one or more vehicle systems to provide a second stimulus to the driver to mitigate the drowsiness being exhibited by the user based at least on the first drowsiness level for the driver exceeding the first predetermined time frame, wherein the first stimulus is different than the second stimulus to mitigate habituation for the user.

19. The computer-program product of claim 18 further comprising instructions to provide the second stimulus based at least on historical information that corresponds to which stimulus has been most successful in mitigating the drowsiness for the driver.

20. The computer-program product of claim 18 further comprising instructions to execute a random generator trigger mechanism to randomly control the one or more vehicle systems to provide at least one of the first stimulus and the second stimulus.

Images