Driver monitoring method for monitoring activities of a driver of a vehicle and vehicle for performing the method

Abstract

The present disclosure relates to a driver monitoring method for monitoring activities of a driver of a vehicle, comprising the steps of estimating a fatigue level of the driver and / or a waiting time of the driver, establishing a driving context and / or a vehicle state of the vehicle, outputting one of a plurality of recommended actions for changing an activity of the driver based on the fatigue level, the waiting time, a success rate of the recommended action, the driving context and / or the vehicle state, estimating a response of the driver to the recommended action, and estimating and updating the success rate of the recommended action. The present disclosure also relates to a vehicle comprising a controller for executing a corresponding method, an input device and an output device.

Description

Technical Field

[0001] This invention relates to a driver monitoring method for monitoring the activities of a vehicle driver, comprising the steps of: estimating the driver's fatigue level and / or the time spent waiting by the driver; establishing the vehicle's driving context and / or vehicle status; outputting one of a plurality of recommended actions to change the driver's activities based on the fatigue level, the waiting time, a success rate of the recommended action, the driving context, and / or the vehicle status; estimating the driver's response to the recommended action; and estimating and updating the success rate of the recommended action. The invention also relates to a vehicle including a controller, an input device, and an output device for performing a corresponding method. Background Technology

[0002] Current methods and corresponding systems for monitoring drivers are designed to detect the activities of a vehicle driver, thereby detecting dangerous or undesirable activities, reacting to them, and improving the driver's sense of safety and wellbeing.

[0003] For example, it is possible to detect driver fatigue or drowsiness, or a situation where the driver is forced to remain idle and wait for certain events, and to react accordingly. Clearly, the term "activity" should be understood broadly and can refer to behaviors indicating a fatigued driver, such as unstable control of the vehicle or prolonged eye closure. Furthermore, the driver's monitoring behavior can indicate a waiting situation where the vehicle may be stationary and the driver may not be performing any significant control activities related to the vehicle's trajectory.

[0004] Once a corresponding activity is detected, numerous predefined responses can be generated to reduce the accident risk associated with drowsy drivers or increase the well-being of waiting drivers. However, known methods are insensitive to whether these predefined responses are truly suitable for their respective drivers and achieve the desired results of reducing drowsy driving or increasing driver well-being while idling. This presents a problem because known methods are ineffective in reducing drowsy driving and improving driver well-being, or worse, the driver may actually find the multiple predefined responses to be a nuisance, potentially rendering the methods ineffective altogether. Summary of the Invention

[0005] The purpose of this invention is to provide an improved method and an improved vehicle that overcome the above-mentioned problems.

[0006] This objective is achieved through a driver monitoring method for monitoring the activities of a vehicle driver and a vehicle, including a controller, an input device, and an output device for performing a corresponding method.

[0007] A driver monitoring method is provided for monitoring the activities of a driver in a vehicle, the driver monitoring method comprising the following steps:

[0008] Estimate the driver's level of fatigue and / or the driver's waiting time;

[0009] Establish the driving context and / or vehicle status of the vehicle;

[0010] Output one of a plurality of recommended actions to change the driver’s activities, based on the fatigue level, the waiting time, a success rate of the recommended action, the driving context and / or the vehicle status;

[0011] Estimate the driver's response to the recommended action; and

[0012] Estimate and update the success rate of the recommended action.

[0013] The term "monitoring" should be understood in a broad sense, including surveillance of the driver's behavior and suggestions for changing or enhancing the driver's behavior.

[0014] The recommended actions may include changing the vehicle's settings, such as its interior lighting or seat position. They may include playing certain music or initiating games for the driver to play. The recommended actions may include suggestions for activities the driver can perform, such as taking a walk, taking a nap, or engaging in meditation. The recommended actions may be output or executed directly, or the driver may be prompted to confirm a suggested action before actually outputting or executing it. Generally, the output of a recommended action may include outputting optical and / or visual information and / or changing the vehicle's physical configuration.

[0015] The success rate of a given recommended action can be a function of the ratio of the number of times a driver accepts a recommended action to the total number of actions suggested to a driver.

[0016] According to the present invention, a specific driver can be identified, making the method applicable to each unique driver of a given vehicle. The driver identification can be performed based on facial recognition steps and corresponding hardware such as a driver monitoring camera.

[0017] In cases where a driver uses multiple vehicles, the vehicles can be interconnected, allowing the method to be performed on any vehicle driven by the same driver. Therefore, the method is not limited to a single vehicle.

[0018] The method described herein can be continuously executed while the driver is driving the vehicle. Alternatively, the execution of the method can be initiated by some predefined signals, such as the occurrence of a potential driving error, such as a sudden turn and / or breaking or crossing some road markings.

[0019] The method can be executed in parallel, that is, simultaneously and with respect to different suggested actions, such that more than one suggested action can be output to the driver.

[0020] According to the present invention, dangerous driving is reduced by effectively combating fatigue, improving the effectiveness of methods for combating driver drowsiness, and enhancing the user experience by adapting numerous measures to the unique characteristics of the driver. Furthermore, the driver's overall well-being can be enhanced because it supports the driver in managing potential wearing waiting periods.

[0021] Compared to current driver monitoring systems, this solution integrates multiple escalation states and provides numerous countermeasures (e.g., multiple modes) based on: first, the type of the driver's current activity or state and her fatigue level; and second, the previously established success rate of a proposed solution or recommended action. Based on these, the algorithm can learn and adapt the most effective methods to counteract individual fatigue and / or improve her overall well-being. To achieve this, the method may include a driver identification step, where the driver's identity is established. Therefore, the method is applicable to a variety of drivers and their individual characteristics and preferences.

[0022] The degree of fatigue may be associated with the driver's actual fatigue and may be a fraction of a fatigue function, which indicates the driver's behavior, such as yawning, eye movements, and / or incorrect driving behavior. This behavior can be detected by the methods described herein and can be used to estimate or calculate the degree of fatigue.

[0023] This invention provides a context-based recommendation algorithm, in which a vehicle can perform some currently described remedies if a driver's fatigue or waiting status or condition is detected.

[0024] In a preferred embodiment of the invention, the driver's fatigue level and / or the driver's waiting time are estimated by using a driver monitoring camera and / or by estimating the lateral and longitudinal vehicle control performed by the driver.

[0025] In another preferred embodiment of the present invention, establishing the driving context and / or vehicle status of the vehicle includes: establishing a current workday, a time period during the day, a driving time, a weather, a road type, a driving history, responses of multiple advanced driver assistance systems, a battery charge, and / or a location of the vehicle.

[0026] In another preferred embodiment of the invention, the plurality of recommended actions includes suggesting a refresh mode and / or a rest mode. Numerous other modes may be applied additionally or alternatively. The suggestion of a mode or a plurality of actions associated with a mode may include changing some vehicle settings, outputting some information, prompting the driver to confirm and initiate the output of some information, and / or many other methodological steps that can improve driving safety and increase the driver's well-being. The plurality of modes can be understood as a collection of various recommended actions, wherein each mode may be associated with a particular driver's fatigue level and / or driver activity.

[0027] In another preferred embodiment of the invention, the refresh mode is output when the driver's fatigue level is lower than that of the rest mode. If the fatigue level exceeds a threshold or fatigue is considered possible due to detection, for example, a long driving time on a rural road, a corresponding counteracting measure, i.e., a recommended action, is proposed.

[0028] In another preferred embodiment of the present invention, the rest mode includes:

[0029] It is recommended to have a rest area, especially one with a battery charging station;

[0030] It is recommended to set a timer for outputting an alarm, output an alarm based on the battery's charging status, play music and / or provide a massage to signal the driver that the rest mode has ended;

[0031] It is recommended to perform noise cancellation, suppress irrelevant messages, and set ambient light and / or temperature.

[0032] In another preferred embodiment of the present invention, the refresh mode includes:

[0033] Suggestion 1: Recreational activities.

[0034] The refresh mode can be output as a recommended action, especially when the driver monitoring method has estimated that the driver has spent more than a predetermined amount of time waiting and / or has not significantly controlled the vehicle. This may occur when the driver is caught in a traffic jam or waiting for other passengers to board. Alternatively or additionally, the refresh mode can be output when the driver monitoring method has estimated a low level of fatigue.

[0035] The recommended actions output in the refresh mode may include various meditation instructions, allowing the driver to utilize the waiting time to increase psychological well-being. In this case, the vehicle may be configured to provide a relaxing environment, such as by playing suitable music or natural sounds, providing appropriate ambient lighting, and / or a massage function. Similarly, instructions for practicing light yoga (e.g., Pranayama) may be output in the refresh mode.

[0036] In principle, any type of information that the driver might find helpful can be output in the refresh mode. This could include reading an e-book, enabling a game such as chess or a card game on the vehicle's computer system, or suggesting a walk in the vicinity. The driver can accept a suitable recommended action, and the method ensures that the success rate of the selected recommended action is updated accordingly.

[0037] In another preferred embodiment of the invention, estimating the driver's response to the recommended action includes: determining which recommended actions have been accepted by the driver and to what extent they have been accepted.

[0038] Therefore, the success rate of accepting a recommended action can be increased each time the driver accepts it. Thus, the driver's response to the recommended action is estimated. In future scenarios, the method may suggest certain recommended actions more frequently if the success rates of these actions are higher than those of other actions. A recommended action that has been frequently selected or accepted by a driver and has improved the driver's fatigue level can be stored and / or rated and can be suggested as a favorite, for example, more frequently in future scenarios, especially in similar contexts and / or when similar levels of fatigue are detected.

[0039] The method can monitor how long a recommended action has been accepted or performed by the driver in order to determine the extent to which the recommended action has been accepted.

[0040] In another preferred embodiment of the invention, estimating the driver's response to the recommended action includes estimating the driver's level of fatigue after the recommended action has been completed.

[0041] The present invention also relates to a vehicle comprising a controller, an input device, and an output device for performing the methods described above. The vehicle may be designed to perform any of the method steps currently outlined; that is, the vehicle may include any hardware necessary to perform the method steps currently described.

[0042] Therefore, the present invention provides numerous benefits of a contextual and intelligent method for monitoring a driver, including improved driving safety, enhanced effectiveness of countermeasures, and improved driver satisfaction due to a highly personalized and adaptable monitoring method. Furthermore, the present invention enhances the perception capabilities of an intelligent vehicle. Attached Figure Description

[0043] Other features, details and advantages of the invention are disclosed in the following description of the drawings and the drawings themselves.

[0044] The accompanying drawings show:

[0045] Figure 1 This is a flowchart of some steps of the method according to the present invention. Detailed Implementation

[0046] exist Figure 1 In one step of the method shown, the vehicle's context can be examined. This may correspond to establishing the vehicle's driving context and / or vehicle status. In this step, the road type in which the vehicle is located can be retrieved from some accessible map data and / or from numerous vehicle sensors, such as a camera capturing the environment around the vehicle. Further data, such as daytime hours, traffic density, driver behavior, and / or regularity of certain driver behaviors, can be obtained from numerous vehicle sensors, from numerous vehicle storage devices such as hard disk drives, and / or from other sources connected to the vehicle and accessible by the method of the present invention, such as the Internet and / or suitable cloud storage sources.

[0047] In another method step, the driver's fatigue level and / or the driver's waiting time can be estimated based on data retrieved from a driver monitoring camera and / or from various sensors that measure the driving characteristics of the vehicle.

[0048] If both the driving context of the vehicle and the driver's current activity indicate undesirable situations, such as a drowsy driver driving the car unsafely, or a driver waiting inside the car while it is stationary, the method can output some of a plurality of recommended actions. These recommended actions aim to alter the driver's activity based on factors such as fatigue level, waiting time, the success rate of the recommended action, the driving context, and / or the vehicle state.

[0049] Initially, the success rate of untried recommended actions can be set to a predetermined value, which may initially be the same for all recommended actions. Once the driver's response to the recommended action is estimated, the success rate of the recommended action can be estimated and updated. Therefore, after executing the method multiple times for various different recommended actions, the success rates of the multiple recommended actions can be automatically updated to reflect their actual effectiveness for a particular driver.

[0050] As an example, the method can detect a drowsy driver driving the vehicle. This could stem from, for example, the driver closing their eyes more regularly and for longer periods than average and / or from the driver making erroneous controls of the vehicle. If the method determines that the vehicle is in motion, a rest can be suggested. Driver drowsiness can be detected based on direct driver behavior such as the driver's eye movements. Alternatively, it can be detected from indirect data, such as when drowsy driving patterns are anticipated, for example from regular driving behavior or driving trajectories detected and stored by the method, or when drowsy environments are actually detected, for example, if the vehicle's current route is a rural road with low traffic density.

[0051] Alternatively, the method can detect that the car is stationary and the driver is not exercising any substantial control over it, indicating that the driver may be waiting for an event, such as the arrival of another passenger in the vehicle. In this case, other actions can be performed, such as issuing yoga instructions.

[0052] Once the recommended action has been completed or rejected by the driver, the driver's activity or state can be checked again, whereby the driver's response to the recommended action is estimated for its validity. If the given recommended action is found to be valid, for example, if the driver accepts the recommended action and her drowsiness decreases after performing the recommended action, then the success rate of this recommended action can be increased from a previous value to a new, higher value, and vice versa. Thus, the success of a proposal can be effectively checked. If no drowsy or waiting driver is found in the vehicle, then according to the method, no proposal may be output.

[0053] In fact, the numerous algorithms included in this invention are capable of retrieving information about the driver's state and the current status of the vehicle. Based on the retrieved information, the numerous algorithms recommend appropriate means or actions to improve the driver's current state and estimate the success or failure of the multiple recommended actions.

[0054] If a similar situation is detected in another instance, i.e., a similar driver state and vehicle environment, a previously successful action can be recommended again. The similarity of various situations can refer to similar vehicle routes and / or some routine phenomena or recurring events, such as repeated similar waiting times to encourage more passengers to board at certain pick-up locations. The algorithms can be designed to iteratively learn a driver's personal preferences and the efficiency of a given suggested action.

[0055] This invention addresses two important issues concerning numerous vehicles driven by numerous drivers.

[0056] On the one hand, research shows that many drivers spend a significant amount of time (averaging about 75 minutes per week in some cases) waiting behind the wheel, such as when picking up children from school, taking children to clubs, or waiting for others during activities such as shopping or a doctor's appointment. This is especially true during pandemics, when waiting in the vehicle is more likely for purposes such as a doctor's appointment or a medical check-up. Drivers might use this waiting time for things like reading, engaging in some form of online activity such as using YouTube or Facebook, listening to music, or planning meals. However, few drivers use this time to actually improve their well-being.

[0057] On the other hand, a significant number of vehicle accidents are caused by drowsy drivers who continue to control the vehicle far beyond the safe operating conditions of any machinery.

[0058] The present invention provides a method for simultaneously detecting a driver’s drowsiness and waiting period, thereby improving many related problems and the driver’s condition.

[0059] In a practical example, the method can detect driver fatigue and suggest switching to the refresh mode, including several possible suggested actions associated with the refresh mode. The driver can accept the suggestion, and the vehicle can take appropriate action. After a given amount of time, such as 15 minutes, the driver's fatigue level can be re-estimated, and an increased fatigue level can still be detected. In this case, it may be found that the suggested action was unsuccessful. As a result, the method can output several suggested actions associated with the rest mode. The driver may be prompted to confirm the suggested selection. If the driver accepts the suggestion, the method can indicate the next possible rest area on a navigation device so that the driver can select a suitable rest area. When suggesting a rest area, the method can consider the vehicle battery's state of charge. If the state of charge is below a given value (e.g., 60%), the method can suggest an area to charge the vehicle. Preferably, a rest area that can be reached within a given time (e.g., 15 minutes) can be suggested. Once the area is reached, the method can suggest that the driver set a timer or consider using a wake-up timer to complete the charging. During rest periods, in addition to setting the vehicle to conditions including pleasant ambient lighting and suitable temperature, the method can also provide noise cancellation and irrelevant message suppression. Once the timer has expired, the method gently awakens the driver by, for example, playing quiet music and / or providing a massage. The method can then re-examine the driver's fatigue level and update the success rate of the recommended actions accordingly. Over time, the algorithm establishes the driver's preferences and characteristics based on their responses to numerous recommended outputs—whether the driver accepts or rejects the recommendations, the current context (i.e., weekday, daytime, weather, driving time, etc.), and the success or failure of the proposed pattern—i.e., the re-estimated fatigue level). Therefore, many future suggestions can be improved, and the driver can experience a highly personalized and efficient vehicle.

[0060] Therefore, the present invention provides numerous benefits of a contextual and intelligent method for monitoring a driver, including improved driving safety, enhanced effectiveness of countermeasures, and improved driver satisfaction due to a highly personalized and adaptable monitoring method. Furthermore, the present invention enhances the perception capabilities of an intelligent vehicle.

[0061] This invention is not limited to the embodiments currently described, but can be varied in different ways. All features defined in the claims, specification, and drawings, including structural details and spatial arrangements, may be relevant to this invention.

Claims

1. A driver monitoring method for monitoring the activities of a vehicle driver, characterized in that: The driver monitoring method includes the following steps: Estimate the driver's level of fatigue and / or the driver's waiting time; Establish the driving context of the vehicle and / or the state of the vehicle; Based on the success rate of multiple recommended actions, and at least one of the fatigue level, the waiting time, the driving context, or the vehicle state, output one of the multiple recommended actions for changing the driver's activities; wherein the success rate of the recommended action is a function of the ratio of the number of times the driver accepts the recommended action to the total number of recommended actions suggested to the driver; Estimate the driver's response to one of the plurality of recommended actions; and Estimate and update the success rate of one of the plurality of recommended actions; The success rate of the plurality of recommended actions is set to a predetermined value, which is initially the same for all the plurality of recommended actions; Each time the driver accepts a recommended action, the success rate of that recommended action is increased. If the success rate of the recommended action is higher than the success rate of other recommended actions among the plurality of recommended actions, then the recommended action is suggested more frequently than the other recommended actions.

2. The driver monitoring method according to claim 1, characterized in that: The driver's fatigue level and / or the driver's waiting time are estimated by means of a driver monitoring camera and / or by means of estimation of lateral and longitudinal vehicle control performed by the driver.

3. The driver monitoring method according to claim 1, characterized in that: The process of establishing the driving context and / or the state of the vehicle includes: establishing the driving context and / or the state of the vehicle based on the current workday, daytime hours, driving time, weather, road type, driving history, response of multiple advanced driver assistance systems, battery charge, and / or the location of the vehicle.

4. The driver monitoring method according to claim 1, characterized in that: The recommended actions include suggested refresh mode and / or rest mode.

5. The driver monitoring method according to claim 4, characterized in that: The refresh mode is output when the driver's fatigue level is lower than that of the rest mode.

6. The driver monitoring method according to claim 4 or 5, characterized in that: The rest modes include: A rest area is recommended, which is equipped with a battery charging station; It is recommended to set a timer for outputting alarms, output alarms based on battery charging status, play music and / or provide massage to signal the driver that the rest mode has ended; and / or It is recommended to perform noise cancellation, suppress irrelevant messages, and / or set ambient light and / or temperature.

7. The driver monitoring method according to claim 4 or 5, characterized in that: The refresh mode includes: suggested entertainment activities.

8. The driver monitoring method according to claim 1, characterized in that: After estimating the driver's response to one of the plurality of recommended actions, the method further includes: determining which recommended actions have been accepted by the driver and to what extent they have been accepted.

9. The driver monitoring method according to claim 1, characterized in that: After estimating the driver's response to one of the plurality of recommended actions, the method further includes estimating the driver's level of fatigue after one of the plurality of recommended actions has been completed.

10. A vehicle comprising a controller, an input device, and an output device for performing the driver monitoring method according to any one of claims 1 to 9.

Images