Method and device for assisting in the performance of driving tasks

Abstract

A device is described for assisting the driver of a vehicle in performing driving tasks. The device is configured to detect, at a specific checkpoint, that the maximum permissible time for not performing a first driving task has been exceeded, and to identify one or more further driving tasks that the driver is not performing at that checkpoint. The device is further configured to issue a combined warning to the driver regarding the first driving task and the one or more further driving tasks.

Description

[0001] The invention relates to a method and a corresponding device that supports the driver of a vehicle in carrying out or performing driving tasks that are to be carried out within the scope of the operation of the vehicle.

[0002] A vehicle may have one or more driving functions, e.g., driver assistance functions designed to automatically guide the vehicle longitudinally and / or laterally. The vehicle may, for example, have a cruise control and / or distance control system (e.g., an adaptive cruise control, ACC, assistance system). Alternatively or additionally, the vehicle may have a lane departure warning system and / or a lane guidance assistant.

[0003] The driver is typically required to perform one or more driving tasks during the operation of one or more automated driving functions to ensure safe operation of the one or more driving functions. Examples of driving tasks include maintaining a sufficiently high level of attention regarding the vehicle's operation and / or holding the steering wheel with one or both hands.

[0004] This document deals with the technical task of providing particularly comfortable and reliable support to the driver of a vehicle when performing one or more driving tasks.

[0005] The problem is solved by each of the independent claims. Advantageous embodiments are described, among other things, in the dependent claims. It should be noted that additional features of a patent claim dependent on an independent patent claim can form a separate invention, independent of the combination of all features of the independent patent claim, without the features of the independent patent claim or only in combination with a subset of the features of the independent patent claim, which invention can be made the subject of an independent claim, a divisional application, or a subsequent application. This applies equally to technical teachings described in the description, which can form an invention independent of the features of the independent patent claims.

[0006] According to one aspect, a device for assisting the driver of a (motor) vehicle in performing driving tasks is described, wherein the driving tasks are to be performed by the driver. It may be necessary for the driver to perform a certain number of different driving tasks (e.g., 2 or more, or 3 or more driving tasks). Examples of driving tasks are: • monitoring of the vehicle's operation by the driver; • the presence of a minimum level of attention by the driver with regard to the operation of the vehicle; and / or • touching and / or gripping the vehicle’s steering wheel with one or both hands.

[0007] The device can be configured to acquire sensor data from one or more driver sensors of the vehicle, wherein the one or more driver sensors comprise, for example, an interior camera and / or a hands-on sensor. Based on the sensor data from the one or more driver sensors, it can be determined whether (at a specific point in time) a driving task is being performed by the driver or not. This can be determined for each individual driving task in the set of driving tasks.

[0008] Each individual driving task can be associated with a maximum permissible period of time for the driver to not perform the respective driving task. In other words, a period of time can be defined for each individual driving task, for which the driver may refrain from performing the respective driving task at most for a period of time. These maximum permissible periods of time can vary for different driving tasks. Furthermore, the maximum permissible periods of time can depend on whether an automatic driving function is being used for automated longitudinal and / or lateral guidance of the vehicle, and if so, which one.

[0009] The device can be configured to repeatedly, in particular periodically, update a time counter for the plurality and / or set of different driving tasks, wherein the time counter for a driving task indicates the period of time since which the driving task has not been (continuously) performed by the driver. The individual time counters can be updated based on the sensor data of one or more driver sensors. Based on the time counters of the individual driving tasks, it can be checked whether or not the maximum permissible period of time for not performing the respective driving task is exceeded at a specific point in time. This can be checked repeatedly, in particular periodically, at successive check times.

[0010] The device is configured to detect at a (specific) check point in time that the maximum permissible period for not performing a first driving task has been exceeded. In particular, during the operation of an automated driving function for automated longitudinal and / or lateral guidance of the vehicle, it can be detected that the maximum permissible period for not performing a first driving task has been exceeded at the (specific) check point in time.

[0011] The device is further configured to identify one or more additional (in particular all additional) driving tasks (from the set of driving tasks) that are not being performed by the driver at the check time. In this case, one or more additional driving tasks can be identified that are not being performed by the driver at the check time, but for which the maximum permissible period for non-performance has not yet been exceeded at the check time.

[0012] It can be caused (in response to the identification) that a combined indication relating to the first driving task and relating to the one or more further driving tasks is issued to the driver (e.g. via a user interface of the vehicle). The combined indication can relate to the one or more further driving tasks even if the maximum permissible period of time for not performing the one or more further driving tasks has not yet been exceeded at the time of the check. The combined indication can be aimed at prompting the driver to perform the first driving task and the one or more further driving tasks. For example, the indication can state, in particular indicate, that the first driving task and the one or more further driving tasks are not being performed.

[0013] This allows the driver to be given a combined set of instructions, informing them of the entire spectrum of tasks they are not currently performing and those they should perform. This can be a convenient way to increase the vehicle's driving safety.

[0014] The device can be configured to identify a third driving task (from the set of driving tasks) that is being performed by the driver at the review time. The combined indication can be configured to have no reference to the third driving task. In other words, the combined indication can be configured to refer only to the one or more driving tasks that are not being performed by the driver at the review time. This can further increase the driver's comfort.

[0015] The device can be configured to detect, at a time subsequent to the check time, that a driving task (e.g., the first driving task or one of the one or more further driving tasks) that was not performed by the driver at the check time is performed by the driver at the subsequent time. In response, the outputted indication can be adjusted such that the reference to the driving task that is again performed by the driver at the subsequent time is removed from the indication. This can further increase the driver's comfort.

[0016] According to a further aspect, a (road) motor vehicle (in particular a passenger car or a truck or a bus or a motorcycle) is described which comprises the device described in this document.

[0017] According to a further aspect, a method is described for assisting a driver of a vehicle in performing driving tasks that are to be performed by the driver. The method comprises detecting, at a check point in time, that a maximum permissible period for not performing a first driving task has been exceeded, and identifying one or more additional driving tasks that are not being performed by the driver at the check point in time. Furthermore, the method comprises causing a combined indication relating to the first driving task and the one or more additional driving tasks to be output to the driver.

[0018] According to another aspect, a software (SW) program is described. The SW program can be configured to be executed on a processor (e.g., on a vehicle control unit) and thereby to carry out the method described in this document.

[0019] According to a further aspect, a storage medium is described. The storage medium can comprise a software program configured to be executed on a processor and thereby to carry out the method described in this document.

[0020] It should be noted that the methods, devices, and systems described in this document can be used both alone and in combination with other methods, devices, and systems described in this document. Furthermore, any aspects of the methods, devices, and systems described in this document can be combined in a variety of ways. In particular, the features of the claims can be combined in a variety of ways. Furthermore, features listed in parentheses are to be understood as optional features.

[0021] The invention will be described in more detail below using exemplary embodiments. Fig. 1 exemplary components of a vehicle; Fig. 2 exemplary periods in which the driver refrains from performing various driving tasks; and Fig. 3 a flowchart of an exemplary method for assisting the driver of a vehicle in performing required driving tasks.

[0022] As stated at the beginning, this document deals with increasing the comfort and / or safety of a vehicle's driving operation. In this context, Fig. 1 shows an exemplary vehicle 100 with one or more environmental sensors 102, each configured to collect sensor data (also referred to as environmental data) relating to the surroundings of the vehicle 100. Exemplary environmental sensors 102 include a camera, a radar sensor, a lidar sensor, an ultrasonic sensor, etc.

[0023] A (control) device 101 of the vehicle 100 can be configured to evaluate the environmental data, e.g., to detect one or more objects (e.g., other vehicles) in the environment of the vehicle 100. The device 101 can further be configured to effect automated longitudinal and / or lateral guidance of the vehicle 100 based on the environmental data, in particular based on the one or more detected objects. For this purpose, one or more longitudinal and / or lateral guidance actuators 103 (e.g., a drive motor, a braking device, and / or a steering device) of the vehicle can be controlled. The automatic longitudinal guidance of the vehicle 100 can be effected, for example, by a speed controller and / or distance controller. The automated lateral guidance can be effected, for example, by a lane keeping and / or lane guidance assistant.

[0024] The driver of the vehicle 100 is typically required to perform one or more driving tasks during the operation of one or more driving functions (which effect automated longitudinal and / or lateral guidance of the vehicle 100). Examples of driving tasks include: • monitoring the operation of the driving function with a sufficiently high degree of attention; • holding the steering wheel 107 of the vehicle 100 with one hand or with both hands; and / or • the (possibly regular) operation of a control element.

[0025] The vehicle 100 may include one or more driver sensors 106, each configured to determine sensor data related to the driver. Example driver sensors 106 are • an interior camera designed to capture camera images relating to the driver, e.g. to determine the driver’s level of attention based thereon; and / or • a hands-on sensor configured to detect whether the driver is touching the steering wheel 107 with none, one, or both hands.

[0026] It is typically permissible for the driver of vehicle 100 to refrain from performing a specific driving task for a specific maximum permissible period of time. The maximum permissible period of time may vary for different driving tasks. For example, it may be permissible for the driver to take their hands off the steering wheel 107 for a specific maximum permissible first period of time and / or for the driver to be inattentive for a specific maximum permissible second period of time.

[0027] The (control) device 101 of the vehicle 100 can be configured to monitor the failure to perform one or more driving tasks based on the sensor data from the one or more driver sensors 106. Furthermore, the device 101 can be configured to issue a notification to the driver of the vehicle 100 (e.g., via the user interface 104 of the vehicle 100) if it detects that the performance of a specific driving task has already been omitted for the maximum permissible period of time. The notification can prompt the driver to perform this driving task.

[0028] The output of individual instructions for the individual driving tasks may be perceived as uncomfortable and confusing by the driver of the vehicle 100.

[0029] Fig. 2 shows exemplary time periods in which individual driving tasks 201, 202, 203 were not performed by the driver of the vehicle 100. The individual rectangles each represent a period in which the associated driving task 201, 202, 203 was not performed. The individual periods, in particular the duration of the individual periods, can be detected or determined based on the sensor data from one or more driver sensors 106.

[0030] In the Fig. In the example shown in Figure 2, the first driving task 201 was not executed for a relatively long period. The second driving task 202 was not executed for several relatively short periods, and the third driving task 203 was not executed for a single period. As already explained above, a maximum permissible period for non-execution of the respective driving task 201, 202, 203 can be defined for each of the individual driving tasks 201, 202, 203.

[0031] It can be checked (e.g., using timers or time counters) whether the maximum permissible time duration for at least one driving task 201, 202, 203 is exceeded. If this is the case, a notification can be issued to the driver via the user interface 104 to prompt the driver to perform the driving task 201, 202, 203.

[0032] In the Fig. 2, it is detected at a check time 205 that the maximum permissible time period for not performing the first driving task 201 has been exceeded. It can be checked whether one or more further driving tasks 202, 203 are not performed at the check time 205. In the Fig. In the example shown in Figure 2, the second driving task 202 is also not carried out at the check time 205.

[0033] It can be arranged that (at or after the check time 205) a notice is issued regarding the first driving task 201 and the second driving task 202 (even if the maximum permissible time period for not performing the second driving task 202 has not yet been exceeded at the check time 205). This allows the driver to be informed in a convenient and reliable manner about all driving tasks 201, 202 that are not currently being performed but that must be performed.

[0034] Thus, a uniform system is provided in which the warning states and the resulting information can be communicated to the driver clearly and unambiguously and / or in which the different warnings regarding the driving tasks 201, 202, 203 to be performed by the driver are communicated clearly and unambiguously.

[0035] In a central software module, the attention warnings during the transition from driving mode according to SAE Level L0 to driving mode according to SAE Level L2 and / or back can be consolidated and controlled. At the same time, it can be monitored whether the driver has their hands on the steering wheel 107. Required warnings due to a lack of attention and / or missing hands on the steering wheel 107 can be combined in this module, and the corresponding correct and clear instruction can be issued to the driver via the user interface 104. This combination allows the driver to be granted higher limits regarding the (maximum permissible) inattention periods and / or hands-off periods tolerated by the system, while maintaining the same level of safety. This results in a more sophisticated automation experience for the driver.

[0036] In one example, the driver is driving manually. The tolerated time (e.g., as permitted by EuroNCAP) until a warning due to a lack of attention must be issued is, for example, 3 seconds after a loss of eye contact is detected. At t0, the driver takes their eyes off the road. The warning should be issued at tw_L0=3 seconds. For example, at t=2 seconds, the driver activates an SAE-Level L2 system (i.e., an automated driving function). The tolerated time of a loss of eye contact during the SAE-Level L2 system is, for example, 5 seconds. The switch to the L2 system is registered, and the warning at tw_L0=3 seconds is suppressed.

[0037] At t=6s, the driver returns their gaze to the road and simultaneously takes their hands off the steering wheel 107. After a further 5s (--> t=11s), a hands-on warning is issued due to excessive hands-off. This warning remains active for another 15s (until t=26s), for example, until it is escalated into a takeover request. At t=13s, for example, the driver also takes their gaze away from the road. The tolerated gaze aversion time during operation of the SAE Level L2 system is 5s, for example, so that at t=18s, the previous hands-on warning is supplemented by the attention warning, and the driver is clearly and precisely informed that driving tasks 201, 202 consist of gaze monitoring and steering. At t=20s, the driver returns their hands to the steering wheel 107. The warning is now reduced to a warning regarding the lack of attention. This means that the driver is always aware of the extent to which of the driving tasks 201, 202, 203 he is currently unable to perform.The warnings are combined into clear instructions, without the different warnings being issued to the driver in isolation. This conveniently increases driving safety.

[0038] Fig. 3 shows a flowchart of a (possibly computer-implemented) method 300 for assisting the driver of a (motor) vehicle 100 in performing driving tasks 201, 202, 203 to be performed by the driver. For example, it may be necessary for the driver to perform a certain number of different driving tasks 201, 202, 203 (e.g., two or more, or three or more different driving tasks). The individual driving tasks 201, 202, 203 can each be associated with a (possibly different) maximum permissible period of time for not performing the respective driving task 201, 202, 203.

[0039] The method 300 comprises detecting 301, at a check time 205, that the maximum permissible time period for not performing a first driving task 201 (from the set of driving tasks 201, 202, 203) is exceeded.

[0040] Furthermore, method 300 includes identifying 302 one or more additional driving tasks 202 (from the set of driving tasks 201, 202, 203) that are not being performed by the driver at the check time 205. For the one or more additional driving tasks 202, the maximum permissible time period for not performing the respective driving task 202 may not yet have been exceeded.

[0041] Method 300 further includes causing 303 a combined notification to be issued to the driver regarding the first driving task 201 and the one or more additional driving tasks 202. The driver of vehicle 100 can be informed of the entire list of driving tasks 201, 202 that are not currently being performed and those that are to be performed. This conveniently increases the safety of driving operation of vehicle 100.

[0042] The present invention is not limited to the embodiments shown. In particular, it should be noted that the description and figures are intended only to illustrate the principle of the proposed methods, devices, and systems by way of example.

Claims

[1] Device (101) for assisting a driver of a vehicle (100) in carrying out driving tasks (201, 202, 203) to be carried out by the driver; wherein the device (101) is arranged - to detect at a check point in time (205) that a maximum permissible period of time for not performing a first driving task (201) is exceeded; - to identify one or more further driving tasks (202) that are not performed by the driver at the check time (205); and - to cause a combined indication to be issued to the driver with respect to the first driving task (201) and with respect to the one or more further driving tasks (202). [2] Device (101) according to claim 1, wherein the combined indication also relates to the one or more further driving tasks (202) if a maximum permissible period of time for not carrying out the one or more further driving tasks (202) has not yet been exceeded at the checking time (205). [3] Device (101) according to one of the preceding claims, wherein the combined indication is directed to induce the driver to perform the first driving task (201) and the one or more further driving tasks (202). [4] Device (101) according to one of the preceding claims, wherein the driving tasks (201, 202, 203) comprise - monitoring of a driving operation of the vehicle (100); - a minimum level of attention required with regard to the operation of the vehicle (100); and / or - touching and / or gripping a steering wheel (107) of the vehicle (100) with one or both hands. [5] Device (101) according to one of the preceding claims, wherein the device (101) is arranged - to determine sensor data from one or more driver sensors (106) of the vehicle (100); wherein the one or more driver sensors (106) comprise in particular an interior camera and / or a hands-on sensor; and - to determine on the basis of the sensor data of the one or more driver sensors (106) whether a driving task (201, 202, 203) is being carried out by the driver or not. [6] Device (101) according to one of the preceding claims, wherein the device (101) is arranged - to update a time counter for a plurality of different driving tasks (201, 202, 203) repeatedly, in particular periodically; wherein the time counter for a driving task (201, 202, 203) indicates the period of time since which the driving task (201, 202, 203) has not been performed by the driver; and - to detect, on the basis of the time counter for the first driving task (201, 202, 203), that the maximum permissible time period for not carrying out the first driving task (201) is exceeded at the check time (205). [7] Device (101) according to one of the preceding claims, wherein the device (101) is arranged - to detect at a time subsequent to the checking time (205) that a driving task (201, 202) which was not performed by the driver at the checking time (205) is performed by the driver at the subsequent time; and - in response thereto, adapt the issued indication such that the reference to the driving task (201, 202) which is again carried out by the driver at the subsequent time is removed from the indication. [8] Device (101) according to one of the preceding claims, wherein the device (101) is configured to detect, during operation of an automated driving function for automated longitudinal and / or transverse guidance of the vehicle (100), that the maximum permissible time period for not performing a first driving task (201) is exceeded at the checking time (205). [9] Device (101) according to one of the preceding claims, wherein the device (101) is arranged - to identify a third driving task (203) performed by the driver at the check time (205); and - to ensure that the combined indication has no reference to the third driving task (203). [10] Method (300) for assisting a driver of a vehicle (100) in performing driving tasks (201, 202, 203) to be performed by the driver; wherein the method (300) comprises - detecting (301), at a check time (205), that a maximum permissible time period for not performing a first driving task (201) is exceeded; - identifying (302) one or more further driving tasks (202) that are not performed by the driver at the check time (205); and - causing (303) a combined indication to be output to the driver with respect to the first driving task (201) and with respect to the one or more further driving tasks (202).

Images