Method and device for assisting a driver of a vehicle in triggering an automated driving manoeuvre

Abstract

The invention relates to a device for operating a driving function for the automated longitudinal and / or lateral guidance of a vehicle. The device is designed, during operation of the driving function, to determine that the driver of the vehicle can initiate the execution of an automated driving manoeuvre by directing their gaze towards a specific location. The device is also designed, in response to the determination, to cause a virtual assistant presented via a user interface of the vehicle to move their gaze towards the specific location.

Description

[0001] 24-3125

[0002] - 1 -

[0003] Method and device for assisting a driver of a vehicle in initiating an automated driving maneuver

[0004] The invention relates to a method and a corresponding device aimed at assisting the driver of a vehicle in carrying out, in particular initiating, an automated driving maneuver.

[0005] A vehicle may have one or more driving systems, in particular driver assistance systems, designed to automatically guide the vehicle longitudinally and / or laterally. The vehicle may, in particular, have a lane change assistant designed to initiate an automated lane change of the vehicle to another lane. Alternatively or additionally, the vehicle may have a turning assistant designed to initiate an automated turning maneuver of the vehicle at a road intersection.

[0006] This document addresses the technical challenge of enabling a particularly comfortable and reliable execution of an automated driving maneuver, in particular an automated lateral control maneuver, by a vehicle. 24-3125

[0007] - 2 -

[0008] The problem is solved by each of the independent claims. Advantageous embodiments are described, inter alia, in the dependent claims. It should be noted that additional features of a claim dependent on an independent claim, without the features of the independent claim itself or only in combination with a subset of the features of the independent claim, can constitute a separate invention independent of the combination of all features of the independent claim, which can be made the subject of an independent claim, a divisional application, or a subsequent application. This applies equally to technical teachings described in the description, which can constitute an invention independent of the features of the independent claims.

[0009] According to one aspect, a device for operating a driving function for the automated longitudinal and / or lateral control of a (motor) vehicle is described. Within the scope of operating the driving function, automated longitudinal and / or lateral control of the vehicle can be effected (e.g., to keep the vehicle automatically within its current lane). Alternatively or additionally, the driving function can perform one or more automated driving maneuvers. The execution of the one or more automated driving maneuvers may depend on whether the execution was previously initiated or triggered by the vehicle's driver. If necessary, an automated driving maneuver can only be performed if it has been previously initiated by the driver. This initiation can, if applicable, be triggered by the driver's gaze.

[0010] The device may be configured to recognize, during the operation of the driving function (in which automated longitudinal and / or lateral control of the vehicle may already be taking place), that an automated driving maneuver can be performed. Furthermore, the device may be configured to

[0011] - 3 -

[0012] To determine that the driver of the vehicle can initiate the automated driving maneuver by focusing their gaze on a specific point (of the vehicle). In particular, it may be necessary for the driver to first focus their gaze on a specific point (e.g., to perform a check) in order to execute the automated driving maneuver. This can ensure a particularly safe execution of the automated driving maneuver.

[0013] The specified location may include a rearview mirror, in particular the left or right-hand rearview mirror of the vehicle. The driving maneuver may include a lateral maneuver, in particular a lane change or a turning maneuver.

[0014] The device is further configured, in response to the driver's stipulation (that the driver can initiate the automated driving maneuver by directing their gaze to a specific point), to cause a virtual assistant, displayed via the vehicle's user interface (in particular, visually represented), to move its gaze (starting from a neutral position) towards that specific point. The virtual assistant can take the form of a (human) avatar. The virtual assistant can be displayed on a screen of the user interface, preferably facing the driver.

[0015] The device can be configured to cause a virtual assistant with one or more eyes to be displayed via the user interface. Furthermore, in response to the determination (that the driver of the vehicle can initiate the execution of the automated driving maneuver by directing their gaze to a specific location), it can cause the one or more eyes of the virtual assistant (starting from a neutral position) to move towards that specific location.24-3125

[0016] - 4 -

[0017] so that the viewer of the virtual assistant (especially the driver of the vehicle) gets the impression that the virtual assistant is moving its gaze towards the specific spot.

[0018] The one or more eyes can be positioned horizontally midway between a maximum rightward and a maximum leftward deflection in the neutral position, and / or vertically midway between a lower and an upper maximum deflection of the one or more eyes. Alternatively, the one or more eyes can be directed towards the driver of the vehicle in the neutral position.

[0019] The gaze direction of the virtual assistant can thus be used to guide the driver of the vehicle comfortably and reliably to direct their gaze (corresponding to the gaze of the virtual assistant) to a specific point, in particular to perform a check and / or to trigger the execution of an automated driving maneuver. This allows for a particularly comfortable and safe execution of the automated driving maneuver.

[0020] The device can be configured to determine position data relating to the driver, in particular the position of the driver's head, and the vehicle based on camera data from an interior camera (facing the driver). Depending on this position data, the device can then neutralize the gaze of the virtual assistant, specifically by directing its gaze towards the driver, and in particular towards the driver's head. Thus, the device can ensure that the virtual assistant, in its neutral position, directs its gaze towards the driver (and looks "in the driver's eyes"). 24-3125

[0021] - 5 -

[0022] The device can further be configured to cause the virtual assistant to move its gaze from the neutral position to that specific point in response to a setting (that the driver can initiate the automated driving maneuver by directing their gaze to a particular location). This ensures, with particular reliability, that the driver's gaze follows that of the virtual assistant in order to perform the check and / or initiate the automated driving maneuver.

[0023] The device can be configured to determine gaze information regarding the driver's gaze (resulting from changes in the virtual assistant's gaze) based on camera data from the vehicle's interior camera. This gaze information can then be used to determine whether the automated driving maneuver was initiated by the driver. The automated driving maneuver can be executed (possibly only if it has been determined that the driver initiated it (by directing their gaze towards the specific point).

[0024] On the other hand, the automated driving maneuver can be prevented if it is determined that it was not initiated by the driver (because the driver did not focus their gaze on the specific point). Monitoring the driver's gaze can ensure a comfortable, reliable, and safe execution of the automated driving maneuver.

[0025] The device can be configured to repeatedly check, during the operation of the driving function, at a sequence of successive time points, whether the automated driving maneuver can still be carried out at the respective time point in the sequence of successive time points (e.g., whether the gap for a lane change is still available at the respective time point or not). 24-3125

[0026] - 6 -

[0027] At each point in the sequence of successive time points, the virtual assistant can be made to refocus its gaze from the neutral position to the specified location if it has been determined that the automated driving maneuver can still be performed at that particular time. Conversely, at each point in time, the virtual assistant can be made to maintain its gaze in the neutral position if it has been determined that the automated driving maneuver can no longer be performed at that particular time. By repeatedly adjusting the virtual assistant's gaze to the changing traffic situation, a particularly comfortable and safe execution of the automated driving maneuver can be achieved.

[0028] According to another aspect, a (road) motor vehicle (in particular a passenger car or a truck or a bus or a motorcycle) is described that includes the device described in this document.

[0029] According to one aspect, a method for operating a driving function for automated longitudinal and / or lateral control of a (motor) vehicle is described. The method includes determining, within the context of operating the driving function, that the driver of the vehicle can initiate the execution of an automated driving maneuver by directing their gaze to a specific point. Furthermore, in response to this determination, the method includes causing a virtual assistant, displayed via a user interface of the vehicle, to move its gaze to the specified point (in order to induce the driver of the vehicle to imitate the virtual assistant and also direct their gaze to the specified point).

[0030] It should be noted that the aspects described in connection with the device, in particular those described in connection with the device24-3125

[0031] - 7 -

[0032] Claims, as corresponding procedural features, are also applicable to the procedure.

[0033] Another aspect described is a software (SW) program. The SW program can be configured to run on a processor (e.g., on a vehicle's control unit) and thereby execute the procedure described in this document.

[0034] Another aspect describes a storage medium. This storage medium can include a software program configured to run on a processor and thereby execute the procedure described in this document.

[0035] It should be noted that the methods, devices, and systems described in this document can be used both alone and in combination with other methods, devices, and systems described in this document. Furthermore, any aspect of the methods, devices, and systems described in this document can be combined with one another in a variety of ways. In particular, the features of the claims can be combined with one another in a variety of ways. Features listed in parentheses are to be understood as optional features.

[0036] The invention will now be described in more detail using exemplary embodiments.

[0037] Figures 1a and 1b are exemplary components of a vehicle;

[0038] Figures 2a and 2b show an example of a vehicle's virtual assistant; and

[0039] Figure 3 shows a flowchart of an exemplary procedure for assisting the driver of a vehicle in initiating an automated

[0040] Driving maneuvers.24-3125

[0041] - 8 -

[0042] As stated at the outset, this document deals with increasing comfort and / or reliability when initiating an automated driving maneuver. In this context, Fig. 1a shows an exemplary vehicle 100 with one or more environmental sensors 102, each configured to acquire sensor data (also referred to as environmental data) relating to the vehicle 100's surroundings. Exemplary environmental sensors 102 are a camera, a radar sensor, a lidar sensor, and / or an ultrasonic sensor.

[0043] A (control) device 101 of the vehicle 100 can be configured to evaluate environmental data, e.g., to detect one or more objects (e.g., other vehicles) in the vicinity of the vehicle 100. The device 101 can further be configured to effect automated longitudinal and / or lateral control of the vehicle 100 based on the environmental data, in particular based on the one or more detected objects. For this purpose, one or more longitudinal and / or lateral control actuators 103 (e.g., a drive motor, a braking device, and / or a steering device) of the vehicle 100 can be controlled.

[0044] The (control) device 101 can thus be configured to provide a driving function for automated longitudinal and / or lateral control of the vehicle 100. During operation of the driving function, automated lateral control maneuvers can be performed, such as an automated lane change to another lane or an automated turning maneuver. The (control) device 101 can be configured to detect that an automated lateral control maneuver is to be performed or can be performed. In response, the driver of the vehicle 100 can receive a (visual and / or acoustic) indication regarding the automated lateral control maneuver to be performed via the user interface 104 of the vehicle 100. Furthermore, the driver can be notified via the user interface 10424-3125

[0045] - 9 -

[0046] The user will be prompted to provide input indicating whether or not the automated lateral guidance maneuver should be performed.

[0047] The user input for triggering the automated lateral control maneuver can, for example, consist of the driver looking in a rearview mirror 105 of the vehicle 100. A glance in the right rearview mirror 105 can trigger a lateral control maneuver to the right, and / or a glance in the left rearview mirror 105 can trigger a lateral control maneuver to the left. The direction of the driver's gaze in vehicle 100 can be determined based on camera data from an interior camera 106 of vehicle 100 that is directed towards the driver.

[0048] If it is detected that the driver of vehicle 100 is making a user input to trigger the lateral control maneuver, e.g., by looking in the corresponding rearview mirror 105, the control device 101 can initiate the automated lateral control maneuver (with automated longitudinal and / or lateral control of vehicle 100). Conversely, if no triggering user input is detected, the automated lateral control maneuver can be prevented.

[0049] The output of an audible warning regarding a required lateral maneuver may be perceived as unpleasant and / or disruptive by the driver (e.g., if music is playing in the vehicle at the same time). Furthermore, the mental processing of a textual warning regarding a required lateral maneuver may be perceived as too demanding by the driver. This document describes measures that enable a particularly convenient and efficient initiation of an automated lateral maneuver.

[0050] The user interface 104 of the vehicle 100 can be configured (e.g., via a screen of the user interface 104) to display a virtual assistant 200, as exemplified in Figures 2a and 2b. The virtual assistant 200 can be represented as a (human) avatar.

[0051] In particular, the virtual assistant 200 can have a head with eyes 204. Alternatively or additionally, the virtual assistant 200 can be designed such that it has a gaze with a direction of view, whereby the direction of view can be changed.

[0052] The control device 101 can be configured to determine that the driver of vehicle 100 must direct their gaze to a specific point (e.g., a specific rearview mirror 105 of vehicle 100) in order to trigger an automated lateral control maneuver. It can then be instructed that the virtual assistant 200 directs its gaze to that specific point (in particular, moves it from a neutral position towards that specific point). For example, the eyes 204 of the virtual assistant 200 can be directed (from a neutral position) towards the specific point, e.g., towards the left or right rearview mirror 105. In this way, the driver of vehicle 100 can be conveniently and reliably informed of the user input required to initiate the automated lateral control maneuver.

[0053] For example, if the driving function suggests a lane change that the driver needs to confirm by looking in a specific direction, the virtual assistant 200 can be made to look in that direction by turning its head. The driver will then follow the changing gaze of the virtual assistant 200 (e.g., due to mirror neurons) and thus confirm the lane change. This mechanism can be used more generally for automated driving maneuvers that require the driver to keep their eyes on the road in a specific direction. For example, the driving function can determine that another vehicle needs to be overtaken and that a lane change to the left is necessary. Once a suitable gap for the lane change has been found, the virtual assistant 200 can be made to look to the left. The driver follows the virtual assistant 200 with their gaze and thus confirms the lane change.In response to the driver's safety check, the automated lane change to the left can be initiated.

[0054] Fig. 3 shows a flowchart of an exemplary (possibly computer-implemented) method 300 for operating a driving function for automated longitudinal and / or lateral guidance of a (motor) vehicle 100. The method 300 can be executed by a control device 101 of the vehicle 100.

[0055] Procedure 300 comprises determining 301, within the context of operating the driving function, that the driver of vehicle 100 can initiate the execution of an automated driving maneuver, in particular an automated lateral control maneuver, (if necessary) only by directing their gaze to a specific point (e.g., to a specific rearview mirror 105 of vehicle 100). It may be necessary for the driver to perform a preliminary check of a specific point in order to initiate the execution of the automated driving maneuver.

[0056] Furthermore, in response to the determination 301, the method 300 comprises causing 302 a virtual assistant 200, in particular an avatar, displayed via the user interface 104 of the vehicle 100, to move its gaze towards the specified location. The change in the gaze of the virtual assistant 200 (starting from a neutral position) towards the specified location can be caused to induce the driver of the vehicle 100 to imitate the virtual assistant 200, so that the driver also moves his gaze towards the specified location (in order to perform the check and / or to thereby initiate the execution of the automated driving maneuver).

[0057] The measures described in this document enable a particularly comfortable, reliable and safe initiation of automated driving maneuvers, especially lateral control maneuvers, of a vehicle 100.

[0058] The present invention is not limited to the embodiments shown. In particular, it should be noted that the description and the figures are intended only to illustrate the principle of the proposed methods, devices, and systems by way of example.

Claims

24-3125 - 13 - Claims 1) Device (101) for operating a driving function for automated longitudinal and / or lateral guidance of a vehicle (100); wherein the device (101) is configured, - within the framework of the operation of the driving function, to determine that the driver of the vehicle (100) can initiate the execution of an automated driving maneuver by directing his gaze to a specific point; and - in response to the determination, to cause a virtual assistant (200) displayed via a user interface (104) of the vehicle (100) to move its gaze towards the determined location. 2) Device (101) according to claim 1, wherein the device (101) is configured to cause that - a virtual assistant (200) with one or more eyes (204) is displayed via the user interface (104); and - in response to the determination, the one or more eyes (204) of the virtual assistant (200) are moved towards the specified location, so that an observer of the virtual assistant (200) has the impression that the virtual assistant (200) is moving its gaze towards the specified location. 3) Device (101) according to claim 2, wherein - the device (101) is configured to cause the one or more eyes (204) to move from a neutral position to the specified location; and - the one or more eyes (204) in neutral position - horizontally midway between a right-sided maximum deflection and a left-sided maximum deflection 24-3125 - 14 - maximum displacement, and / or vertically midway between a lower maximum displacement and an upper maximum displacement of one or more eyes (204) arranged; or - are directed at the driver of the vehicle (100). 4) Device (101) according to one of the preceding claims, wherein the device (101) is configured, - to determine position data relating to a position of the driver, in particular a position of the driver's head, of the vehicle (100) based on camera data from an interior camera (106) of the vehicle (100); - to effect a neutral gaze position of the virtual assistant (200) depending on the position data, in particular such that the gaze of the virtual assistant (200) in the neutral position is directed towards the driver, in particular towards the driver's head; and - in response to the determination, to cause the virtual assistant (200) to move its gaze from the neutral position to the determined location. 5) Device (101) according to one of the preceding claims, wherein the specified location comprises a rearview mirror (105), in particular a left-side or a right-side rearview mirror (105), of the vehicle (100). 6) Device (101) according to one of the preceding claims, wherein the device (101) is configured, - to determine gaze information relating to the gaze of the driver of the vehicle (100) based on camera data from an interior camera (106) of the vehicle (100);24-3125 - 15 - - to determine, based on eye-tracking information, whether the automated driving maneuver was initiated by the driver or not; and - to initiate the automated driving maneuver if it has been determined that the automated driving maneuver was initiated by the driver; and / or - to prevent the automated driving maneuver from being executed if it has been determined that the automated driving maneuver was not initiated by the driver. 7) Device (101) according to one of the preceding claims, wherein the device (101) is configured, - to repeatedly check at a sequence of successive points in time during the operation of the driving function whether the automated driving maneuver can still be carried out at the respective point in time or not; - to cause the virtual assistant (200) to move its gaze again from a neutral position to the specified location at the relevant time, if it has been determined that the automated driving maneuver can continue to be carried out at that time; and - to cause the virtual assistant (200) to maintain its gaze in the neutral position at the relevant time when it has been determined that the automated driving maneuver can no longer be carried out at that time. 8) Device (101) according to one of the preceding claims, wherein the driving maneuver comprises a lateral guidance maneuver, in particular a lane change or a turning maneuver. 9) Device (101) according to one of the preceding claims, wherein 24-3125 - 16 - - the device (101) is configured to cause the virtual assistant (200), in particular in the form of an avatar, to be displayed on a screen of the user interface (104); and - the screen is in particular facing the driver of the vehicle (100). 10) Method (300) for operating a driving function for automated longitudinal and / or lateral control of a vehicle (100); wherein the method (300) comprises, - Determine (301), within the context of the operation of the driving function, that the driver of the vehicle (100) can initiate the execution of an automated driving maneuver by directing his gaze to a specific location; and - in response to determining (301), causing (302) a virtual assistant (200) displayed via a user interface (104) of the vehicle (100) to move its gaze towards the specified location.

Images