Method for transmitting information to a driver of a motor vehicle and adaptive driver assistance system

Abstract

Method (10) for transmitting information (11) to a driver (1) of a motor vehicle (3), wherein at least one first information channel is provided for transmitting information (11), which is designed as a first visual information transmission point (6), wherein, - the information (11) is presented to the driver (1) via the first information channel, characterized in that - a determination of a perception variable representing a perception of the driver (1) with regard to the information (11) presented by means of the first information channel is carried out, wherein a modeling of the visual perception is carried out to determine the perception variable, wherein a gaze dwell time is taken into account, and at least one further information channel is provided which is designed as a second information transmission point (6, 6', 7, 8), wherein the first information channel and the at least one further information channel are different information channels, wherein - a definition of a transmission of the information (11) by means of the further information channel is carried out on the basis of the determined perception variable with regard to the information (11) presented by means of the first information channel, wherein, if the perception variable represents a perception of the information (11) by the driver (1), the definition of the transmission of the information (11) by means of the further information channel comprises at least one of the following steps: - Failure to transmit the information (11) via another information channel, and / or - Adaptation of the transmission of information via another information channel.

Description

The present invention relates to a method for transmitting information to a driver of a motor vehicle, at least two information channels being provided for transmitting information. The invention further relates to an apparatus and a computer program which is configured to carry out the method and to a machine-readable storage medium on which the computer program is stored.Prior ArtEye tracking systems are known from the prior art, for example. Eye tracking is the recording of a person's gaze movements, which consist mainly of fixations (points that are observed accurately), saccades (fast eye movements) and regressions. The term "eye tracker" refers to devices and systems that record and allow analysis of gaze movements and gaze directions. Modern eye trackers consist of near-infrared cameras which generate reflections on the retina with an illumination unit. The calibrated positions of pupil and reflections allow accurate detection of the driver's gaze direction. FIG. 1 shows a schematic model for calibrating such a system. In this case, different positions of a pupil with reflections are shown by means of A. C is an illumination unit for illuminating the pupil. B shows a camera system for recording the pupil and the reflections. D represents a visual focus field. Here, for different image points on the focus field, the respective pupil position and reflection are recorded with the camera. From these data, an assignment of the different image points and pupil positions takes place by means of a transfer function E. The correspondingly observed image points can be determined from this for future pupil positions.Furthermore, the prior art discloses the patent application EP 2 325 722 A. This document relates to a method of modulating operation of a device, comprising:arranging a hardware sensor in or on the device for detecting attention of a user with respect to the device;processing a signal from the hardware sensor and outputting a metric or index of attention of the user with respect to the device; andadapting the operation of the device based on the metric or index of attention of the user to the device;wherein the adjustment is initiated by the device and comprises a notification and / or information and / or communication to the user based on the attention of the user to the device.Furthermore, the prior art discloses the patent application GB 2 500 690 A. This document relates to a method for assisting the driver of a motor vehicle. In this case, for example for transmitting information to the driver, the information suitable for the respective situation is selected from a plurality of possible information channels, for example visually, haptically, acoustically.Furthermore, the prior art discloses the patent application EP 2 513 882 B1. This document relates to methods for providing an accident warning alarm in a vehicle. In this case, for example, the intensity of a warning is increased if the driver is not looking at the roadway, or a warning is reduced if the driver is looking at the roadway.US 2015 / 0 015 479 A1 relates to a mobile terminal which is connectable to a video display device arranged in a vehicle and to a control device for controlling the same. A mobile terminal capable of controlling the output of a screen displayed on a video display device disposed in a vehicle according to the line of sight of the driver without interfering with driving and a control method thereof are provided. It is also provided to provide a mobile terminal capable of outputting video displayed on a display on which the eyes of the driver are fixed to another video display device within a vehicle to enable a passenger to share the video seen by the driver, and a control method thereof.DE 10 2012 221 090 A1 relates to a system and method for closed-loop driver attention management in order to manage the attention of the driver during driving. The driver may drive using an autonomous driving system or an autonomous driving capability limited system. The method may detect characteristics of a driver by a sensor; determine whether the driver is exerting sufficient monitoring control of the vehicle based on the detected characteristics of the driver; and provide a series of one or more prompts, wherein the infiltrateability of each prompt is related to the determination whether the driver is exerting sufficient monitoring control of the vehicle.US 2009 / 0 022 368 A1 relates to a monitoring device, a monitoring method, a control device, a control method and a corresponding program. When a driver in a vehicle traveling straight is looking sideways and drives his gaze in a direction that is clearly different from the forward direction for a long time, this means that the driver is driving inattentively, but a method of recognizing the state of the driver only from the face direction cannot recognize the state of inattentive driving. However, by independently detecting the face direction and the gaze direction of the driver, it can be determined that the driver does not adequately watch the front side when the gaze direction changes by deviating from a predetermined range in the front side even when the face direction is directed to a substantially constant direction of the front side of the vehicle.Disclosure of the InventionIn contrast, the method according to the invention advantageously enables an information presentation adapted to the actual currently present demand for information of the driver. Such an adaptive information transmission advantageously takes into account whether or not the driver has already perceived information.This is made possible according to the invention by the features specified in patent claim 1 and patent claims 7 to 10. Further embodiments of the invention are the subject matter of dependent claims.The method according to the invention for transmitting information to a driver of a motor vehicle, wherein at least two information channels are provided for transmitting information, is characterized by the following steps:presenting the information to the driver by means of at least one first information channel,determining a perception variable representing a perception of the driver with respect to the presented information,defining a transmission of the information by means of a further information channel on the basis of the determined perception variable.This is understood to mean that advantageously a transmission of information adapted to the actual need of the driver can take place. This can also be referred to as adaptive information transmission. For this purpose, the driver is first provided with the corresponding information about, i.e., by means of, a first information channel. An information channel can be understood to mean any way of offering and transmitting information. The potential information channels are defined by the sensory perception options of the driver. Therefore, visual, audio, haptic, or even olfactory channels and others are possible. The information channels that can actually be used are, of course, limited by the technical devices present in the vehicle. The practical implementation of an information channel is effected by means of an information transmission point.A visual presentation of the information can be effected, for example, by means of a display. For this purpose, it is advantageously possible to use displays already installed in the vehicle, such as, for example, the instrument cluster or the display of the center console or else a head-up display.The presentation of the information by means of the first information channel can take place, as indicated, via one of these information channels. Furthermore, it is of course also possible for the presentation of the information to take place via a plurality of channels. Due to the technical implementation of the channels by means of information transmission points, different information channels can differ both with regard to their sensory perception-for example visually (display) and auditively (loudspeaker)-and with regard to their structural implementation-for example two displays as two different visual information channels.Furthermore, the method according to the invention advantageously provides that a check is made as to whether the driver has perceived or is perceiving the presented information. For this purpose, a perception variable representing the perception is determined and analyzed. Various approaches are possible for determining whether the information offered has been perceived. In this case, for example, a focusing of the driver on the information presentation, for example the viewing direction and the duration of the view stay on a visual presentation, can be analyzed. Alternatively or additionally, a reaction of the driver can also be observed and analyzed, i.e. whether he carries out an action which indicates a perception of the presented information. In the context of estimating the perception, both a positive perception and a missing perception by the driver can be recognized.On the basis of the estimated perception, a transmission of the information can be defined by means of a further information channel. The definition of the transmission can provide, for example, if a lack of perception is detected, a presentation and, if applicable, an amplification of the presentation of the information by means of a further information channel. This advantageously makes it possible to ensure that the driver perceives the corresponding information. However, the definition of the transmission can also provide, for example in the case of a recognized positive perception, omission, or, if appropriate, attenuation or deactivation of an additional presentation. This advantageously allows presentation of redundant information to be reduced or even avoided. Redundant information is to be understood as the information which is presented once again to the driver, although these have already been perceived by the driver or are aware of the driver.The definition of the transmission can also comprise the step of implementing the definition of the transmission, i.e. executing the transmission of the information by means of the further information channel. Alternatively, this step may advantageously follow the definition step. This realizes the actual presentation of the information by means of the further information channel and thereby optimizes and finalizes the transmission of the information to the driver.The term "transmission of the information" is to be understood to mean that this is in particular the same information which is to be transmitted to the driver, if appropriate adapted to the respective information channel. Therefore, the perception or the perception variable representing the perception of the driver can also be understood as a perception with respect to the information transmitted by means of the first information channel.The method is further characterized in that the first information channel is configured as a first visual information transmission point.This is understood to mean that the presentation of the information takes place within the scope of the first step of the method by means of a visual information transmission. In this case, displays can be used advantageously. In addition to the instrument cluster, it is also possible, for example, to use head-up displays or displays which are positioned in or around the central console. The visual information transmission point thereby enables a unidirectional information flow. Of course, it is also possible to use displays which enable a bidirectional flow of information, for example touch displays. In additional or alternative embodiments, audio or haptic information transmission points can also be used as the first information channel.In a preferred development, the method is characterized in that the further information channel is configured asan audio information transmission point and / oras a second visual information transmission point and / oras a haptic information transmission point.The term "further information channel" is to be understood with reference to step 3 of the method, in which a transmission of the information by means of a further information channel is defined on the basis of the determined perception. The described development is to be understood such that the definition of the transmission of the information is advantageously carried out by means of an audio information transmission point, for example a loudspeaker. In an advantageous alternative, the further information channel can likewise be designed as a visual information transmission point, for example as a further display. In an alternative embodiment, the further information channel is designed for transmitting the information as a haptic information transmission point. Suitable haptic information transmission points are, in particular, components with which the driver is in contact, for example the steering wheel, the brake or accelerator pedal, and the driver's seat. For example, information can be transmitted to the driver by a predefined steering movement or independent movement of the brake pedal.The method is also characterized in that, if the perception variable represents a perception of the information by the driver, the definition of the transmission of the information by means of the further information channel comprises at least one of the following steps:omitting a transmission of the information by means of a further information channel, and / oradapting a transmission of the information by means of a further information channel, wherein in particular the presentation of the information is attenuated.This is understood to mean that, if, within the scope of the estimation of the perception, a positive, i.e. actual perception by the driver is determined, one of the steps specified is carried out. These steps can be understood as defining and converting the transmission of the information by means of the further information channel. In this case, it is provided that no transmission of the information takes place by means of a further information channel. If a presentation by means of a further information channel is already present, it can alternatively also be provided that the transmission is adapted by means of this information channel. In this case, for example, an attenuation of the presentation can advantageously be provided. This can be effected, for example, in the case of visual representations by changing the colors, dimming the representation, or (semi) Of the elements shown. The method thus advantageously enables redundant information to be determined on the basis of the perception variable and redundant information to be omitted by means of a further information channel and / or redundant information to be adapted by means of a further information channel, wherein in particular the presentation of the redundant information is attenuated. Redundant information is to be understood as meaning duplicated information which has already been perceived by the driver and / or is known to the driver. The redundancy is of course not limited to a specific information channel, i.e. even when the same information is presented by means of different information channels, redundancy can occur as soon as the driver has perceived the information. A presentation of already perceived, i.e. redundant, information via a further information channel is superfluous. By avoiding the presentation of such information, a reduction in the information burden on the driver can be achieved. Furthermore, this advantageously reduces driver distraction by means of further information. It is also conceivable that new functions can thereby achieve a higher user acceptance. A further advantage is the energy saving in electrical display devices-these are only switched on or switched to full brightness if the presentation of the information for the driver is actually necessary, since the driver has not yet perceived it. Furthermore, avoiding disruptive light sources and reflections is achieved by means of display devices in a vehicle, in particular during night driving.In an alternative advantageous embodiment, the method is characterized in that if the perception variable does not represent a perception of the information by the driver, the definition of the transmission of the information by means of the further information channel comprises at least one of the following steps:transmitting the information by means of a further information channel and / oradapting the transmission of the information by means of a further information channel, wherein in particular an amplification of a presentation of the information takes place.This is understood to mean that if, in the context of estimating the perception, a negative, i.e. no perception by the driver is determined, one of the steps specified is carried out. In this case, it is provided that additional transmission of the information takes place by means of a further information channel. If a presentation by means of a further information channel is already present, it can alternatively also be provided that the transmission is adapted by means of this information channel. Here, for example, an enhancement of the presentation can be provided advantageously. This can be achieved, for example, in the case of visual representations by changing the colors, lightening the presentation, stronger contrasts or other visual effects, for example. The elements shown are flashed. In the case of audio presentations, it is advantageously possible to provide for repetition and / or an increase in the volume. It should also be noted that the terms "no perception" or "perception" relate in particular to the perception by the driver on the basis of the transmission by means of the at least first information channel.In a preferred embodiment, the method is characterized in that at least one of the following factors is taken into account for ascertaining the perception variable:a viewing direction of the driver and / oran action of the driver.This is understood to mean that the ascertainment of the perception can take place separately from an active confirmation by the driver. Such a confirmation could be effected, for example, by means of a voice comment or the pressing of a confirmation button. By avoiding such an active confirmation, the distraction of the driver as well as the working actions of the driver can be advantageously reduced. Indirect determination of the perception of presented information can therefore reduce the burden on the driver and increase the safety of the vehicle guidance. An indirect determination of an expected perception can be made, for example, by means of the viewing direction of the driver. This can be used advantageously in particular for visual information transmission points, wherein the viewing direction has to be made, for example, to the presented information of the information interface. The viewing direction can be determined, for example, by means of an interior camera and conventional eye tracking methods. If the driver, for example, looks at visual information represented by means of the first information channel, a perception of the information can be assumed in an estimate.In an alternative embodiment, an indirect determination of a probable perception can be made, for example, by means of an observation and analysis of the behavior and the actions of the driver. If an action takes place within a defined time period after presentation of an information item, which action can be evaluated as a reaction to the transmitted information item, this can be seen as evidence for perception in a first estimate. By way of example, the following actions for the information "Pending change of direction" can be seen within the scope of navigation: placing in a suitable lane for turning, turning signal, setting shoulder gaze, etc.In addition, the method is characterized in that, in order to determine the perception variable, the visual perception is modeled, a duration of eye stay being taken into account.This is understood to mean that algorithms for modelling visual attention with respect to objects can advantageously be used within the scope of the method. Such algorithms run, for example, on a control device. Here, an estimation of the perception can be made, for example, by the viewing direction and the dwell time of the view. Here, the measurement of the eye dwell time refers in particular to what are known as ROIs (regions of interest), for example the positions at which the relevant information is displayed. It is clear from this that the eye stay duration relates to the duration of the driver's eye looking in a specific direction or to a specific point. In this case, a dwell time of greater than or equal to 1 second, for example, can indicate an attentive user. This corresponding presented information could thus be assumed to be "perceived".In one possible embodiment, the method is characterized in that the perception variable is determined by means of an interior sensor system, in particular by means of an interior camera.This is understood to mean that, for example, an interior camera is used within the scope of the method. This camera enables at least parts of the driver, in particular his face and his eyes, to be recorded. In other words, it is possible, with the aid of an interior sensor system (for example a driver observation camera for eye tracking), to recognize the driver's gaze direction (=visual attention) with respect to objects and / or displays and to suppress or reinforce audio and / or visual and / or haptic indications by a human-machine interface accordingly.Two examples may be given by way of example:Navigation system has taken a route planning-in 100 m, the driver is to turn right. The driver looks at route description which is displayed on the display of the navigation device. A separate audio output of the directions ("turn right now") is suppressed, since the driver has demonstrated his visual attention by looking at the display.Driver continuously looks at speed indication in the HUD. The speed display in the instrument cluster has been left out of view for some time. Therefore, the display in the instrument cluster is shaded stepwise and is only lightened again when the driver explicitly looks at the instrument cluster.The method according to the invention, which is also provided for controlling a driver assistance system with a transmission of information to a driver of a motor vehicle, wherein at least two information channels are provided for transmitting information, is characterized in that using the method described abovepresenting the information to the driver by means of at least one first information channel,determining a perception variable representing a perception of the driver with respect to the information presented, anda definition of a transmission of the information by means of a further information channel is effected on the basis of the detected perception variable.This is to be understood as meaning a driver assistance system by means of which, by using the method described, transmission of non-perceived information to the driver is assisted and transmission of redundant information can be avoided.Furthermore, according to the invention, a device is provided which has means and is set up to carry out the method. A device for a driver assistance system for implementing the described method may be considered as a device, for example. Furthermore, an interior camera, for example, for implementing the described method can be considered as a device.The method can be implemented, for example, in software or hardware or in a mixed form of software and hardware, for example in a control device. The approach presented here furthermore creates a device which is designed to carry out, actuate or implement the steps of a variant of a method presented here in corresponding devices. This embodiment variant of the invention in the form of a device also enables the object on which the invention is based to be achieved quickly and efficiently.Such a device can be understood in the present case as an electrical device which processes sensor signals and outputs control and / or data signals as a function thereof. The device can have an interface which can be designed as hardware and / or software. In the case of a hardware configuration, the interfaces can be part of a so-called system ASIC, for example, which contains a wide variety of functions of the device. However, it is also possible for the interfaces to be dedicated, integrated circuits or to consist at least partially of discrete components. In the case of a software configuration, the interfaces can be software modules which are present, for example, on a microcontroller in addition to other software modules.A computer program product or computer program with program code which can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and is used for carrying out, implementing and / or controlling the steps of the method according to one of the embodiments described above is also advantageous, in particular if the program product or program is executed on a computer or a device.EmbodimentsIt should be noted that the features individually listed in the description can be combined with one another in any technically meaningful manner and reveal further embodiments of the invention. Further features and functionality of the invention will become apparent from the description of exemplary embodiments with reference to the appended figures.Of the figures, FIG. 1 shows a schematic model for calibrating an eye tracking system; and FIG. 2 shows a schematic section of a driver in a motor vehicle in an embodiment of the method, and FIG. 3 shows a schematic flow chart of the steps of the method.FIG. 1 shows a schematic model for calibrating an eye tracking system. A description of this calibration as well as the technology of eyetracking can be found in the statements relating to the prior art.FIG. 2 shows a schematic section of a driver 1 in a motor vehicle 3 in an embodiment of the method 10, or 10'. In this case, the driver 1 is positioned in the seat position in the vehicle 3. The vehicle 3 further contains a driver assistance system 4. According to method 10 or 10', information 11 is transmitted to driver 1 by means of a first information channel. This information channel is designed in the exemplary embodiment as a visual information channel by means of a display 6. The display 6 is, for example, the display of the center console, a display of the instrument cluster or a separate display.The eyes 2 of the driver 1 are monitored by means of an interior sensor system 5. This interior sensor system 5 is designed as a camera with active infrared illumination for ascertaining the viewing direction ("Eyetracking"). This allows the ascertainment of gaze vectors and the eye position to be carried out. By using eye tracking methods, it is possible to check where the driver 1 is looking and which elements the driver 1 is looking at. This makes it possible, for example, to check whether the driver 1 has viewed information 11 visually displayed by means of an HMI driver display 6. Furthermore, an estimate is made as to whether the driver 1 has perceived the information 11. For this purpose, the time interval of the observation will be taken into account, among other things.In addition, a further information channel is provided. In the exemplary embodiment, this is designed as an audio information channel by means of the loudspeakers 7 of the vehicle. According to the perception by the driver 1 of the information 11 transmitted via the first information channel, i.e. the display 6, a transmission of the information 11 via a second information channel is defined and implemented. If, for example, the driver cannot assume that the information 11 displayed on the display 6 is perceived on the basis of the data ascertained by eye tracking, the information 11 can be offered further via the audio path through the loudspeaker 7 to the driver 1 by voice output. However, if it is determined that the driver 1 already perceives the information 11 during a presentation by means of the display 6, an additional audio presentation of the information by means of the loudspeaker 7 can be dispensed with.Furthermore, an additional information channel is provided. In the exemplary embodiment, this is represented as a haptic information channel by means of a further information interface 8, which is configured, for example, as a steering wheel vibration and / or a steering wheel movement. Information is also provided via this route only if the evaluation of the data suggests that the driver 1 has not perceived the information 11 presented on the display.Furthermore, a further visual information interface 6' is represented by a further display. If, for example, it is determined that the driver 1 has not perceived information 11, this information can also be duplicated via this information channel, which is also visual, and can additionally be presented to the driver 1. This is omitted if the driver 1 has already perceived the information 11.The control of the information channels is effected by means of a regulating and control unit 9. This can be the same or a different regulating and control device 9. The control device 9 transmits, for example, via an interface (for example. CAN) commands for HMI adaptation to the control units of affected objects (for example. Navigation system, instrument cluster).FIG. 3 shows a schematic flow chart of the steps of the method 10, 10' according to an exemplary embodiment. In a first step S 1, the presentation of the information to the driver takes place by means of at least one first information channel. In a second step S 2, a perception variable representing a perception of the driver is determined with respect to the presented information. In a third step S 3, the information is defined by means of a further information channel on the basis of the determined perception variable. This third step S 3 can be followed by a fourth step S 4, which comprises the transmission of the information by means of a further information channel.

Claims

Method (10) for transmitting information (11) to a driver (1) of a motor vehicle (3), wherein at least one first information channel is provided for transmitting information (11), which is designed as a first visual information transmission point (6), wherein - the information (11) is presented to the driver (1) by means of the first information channel, characterized in that - a determination of a perception variable representing a perception of the driver (1) is made with respect to the information (11) presented by means of the first information channel, wherein a modeling of the visual perception is carried out for determining the perception variable, wherein a viewing dwell time is taken into account, and at least one further information channel is provided, which is designed as a second information transmission point (6, 6', 7, 8), wherein the first information channel and the at least one further information channel are different information channels, wherein - a definition of a transmission of the information (11) by means of the further information channel is carried out on the basis of the ascertained perception variable with respect to the information (11) presented by means of the first information channel, wherein, if the perception variable represents a perception of the information (11) by the driver (1), the definition of the transmission of the information (11) by means of the further information channel comprises at least one of the following steps: - omission of a transmission of the information (11) by means of a further information channel, and / or - adaptation of a transmission of the information by means of a further information channel.Method (10) according to claim 1, characterised in that the further information channel is configured as - an audio information transmission point (7) and / or - a second visual information transmission point (6, 6') and / or - a haptic information transmission point (8).Method (10) according to one of the preceding claims, characterized in that - the presentation of the information (11) is attenuated when the transmission of the information is adjusted by means of a further information channel.Method (10) according to one of Claims 1 to 3, characterized in that, if the perception variable does not represent a perception of the information (11) by the driver (1), the definition of the transmission of the information (11) by means of the further information channel comprises at least one of the following steps: - transmission of the information (1) by means of a further information channel and / or - adaptation of the transmission of the information (11) by means of a further information channel, wherein in particular an amplification of a presentation of the information (11) takes place.Method (10) according to one of the preceding claims, characterized in that at least one of the following factors is taken into account for determining the perception variable: - a viewing direction of the driver (1) and / or - an action of the driver (1).Method (10) according to one of the preceding claims, characterized in that the perception variable is determined by means of an interior sensor system (5), in particular by means of an interior camera.Method (10') for controlling a driver assistance system with a transmission of information (11) to a driver (1) of a motor vehicle (3), wherein at least two information channels are provided for transmitting information (11), characterized in that, using the method of at least one of Claims 1 to 6, - the information (11) is presented to the driver (1) by means of at least one first information channel, - a determination of a perception variable representing a perception of the driver (1) with respect to the presented information (11), and - a definition of a transmission of the information (11) is carried out by means of a further information channel on the basis of the determined perception variable.Device (9) which is configured to carry out the method (10) according to one of Claims 1 to 6 and / or the method (10') according to Claim 7.Computer program configured to execute the method (10) according to one of claims 1 to 6 and / or the method (10') according to claim 7.A machine readable storage medium having stored thereon the computer program of claim 9.

Images