Method for outputting a driver attention warning, computing device, and computer-readable storage medium

The method adjusts driver attention warnings based on passenger attention data, enhancing safety by reducing unnecessary alerts and improving the system's responsiveness to dynamic vehicle situations.

WO2026238332A1PCT designated stage Publication Date: 2026-11-19BAYERISCHE MOTOREN WERKE AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2026/060693
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-12
Filing Date
2026-04-14
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Existing driver attention monitoring systems in vehicles do not adequately consider the influence of passengers on the driver's attention, leading to potential premature or unnecessary warnings, especially in semi- and fully-automated driving scenarios.

Method used

A method that adjusts the driver attention warning threshold based on passenger attention data, collected through cameras and sensors, allowing for a more dynamic and situation-specific warning system by considering the interaction between the driver and passenger.

Benefits of technology

Enhances driving safety by reducing unnecessary warnings and improving the system's flexibility to handle varying attention levels within the vehicle, ensuring appropriate and timely warnings are issued.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2026060693_19112026_PF_FP_ABST
    Figure EP2026060693_19112026_PF_FP_ABST
Patent Text Reader

Abstract

One aspect of the invention relates to a method for outputting a driver attention warning to a driver of a vehicle. The method can include receiving driver attention data that can describe the driver's state of attention or the attention of the driver with respect to a traffic event. In addition, the method can include receiving passenger data that can describe physical and / or behavior-related characteristics of a passenger. Furthermore, the passenger's state of attention can be evaluated on the basis of the passenger data, wherein the passenger's state of attention can describe the attention of the passenger with respect to the traffic event. The method can also include adapting a limit value for the driver attention warning on the basis of the evaluated state of attention of the passenger. Finally, the driver attention warning can be output on the basis of the driver's state of attention and the adapted limit value.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] 25-1213

[0002] 1

[0003] METHOD FOR ISSUING A DRIVER ATTENTION WARNING, CALCULATION DEVICE AND COMPUTER-READABLE STORAGE MEDIUM

[0004] The present invention relates to a method for issuing a driver attention warning to a driver of a vehicle, a computing device for a vehicle and a computer-readable storage medium.

[0005] In particular, the invention relates to an adaptive system for monitoring driver attention while taking into account the attention of a passenger.

[0006] Driver attention monitoring systems have been known in the field of driver assistance systems for some time. These systems serve to monitor the driver's attention to traffic and, if necessary, issue warnings if the driver is deemed inattentive. However, such systems are not only important in driver assistance systems, but also during manual (non-assisted) driving as part of safety systems.

[0007] An example of this are SAE Level 2 systems, which can assist the driver with longitudinal and / or lateral control of the vehicle, provided the driver is attentive. These systems allow the vehicle to drive semi-autonomously in certain situations, but the driver must be ready to take control at any time. Continuous monitoring of driver attention is therefore a crucial safety aspect when implementing such systems.

[0008] In this document, the term "automated driving" refers to driving with automated longitudinal and / or lateral control. Automated driving can, for example, involve extended periods of driving on the highway or time-limited driving during parking maneuvers. The term "automated driving" encompasses automated driving at any level of automation. Examples of automation levels include assisted, partially automated, conditionally automated, highly automated, and fully automated driving (each with an increasing degree of automation). The five automation levels mentioned above correspond to SAE Levels 1 to 5 of the SAE J3016 standard (SAE - Society of Automotive Engineering) as of April 30, 2021. In assisted driving (SAE Level 1), the system performs longitudinal or lateral control in 25-1213

[0009] 2

[0010] Specific driving situations are addressed. In partially automated driving (SAE Level 2), the system takes over longitudinal and lateral control in certain driving situations, although the driver must continuously monitor the system, as with assisted driving. In conditionally automated driving (SAE Level 3), the system takes over longitudinal and lateral control in certain driving situations without the driver needing to continuously monitor the system; however, the driver must be able to take over vehicle control within a certain timeframe if requested by the system. In highly automated driving (SAE Level 4), the system takes over vehicle control in certain driving situations, even if the driver does not respond to a request to intervene, thus eliminating the driver as a fallback option.In fully automated driving (SAE Level 5), the system can perform all aspects of the dynamic driving task under any road and environmental conditions that can also be mastered by a human driver.

[0011] The object of the present invention is to provide a solution for improving driver attention monitoring in vehicles.

[0012] This problem is solved by the features of the independent claims. Further advantageous embodiments of the invention are specified in the dependent claims.

[0013] One aspect of the invention relates to a method for issuing a driver attention warning to the driver of a vehicle. The method can include receiving driver attention data, which may describe the driver's level of attention to a traffic situation. Furthermore, the method can include receiving passenger data, which may describe the physical and / or behavioral characteristics of a passenger. In addition, passenger attention can be assessed based on the passenger data, with the passenger attention describing the passenger's level of attention to the traffic situation. Finally, the method can include adjusting a threshold for the driver attention warning based on the assessed passenger attention.Finally, the driver attention warning can be issued depending on the driver's attention level and the adjusted threshold.

[0014] The procedure can be carried out, for example, using a computing device. The computing device can be, for example, at least one electronic 25-1213

[0015] 3

[0016] The vehicle's control unit, which includes one or more programmable processors, may also be configured as a computing device. Furthermore, the computing device may have a computer-readable storage medium on which a computer program is stored. To execute corresponding process steps, such as receiving driver attention data or evaluating passenger attention, the computer program can be run on the computing device.

[0017] Driver attention data can be collected in various ways, such as through camera systems that monitor the driver's face and gaze, or through sensors that detect steering wheel movements and other driver actions. This data can provide insights into how attentively the driver is following the traffic situation. Passenger data can also be collected through various sensors. These can include cameras that capture the passenger's posture, facial expressions, or gaze direction, as well as objects used by the passenger, and microphones that can record conversations or other sounds in the vehicle. When assessing passenger attention, it is possible to analyze the extent to which the passenger is also paying attention to the traffic situation. This can be done, for example, by evaluating the passenger's gaze direction or body posture.

[0018] Adjusting the threshold for the driver attention warning based on the assessed passenger attention allows for consideration of the overall situation in the vehicle. The driver attention warning can then be issued taking into account both the driver's attention and the adjusted threshold.

[0019] In particular, the driver attention warning can be issued when the driver's attention falls below the adjusted threshold. This allows for a more differentiated and situation-specific warning. This method enables driver attention monitoring that can take into account not only the driver but also the influence of the passenger.

[0020] Another embodiment of the method provides that adjusting the threshold may include lowering the threshold if the assessed passenger attention increases. Lowering the threshold allows a lower level of driver attention to be tolerated before the driver attention warning is issued.25-1213

[0021] 4

[0022] This design allows for a dynamic adjustment of the threshold value, reacting to the current situation in the vehicle. If the passenger's attention increases, the driver attention threshold can be lowered. This could mean that the system can tolerate a slightly lower level of driver attention if the passenger is particularly attentive.

[0023] This adaptation allows the system to react more flexibly to different situations within the vehicle. For example, it can take into account that in situations where the passenger is very attentive, a temporary decrease in the driver's attention may be acceptable. Depending on the situation, it can be assumed that an attentive passenger, keeping an eye on the traffic situation, can inform the driver about important aspects of the traffic situation and warn them if necessary. Therefore, a warning from the vehicle system may be required later or not at all in such situations. This can help avoid premature or unnecessary warnings.

[0024] Another embodiment of the method uses camera data as the passenger data, describing the passenger's posture, facial features, gaze direction, and / or head position, or objects being used. Using camera data can be a precise method for assessing passenger attention. Posture can indicate whether the passenger is focused on the traffic or engaged in another activity. Analyzing facial features can reveal signs of fatigue or distraction, for example. Gaze direction can indicate whether the passenger is observing the traffic or otherwise distracted. Head position can also provide clues about the passenger's attention by showing whether they are paying attention to the traffic or engaged in another activity.The detection of used objects, such as smartphones, books or magazines, can, in conjunction with other data, provide indications of the passenger's attention to other activities.

[0025] This data can be captured and analyzed in real time, enabling a continuous assessment of passenger attention. The combination of these various aspects—body posture, facial expressions, gaze direction, and head position—25-1213

[0026] 5

[0027] and / or objects used - can enable a comprehensive assessment of passenger attention.

[0028] Using camera data can offer the advantage that no additional sensors or devices need to be installed in the vehicle, as interior cameras are usually already present. This can simplify system implementation and simultaneously enable reliable data acquisition without incurring additional hardware costs.

[0029] Another embodiment of the method involves recording the interaction between the driver and the passenger using camera data and evaluating the passenger's attention based on this recorded interaction. An interaction deemed distracting can lead to a reduction in the assessed passenger attention.

[0030] This implementation can add an additional dimension to the assessment of passenger attention. By analyzing the interaction between driver and passenger, the system can consider not only the passenger's individual attention but also the impact of this interaction on the overall situation in the vehicle. The recording of this interaction can encompass various aspects.

[0031] For example, conversations, gestures, or other forms of communication between the driver and passenger can be recognized using camera data. The system can then assess whether this interaction might be distracting.

[0032] An interaction deemed distracting, which can lead to a decrease in the assessed passenger attention, could be, for example, an intense conversation during which the passenger takes their eyes off the road. In such cases, the system may rate the passenger's attention lower, which in turn can influence the adjustment of the threshold for the driver attention warning. Considering this interaction can help obtain a more accurate picture of the actual attention situation in the vehicle and adjust the system's warnings accordingly.

[0033] Another embodiment of the method provides that the driver attention warning can be issued in several escalation levels, wherein a first escalation level can include a visual driver attention warning and a second escalation level can include an audible and / or haptic driver attention warning. This multi-stage approach25-1213

[0034] 6

[0035] The system can enable an appropriate and gradual escalation of the driver attention warning. By using various modalities – visual, audible, and / or haptic – the system can address the driver's attention in different ways.

[0036] The first escalation level, a visual driver attention warning, can be a subtle yet effective method of alerting the driver to potential inattention. This visual driver attention warning can be issued independently of the passenger's attention. This is because a visual warning is generally perceived as less intrusive. The visual driver attention warning can be delivered, for example, by a symbol illuminating in the instrument cluster or on a head-up display.

[0037] Should the visual driver attention warning prove insufficient to restore the driver's focus, the second escalation level, an audible driver attention warning, can be activated. Unlike the visual warning, the audible driver attention warning can be issued based on the driver's level of attention and an adjusted threshold. This means the system considers passenger attention before issuing a more salient audible warning. An audible signal can be more effective in regaining the driver's attention, especially if they are distracted. This can be a tone, a spoken warning, or another type of acoustic signal.

[0038] The use of escalation levels, where the visual warning is issued regardless of the passenger's attention and the audible and / or haptic warning is issued depending on the passenger's attention, can help avoid unnecessary distractions while ensuring that the driver is effectively warned if inattention persists. Furthermore, this system can flexibly adapt to different situations and levels of inattention, taking into account the overall situation in the vehicle, including the passenger's attention.

[0039] Another aspect of the invention relates to a computing device for a vehicle, which is configured to execute a method according to the invention and its advantageous embodiments. The computing device can serve as a central element of the driver attention warning system. It can be designed to perform the 25-1213

[0040] 7

[0041] The computing device processes various input data, performs the calculations, and controls the corresponding outputs. The computing device can, for example, comprise one or more processors that can be programmed to execute the steps of the method according to the invention. This can include processing the driver attention data and the passenger data, evaluating the passenger attention, adjusting the threshold for the driver attention warning, and outputting the driver attention warning.

[0042] Furthermore, the computing unit can have interfaces to various sensors and output devices of the vehicle. This can include connections to cameras for capturing driver and passenger data, to displays for outputting visual driver attention warnings, to loudspeakers for audible driver attention warnings, and to actuators for haptic warnings (e.g., steering wheel / seat vibration, seatbelt tensioner). Integrating such a computing unit into a vehicle can enable the implementation of the method according to the invention into existing vehicle systems. This can contribute to improved driving safety by allowing the system to monitor the driver's attention, taking into account the passenger's situation, and to issue warnings as necessary.

[0043] Another aspect of the invention relates to a computer-readable storage medium comprising instructions which, when executed by a computing device, can cause it to execute a method according to the invention for issuing a driver attention warning to a driver of a vehicle, and its advantageous embodiments. The computer-readable storage medium can take various forms, such as a hard drive, a solid-state drive (SSD), a USB flash drive, an SD card, or other non-volatile memory. It can serve to store the computer program containing the instructions for executing the method according to the invention.

[0044] The present invention also relates to a computer program comprising commands which, when the program is executed by a computing device, cause the computing device to execute a method according to the invention for issuing a driver attention warning to a driver of a vehicle and the advantageous embodiments thereof. Furthermore, the present invention relates to a vehicle equipped with a computing device according to the invention. 25-1213

[0045] 8

[0046] In summary, the invention is based on the understanding that considering passenger attention when assessing and warning of driver inattention can lead to improved driving safety. The method according to the invention comprises receiving driver attention data and passenger data, whereby the passenger attention is assessed and a threshold for the driver attention warning can be adjusted. The invention can enhance existing attention monitoring systems by evaluating the interaction between driver and passenger as a whole. The threshold can be lowered when passenger attention increases, thus allowing for a more tolerant level of driver attention. The passenger data can be acquired as camera data describing the passenger's posture, facial expressions, gaze direction, head position, and / or objects being used.The driver attention warning can be issued at several escalation levels, which can lead to a more flexible response to different situations in the vehicle.

[0047] The preferred embodiments and their advantages presented with reference to the method according to the invention apply accordingly to the computing device and the computer-readable storage medium according to the invention. Furthermore, the preferred embodiments and their advantages presented with reference to the method according to the invention also apply to the computer program and the vehicle.

[0048] The invention will now be explained in more detail with reference to preferred embodiments and the accompanying drawings. These show:

[0049] Fig. 1 shows a schematic representation of a vehicle with a system for issuing a driver attention warning, taking into account the passenger's attention; and

[0050] Fig. 2 is a diagram illustrating the adjustment of the threshold for the driver attention warning depending on the passenger attention.

[0051] Fig. 1 shows a schematic representation of a vehicle 1 with a system for issuing a driver attention warning, taking into account the passenger's attention. The vehicle 1 may include a computing unit 2, which 25-1213

[0052] 9

[0053] The vehicle can be configured to carry out the method according to the invention. A camera 3 can be arranged in the vehicle interior and serve to capture both the driver 4 and the passenger 5. The camera 3 can capture driver attention data and passenger data that can describe physical and / or behavioral characteristics of the driver 4 and the passenger 5. This data can include posture, facial features, gaze direction, head position, and / or objects used.

[0054] The computing unit 2 can process the data captured by the camera 3 and use it to assess driver and passenger attention. Based on this assessment, the computing unit 2 can adjust a threshold for the driver attention warning. An output unit 6 can be provided in the vehicle 1 to issue the driver attention warning. This output unit 6 can be, for example, a display in the instrument cluster, a head-up display, a loudspeaker, and / or an actuator, depending on whether a visual, audible, and / or haptic warning is to be issued.

[0055] Figure 2 shows a diagram illustrating the adjustment of the driver attention warning threshold depending on passenger attention. The horizontal axis represents passenger attention, while the vertical axis represents driver attention.

[0056] The diagram shows a threshold value of 7, which changes depending on the passenger's attention span. This threshold value can be lowered if the assessed passenger attention span increases. This could mean that with higher passenger attention, a lower level of driver attention can be tolerated before the driver attention warning is issued.

[0057] A warning zone 8 is shown below the threshold 7. If the driver's attention falls within this zone, a driver attention warning may be issued. The size and shape of this warning zone 8 can change depending on the passenger's attention. The relationship between passenger attention and the threshold is shown linearly in Fig. 2. However, this is not mandatory; the relationship can be arbitrary and, for example, also exhibit non-linear curves.

[0058] This illustration demonstrates how the system can react flexibly to different situations in the vehicle. With a high level of passenger attention, a slightly lower level of driver attention can be tolerated, since (depending on the situation) it can be assumed that an attentive passenger can inform the driver about important aspects of the traffic situation and warn them if necessary. [Reference symbol list]

[0059] vehicle

[0060] Computer facility

[0061] camera

[0062] driver

[0063] passenger

[0064] Output unit

[0065] threshold

[0066] Warning area

Claims

Claims 1. Method for issuing a driver attention warning to a driver (4) of a vehicle (1), comprising the steps: - Receiving driver attention data which describes driver attention or attention (4) in relation to a traffic event; - Receiving passenger data describing physical and / or behavioral characteristics of a passenger (5); - Evaluating passenger attention based on passenger data, where passenger attention describes the passenger's attention (5) to the traffic situation; - Adjusting a threshold (7) for the driver attention warning depending on the assessed passenger attention; - Issuing the driver attention warning depending on the driver attention level and the adjusted threshold (7).

2. Method according to claim 1 , characterized by the fact that Adjusting the threshold (7) includes lowering the threshold (7) if the assessed passenger attention increases, whereby lowering the threshold (7) tolerates a lower level of driver attention until the driver attention warning is issued.

3. Method according to claim 1 or 2, characterized by the fact that as the passenger data camera data are used and the camera data describe the posture, facial features, direction of gaze and / or head position of the passenger (5) and / or objects used by the passenger (5).

4. Method according to claim 3, characterized by the fact that an interaction between the driver (4) and the passenger (5) is recorded using camera data and the passenger's attention is assessed depending on the recorded interaction, whereby one is considered distracting13 The assessed interaction leads to a reduction in the assessed passenger attention.

5. Method according to any one of the preceding claims, characterized by the fact that The driver attention warning is issued in several escalation levels, with a first escalation level including a visual warning and a second escalation level including an audible and / or haptic warning.

6. Computing device (2) for a vehicle (1) which is configured to perform a method according to one of the preceding claims.

7. Computer-readable storage medium comprising instructions which, when executed by a computing device (2), cause it to execute a method according to any one of claims 1 to 5.