Driver monitoring system for vehicle

Abstract

The present disclosure relates to a driver monitoring system for a vehicle, comprising: at least one internal sensor configured to monitor an interior of the vehicle, at least one external sensor configured to monitor an environment of the vehicle, and a computing unit, the computing unit is configured to obtain, analyze and process sensor data from the internal sensor and the external sensor; identifying a perceived change in sensor data from the internal sensor and / or the external sensor, the perceived change being within the field of view of the vehicle occupant; assigning a virtual focus to the recognized perceived change; deriving a prediction of an occupant gaze vector based on the virtual focus, estimating a gaze vector and / or a head pose of the occupant using sensor data of the internal sensor, and determining a position of the occupant by comparing the occupant gaze vector estimated from the internal sensor data with the prediction of the occupant gaze vector using the virtual focus, and the estimation of the sight line vector of the passenger is refined.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a driver monitoring system (DMS) for a vehicle and a method of operating the same. BACKGROUND

[0002] Vehicle accidents are often caused by inattentive or drowsy drivers. Current driver monitoring systems (DMS) focus on using on-board cameras and sensors in combination with computer vision techniques to detect such states. Such systems are typically integrated in high-end vehicles, often using a single interior camera to capture and process images to monitor driver behavior. Most DMS solutions can not effectively address special cases, such as visual impairments related to strabismus or monocular vision. They also tend to overlook the critical relationship between a driver’s gaze and the external environment of the vehicle. This limitation leads to difficulties in accurately assessing a driver’s attention, especially in complex driving scenarios.

[0003] Therefore, there is a need for a more inclusive, cost-effective, and comprehensive DMS system that can take into account different driver conditions and external interactions. SUMMARY

[0004] The present disclosure provides a driver monitoring system (DMS) for a vehicle, a method of operating the same, and a non-transitory computer readable medium according to the independent claims. Embodiments are given in the dependent claims, the description and drawings.

[0005] In a first aspect, the present disclosure relates to a driver monitoring system (DMS) for a vehicle, the driver monitoring system comprising at least one interior sensor configured to monitor an interior of the vehicle, at least one exterior sensor configured to monitor an environment of the vehicle, and a computing unit, wherein the computing unit is configured to: obtain, process, and analyze sensor data from the interior sensor and the exterior sensor; identify a perceived change in the sensor data from the exterior sensor, wherein the perceived change is within a field of view of an occupant of the vehicle; assign a virtual focal point to the identified perceived change; derive a prediction of a gaze vector of the occupant based on the virtual focal point; estimate a gaze vector and / or a head pose of the occupant using the sensor data of the interior sensor; and refine the estimate of the gaze vector of the occupant by comparing the gaze vector of the occupant estimated from the sensor data of the interior sensor with the prediction of the gaze vector of the occupant using the virtual focal point.

[0006] The occupant can be the driver of the vehicle, but the system is not limited to predicting the driver's gaze vector. The system can also be applied to predict the gaze vector of other occupants or passengers with different focal points inside and outside the vehicle. The system is versatile and applicable to various vehicle types, including passenger cars, commercial vehicles, engineering vehicles, and motorcycles. The system is also applicable to fully autonomous vehicles. In such vehicles, the focus of the system shifts to applications with lower safety requirements, with more focus on human-machine interfaces, augmented reality, communication, and comfort. Overall, the system is applicable in situations where it is intended to estimate or determine a person's gaze vector. Therefore, the system is also applicable to aircraft and can be used in augmented reality technology.

[0007] The system collects and analyzes data from both internal and external sensors. The at least one internal sensor can be a single sensor or can be composed of multiple similar or different sensors. The internal sensor is configured to monitor the interior of the vehicle. The interior of the vehicle can include the driver, other occupants, the cabin, the dashboard, or other displays inside. The internal sensor can monitor the driver and / or other occupants of the vehicle. The internal sensor can be a cabin sensor that is directed at the driver. Specifically, the internal sensor can continuously monitor the driver's eyes, gaze, and / or overall posture. The internal sensor can also be a cabin sensor configured to monitor the overall interior conditions of the vehicle, including other occupants or passengers. The internal sensor can be a cabin sensor configured to monitor the dashboard and / or certain displays inside the vehicle. The internal sensor can be installed at different locations within the cabin, such as at the rear window mirror.

[0008] The at least one external sensor can be a single sensor or can be composed of multiple similar or different sensors. The external sensor can continuously monitor the environment of the vehicle. The external sensor can employ various technologies to capture detailed information about the external environment. The data acquired by the external sensor can include, for example, traffic flow, road conditions, and distance from other vehicles, and can also include road signs or the presence of pedestrians near the vehicle's travel path.

[0009] The computing unit is configured to take, process, and analyze sensor data from both internal and external sensors. The computing unit uses the sensor data to identify a change in perception within the driver’s or occupant’s field of view. The identified change is a change in perception of the environment outside the vehicle. An event causing a change in perception within the driver’s or occupant’s field of view is likely to be identified by the relevant person. Responsively, the person can turn their gaze to the event. Accordingly, the system can utilize the location of the identified event and assign a virtual focal point for the driver’s or occupant’s gaze at the event. As such, the virtual focal point can be represented as a direction within the driver’s or occupant’s field of view. More accurately, the virtual focal point can be represented in full 3D, including a range and direction within the driver’s or occupant’s field of view. The identified and assigned virtual focal point is then used by the computing unit to derive a prediction of the driver’s or occupant’s gaze vector.

[0010] In particular, the system utilizes the relationship between the driver’s gaze and the change in the environment outside the vehicle to derive a prediction of the driver’s gaze vector, thereby enabling an accurate assessment of the driver’s attention, especially in complex driving scenarios. Accordingly, the system facilitates a safer, more connected driving experience.

[0011] The DMS system is inclusive, cost-effective, and comprehensive, as the system is able to account for a variety of driver conditions and external interactions.

[0012] The computing unit is configured to estimate the occupant’s gaze vector and / or head pose using sensor data of internal sensors. The sensor data can be acquired by monitoring the occupants of the vehicle, particularly by monitoring the driver of the vehicle. For example, the internal sensors can be cabin sensors that are directed at the driver. The sensor data of the internal sensors can be used to monitor the driver’s head pose and continuously track the driver’s eye position and gaze direction, thereby estimating or determining the driver’s gaze vector. The head pose or head posture can also be used to identify additional virtual focal points that are not dependent on changes in perception. And the additional virtual focal points can also be used to predict the driver’s gaze vector.

[0013] The computing unit is configured to refine the estimate of the occupant’s gaze vector by comparing the occupant’s gaze vector estimated from internal sensor data to the occupant’s gaze vector predicted using the virtual focal point. In doing so, the computing unit confirms that the occupant’s eyes and gaze are indeed moving in response to the change in perception, ensuring that the virtual focal point accurately reflects the occupant’s attention and can thus be used to refine the occupant’s gaze vector estimated from internal sensor data. Accordingly, the system employs an automatic calibration method that continuously aligns the occupant’s gaze vector with both internal and external data points, thereby adjusting the calibration in real-time based on the dynamic driving environment.

[0014] Therefore, this system overcomes the limitations of current DMS systems by integrating both internal and external sensing technologies and employing an automatic calibration method that effectively aligns targets within the occupant's likely field of vision with the occupant's eye position and line of sight. This dual-focusing approach enables continuous calibration and refined self-evaluation, thereby enhancing the system's accuracy in real-world driving scenarios.

[0015] This system contributes to a safer and more connected driving experience. By integrating internal and external data and ensuring non-invasive calibration, the system provides an accurate and safe monitoring solution applicable to a variety of drivers, passengers, and driving scenarios.

[0016] Changes in perception are correlated with changes in sensor data acquired by external sensors. Therefore, the system identifies changes in perceived external environment as the occupant's virtual focus. Changes in the external environment include events such as the passing of other vehicles or road signs, changes in traffic light colors, sudden braking of a vehicle ahead, or a pedestrian attempting to cross the road in front of the vehicle. The system can use these events as passive calibration targets to refine the estimation or determination of the occupant's line-of-sight vector.

[0017] According to an embodiment of the first aspect, the computing unit is further configured to identify perceived changes in the sensor data from the internal sensors, wherein the perceived changes are located within the field of vision of the occupants of the vehicle.

[0018] According to an embodiment of the first aspect, the perceived change is related to a change in the appearance of devices or device displays inside the vehicle, particularly a head-up display (HUD) and / or user interface (UI) and / or rearview mirror, wherein the change in appearance is triggered and / or initiated by the driver monitoring system. For situations where external sensor data cannot provide sufficient calibration, the system can employ an active calibration method. This active calibration is subtle and inconspicuous. Unlike prominent cues or alarms that may distract the driver, the system performs calibration, for example, by changing the display of the HUD, UI, or rearview mirror. These changes are designed to naturally attract the driver's gaze, making the calibration process unnecessary for conscious driver involvement. In this way, calibration can be seamlessly integrated into the driving experience, maintaining focus and safety. Active calibration is performed only when the system assesses internal and external conditions as safe, ensuring that the calibration process does not interfere with vehicle operation or affect the driver's attention.

[0019] According to an embodiment of the first aspect, the internal sensors include 2D cameras and / or time-of-flight (ToF) cameras and / or stereo cameras and / or radar systems and / or lidar (LIDAR) systems. Data from such sensors can accurately track the eye movements, head position, and overall posture of the pilot or other occupants. One or more internal sensors can be configured to operate under various lighting conditions and adapt to different facial features and expressions. One or more internal sensors can be mounted in different locations within the cabin, for example, mounted on the rear window mirror and pointed directly at the pilot.

[0020] According to an embodiment of the first aspect, the external sensor includes a 2D camera and / or a radar system and / or a lidar system and / or an ultrasonic sensor. The external sensor can employ a variety of technologies to capture detailed information about the external environment. The data acquired by the external sensor includes, for example, information about traffic flow, road conditions, and distances to other vehicles, and also includes information about the presence of pedestrians near road signs or the vehicle's path.

[0021] According to an embodiment of the first aspect, the computing unit is configured to include a user profile comprising personalized metadata of the occupant in the estimation of the occupant's gaze vector. The metadata of the personalized user profile includes information about individual visual acuity and facial feature differences, such as strabismus or monocular vision. By including a personalized user profile, the system is able to adapt to visual impairments when estimating or determining the occupant's gaze vector. This adaptability can be enhanced by machine learning algorithms that continuously refine the system's understanding of individual occupant behavior and preferences.

[0022] According to an embodiment of the first aspect, the computing unit is configured to update and / or refine the estimation of the occupant's gaze vector by comparing the occupant's gaze vector estimated based on the sensor data from the internal sensors and the personalized metadata with the prediction of the occupant's gaze vector using the virtual focus, and by minimizing the difference between the estimated and predicted occupant gaze vectors. Therefore, the system includes a self-evaluation mechanism that can detect persistent differences between the estimated and predicted gaze vectors, allowing for automatic adjustments during computation. For example, the system might ignore data from the disabled eye obtained by the internal sensors or apply a fixed offset for strabismus.

[0023] According to an embodiment of the first aspect, the computing unit is configured to issue a warning to the driver when the occupant is the driver of the vehicle, in response to an estimated driver's gaze vector and / or head posture indicating that the driver is inattentive and / or drowsy. The estimation of the driver's gaze vector can be based on internal data or a refined estimation using both internal and external data. The warning can be a visual, auditory, or tactile warning. The warning can be issued by various systems of the vehicle in response to the DMS computing unit triggering the warning.

[0024] According to an embodiment of the first aspect, the computing unit is configured to, when the occupant is the driver of the vehicle, classify events causing changes in perception as hazardous events, and, in response to a discrepancy between the estimated driver's gaze vector and a predicted driver's gaze vector based on the virtual focus corresponding to the hazardous event, trigger a warning and / or trigger action by the vehicle safety system. Therefore, the combination of internal and external monitoring sensors can extend beyond self-calibration to act as a fail-safe function in critical driving situations. The system issues an alert when the driver fails to actively notice a critical situation (e.g., a playing child or sudden braking of the vehicle). The alert can be visual, auditory, or tactile. The system can be seamlessly integrated with existing vehicle safety systems (e.g., automatic braking or lane-keeping assist systems). Therefore, it can provide a comprehensive safety response based on the driver's current state of attention.

[0025] According to an embodiment of the first aspect, the computing unit is configured to process and analyze sensor data from the internal and / or external sensors using neural networks and / or other machine learning algorithms and / or rule-based algorithms. Rule-based algorithms can process and analyze sensor data from internal and / or external sensors using predefined rule sets. Machine learning algorithms and neural networks provide efficient and cost-effective methods for processing and analyzing large amounts of data. They are capable of accurately identifying virtual focal points in external sensor data, predicting occupant gaze vectors, and refining estimates of occupant gaze vectors. Therefore, using neural networks or machine learning algorithms can enhance the system's ability to continuously improve its understanding of individual driver or passenger behavior and preferences.

[0026] In a second aspect, this disclosure relates to a method of operating a Driver Monitoring System (DMS) for a vehicle, the driver monitoring system comprising: at least one internal sensor configured to monitor the interior of the vehicle; at least one external sensor configured to monitor the environment of the vehicle; and a computing unit, wherein the computing unit acquires, processes, and analyzes sensor data from the internal sensor and the external sensor; wherein the computing unit identifies perceptual changes in the sensor data from the external sensor, wherein the perceptual changes are located within the field of vision of an occupant of the vehicle; wherein the computing unit assigns a virtual focus to the identified perceptual changes and derives a prediction of the occupant's gaze vector based on the virtual focus; wherein the computing unit uses sensor data from the internal sensor to estimate the occupant's gaze vector and / or head posture; and wherein the computing unit refines the estimation of the occupant's gaze vector by comparing the estimated gaze vector of the occupant based on the sensor data from the internal sensor with the prediction of the occupant's gaze vector using the virtual focus.

[0027] According to an embodiment of the second aspect, the computing unit also identifies perceptual changes in the sensor data from the internal sensors, wherein the perceptual changes are located within the field of vision of the vehicle's occupants.

[0028] This method and its implementation can be performed using the system described above or any of its implementations. The description of the system and its implementations applies accordingly.

[0029] This method uses the relationship between an occupant's line of sight and changes in the vehicle's interior or external environment to predict the occupant's gaze vector, thereby accurately assessing driver or passenger attention, especially in complex driving scenarios. Therefore, this method contributes to a safer, more connected driving experience. It is inclusive, cost-effective, and comprehensive because it considers various driver conditions and external interactions.

[0030] Specifically, by integrating both internal and external sensing technologies and employing an automated calibration method that effectively aligns passive targets (such as other vehicles and road signs) within the driver's likely field of vision with the driver's eye position and gaze direction, this system overcomes the limitations of current Driver Monitoring Systems (DMS). This dual-focus scheme allows for continuous calibration and refined self-evaluation, thereby improving the accuracy of the estimated driver gaze vector.

[0031] In a third aspect, this disclosure relates to a non-transitory computer-readable medium containing instructions for performing one or all of the steps or aspects of the methods described herein. The computer-readable medium can be configured as: an optical medium, such as an optical disc (CD) or digital versatile optical disc (DVD); a magnetic medium, such as a hard disk drive (HDD); a solid-state drive (SSD); a read-only memory (ROM), such as flash memory; and so on. Furthermore, the computer-readable medium can be configured as a data storage device accessible via a data connection (e.g., an internet connection). For example, the computer-readable medium can be an online data repository or cloud storage. Attached Figure Description

[0032] This document describes exemplary embodiments and functions of the present disclosure in conjunction with the following accompanying drawings.

[0033] Figure 1A and Figure 1B A schematic diagram of a vehicle equipped with a DMS system implementation is shown.

[0034] Figure 2A and Figure 2B schematically shown Figure 1A and Figure 1B Two examples of data acquired by the internal sensors in the illustrated implementation;

[0035] Figure 3 schematically shown Figure 1A and Figure 1B The flowchart shown illustrates the implementation method of the DMS system operation method;

[0036] Figure 4 An example view of a scene captured by an external camera is shown schematically;

[0037] Figure 5 schematically shown Figure 4 An aerial view of the scene;

[0038] Figure 6 schematically shown Figure 1A and Figure 1B A simplified flowchart of another implementation of the DMS system operation method;

[0039] Figure 7 An example result illustrating the incorporation of user profiles into the gaze vector estimation of a driver with strabismus is shown.

[0040] List of reference numerals

[0041] 10 vehicles

[0042] 12 Driver Monitoring System

[0043] 14 internal sensors

[0044] 16 external sensors

[0045] 18 computing units

[0046] 20 vehicle occupants / drivers

[0047] 22 pedestrians

[0048] 24. Brake lights of other vehicles

[0049] 26 Virtual Focus / Possible Gaze Targets Detailed Implementation

[0050] Figure 1A and Figure 1B Schematic side and top views of a vehicle 10 equipped with an embodiment of the DMS system 12 of this disclosure are depicted, respectively. System 12 includes an internal sensor 14, a plurality of external sensors 16, and a computing unit 18. The internal sensor 14 may include, for example, a 2D camera and / or a time-of-flight (ToF) camera and / or a stereo camera and / or a radar system and / or a lidar system. The internal sensor 14 may include a near-infrared camera configured to operate under various lighting conditions and may be mounted, for example, on a rearview mirror of the vehicle 10. Figure 2A and Figure 2B The near-infrared camera mounted on the rearview mirror is shown monitoring the occupants 20 of the vehicle 10. Figure 2A ) and the near-infrared camera mounted on the dashboard is pointed directly at the vehicle driver 20 ( Figure 2B Examples of data obtained. The internal sensor 14 continuously monitors the driver 20 or occupant 20 of the vehicle 10, thereby enabling accurate tracking of the driver's or occupant's eye movements, head position, and overall posture.

[0051] Multiple external sensors 16 may include 2D cameras and / or radar systems and / or lidar systems and / or ultrasonic sensors. Therefore, the external sensors 16 can employ a variety of technologies to capture detailed information about the external environment of the vehicle 10. The data acquired by the external sensors 16 includes, for example, information about traffic flow, road conditions, and distances to other vehicles, as well as information about road signs or the presence of pedestrians near the path of the vehicle 10.

[0052] The computing unit 18 is configured to acquire, process, and analyze sensor data from internal sensors 14 and multiple external sensors 16. The computing unit uses the sensor data to identify changes in perception within the driver's or occupant's field of vision 20. The identified changes can be changes in perception of the interior of the vehicle 10 or changes in perception of the external environment by the vehicle 10. Events causing changes in perception within the driver's or occupant's field of vision are likely to be identified by the corresponding person 20. Responsibly, the person 20 may turn their gaze toward the event. Therefore, the DMS system 12 uses the location of the identified event and assigns a virtual focus 26 (see [link to DMS system]) to the driver's or occupant's gaze toward the event. Figure 4 and Figure 5 The virtual focus 26 can be represented as the direction within the field of vision of the driver 20 or the occupant 20. The virtual focus 26 can also be represented in full 3D, including the range and direction within the field of vision of the driver 20 or the occupant 20. The virtual focus 26 identified and assigned by the computing unit 18 can be used by the computing unit 18 to derive a prediction of the line-of-sight vector of the driver 20 or the occupant 20.

[0053] Figure 3 Depicting Figure 1A and Figure 1B A flowchart illustrating an implementation of the operation method of the DMS system 12 in the vehicle 10 is provided. This method aims to estimate or determine the line-of-sight vector of the occupant 20 or driver 20 of the vehicle 10. For simplicity, only the estimation of the driver 20's line-of-sight vector is discussed below. This method can be similarly used to estimate the line-of-sight vector of any other occupant or passenger in the vehicle 10.

[0054] This method employs an automatic calibration estimation method that continuously aligns the driver's line-of-sight vector with both internal and external data points, thereby adjusting the calibration in real time according to the dynamic driving environment.

[0055] The computing unit 18 uses data acquired by the internal sensor 14 to estimate the driver 20's gaze vector and head posture. For example, Figure 2A and Figure 2B In the diagram, dashed arrows mark the estimated line-of-sight vectors for the driver 20 and the occupant 20.

[0056] The gaze vector estimate of driver 20 can be refined by comparing it with the gaze vector of driver 20 predicted using multiple virtual focal points 26 (obtained by analyzing data captured by multiple external sensors 16).

[0057] The computing unit 18 of system 12 uses external sensor data to identify changes in perception within the driver 20's field of vision. The computing unit 18 identifies changes in perception of the external environment of the vehicle 10 and uses these changes as the virtual focus 26 of the driver 20.Figure 3 The phrase "identifying potential targets of gaze" refers to changes in the external environment, such as the passing of other vehicles, changes in traffic light colors, sudden braking of a vehicle in front, or pedestrians attempting to cross the road in front of a vehicle. Figure 3 (Change detection in the context). Furthermore, changes in the external environment may also include static objects detected in external data (…). Figure 3 (Object detection in the text). These objects (e.g., road signs passing by the vehicle 20) may also be identified as changes in the perception of the external environment of the vehicle 10, and thus can be identified as virtual focus 26 for the driver 20. An additional virtual focus, independent of changes in perception, can be provided by estimating the head posture of the driver 20 based on data from the internal sensors 14. This additional virtual focus can also be used to predict the gaze vector of the driver 20.

[0058] Figure 4 An example view of a scene captured by an external camera is shown. The scene shows a second vehicle traveling on a road in front of vehicle 10. The second vehicle is forced to brake suddenly as a pedestrian 22 attempts to cross the road. The computing unit 18 identifies the moving pedestrian 22 and the suddenly illuminated brake light 24 as changes in perception within the driver 20's field of vision and identifies them as separate virtual focal points 26, representing possible gaze targets of the driver 20. The system 12 then uses these possible gaze targets 26 to refine the driver 20's gaze vector estimate.

[0059] In the refinement step, the calculation unit 18 compares each virtual focus 26 with an estimate of the driver 20's gaze vector based on internal sensor data. For example, the calculation unit 18 confirms that the driver 20's eyes do indeed move with changes in perception, thus ensuring that the virtual focus 26 accurately reflects the driver 20's attention and can therefore be used to refine the estimate of the driver 20's gaze vector. If the gaze estimate based on internal sensor data is similar to the gaze vector of the virtual focus 26 identified in external data ( Figure 3 If the question mark indicates "equal?", then both the internal and external datasets are used to refine the estimation or determination of the driver's 20-degree line-of-sight vector. Figure 3 (The phrase "calibrate on the target" in the text).

[0060] As an example of this method, Figure 5 Shown from a bird's-eye view Figure 4 The scene is illustrated with dashed lines marking possible line-of-sight vectors pointing to virtual focus 26, and dotted lines marking the estimated line-of-sight vector of driver 20. For refinement, the central virtual focus 26, located at the center brake light of the second vehicle, is compared and aligned with the estimated line-of-sight vector of driver 20 by minimizing the differences between the individual vectors.

[0061] This method thus integrates both internal and external sensor data, enabling efficient automatic calibration that aligns targets within the driver's likely field of vision (such as other vehicles and road signs) with the driver's eye position and line of sight. This dual-focus scheme allows for continuous calibration and refined self-evaluation, thereby improving the system's accuracy.

[0062] Figure 6 Depicting Figure 1A and Figure 1B A simplified flowchart illustrating another embodiment of the DMS system operation method is shown. This method is similar to... Figure 3 The method shown and described above is similar, but a user profile is additionally added to the estimation of the driver 20's gaze vector. This user profile includes personalized metadata about the driver 20 and includes information about individual visual acuity and facial feature differences, such as strabismus or monocular vision. By including a personalized user profile, the method is able to adapt to visual impairments when estimating the driver 20's gaze vector.

[0063] The computing unit 18 can update and / or optimize the user profile of driver 20, and refine the estimation of the driver 20's gaze vector by comparing the estimated gaze vector of driver 20 using internal sensor data and personalized metadata with the predicted gaze vector of driver 20 using virtual focus 26, and by minimizing the difference between the estimated and predicted gaze vectors. Therefore, the method includes a self-evaluation mechanism that can detect persistent differences between the predicted and actual gaze vectors, allowing for automatic adjustments during computation. For example, the DMS system 12 can ignore data from the disabled eye acquired by internal sensor 14, or apply a fixed offset for strabismus. Thus, the system 12's understanding of the individual driver 20's behavior and preferences can be continuously refined.

[0064] For example, Figure 7 The results of incorporating user profiles into the gaze vector estimation of a driver with strabismus are shown. Figure 7 The driver's line-of-sight vector estimate marked with a dashed arrow in the left panel is incorrect because it does not include the user profile. Figure 7 The estimate of the driver's 20-degree line-of-sight vector, shown in the right panel after applying oblique offset based on the driver's user profile, demonstrates correct behavior.

Claims

1. A driver monitoring system (12) for a vehicle (10), the driver monitoring system comprising: At least one internal sensor (14) configured to monitor the interior of the vehicle. At least one external sensor (16) configured to monitor the environment of the vehicle (10), and Calculation unit (18), The computing unit (18) is configured as follows: Acquire, process and analyze sensor data from the internal sensor (14) and the external sensor (16); Identify perceptual changes in the sensor data from the external sensor (16), wherein the perceptual changes are within the field of vision of the occupant (20) of the vehicle (10); Assign a virtual focus to the identified changes in perception (26); Based on the virtual focus (26), the prediction of the occupant's (20) gaze vector is obtained; The sensor data from the internal sensor (14) is used to estimate the gaze vector and / or head posture of the occupant (20); and The estimation of the occupant's (20) gaze vector is refined by comparing the gaze vector of the occupant estimated based on the sensor data of the internal sensor (14) with the prediction of the occupant's gaze vector using the virtual focus (26).

2. The driver monitoring system (12) according to claim 1, wherein, The computing unit (18) is also configured to identify perceptual changes in the sensor data from the internal sensor (14), wherein the perceptual changes are within the field of vision of the occupant (20) of the vehicle (10).

3. The driver monitoring system (12) according to claim 2, wherein, The perceived change is related to a change in the visual appearance of a device or device display inside the vehicle, wherein the change in appearance is triggered and / or initiated by the driver monitoring system (12).

4. The driver monitoring system (12) according to claim 3, wherein, The device or the device display is a head-up display and / or a user interface and / or a reflector.

5. The driver monitoring system (12) according to claim 1 or 2, wherein, The internal sensors (14) include 2D cameras and / or time-of-flight cameras and / or stereo cameras and / or radar systems and / or lidar systems.

6. The driver monitoring system (12) according to claim 1 or 2, wherein, The external sensor (16) includes a 2D camera and / or a radar system and / or a lidar system and / or an ultrasonic sensor.

7. The driver monitoring system (12) according to claim 1 or 2, wherein, The computing unit (18) is configured to include a user profile that includes personalized metadata of the occupant (20) in the estimation of the occupant's (20) gaze vector.

8. The driver monitoring system (12) according to claim 7, wherein, The computing unit (18) is configured to update and / or refine the estimation of the occupant's (20) gaze vector by comparing the gaze vector of the occupant (20) estimated based on the sensor data of the internal sensor (14) and the personalized metadata with the prediction of the gaze vector of the occupant (20) using the virtual focus (26), and by minimizing the difference between the estimated gaze vector of the occupant (20) and the predicted gaze vector of the occupant.

9. The driver monitoring system (12) according to claim 1 or 2, wherein, The computing unit (18) is configured to issue a warning to the driver (20) in response to the estimated gaze vector and / or head posture of the driver (20) indicating that the driver (20) is inattentive and / or drowsy, when the occupant (20) is the driver of the vehicle (10).

10. The driver monitoring system (12) according to claim 1 or 2, wherein, The computing unit (18) is configured to classify events that cause changes in perception as dangerous events when the occupant (20) is the driver of the vehicle (10), and to trigger a warning and / or trigger action by the vehicle safety system in response to a discrepancy between the estimated gaze vector of the driver (20) and the predicted gaze vector of the driver (20) based on the virtual focus corresponding to the dangerous event.

11. The driver monitoring system (12) according to claim 1 or 2, wherein, The computing unit (18) is configured to use neural networks and / or other machine learning algorithms and / or rule-based algorithms to process and analyze sensor data from the internal sensor (14) and / or the external sensor (16).

12. A method of operating a driver monitoring system (12) for a vehicle (10), the driver monitoring system (12) comprising: At least one internal sensor (14) is configured to monitor the interior of the vehicle (10); At least one external sensor (16) configured to monitor the environment of the vehicle (10); and a computing unit (18), The computing unit (18) acquires, processes, and analyzes sensor data from the internal sensor (14) and the external sensor (16); The computing unit (18) identifies perceptual changes in sensor data from the external sensor (16), wherein the perceptual changes are within the field of vision of the occupant (20) of the vehicle (10). The computing unit assigns a virtual focus (26) to the identified changes in perception and derives a prediction of the occupant's gaze vector based on the virtual focus. The computing unit (18) uses sensor data from the internal sensor (14) to estimate the gaze vector and / or head posture of the occupant (20); and The calculation unit (18) refines the estimation of the occupant's (20) gaze vector by comparing the gaze vector of the occupant (20) estimated based on the sensor data of the internal sensor (14) with the prediction of the gaze vector of the occupant (20) using the virtual focus (26).

13. The method according to claim 12, wherein, The computing unit (18) also identifies perceptual changes in the sensor data from the internal sensor (14), wherein the perceptual changes are within the field of vision of the occupant (20) of the vehicle (10).

14. A non-transitory computer-readable medium comprising instructions for performing the method of claim 12 or 13.

Images