Method and device for locating the head of a vehicle occupant

Abstract

Method (500) for locating the head (108) of a vehicle occupant (106), wherein the vehicle occupant (106) is wearing glasses (112), the method (500) comprising the following steps: Reading (510) spectacle information (315) representing the spectacles (112) wherein in the reading step (510) a position and / or orientation of the spectacles (112) is read as the spectacle information (315) and a reference position and / or reference orientation of the spectacles (112) is read as a vehicle occupant-specific parameter (335); and Processing (520) the spectacle information (315) to locate the head (108), wherein in the processing step (520) the head (108) is located by comparing the position and / or location of the spectacles (112) with the reference position and / or reference location.

Description

State of the art

[0001] The invention relates to a device or a method according to the preamble of the independent claims. The present invention also relates to a computer program.

[0002] A driver's head orientation plays a crucial role in safety and comfort features. For example, it can predict a driver's intention to change lanes. Furthermore, head orientation serves as an important parameter for visually assessing a driver's level of attention or fatigue.

[0003] Several methods for estimating head poses are known, such as 2D image-based methods or methods where a head pose can be estimated in real time using depth data.

[0004] Various methods for head pose estimation are known from the prior art. For example, documents DE 10 2014 206 626 A1, JP 2008 - 269 182 A, and the publication "Driver Gaze Tracking and Eyes Off the Road Detection System" by Francisco Vicente et al. in IEEE Transactions on Intelligent Transportation Systems describe approaches for monitoring vehicle occupants. Further relevant methods are known, for example, from document DE 10 2010 006 293 A1. Disclosure of the invention

[0005] Against this background, the approach presented here comprises a method for locating the head of a vehicle occupant, a device that uses this method, and finally a corresponding computer program according to the main claims. Advantageous further developments and improvements of the device specified in the independent claim are possible through the measures listed in the dependent claims.

[0006] A method for locating the head of a vehicle occupant is presented, wherein the vehicle occupant is wearing glasses, and wherein the method comprises the following steps: Reading in information about the glasses; and Processing the information from the glasses to locate the head.

[0007] The vehicle occupant can be a driver or passenger. The eyeglass information can, for example, represent the position or orientation of the glasses. For instance, the eyeglass information can be processed to determine the position or orientation of the head based on the position or orientation of the glasses.

[0008] The approach presented here is based on the finding that data relating to glasses worn by a vehicle occupant can be used to locate the occupant's head inside the vehicle. Thus, the occupant's head can be located accurately and reliably even if their face is partially obscured by the glasses.

[0009] According to one embodiment, during the reading step, the position and, additionally or alternatively, the orientation of the glasses can be read as the glasses information. The position of the glasses can be understood as data representing their location within the vehicle interior. The orientation of the glasses can be understood as data representing their orientation within the vehicle interior. For example, the position or orientation of the glasses can be determined using a three-dimensional coordinate system. This embodiment enables precise and reliable localization of the head based on the location of the glasses.

[0010] According to a further embodiment, at least one vehicle occupant-specific parameter can be read in the input step. In the processing step, the eyeglass information can be processed using the vehicle occupant-specific parameter. A vehicle occupant-specific parameter can be, for example, a parameter that can be used to identify the vehicle occupant. The vehicle occupant-specific parameter can, for example, indicate whether the vehicle occupant wears eyeglasses or not. Furthermore, the vehicle occupant-specific parameter can represent, for example, a habitual or average position or orientation of the eyeglasses in the vehicle interior. This embodiment can increase the accuracy in locating the head.

[0011] It is advantageous if, during the initial reading step, a reference position and / or position of the glasses is entered as the vehicle occupant-specific parameter. During the processing step, the head can be located by comparing the position and / or position of the glasses with the reference position and / or position. The reference position or position could, for example, be a familiar or average position of the glasses when the vehicle occupant is wearing them in the vehicle. In particular, the reference position or position could be a position of the glasses that corresponds to a position of the head when the vehicle occupant is sufficiently alert or attentive. This design allows the head to be located reliably and accurately with minimal computational effort.

[0012] Furthermore, it is advantageous if the spectacle information is processed during the processing step to determine the position and / or orientation of the head. This embodiment enables unambiguous localization of the head.

[0013] According to a further embodiment, in the reading step, information generated using at least one image of the vehicle occupant can be read in as the glasses information. This enables continuous generation of the glasses information.

[0014] The method can also include a step of capturing an image of the vehicle occupant using a capture device. This capture device could, for example, be a camera for monitoring the vehicle interior. This embodiment allows the image of the vehicle occupant to be captured.

[0015] Furthermore, in the reading step, information generated using at least one algorithm can be read in as the spectacle information. This embodiment enables fast and efficient generation of the spectacle information.

[0016] The method can further include a step of generating an attention value representing the vehicle occupant's level of attention, based on the result of processing the information from the glasses. Additionally or alternatively, a control signal for activating a vehicle control unit can be generated in this step, also based on the processing result. This enables a reliable determination of the vehicle occupant's attention or reliable vehicle control.

[0017] This process can be implemented, for example, in software or hardware, or in a hybrid form of software and hardware, such as in a control unit.

[0018] The approach presented here further creates a device designed to perform, control, and implement the steps of a variant of the method presented here in appropriate facilities. This embodiment of the invention in the form of a device also allows the problem underlying the invention to be solved quickly and efficiently.

[0019] For this purpose, the device may include at least one processing unit for processing signals or data, at least one storage unit for storing signals or data, at least one interface to a sensor or actuator for reading sensor signals from the sensor or for outputting data or control signals to the actuator, and / or at least one communication interface for reading or outputting data embedded in a communication protocol. The processing unit may, for example, be a signal processor, a microcontroller, or the like, and the storage unit may be flash memory, an EPROM, or a magnetic storage device.The communication interface can be configured to read or output data wirelessly and / or via wired connections, whereby a communication interface that can read or output wired data can, for example, read this data electrically or optically from or output it into a corresponding data transmission line.

[0020] In this context, a device can be understood as an electrical device that processes sensor signals and outputs control and / or data signals accordingly. The device may have an interface, which can be implemented in hardware and / or software. In the case of a hardware-based interface, the interfaces can, for example, be part of a so-called system ASIC, which incorporates various functions of the device. However, it is also possible that the interfaces are separate integrated circuits or consist at least partially of discrete components. In the case of a software-based interface, the interfaces can be software modules, which, for example, are present on a microcontroller alongside other software modules.

[0021] In an advantageous embodiment, the device controls a driver assistance system of a vehicle. For this purpose, the device can, for example, access sensor signals such as image signals from at least one camera for monitoring the interior of the vehicle, in particular the head area of ​​the vehicle occupant, environmental sensor signals, steering angle sensor signals, or wheel speed sensor signals.

[0022] Also advantageous is a computer program product or computer program with program code that can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and is used to carry out, implement and / or control the steps of the method according to one of the embodiments described above, in particular if the program product or program is executed on a computer or device.

[0023] Exemplary embodiments of the invention are shown in the drawings and explained in more detail in the following description. It shows: Fig. 1 a schematic representation of a vehicle with a device according to an exemplary embodiment; Fig. 2 a schematic representation of the head of a vehicle occupant Fig. 1; Fig. 3 a schematic representation of a device according to an exemplary embodiment; Fig. 4 a schematic representation of a process for locating a head using a device according to an exemplary embodiment; and Fig. 5 a flowchart of a procedure according to an exemplary embodiment.

[0024] In the following description of favorable embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the various figures and acting similarly, without repeating these elements.

[0025] Fig. Figure 1 shows a schematic representation of a vehicle 100 with a device 102 according to an exemplary embodiment. The vehicle 100 is equipped with a detection device 104, which is configured to detect a vehicle occupant 106, here a driver of the vehicle 100, in particular a part of the head 108 of the vehicle occupant 106, and to transmit an image signal 110, which represents at least one image of the vehicle occupant 106 captured during detection, to the device 102. The detection device 104 is, for example, a camera for monitoring the interior of the vehicle 100. The vehicle occupant 106 wears glasses 112. The device 102 is configured to detect the glasses 112 using the image signal 110 and, depending on the detection result, to locate the head 108 in the vehicle interior.

[0026] According to an optional embodiment, the device 102 is designed to determine a position or orientation of the glasses 112 when the glasses 112 are detected, and to determine a position or orientation of the head 108 depending on the position or orientation of the glasses 112.

[0027] According to a further optional embodiment, the device 102 is configured to generate a control signal 114, depending on the result of the localization of the head 108, and to send it to a vehicle control unit 116. The vehicle control unit 116 is configured, for example, to use the control signal 114 to control a driver assistance function to support the vehicle occupant 106 in driving the vehicle 100. Optionally, the device 102 is configured to generate an attention value 118, representing the attention of the vehicle occupant 106, depending on the result of the localization, and to send it to the vehicle control unit 116. In this case, the vehicle control unit 116 is configured to control the vehicle 100 using the attention value 118.

[0028] Fig. Figure 2 shows a schematic representation of the head 108 of a vehicle occupant 106. Fig. 1. In the case of the Fig. The image of head 108 shown in Figure 2 is, for example, an image of head 108 captured by the detection device. A frontal view of head 108 is shown, in which the glasses 112 are visible. The device is designed, for example, to extract the position or orientation of the glasses 112 from the image represented by the image signal using a three-dimensional coordinate system. For this purpose, the device determines the position or orientation of the glasses 112 relative to an x-, y-, or z-axis. Based on x-, y-, and z-coordinates representing the position or orientation of the glasses 112, the device determines x-, y-, and z-coordinates representing the position or orientation of head 108.

[0029] Fig. Figure 3 shows a schematic representation of a device 102 according to an exemplary embodiment. The device 102 can be a previously designed device based on the Fig. 1 and Fig. The device described in Section 2 is a device 102. The device 102 comprises a reading unit 310 configured to read in spectacle information 315 representing the spectacles and forward it to a processing unit 320. The processing unit 320 is configured to locate the head of the vehicle occupant using the spectacle information 315 and to generate a localization value 325 representing a result of the localization. The localization value 325 represents, for example, the position or orientation of the head determined as a function of the position or orientation of the spectacles.

[0030] According to one embodiment, the reading unit 310 is configured to further read the image signal 110 via an interface to the detection device and forward it to an optional generation unit 330. The generation unit 330 is configured to generate the spectacle information 315 using the image signal 110. The reading unit 310 reads the spectacle information 315 via the generation unit 330.

[0031] According to one embodiment, the generating unit 330 is designed to extract the spectacle information 315 from the image signal 110 using at least one algorithm.

[0032] Optionally, the generation unit 330 is designed to further receive the localization value 325 from the processing unit 320, to generate the control signal 114 or the attention value 118 using the localization value 325 and to output the control signal 114 or the attention value 118 to an interface to the vehicle control unit.

[0033] According to a further embodiment, the reading unit 310 also serves to read in at least one vehicle occupant-specific parameter 335. The reading unit 310 passes the parameter 335 to the processing unit 320, which is configured to further generate the localization value 325 using the parameter 335. The parameter 335 represents, for example, a reference position or reference position of the glasses. A reference position or position can be understood as a target position or target position of the glasses, which may, for example, correspond to a normal or average position or position of the glasses. In this case, the processing unit 320 uses the parameter 335 to determine the position or position of the head by comparing the position or position of the glasses with the reference position or position.A robust head orientation estimation can be performed based on such a spectacle orientation estimation.

[0034] Fig. Figure 4 shows a schematic representation of a process for locating a head using a device according to an exemplary embodiment, such as a device as previously described in the Fig. 1, Fig. 2 to Fig. 3 is described. Here, a block 402 represents the driver of the vehicle, a block 402 an input, a block 404 a sensor for detecting the driver, a block 406 a glasses detection, a block 408 a glasses segmentation, a block 410 an optional driver ID, a block 412 at least one optional further parameter, a block 414 a head orientation, a block 416 a processing unit, a block 418 an output, a block 420 a direction of gaze of the driver, a block 422 a sleepiness level of the driver, a block 424 a level of attention of the driver and a block 426 an airbag setting.

[0035] Fig. Figure 5 shows a flowchart of a method 500 according to an exemplary embodiment. The method 500 can, for example, be used in conjunction with a previously described method based on the Fig. 1, Fig. 2, Fig. 3 to Fig.The device described in section 4 is used. In step 510, the eyeglass information is read. In step 520, the eyeglass information is processed to locate the head of the vehicle occupant.

[0036] Steps 510 and 520 can be performed continuously.

[0037] In cases of significant head obstruction, it can be difficult to reliably determine head orientation using existing algorithms. For example, (sun)glasses can heavily obscure a significant eye region that is important for estimating head pose. Additional obstructions around the mouth can also make it difficult or even impossible to reliably determine head orientation.

[0038] Using the 500 procedure, it is now possible to support the driver with enhanced safety and comfort features. Robust head orientation is a fundamental variable for these functions.

[0039] Current fatigue estimation systems are generally based on non-visual parameters, such as steering angle or lane departure warning sensor data. These parameters are significantly further removed in the signal chain than directly measurable physiological changes. Furthermore, steering behavior often does not provide a clear indication of driver fatigue. For example, driving with a trailer can generate a similar steering angle signal to that produced when driving with fatigue.

[0040] The previously described method for estimating head pose also works when the driver is wearing glasses or sunglasses. In this case, the driver's eyes are covered. By determining the head orientation based on the orientation of the glasses, the head orientation can still be reliably determined.

[0041] According to one embodiment, the head position or orientation is determined using 3D data from a depth image of the driver. The glasses are detected and segmented from the depth image. The orientation of the glasses is determined, and the head position and orientation are inferred from this. The position of the glasses provides a robust indication of the head position and orientation.

[0042] For this purpose, the driver is continuously tracked by a sensor. The sensor records both 2D near-infrared data and depth data. The glasses are detected and segmented from the acquired data. This glasses segmentation can be implemented using simple, existing algorithms. Determining the position and orientation of the glasses is significantly easier than determining the position and orientation of the head, since glasses exhibit specific features, particularly in the depth image, such as a planar surface or recognizable edges.

[0043] Optionally, additional person-specific parameters are taken into account via an existing person ID, such as the usual position or orientation of the driver's glasses. Alternatively, an average position or orientation of glasses can be used. Using these parameters, the position or orientation of the head is robustly determined from the position or orientation of the glasses. Optionally, this can be used to determine, for example, the driver's gaze direction, drowsiness level, or attention level.

[0044] If an embodiment includes an “and / or” connection between a first feature and a second feature, this is to be read as meaning that the embodiment according to one embodiment has both the first feature and the second feature, and according to another embodiment either only the first feature or only the second feature.

Claims

[1] Method (500) for locating the head (108) of a vehicle occupant (106) wherein the vehicle occupant (106) is wearing glasses (112), the method (500) comprising the following steps: Reading (510) spectacle information (315) representing the spectacles (112) wherein in the reading step (510) a position and / or orientation of the spectacles (112) is read as the spectacle information (315) and a reference position and / or reference orientation of the spectacles (112) is read as a vehicle occupant-specific parameter (335); and Processing (520) the spectacle information (315) to locate the head (108), wherein in the processing step (520) the head (108) is located by comparing the position and / or location of the spectacles (112) with the reference position and / or reference location. [2] Method (500) according to claim 1, wherein in the processing step (520) the spectacle information (315) is processed to determine a position and / or orientation of the head (108). [3] Method (500) according to claim 1 or 2, wherein in the reading step (510) information generated using at least one image of the vehicle occupant (106) is read in as the glasses information (315). [4] Method (500) according to claim 3, comprising a step of capturing the image of the vehicle occupant (106) by a capture device (104) for capturing the vehicle occupant (106). [5] Method (500) according to one of the preceding claims, wherein in the reading step (510) information generated using at least one algorithm is read in as the spectacle information (315). [6] Method (500) according to one of the preceding claims, comprising a step of generating an attention value (118) representing the attention of the vehicle occupant (106) and / or a control signal (114) for controlling a vehicle control unit (116) depending on a result of processing (520) the spectacle information (315). [7] Device (102) comprising units (310, 320, 330) configured to perform and / or control the method (500) according to any of the preceding claims. [8] Computer program configured to execute and / or control the method (500) according to any one of claims 1 to 6. [9] Machine-readable storage medium on which the computer program according to claim 8 is stored.

Images