Method for determining a state of a driver monitoring device, and driver monitoring device

Abstract

A method for determining a state of a driver monitoring device, having an illumination device for illuminating a passenger compartment of a vehicle, a diffuser, and an image-capturing device, comprises the following steps: b) illuminating the passenger compartment and generating at least one first image of the passenger compartment that is based on an image, captured by the image-capturing device, of the passenger compartment illuminated by way of the illumination device, c) dividing the at least one first image into multiple image sections, d) determining a maximum intensity of pixels of a first group of image sections of the at least one first image, and determining a mean intensity of a second group of image sections of the at least one first image, e) determining an illumination difference in the at least one first image by dividing the maximum intensity of the first group by the mean intensity of the second group, and f) determining that the diffuser is operational and arranged correctly and activating the illumination device for driver monitoring when the illumination difference in the at least one first image is less than a lower threshold value, or determining that the diffuser is not operational and arranged correctly and deactivating the illumination device when the illumination difference is greater than an upper threshold value.

Description

[0001] METHOD FOR DETERMINING A STATE OF A DRIVER MONITORING DEVICE AND DRIVER MONITORING DEVICE

[0002] The present invention relates to a method for determining a state of a driver monitoring device and a driver monitoring device.

[0003] Driver monitoring devices comprising an illumination device, for example in the form of a laser, in particular in the form of a VCSEL (Vertical Cavity Surface Emitting Laser), for illuminating an area of ​​a vehicle interior in which the driver's head is located, and an image capture device for capturing an image of the driver's head illuminated by the illumination device, serve to determine, based on the image of the driver's head, in particular of the eyes, whether the driver is distracted or tired, and, if necessary, to issue a warning signal to warn the driver or request that they take a break. Since direct irradiation of the light from the illumination device onto the driver's head would damage the eyes, a diffuser is provided in the light beam path to scatter the light in such a way that the intensity of the light hitting the eyes is too low to cause eye damage.However, if the diffuser becomes loose and falls off, or is otherwise impaired in its function, there is a risk of damage to the driver's eyes.

[0004] It is an object of the present invention to provide a method for determining a state of a driver monitoring device and a corresponding driver monitoring device with which the safety of the driver can be increased.

[0005] This object is achieved by the features of the independent claims. Advantageous developments of the invention are the subject of the dependent claims.

[0006] A method for determining a state of a driver monitoring device, which has a lighting device for illuminating an interior of a vehicle, a diffuser arranged in a beam path of the lighting device when the driver monitoring device is in proper condition for scattering a light emitted by the lighting device, and an image capture device for capturing an image of the interior, comprises, according to one embodiment, the following steps: b) illuminating the interior by means of the lighting device and generating at least one first image of at least a portion of the interior, which image is based on an image of the interior illuminated by means of the lighting device captured by the image capture device, c) dividing the at least one first image into a left upper image portion, a right upper image portion, and a left middle image portion,a right middle image section, a left lower image section and a right lower image section, d) determining a maximum intensity of pixels of a first group of image sections of the at least one first image, which has the left middle image section and the right middle image section, and determining an average intensity of a second group of image sections of the at least one first image, which has the left upper image section, the right upper image section and the right lower image section or the left upper image section, the right upper image section and the left lower image section, e) determining an illumination difference of the at least one first image by dividing the maximum intensity of the first group of image sections of the at least one first image by the average intensity of the second group of image sections of the at least one first image, and f) determining,that the diffuser is functional and properly arranged in the beam path of the lighting device and activating the lighting device for driver monitoring if the illumination difference of the at least one first image is less than a lower threshold, or determining that the diffuser is not functional and properly arranged in the beam path of the lighting device and deactivating the lighting device if the illumination difference is greater than an upper threshold.

[0007] The illumination device can, in particular, comprise a laser, preferably a VCSEL. In this case, the image capture device is configured such that the driver's head is located substantially in the center of the field of view of the image capture device, while the illumination device is configured such that, without a diffuser connected upstream, the majority of the light falls on the driver's head.

[0008] Furthermore, in step c), the at least one image is preferably divided into six image sections of equal size, wherein in step d) the second group is selected in particular such that only a small section of an image of a side window of the vehicle is contained in the image sections of the group in order to minimize the influence of extraneous light.

[0009] The upper threshold value and the lower threshold value, with which the illumination difference is compared in step f) in order to determine whether the diffuser is functional and properly arranged in the beam path of the illumination device, are preferably determined or set using an empirical method in which images of a driver's head located in the interior are captured with the image capture device in states in which the interior is illuminated with the illumination device under different (external) lighting conditions and the diffuser is either functional and properly arranged in the beam path of the illumination device or not.

[0010] Due to this configuration of the driver monitoring device, in many cases it is already possible to determine, by comparing the difference in illumination with the upper or lower threshold value, whether the diffuser is functional and properly arranged in the beam path of the lighting device or not, and thus whether or not there is a risk of damage to the driver's eyes. In the event of a potential risk to the eyes, the lighting device is deactivated to prevent (further) damage.

[0011] According to one embodiment, the method further comprises the steps of: g) applying a Laplace filter to the at least one first image if the illumination difference is greater than the lower threshold and less than the upper threshold to obtain a modified at least one first image, and detecting at least one edge in the modified at least one first image, h) setting an intensity value of pixels of the modified at least one first image that do not have a portion of the at least one edge to a maximum intensity value, and setting the intensity value of pixels of the modified at least one first image that have a portion of the at least one edge to a minimum intensity value to obtain a modified at least one first image, i) applying a median filter to the modified at least one first image to obtain a filtered at least one first image,j) removing one or more sections of the filtered at least one first image, wherein at least one section contains an image of at least a section of a steering wheel area of ​​the vehicle, in order to obtain a test image section of the filtered at least one first image, and k) determining a number of pixels with maximum intensity in the test image section of the filtered at least one first image and determining, based on the number of pixels with maximum intensity in the test image section of the filtered at least one first image, in particular based on a number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, whether the diffuser is functional and properly arranged in the beam path of the illumination device or not,and storing corresponding information in a memory unit of a control unit of the vehicle.,

[0012] In one embodiment, it is thereby possible to correctly determine with a high degree of certainty whether the diffuser is functional and properly arranged in the beam path of the illumination device, even under unfavourable (external) lighting conditions.

[0013] The threshold value can be determined or adjusted in an analogous manner as described above using an empirical method, preferably using a confusion matrix.

[0014] According to one embodiment, the method further comprises the following steps: b1) generating at least one second image of the at least one section of the interior, which is based on an image that is captured by the image capture device while the interior is not illuminated by the illumination device, c1) dividing the at least one second image into a left upper image section, a right upper image section, a left middle image section, a right middle image section, a left lower image section and a right lower image section, d1) determining a maximum intensity of pixels of a first group of image sections of the at least one second image, which has the left lower image section and the right lower image section, and determining an average intensity of a second group of image sections of the at least one second image, which has the left upper image section,the upper right image section and the lower right image section or the upper left image section, the upper right image section and the lower left image section, e1) determining an illumination difference of the at least one second image by dividing the maximum intensity of the first group of image sections of the at least one second image by the average intensity of the second group of image sections of the at least one second image, g1) applying a Laplace filter to the at least one second image if the illumination difference is greater than the lower threshold value and less than the upper threshold value to obtain a modified at least one second image, and detecting at least one edge in the modified at least one second image, h1) setting an intensity value of pixels of the modified at least one second image that do not have a section of the at least one edge,to a maximum intensity value, and setting the intensity value of pixels of the modified at least one second image that have a portion of the at least one edge to a minimum intensity value to obtain a modified at least one second image, i1) applying a median filter to the modified at least one second image to obtain a filtered at least one second image, j1) removing one or more portions of the filtered at least one second image, wherein at least one portion includes an image of at least a portion of a steering wheel area of ​​the vehicle, to obtain a test image portion of the filtered at least one second image, and k1) determining a number of pixels with maximum intensity in the test image portion of the filtered at least one second image,wherein in step k) it is determined whether the diffuser is functional and properly arranged in the beam path of the illumination device or not based on the number of pixels with maximum intensity in the test image section of the filtered at least one first image and the number of pixels with maximum intensity in the test image section of the filtered at least one second image, in particular based on a number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value.

[0015] According to one embodiment, in step b), a first image is generated which is based on an image acquired by the image capture device with a first exposure time of preferably 36 gs, and another first image is generated which is based on an image acquired by the image capture device with a second exposure time of preferably 640 gs, in step b1), a second image is generated which is based on an image acquired by the image capture device with the first exposure time, and another second image is generated which is based on an image acquired by the image capture device with the second exposure time, and in step k), based on the number of pixels with maximum intensity in the test image section of the filtered one first image, the number of pixels with maximum intensity in the test image section of the filtered other first image,the number of pixels with maximum intensity in the test image section of the one filtered second image and the number of pixels with maximum intensity in the test image section of the other filtered second image determines whether the diffuser is functional and properly arranged in the beam path of the illumination device or not.

[0016] According to one embodiment, in step b), the interior is illuminated by means of the lighting device with light of an intensity that is lower than the intensity of a light with which the interior is illuminated during normal operation of the driver monitoring device. In this case, in step b), the intensity of the light can advantageously be selected such that even if the light hits the driver's eyes directly, i.e., without it having been previously scattered by a diffuser, the eyes are not damaged. According to one embodiment, generating the at least one first image and / or the at least one second image comprises extracting an image section of an image based on the image of the interior captured by the image capture device and / or reducing a file size of an image based on the image of the interior captured by the image capture device.

[0017] According to one embodiment, in step b) and / or in step b1), a left-side image section is extracted when the vehicle is configured as a right-hand drive vehicle, and a right-side image section is extracted when the vehicle is configured as a left-hand drive vehicle, wherein in step d) and / or in step d1), the second group comprises the upper left image section, the upper right image section and the lower right image section when the vehicle is configured as a right-hand drive vehicle, and the second group comprises the upper left image section, the upper right image section and the lower left image section when the vehicle is configured as a left-hand drive vehicle.

[0018] According to one embodiment, the method further comprises the following step, carried out before step b): a) communicating with the control unit of the vehicle in order to determine, based on the information stored in the memory unit, whether the diffuser is functional and properly arranged in the beam path of the lighting device or not, and permanently deactivating the lighting device and terminating the method if the information is stored in the memory unit that the diffuser is not functional and properly arranged in the beam path of the lighting device, or continuing the method with step b) if the information is stored in the memory unit that the diffuser is functional and properly arranged in the beam path of the lighting device.

[0019] A driver monitoring device according to one embodiment comprises a lighting device for illuminating an interior of a vehicle, a diffuser arranged in a beam path of the lighting device when the driver monitoring device is in proper condition for scattering a light emitted by the lighting device, and an image capture device for capturing an image of the interior, wherein the driver monitoring device is configured to b) illuminate the interior by means of the lighting device and to generate at least a first image of at least a portion of the interior, which image is based on an image captured by the image capture device of the interior illuminated by the lighting device, c) divide the at least one first image into a left upper image portion, a right upper image portion, a left middle image portion, a right middle image portion,to divide a lower left image section and a lower right image section, d) to determine a maximum intensity of pixels of a first group of image sections of the at least one first image, which has the left middle image section and the right middle image section, and an average intensity of a second group of image sections of the at least one first image, which has the upper left image section, the upper right image section and the lower right image section or the upper left image section, the upper right image section and the lower left image section, e) to determine an illumination difference of the at least one first image by dividing the maximum intensity of the first group of image sections of the at least one first image by the average intensity of the second group of image sections of the at least one first image, and f) to determinethat the diffuser is functional and properly arranged in the beam path of the lighting device and to activate the lighting device for driver monitoring if the illumination difference of the at least one first image is less than a lower threshold, or to determine that the diffuser is not functional and properly arranged in the beam path of the lighting device and to deactivate the lighting device if the illumination difference is greater than an upper threshold.

[0020] According to one embodiment, the driver monitoring device is further configured to: g) apply a Laplace filter to the at least one first image if the illumination difference is greater than the lower threshold and less than the upper threshold, in order to obtain a modified at least one first image, and to detect at least one edge in the modified at least one first image, h) set an intensity value of pixels of the modified at least one first image that do not have a portion of the at least one edge to a maximum intensity value, and set the intensity value of pixels of the modified at least one first image that have a portion of the at least one edge to a minimum intensity value, in order to obtain a modified at least one first image, i) apply a median filter to the modified at least one first image to obtain a filtered at least one first image,j) removing one or more sections of the filtered at least one first image, wherein at least one section contains an image of at least a section of a steering wheel area of ​​the vehicle, in order to obtain a test image section of the filtered at least one first image, and k) determining a number of pixels with maximum intensity in the test image section of the filtered at least one first image and, based on the number of pixels with maximum intensity in the test image section of the filtered at least one first image, in particular based on a number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, determining whether the diffuser is functional and properly arranged in the beam path of the illumination device or not,and to store corresponding information in a memory unit of a control unit of the vehicle.,

[0021] According to one embodiment, the driver monitoring device is further configured to b1) generate at least one second image of the at least one section of the interior, which is based on an image captured by the image capture device while the interior is not illuminated by the illumination device, c1) divide the at least one second image into a left upper image section, a right upper image section, a left middle image section, a right middle image section, a left lower image section and a right lower image section, d1) determine a maximum intensity of pixels of a first group of image sections of the at least one second image, which group comprises the left lower image section and the right lower image section, and an average intensity of a second group of image sections of the at least one second image, which group comprises the left upper image section,the upper right image section and the lower right image section or the upper left image section, the upper right image section and the lower left image section, e1) to determine an illumination difference of the at least one second image by dividing the maximum intensity of the first group of image sections of the at least one second image by the average intensity of the second group of image sections of the at least one second image, g1) to apply a Laplace filter to the at least one second image if the illumination difference is greater than the lower threshold value and less than the upper threshold value in order to obtain a modified at least one second image, and to detect at least one edge in the modified at least one second image, h1) an intensity value of pixels of the modified at least one second image which do not have a section of the at least one edge,to a maximum intensity value, and to set the intensity value of pixels of the modified at least one second image that have a portion of the at least one edge to a minimum intensity value to obtain a modified at least one second image, i1) to apply a median filter to the modified at least one second image to obtain a filtered at least one second image, j1) to remove one or more portions of the filtered at least one second image, wherein at least one portion includes an image of at least a portion of a steering wheel area of ​​the vehicle, to obtain a test image portion of the filtered at least one second image, and k1) to determine a number of pixels with maximum intensity in the test image portion of the filtered at least one second image, wherein the driver monitoring device is configured toin k) based on the number of pixels with maximum intensity in the test image section of the filtered at least one first image and the number of pixels with maximum intensity in the test image section of the filtered at least one second image, in particular based on a number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, to determine whether the diffuser is functional and properly arranged in the beam path of the illumination device or not.

[0022] According to one embodiment, the driver monitoring device is configured to: in b) generate a first image based on an image captured by the image capture device with a first exposure time of preferably 36 gs and to generate another first image based on an image captured by the image capture device with a second exposure time of preferably 640 gs, in b1) generate a second image based on an image captured by the image capture device with the first exposure time and to generate another second image based on an image captured by the image capture device with the second exposure time, and in k) based on the number of pixels with maximum intensity in the test image section of the filtered first image,the number of pixels with maximum intensity in the test image section of the filtered other first image, the number of pixels with maximum intensity in the test image section of the one filtered second image, and the number of pixels with maximum intensity in the test image section of the filtered other second image to determine whether the diffuser is functional and properly arranged in the beam path of the illumination device or not.

[0023] According to one embodiment, the driver monitoring device is configured to illuminate the interior by means of the lighting device with light of an intensity that is lower than an intensity of a light with which the interior is illuminated during normal operation of the driver monitoring device.

[0024] According to one embodiment, the driver monitoring device is configured to generate the at least one first image and / or the at least one second image by extracting an image portion of an image based on the image of the interior captured by the image capture device and / or reducing a file size of an image based on the image of the interior captured by the image capture device.

[0025] According to one embodiment, the driver monitoring device is configured to extract a left-side image section in b) and / or b1) when the vehicle is configured as a right-hand drive vehicle and to extract a right-side image section when the vehicle is configured as a left-hand drive vehicle, wherein in d) and / or d1) the second group comprises the upper left image section, the upper right image section and the lower right image section when the vehicle is configured as a right-hand drive vehicle, and the second group comprises the upper left image section, the upper right image section and the lower left image section when the vehicle is configured as a left-hand drive vehicle.

[0026] According to one embodiment, the driver monitoring device is configured to communicate with the control unit of the vehicle before b) a) in order to determine, based on the information stored in the memory unit, whether the diffuser is functional and properly arranged in the beam path of the lighting device or not, and to permanently deactivate the lighting device and terminate the driver monitoring operation if the information is stored in the memory unit that the diffuser is not functional and properly arranged in the beam path of the lighting device, or to continue with b) if the information is stored in the memory unit that the diffuser is functional and properly arranged in the beam path of the lighting device.

[0027] Further advantages, features and possible applications of the present invention will become apparent from the following detailed description in conjunction with the figures.

[0028] It shows:

[0029] Fig. 1 schematically shows a vehicle with a driver monitoring device according to an embodiment of the invention;

[0030] Fig. 2 shows an image of an interior of a vehicle captured by an image capture device; Fig. 3 shows an image of the interior generated based on the image captured by the image capture device, divided into six image sections;

[0031] Fig. 4 shows an image of the interior generated based on an image captured by the image capture device, which image is divided into six image sections, in which a diffuser for scattering a light emitted by a lighting device for illuminating the interior is not functional and is properly arranged in a beam path of the lighting device;

[0032] Fig. 5 shows an image of the interior generated based on an image captured by the image capture device, which image is divided into six image sections, in which the diffuser is functional and properly arranged in the beam path of the illumination device;

[0033] Fig. 6 shows an image of the interior generated based on an image captured by the image capture device, in which the diffuser is not functional and is properly arranged in the beam path of the lighting device;

[0034] Fig. 7 shows the image shown in Fig. 6 to which a Laplace filter has been applied;

[0035] Fig. 8 shows an image modified based on the image shown in Fig. 7;

[0036] Fig. 9 is a filtered image based on the image shown in Fig. 8 to which a median filter has been applied;

[0037] Fig. 10 shows an image of the interior generated based on an image captured by the image capture device;

[0038] Fig. 11 shows an image section of an image based on the generated image shown in Fig. 10; and

[0039] Fig. 12 shows the image shown in Fig. 1 1 from which an image of a steering wheel area has been removed.

[0040] Fig. 1 shows schematically a vehicle with a driver monitoring device according to an embodiment of the invention.

[0041] The vehicle 10, for example a passenger car, has a control unit 13, which has a memory unit 14, and a driver monitoring device 100. The driver monitoring device 100 has an illumination device 110, for example a laser, in particular a VCSEL, for illuminating an interior 11 of the vehicle 10, illustrated in Fig. 2, an image capture device 130, and a diffuser, which, when the driver monitoring device 100 is in a proper state, is arranged in a beam path of the illumination device 110 in order to scatter the light emitted by the illumination device 110.

[0042] The image capture device 130 is configured to capture an image of the head, in particular the eyes, of a driver of the vehicle 10 in order to determine, based on the image, whether the driver is tired or distracted. The illumination device 110 is configured to illuminate, in particular, the driver's head.

[0043] The storage unit 14 is configured to store information about whether the diffuser 120 is functional and properly arranged in the beam path of the illumination device 130 or not.

[0044] In the method according to the invention or by means of the driver monitoring device 100 according to the invention, communication is first carried out with the control unit 13 of the vehicle 10 in order to determine, based on the information stored in the memory unit 14, whether the diffuser 120 is functional and properly arranged in the beam path of the illumination device 130 or not.

[0045] If it is determined that the diffuser 120 is not functional and is properly positioned in the beam path of the lighting device 130, the lighting device 110 is permanently deactivated and the method or driver monitoring operation is terminated. However, if it is determined that the diffuser 120 is functional and properly positioned in the beam path of the lighting device 130, the method according to the invention continues with the steps described below.

[0046] Fig. 2 shows an image 20 of the interior 11 of a vehicle 10 configured as a right-hand drive, captured by the image capture device 130, which image has a left-side image section 21 and a right-side image section 22, which is of interest in the present case because it contains an image of the driver's head if the driver is sitting in the vehicle 10. Fig. 3 shows an image 30 of the interior 11 generated based on the image 20 shown in Fig. 2, wherein a file size of the image 20 was reduced, the right-side image section 22 was extracted, and the image was divided into six image sections, namely a left upper image section 31, a right upper image section 32, a left middle image section 33, a right middle image section 34, a left lower image section 35, and a right lower image section 36.

[0047] Fig. 4 shows an image 30 of the interior 11 generated based on an image 20 captured by the image capture device 130, which image contains an image of the driver. The generated image 30 is divided into six image sections 31-36, and the diffuser 120 was not functional and properly positioned in the beam path of the illumination device when the image 20 was captured. As can be seen from Fig. 4, in this case, the driver's head is brightly illuminated by the illumination device 110, which could damage the driver's eyes, while the rest of the image 30 is essentially dark.

[0048] Fig. 5 shows an image 30 of the interior 11 generated based on an image 20 captured by the image capture device 130, which image contains an image of the driver. The generated image 30 is divided into six image sections 31-36, and the diffuser 120 was functional and properly arranged in the beam path of the illumination device when the image 20 was captured. As can be seen from Fig. 5, in this case, the brightness of the generated image 30 is essentially homogeneous due to the scattering of the light from the illumination device 110 by means of the diffuser 120, thereby preventing damage to the driver's eyes.

[0049] In the method according to the invention, a maximum intensity of pixels of a first group 33, 34 of image sections of the image 30, which group comprises the left-center image section 33 and the right-center image section 34, and an average intensity of a second group 31, 32, 36; 31, 32, 35 of image sections of the image 30 are determined, which group comprises the upper left image section 31, the upper right image section 32 and the lower right image section 36 or the upper left image section 31, the upper right image section 32 and the lower left image section 35. Since in the present case the vehicle 10 is configured as a right-hand drive, the second group comprises the upper left image section 31, the upper right image section 32 and the lower right image section 36.

[0050] Then, an illumination difference of the image 30 is determined by dividing the maximum intensity of the first group 33, 34 of image sections of the image 30 by the average intensity of the second group 31, 32, 36; 31, 32, 35 of image sections of the image 30. Based on the determined illumination difference, it is then determined that the diffuser 120 is functional and properly arranged in the beam path of the illumination device 110, and that the illumination device 110 is activated for driver monitoring if the illumination difference of the image 30 is less than a lower threshold, or it is determined that the diffuser 120 is not functional and properly arranged in the beam path of the illumination device 110, and that the illumination device 110 is deactivated if the illumination difference is greater than an upper threshold.

[0051] If the illumination difference is greater than the lower threshold and less than the upper threshold, the image undergoes 30 further processing steps, as explained below.

[0052] Fig. 6 shows an image 30 of the interior 11 generated based on an image 20 captured by the image capture device 130, which image contains an image of the driver, and in which the diffuser 120 is not functional and is properly arranged in the beam path of the illumination device 110. Here, the illumination difference is greater than the lower threshold and smaller than the upper threshold.

[0053] Fig. 7 shows an image 30' modified based on the image 30 shown in Fig. 6, wherein a Laplace filter has been applied to the image shown in Fig. 6, whereby at least one edge 37 becomes visible in the modified image 30', which edge is detected by means of the method according to the invention or the driver monitoring device 100.

[0054] Thereafter, an intensity value of pixels of the modified image 30' that do not include a portion of the at least one edge 37 is set to a maximum intensity value, and the intensity value of pixels of the modified image 30' that include a portion of the at least one edge 37 is set to a minimum intensity value to obtain a modified image 30" illustrated in Fig. 8.

[0055] A median filter is then applied to the modified image 30" illustrated in Fig. 8 to obtain a filtered image 30'" illustrated in Fig. 9. One or more sections are removed from the filtered image 30'" illustrated in Fig. 9, for example by setting the pixels of these sections to the minimum intensity value, wherein at least one of these sections contains an image of at least a section of a steering wheel area 12 of the vehicle 10, on which, as illustrated in Fig. 12, a driver's hand is usually positioned and which, due to the reflection of the light emitted by the illumination device 110 by the skin of the hand, leads / would lead to pixels with high intensity in order to obtain a test image section of the filtered image 30'". Fig. 11 shows an image section of an image based on the generated image shown in Fig. 10; and Fig. 12 shows the image shown in Fig.11, from which the steering wheel area 12 has been removed.

[0056] Thereafter, a number of pixels with maximum intensity in the test image section of the filtered image 30'" is determined. Based on the number of pixels with maximum intensity in the test image section of the filtered image 30'", in particular based on a number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, it is then determined whether the diffuser 120 is functional and properly arranged in the beam path of the lighting device 110 or not, and corresponding information is stored in the memory unit 14 of the control unit 13 of the vehicle 10.

[0057] Preferably, in the method according to the invention / by means of the driver monitoring device 110 according to the invention, several images of the interior 11, in particular four images, are captured by the image capture device 130, wherein two of the images are captured with different exposure times while the interior 11 is illuminated by the illumination device 110, and two of the images are captured with the different exposure times while the interior 11 is not illuminated by the illumination device 110. The four images are then processed as described above, and based on the number of pixels with maximum intensity in the respective test image sections, it is determined whether the diffuser 120 is functional and properly arranged in the beam path of the illumination device 110 or not. LIST OF REFERENCE SYMBOLS

[0058] 10 vehicles

[0059] 11 Interior 12 Steering wheel area

[0060] 13 Control unit

[0061] 14 storage unit

[0062] 20 Image Interior

[0063] 21 left section of the interior image 22 right section of the interior image

[0064] 30 first picture

[0065] 37 edge

[0066] 100 Driver monitoring device

[0067] 110 Lighting device 120 Diffuser

[0068] 130 Image capture device

Claims

CLAIMS Method for determining the state of a driver monitoring device (100), which comprises a lighting device (110) for illuminating an interior (1 1 ) of a vehicle (10), a diffuser (120) arranged in a beam path of the lighting device (1 10) when the driver monitoring device (100) is in proper working order, for scattering light emitted by the lighting device (110), and an image acquisition device (130) for capturing an image (20) of the interior (11 ), wherein the method comprises the following steps: b) illuminating the interior (11 ) by means of the lighting device (110), and generating at least a first image (30) of at least a section of the interior (1 1 ), which is based on an image (20) of the interior (11 ) illuminated by means of the lighting device (110) captured by the image acquisition device (130),c) Dividing the at least one first image (30) into an upper left image section (31), an upper right image section (32), a middle left image section (33), a middle right image section (34), a lower left image section (35), and a lower right image section (36); d) Determining a maximum intensity of pixels of a first group (33, 34) of image sections of the at least one first image (30) comprising the middle left image section (33) and the middle right image section (34), and determining a mean intensity of a second group (31, 32, 36; 31, 32, 35) of image sections of the at least one first image (30) comprising the upper left image section (31), the upper right image section (32), and the lower right image section (36), or the upper left image section (31), the upper right image section (32), and the lower left image section (35).e) Determining an illumination difference of the at least one first image (30) by dividing the maximum intensity of the first group (33, 34) of image sections of the at least one first image (30) by the mean intensity of the second group (31, 32, 36; 31, 32, 35) of image sections of the at least one first image (30), and, f) Determine that the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110) and activate the lighting device (110) for driver monitoring when the illumination difference of the at least one first image (30) is less than a lower threshold, or determine that the diffuser (120) is not functional and properly arranged in the beam path of the lighting device (110) and deactivate the lighting device (110) when the illumination difference is greater than an upper threshold. Method according to claim 1, further comprising the steps of: g) Applying a Laplace filter to the at least one first image (30) when the illumination difference is greater than the lower threshold and less than the upper threshold to obtain a modified at least one first image (30'),and capturing at least one edge (37) in the modified at least one first image (30'), h) setting the intensity value of pixels of the modified at least one first image (30') that do not have a section of the at least one edge (37) to a maximum intensity value, and setting the intensity value of pixels of the modified at least one first image (30') that have a section of the at least one edge (37) to a minimum intensity value, in order to obtain a modified at least one first image (30"), i) applying a median filter to the modified at least one first image (30") to obtain a filtered at least one first image (30"'), j) removing one or more sections of the filtered at least one first image (30"'), wherein at least one section contains an image of at least one section of a steering wheel area (12) of the vehicle (10),to obtain a test image section of the filtered at least one first image (30"'), and k) determining a number of pixels with maximum intensity in the test image section of the filtered at least one first image (30"') and determining, based on the number of pixels with maximum intensity in the test image section of the filtered at least one first image (30"'), in particular, based on a number of pixels scaled to a predetermined image size at maximum intensity and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110) or not, and storing corresponding information in a storage unit (14) of a control unit (13) of the vehicle (10). Method according to claim 2, further comprising the following steps: b1) generating at least one second image of the at least one section of the interior (11), which is based on an image that is captured by the image acquisition device (130) while the interior (11) is not illuminated by the lighting device (110), c1) dividing the at least one second image into a left upper image section, a right upper image section, and a left middle image section.a right middle image section, a left lower image section and a right lower image section, d1) Determining a maximum intensity of pixels of a first group of image sections of the at least one second image, which includes the left lower image section and the right lower image section, and determining a mean intensity of a second group of image sections of the at least one second image, which includes the left upper image section, the right upper image section and the right lower image section or the left upper image section, the right upper image section and the left lower image section, e1) Determining an illumination difference of the at least one second image by dividing the maximum intensity of the first group of image sections of the at least one second image by the mean intensity of the second group of image sections of the at least one second image,g1) Applying a Laplace filter to at least one second image when the illumination difference is greater than the lower threshold and less than the upper threshold, to obtain a modified at least one second image, to obtain, and capture at least one edge in the modified at least one second image, h1) setting an intensity value of pixels of the modified at least one second image which do not have a section of the at least one edge to a maximum intensity value, and setting the intensity value of pixels of the modified at least one second image which have a section of the at least one edge to a minimum intensity value, in order to obtain a modified at least one second image, i1) applying a median filter to the modified at least one second image in order to obtain a filtered at least one second image, j1) removing one or more sections of the filtered at least one second image, wherein at least one section contains an image of at least one section of a steering wheel area (12) of the vehicle (10), in order to obtain a test image section of the filtered at least one second image,and k1) Determining a number of pixels with maximum intensity in the test image section of the filtered at least one second image, wherein in step k) based on the number of pixels with maximum intensity in the test image section of the filtered at least one first image (30) and the number of pixels with maximum intensity in the test image section of the filtered at least one second image, in particular based on a number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, it is determined whether the diffuser (120) is functional and properly arranged in the beam path of the illumination device (110) or not. Method according to claim 3, wherein in step b) a first image (30) is generated which is based on an image (20),which is captured by the image acquisition device (130) with a first exposure time of preferably 36 ps and another first image is generated which is based on an image that is captured by the image acquisition device with a second exposure time of preferably 640 ps, ​​in step b1) a second image is generated which is based on, based on an image captured by the image acquisition device (130) with the first exposure time and another second image is generated, which is based on an image captured by the image acquisition device (130) with the second exposure time, and in step k) based on the number of pixels with maximum intensity in the test image section of the filtered first image, the number of pixels with maximum intensity in the test image section of the other filtered first image, the number of pixels with maximum intensity in the test image section of one filtered second image and the number of pixels with maximum intensity in the test image section of the other filtered second image, it is determined whether the diffuser (120) is functional and properly arranged in the beam path of the illumination device (110) or not.

5. Method according to one of the preceding claims, wherein in step b) the interior (11 ) is illuminated by means of the lighting device (110) with light of an intensity which is lower than an intensity of light with which the interior (11 ) is illuminated during normal operation of the driver monitoring device (100).

6. Method according to one of the preceding claims, wherein the generation of the at least one first image (30) and / or the at least one second image comprises extracting an image section (21, 22) of an image based on the image (20) of the interior (11) captured by the image capture device (130) and / or reducing a file size of an image based on the image of the interior captured by the image capture device (130).

7. Method according to the preceding claim, wherein in step b) and / or in step b1) a left-side image section (21) is extracted when the vehicle (10) is configured as a right-hand drive vehicle, and a right-side image section (22) is extracted when the vehicle is configured as a left-hand drive vehicle, and in step d) and / or in step d1) the second group comprises the upper left image section (31), the upper right image section (32) and the lower right image section (36) when the vehicle (10) is configured as a right-hand drive vehicle, and the second group comprises the upper left image section (31), the upper right image section (32) and the lower left image section (35) when the vehicle (10) is configured as a left-hand drive vehicle. A method according to any of the preceding claims and claim 2, further comprising the following step performed prior to step b): a) communicating with the control unit (13) of the vehicle (10) to determine, based on the information stored in the storage unit (14), whether the diffuser (120) is functional and properly positioned in the beam path of the lighting device (130) or not, and permanently deactivating the lighting device (110) and terminating the method if the information stored in the storage unit (14) indicates that the diffuser (120) is not functional and properly positioned in the beam path of the lighting device (110), or continuing the method with step b) if the information stored in the storage unit (14) indicates that the diffuser (120) is functional and properly positioned in the beam path of the lighting device (110). Driver monitoring device (100),which comprises a lighting device (110) for illuminating an interior (11) of a vehicle (10), a diffuser (120) arranged in a beam path of the lighting device (110) when the driver monitoring device (100) is in proper working order, for scattering light emitted by the lighting device (110), and an image acquisition device (130) for acquiring an image (20) of the interior (11), wherein the driver monitoring device (100) is configured to: b) illuminate the interior (11) by means of the lighting device (110), and to generate at least a first image (30) of at least a section of the interior (11), which is based on an image (20) of the interior (11) illuminated by means of the lighting device (110) acquired by the image acquisition device (130); c) divide the at least one first image (30) into an upper left image section (31), a upper right section of the image (32),d) to divide a left middle image section (33), a right middle image section (34), a left lower image section (35) and a right lower image section (36), d) a maximum intensity of pixels of a first group (33, 34) of image sections of the at least one first image (30) which has the left middle image section (33) and the right middle image section (34), and to determine an average intensity of a second group (31, 32, 36; 31, 32, 35) of image sections of the at least one first image (30), comprising the upper left image section (31), the upper right image section (32) and the lower right image section (36) or the upper left image section (31), the upper right image section (32) and the lower left image section (35), e) to determine an illumination difference of the at least one first image (30) by dividing the maximum intensity of the first group (33, 34) of image sections of the at least one first image (30) by the average intensity of the second group (31, 32, 36; 31, 32, 35) of image sections of the at least one first image (30), and f) to determine that the diffuser (120) is functional and properly arranged in the beam path of the illumination device (110). is and to activate the lighting device (110) for driver monitoring,if the illumination difference of the at least one first image (30) is less than a lower threshold, or to determine that the diffuser (120) is not functioning and is properly arranged in the beam path of the lighting device (110) and to deactivate the lighting device (110) if the illumination difference is greater than an upper threshold. Driver monitoring device (100) according to claim 9, wherein the driver monitoring device (100) is further configured to: g) apply a Laplace filter to the at least one first image (30) if the illumination difference is greater than the lower threshold and less than the upper threshold, in order to obtain a modified at least one first image (30'), and to detect at least one edge (37) in the modified at least one first image (30'), h) determine an intensity value of pixels of the modified at least one first image (30'),to set the intensity value of pixels of the modified at least one first image (30') which do not have a section of the at least one edge (37) to a maximum intensity value, and to set the intensity value of pixels of the modified at least one first image (30") which have a section of the at least one edge (37) to a minimum intensity value in order to obtain a modified at least one first image (30"), i) to apply a median filter to the modified at least one first image (30") to obtain a filtered at least one first image (30"), j) to remove one or more sections of the filtered at least one first image (30"), wherein at least one section contains an image of at least one section of a steering wheel area (12) of the vehicle (10) to obtain a test image section of the filtered at least one first image (30"), and k) to determine a number of pixels with maximum intensity in the test image section of the filtered at least one first image (30") and, based on the number of pixels with maximum intensity in the test image section of the filtered at least one first image (30"), in particular based on a number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value,whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110) or not, and to store corresponding information in a storage unit (14) of a control unit (13) of the vehicle (10). Driver monitoring device (100) according to claim 10, wherein the driver monitoring device (100) is further configured to: b1) generate at least one second image of the at least one section of the interior (11), which is based on an image captured by the image acquisition device (130) while the interior (11) is not illuminated by the lighting device (110); c1) divide the at least one second image into a left upper image section, a right upper image section, a left middle image section, a right middle image section, a left lower image section, and a right lower image section.d1) to determine a maximum intensity of pixels of a first group of image sections of the at least one second image, which has the lower left image section and the lower right image section, and a mean intensity of a second group of image sections of the at least, to determine a second image, which has the upper left image section, the upper right image section and the lower right image section, or the upper left image section, the upper right image section and the lower left image section, e1) to determine an illumination difference of the at least one second image by dividing the maximum intensity of the first group of image sections of the at least one second image by the mean intensity of the second group of image sections of the at least one second image, g1) to apply a Laplace filter to the at least one second image if the illumination difference is greater than the lower threshold and less than the upper threshold in order to obtain a modified at least one second image, and to capture at least one edge in the modified at least one second image, h1) an intensity value of pixels of the modified at least one second image,which do not have a section of the at least one edge, to a maximum intensity value, and to set the intensity value of pixels of the modified at least one second image which have a section of the at least one edge to a minimum intensity value in order to obtain a modified at least one second image, i1) apply a median filter to the modified at least one second image in order to obtain a filtered at least one second image, j1) remove one or more sections of the filtered at least one second image, wherein at least one section contains an image of at least one section of a steering wheel area (12) of the vehicle (10) in order to obtain a test image section of the filtered at least one second image, and k1) determine a number of pixels with maximum intensity in the test image section of the filtered at least one second image,wherein the driver monitoring device (100) is configured to perform, in k) based on the number of pixels with maximum intensity in the test image section of the filtered at least one first image (30) and the number of pixels with maximum intensity in the test image section of the filtered, to determine, based on at least a second image, in particular based on a number of pixels scaled to a predetermined image size at maximum intensity and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110) or not. Driver monitoring device (100) according to claim 11, wherein the driver monitoring device (100) is configured to generate, in b), a first image (30) based on an image (20) captured by the image acquisition device (130) with a first exposure time of preferably 36 ps, and to generate another first image based on an image captured by the image acquisition device with a second exposure time of preferably 640 ps, ​​and, in b1), to generate a second image based on an imageto generate a second image based on an image captured by the image acquisition device (130) with the first exposure time, and in k) to determine, based on the number of pixels with maximum intensity in the test image section of the filtered first image, the number of pixels with maximum intensity in the test image section of the other filtered first image, the number of pixels with maximum intensity in the test image section of one filtered second image, and the number of pixels with maximum intensity in the test image section of the other filtered second image, whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110). Driver monitoring device (100) according to one of the preceding claims,wherein the driver monitoring device (100) is configured to illuminate the interior (11) by means of the lighting device (110) with light of an intensity lower than the intensity of light with which the interior (11) is illuminated during normal operation of the driver monitoring device (100). Driver monitoring device (100) according to one of the preceding claims, wherein the driver monitoring device (100) is configured to display the at least one first image (30) and / or the at least one second image by extracting an image section (21, 22) of a sensor located on the sensor of the driver monitoring device (100). Image acquisition device (130) to generate an image (20) of the interior (11) based on the image (20) captured by the image acquisition device (11) and / or to reduce the file size of an image based on the image of the interior captured by the image acquisition device (130).

15. Driver monitoring device (100) according to the preceding claim, wherein the driver monitoring device (100) is configured to extract a left-side image section (21) in b) and / or b1) when the vehicle (10) is configured as a right-hand drive vehicle, and to extract a right-side image section (22) when the vehicle is configured as a left-hand drive vehicle, and wherein in d) and / or d1) the second group comprises the upper left image section (31), the upper right image section (32) and the lower right image section (36) when the vehicle (10) is configured as a right-hand drive vehicle, and the second group comprises the upper left image section (31), the upper right image section (32) and the lower left image section (35) when the vehicle (10) is configured as a left-hand drive vehicle.

16. Driver monitoring device (100) according to one of the preceding claims and claim 10, wherein the driver monitoring device (100) is configured to communicate with the control unit (13) of the vehicle (10) in order to determine, based on the information stored in the storage unit (14), whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (130) or not, and to permanently deactivate the lighting device (110) and terminate the driver monitoring operation if the information stored in the storage unit (14) indicates that the diffuser (120) is not functional and properly arranged in the beam path of the lighting device (110), or to proceed with (b) if the information stored in the storage unit (14) indicates that the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110).

Images