Method for determining state of driver monitoring device and driver monitoring device

Abstract

A method for determining the status of a driver monitoring device comprising an illumination device for illuminating an interior of a vehicle, a diffuser and an image capture device, the method comprising the steps of: b) illuminating the interior and generating at least one first image of the interior, the first image is based on an image of the interior illuminated by means of an illumination device, said image being captured by an image capture device, c) dividing the at least one first image into a plurality of image segments, d) determining a maximum intensity of pixels of a first group of image segments of the at least one first image, and determining an average intensity of a second set of image segments of the at least one first image, e) determining an illumination difference of the at least one first image by dividing the maximum intensity of the first set by the average intensity of the second set, f) if the illumination difference of the at least one first image is less than a lower threshold, if the illumination difference is greater than an upper threshold, it is determined that the diffuser is active and appropriately arranged and the lighting device is enabled for driver monitoring, or if the illumination difference is greater than an upper threshold, it is determined that the diffuser is not active and appropriately arranged and the lighting device is disabled.

Description

Technical Field

[0001] The invention relates to a method for determining the state of a driver monitoring device and to a driver monitoring device. Background Art

[0002] The driver monitoring device comprises an illumination device and an image capture device, the illumination device being for example in the form of a laser, in particular a VCSEL (Vertical Cavity Surface Emitting Laser), the illumination device being used to illuminate the area of ​​the interior of the vehicle where the driver's head is located; the image capture device being used to capture an image of the driver's head illuminated using the illumination device; the driver monitoring device being used to capture an image of the driver's head, in particular an image of the driver's eyes, as a basis for determining whether the driver is distracted or fatigued, and if necessary, outputting a warning signal to warn the driver or prompt them to take a break. Since shining the light of the illumination device directly onto the driver's head can damage his eyes, a diffuser is provided in the beam path of the light, which diffuses the light in such a way that the intensity of the light striking the eyes is too low to damage the eyes. However, if the diffuser becomes loose and falls off or is otherwise restricted in its operation, there is a risk of damaging the driver's eyes. Summary of the invention

[0003] The object of the present invention is to provide a method for determining the state of a driver monitoring device and a corresponding driver monitoring device which can be used to increase the safety of the driver.

[0004] This object is achieved by the features of the independent claim. Advantageous developments of the invention are subject matter of the dependent claims.

[0005] A method for determining the state of a driver monitoring device, the driver monitoring device comprising: a lighting device for illuminating the interior of a vehicle; a diffuser, arranged in a correct state of the driver monitoring device in the light beam path of the lighting device for scattering the light emitted by the lighting device; and an image capture device for capturing an image of the interior; the method according to one embodiment comprises the following steps:

[0006] b) illuminating the interior by means of an illumination device and generating at least one first image of at least a section of the interior, the first image being based on an image of the interior illuminated by the illumination device, said image being captured by an image capture device,

[0007] c) dividing the at least one first image into an upper left image segment, an upper right image segment, a left middle image segment, a right middle image segment, a lower left image segment and a lower right image segment,

[0008] d) determining the maximum intensity of pixels of a first group of image segments of the at least one first image, the group comprising a left middle image segment and a right middle image segment, and determining the average intensity of a second group of image segments of the at least one first image, the group comprising an upper left image segment, an upper right image segment and a lower right image segment, or an upper left image segment, an upper right image segment and a lower left image segment,

[0009] e) determining an illumination difference of the at least one first image by dividing a maximum intensity of a first set of image segments of the at least one first image by an average intensity of a second set of image segments of the at least one first image, and

[0010] f) if the illumination difference of the at least one first image is below a lower threshold, determining that the diffuser is functional and properly arranged in the beam path of the lighting device and enabling the lighting device for driver monitoring, or if the illumination difference is above an upper threshold, determining that the diffuser is not functional and properly arranged in the beam path of the lighting device and deactivating the lighting device.

[0011] The lighting device can in particular comprise a laser, preferably a VCSEL. In this case, the image capture device is in particular configured such that the driver's head is located substantially in the middle of the field of view of the image capture device, and the lighting device is configured such that, without an upstream diffuser, the main part of the light falls on the driver's head.

[0012] Furthermore, in step c), the at least one image is preferably divided into six image segments of equal size, wherein, in step d), the second group is in particular selected such that only a small portion of the image of the side window of the vehicle is contained in the image segments of the group in order to minimize the influence of extraneous light.

[0013] In step f), the illumination difference is compared with an upper threshold and a lower threshold to determine whether the diffuser is functional and properly arranged in the beam path of the lighting device, the upper threshold and the lower threshold being preferably determined or set based on an empirical method in which an image of a driver's head located inside a vehicle is captured using an image capture device in a state in which the interior is illuminated using the lighting device under different (external) lighting conditions, and the diffuser is functional and properly arranged in the beam path of the lighting device, or the diffuser is not functional and properly arranged in the beam path of the lighting device.

[0014] Due to this configuration of the driver monitoring device, in many cases it is already possible to determine, by comparing the lighting difference with an upper or lower threshold value, whether the diffuser is functional and appropriately arranged in the beam path of the lighting device and therefore whether there is a risk of damage to the driver's eyes, wherein in the case of a potential risk to the eyes the lighting device is deactivated to prevent (further) damage.

[0015] According to one embodiment, the method further comprises the following steps:

[0016] g) if the illumination difference is greater than a lower threshold and less than an upper threshold, applying a Laplace filter to the at least one first image so as to obtain a modified at least one first image, and capturing at least one edge in the modified at least one first image,

[0017] h) setting the intensity values ​​of pixels of the modified at least one first image that do not have the at least one edge to a maximum intensity value, and setting the intensity values ​​of pixels of the modified at least one first image that have the at least one edge to a minimum intensity value, so as to obtain the modified at least one first image,

[0018] i) applying a median filter to the modified at least one first image so as to obtain a filtered at least one first image,

[0019] j) removing one or more sections of the filtered at least one first image, wherein at least one section comprises an image of at least one section of a steering wheel area of ​​the vehicle, so as to obtain a test image section of the filtered at least one first image;

[0020] k) determining the number of pixels with maximum intensity in a test image section of the at least one first filtered image and, based on the number of pixels with maximum intensity in the test image section of the at least one first filtered image, in particular based on the number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, determining whether a diffuser is functional and appropriately arranged in the beam path of the lighting device and storing a corresponding piece of information in a memory unit of a control unit of the vehicle.

[0021] Thus, in one embodiment, even under adverse (extraneous) light conditions it can be correctly determined with a high degree of certainty whether the diffuser is functional and properly arranged in the beam path of the lighting device.

[0022] In this case, the threshold may be determined or set using empirical methods in a similar manner as described above, preferably using a confusion matrix.

[0023] According to one embodiment, the method further comprises the following steps:

[0024] b1) generating at least one second image of at least one section of the interior, said second image being based on an image captured by the image capture device when the interior is not illuminated by means of an illumination device,

[0025] c1) dividing the at least one second image into an upper left image segment, an upper right image segment, a left middle image segment, a right middle image segment, a lower left image segment and a lower right image segment,

[0026] d1) determining the maximum intensity of pixels of a first group of image segments of the at least one second image, the group comprising a lower left image segment and a lower right image segment, and determining the average intensity of a second group of image segments of the at least one second image, the group comprising an upper left image segment, an upper right image segment and a lower right image segment, or an upper left image segment, an upper right image segment and a lower left image segment,

[0027] e1) determining the illumination difference of the at least one second image by dividing the maximum intensity of a first set of image segments of the at least one second image by the average intensity of a second set of image segments of the at least one second image,

[0028] g1) if the illumination difference is greater than a lower threshold and less than an upper threshold, applying a Laplacian filter to the at least one second image to obtain a modified at least one second image, and capturing at least one edge in the modified at least one second image,

[0029] h1) setting the intensity values ​​of pixels of the segment of the modified at least one second image without the at least one edge to a maximum intensity value, and setting the intensity values ​​of pixels of the segment of the modified at least one second image with the at least one edge to a minimum intensity value, so as to obtain the modified at least one second image,

[0030] i1) applying a median filter to the modified at least one second image to obtain a filtered at least one second image,

[0031] j1) removing one or more sections of the filtered at least one second image, wherein at least one section comprises an image of at least one section of a steering wheel area of ​​the vehicle, so as to obtain a test image section of the filtered at least one second image; and

[0032] k1) determining the number of pixels having a maximum intensity in a test image section of the at least one second filtered image,

[0033] Therein, in step k), it is determined whether the diffuser is functional and appropriately arranged in the beam path of the lighting device based on the number of pixels with maximum intensity in a test image segment of at least one first filtered image and the number of pixels with maximum intensity in a test image segment of at least one second filtered image, in particular based on the number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to a predetermined image size with a threshold value.

[0034] According to one embodiment, in step b), a first image based on an image captured by an image capture device with a first exposure time of preferably 36 μs is generated, and another first image based on an image captured by the image capture device with a second exposure time of preferably 640 μs is generated, in step b1), a second image based on the image captured by the image capture device with the first exposure time is generated, and another second image based on the image captured by the image capture device with the second exposure time is generated, and in step k), based on the number of pixels with maximum intensity in a filtered test image segment of one first image, the number of pixels with maximum intensity in a filtered test image segment of another first image, the number of pixels with maximum intensity in a filtered test image segment of a second image, and the number of pixels with maximum intensity in a filtered test image segment of another second image, it is determined whether the diffuser is functional and properly arranged in the beam path of the lighting device.

[0035] According to one embodiment, in step b), the interior is illuminated by means of the lighting device with light whose intensity is lower than the intensity of the light with which the interior is illuminated during normal operation of the driver monitoring device. Here, in step b), the intensity of the light can advantageously be selected such that even in the case of light shining directly on the eyes of the driver, that is to say even without prior scattering by a diffuser, the eyes are not damaged.

[0036] According to one embodiment, the generation of the at least one first image and / or the at least one second image includes extraction of image segments of an image based on an internal image captured by an image capture device and / or reduction of a file size of an image based on an internal image captured by an image capture device.

[0037] According to one embodiment, in step b) and / or step b1), a left-hand image segment is extracted when the vehicle is configured for right-hand drive, and a right-hand image segment is extracted when the vehicle is configured for left-hand drive, wherein, in step d) and / or step d1), when the vehicle is configured for right-hand drive, the second group includes an upper left image segment, an upper right image segment and a lower right image segment, and when the vehicle is configured for left-hand drive, the second group includes an upper left image segment, an upper right image segment and a lower left image segment.

[0038] According to one embodiment, the method further comprises, before step b), performing the following steps:

[0039] a) communicating with a control unit of the vehicle in order to determine, based on information stored in a storage unit, whether the diffuser is functional and properly arranged in the beam path of the lighting device, and permanently deactivating the lighting device and terminating the method if the information stored in the storage unit is that the diffuser is not functional and properly arranged in the beam path of the lighting device, or continuing with step b) of the method when the information stored in the storage unit is that the diffuser is functional and properly arranged in the beam path of the lighting device.

[0040] A driver monitoring device according to one embodiment comprises: a lighting device for illuminating an interior of a vehicle; a diffuser, which in a correct state of the driver monitoring device is arranged in a beam path of the lighting device for scattering light emitted by the lighting device; and an image capture device for capturing an image of the interior, wherein the driver monitoring device is configured to:

[0041] b) illuminating the interior by means of an illumination device and generating at least one first image of at least a section of the interior, said first image being based on an image of the interior illuminated by the illumination device, said image being captured by an image capture device,

[0042] c) dividing the at least one first image into an upper left image segment, an upper right image segment, a left middle image segment, a right middle image segment, a lower left image segment and a lower right image segment,

[0043] d) determining the maximum intensity of pixels of a first group of image segments of the at least one first image, the group comprising a left middle image segment and a right middle image segment, and determining the average intensity of a second group of image segments of the at least one first image, the group comprising an upper left image segment, an upper right image segment and a lower right image segment, or an upper left image segment, an upper right image segment and a lower left image segment,

[0044] e) determining an illumination difference of the at least one first image by dividing a maximum intensity of a first set of image segments of the at least one first image by an average intensity of a second set of image segments of the at least one first image, and

[0045] f) if the illumination difference of the at least one first image is less than a lower threshold, determining that the diffuser is functional and properly arranged in the beam path of the lighting device and enabling the lighting device for driver monitoring, or if the illumination difference is greater than an upper threshold, determining that the diffuser is not functional and properly arranged in the beam path of the lighting device and deactivating the lighting device.

[0046] According to one embodiment, the driver monitoring device is further configured to:

[0047] g) if the illumination difference is greater than a lower threshold and less than an upper threshold, applying a Laplacian filter to the at least one first image to obtain a modified at least one first image, and capturing at least one edge in the modified at least one first image,

[0048] h) setting the intensity values ​​of pixels of the modified at least one first image that do not have the at least one edge to a maximum intensity value, and setting the intensity values ​​of pixels of the modified at least one first image that have the at least one edge to a minimum intensity value, so as to obtain the modified at least one first image,

[0049] i) applying a median filter to the modified at least one first image so as to obtain a filtered at least one first image,

[0050] j) removing one or more sections of the filtered at least one first image, wherein at least one section comprises an image of at least one section of a steering wheel area of ​​the vehicle, so as to obtain a test image section of the filtered at least one first image;

[0051] k) determining the number of pixels with maximum intensity in a test image section of the at least one first filtered image and, based on the number of pixels with maximum intensity in the test image section of the at least one first filtered image, in particular based on the number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, determining whether a diffuser is functional and appropriately arranged in the beam path of the lighting device and storing a corresponding piece of information in a memory unit of a control unit of the vehicle.

[0052] According to one embodiment, the driver monitoring device is further configured to:

[0053] b1) generating at least one second image of at least one section of the interior, said second image being based on an image captured by the image capture device when the interior is not illuminated by means of the illumination device,

[0054] c1) dividing at least one second image into an upper left image segment, an upper right image segment, a left middle image segment, a right middle image segment, a lower left image segment and a lower right image segment,

[0055] d1) determining a maximum intensity of pixels of a first group of image segments of the at least one second image, comprising a lower left image segment and a lower right image segment, and determining an average intensity of pixels of a second group of image segments of the at least one second image, the second group of image segments comprising an upper left image segment, an upper right image segment and a lower right image segment, or an upper left image segment, an upper right image segment and a lower left image segment,

[0056] e1) determining the illumination difference of the at least one second image by dividing the maximum intensity of a first set of image segments of the at least one second image by the average intensity of a second set of image segments of the at least one second image,

[0057] g1) if the illumination difference is greater than a lower threshold and less than an upper threshold, applying a Laplacian filter to the at least one second image so as to obtain a modified at least one second image and capturing at least one edge in the modified at least one second image,

[0058] h1) setting the intensity values ​​of pixels of the segment of the modified at least one second image without the at least one edge to a maximum intensity value, and setting the intensity values ​​of pixels of the segment of the modified at least one second image with the at least one edge to a minimum intensity value, so as to obtain the modified at least one second image,

[0059] i1) applying a median filter to the modified at least one second image so as to obtain a filtered at least one second image,

[0060] j1) removing one or more sections of the at least one second filtered image, wherein at least one section comprises an image of at least one section of a steering wheel area of ​​the vehicle, so as to obtain a test image section of the at least one second filtered image; and

[0061] k1) determining the number of pixels having a maximum intensity in a test image section of the at least one second filtered image,

[0062] The driver monitoring device is configured to: in step k), determine whether the diffuser is functional and appropriately arranged in the beam path of the lighting device based on the number of pixels with maximum intensity in a test image segment of at least one first filtered image and the number of pixels with maximum intensity in a test image segment of at least one second filtered image, in particular based on the number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to a predetermined image size with a threshold value.

[0063] According to one embodiment, the driver monitoring device is configured to: in step b), generate a first image based on an image captured by an image capture device with a first exposure time of preferably 36 μs, and generate another first image based on an image captured by an image capture device with a second exposure time of preferably 640 μs, in step b1), generate a second image based on an image captured by the image capture device with the first exposure time, and generate another second image based on an image captured by the image capture device with the second exposure time, and in k), determine whether the diffuser is functional and properly arranged in the beam path of the lighting device based on the number of pixels with maximum intensity in a filtered test image segment of one first image, the number of pixels with maximum intensity in a filtered test image segment of another first image, the number of pixels with maximum intensity in a filtered test image segment of one second image, and the number of pixels with maximum intensity in a filtered test image segment of another second image.

[0064] According to one embodiment, the driver monitoring device is configured to, in b), illuminate the interior by means of the lighting device with light of an intensity lower than that of light illuminating the interior during normal operation of the driver monitoring device.

[0065] According to one embodiment, the driver monitoring device is configured to generate the at least one first image and / or the at least one second image by extracting an image segment based on an image of the interior captured by an image capture device, and / or by reducing the file size of an image based on the image of the interior captured by the image capture device.

[0066] According to one embodiment, the driver monitoring device is configured to, in b) and / or b1), extract a left-hand image segment when the vehicle is configured for right-hand drive, and extract a right-hand image segment when the vehicle is configured for left-hand drive, wherein, in d) and / or d1), when the vehicle is configured for right-hand drive, the second group includes an upper left image segment, an upper right image segment and a lower right image segment, and when the vehicle is configured for left-hand drive, the second group includes an upper left image segment, an upper right image segment and a lower left image segment.

[0067] According to one embodiment, the driver monitoring device is configured, before b):

[0068] a) communicating with a control unit of the vehicle in order to determine, based on information stored in a storage unit, whether the diffuser is functional and properly arranged in the beam path of the lighting device, and to permanently deactivate the lighting device and terminate the driver monitoring operation if the information stored in the storage unit is that the diffuser is not functional and properly arranged in the beam path of the lighting device, or to continue the driver monitoring operation when the information stored in the storage unit is that the diffuser is functional and properly arranged in the beam path of the lighting device. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Further advantages, features and application possibilities of the present invention can be found in the following detailed description in conjunction with the accompanying drawings.

[0070] In the attached picture:

[0071] Figure 1 A vehicle having a driver monitoring device according to an embodiment of the present invention is schematically shown;

[0072] Figure 2 shows an image of the interior of a vehicle captured by means of an image capture device;

[0073] Figure 3 shows an image of the interior generated based on an image captured by an image capture device and divided into six image sections;

[0074] Figure 4 shows an image of an interior generated based on an image captured by an image capture device and divided into six image segments, wherein a diffuser for scattering light emitted by an illumination device for illuminating the interior is not functional and properly arranged in a beam path of the illumination device;

[0075] Figure 5 shows an image of an interior generated based on an image captured by an image capture device and divided into six image segments, wherein a diffuser is active and appropriately arranged in the beam path of the lighting device;

[0076] Figure 6 shows an image of the interior generated based on an image captured by an image capture device, wherein a diffuser is not functional and properly arranged in a beam path of the lighting device;

[0077] Figure 7 shows that the Laplace filter has been applied Figure 6 The image shown;

[0078] Figure 8 Shown based on Figure 7 a modified image of the image shown;

[0079] Fig. 9 Shown based on Figure 8 the image shown and applying a median filter to the filtered image;

[0080] Fig.10 showing an image of the interior generated based on an image captured by an image capture device;

[0081] Fig.11 Shown based on Fig.10 an image segment of an image showing the resulting image; and

[0082] Fig.12 Shows Fig.11 The image shown has the steering wheel area removed from the image. DETAILED DESCRIPTION

[0083] Figure 1 A vehicle having a driver monitoring device according to an embodiment of the present invention is schematically shown.

[0084] The vehicle 10, such as a passenger car, comprises a control unit 13 having a storage unit 14 and comprises a driver monitoring device 100. The driver monitoring device 100 comprises an illumination device 110, such as a laser, in particular a VCSEL, for illuminating Figure 2 Shown is the interior 11 of the vehicle 10 , an image capture device 130 , and a diffuser which, in a correct state of the driver monitoring device 100 , is arranged in the beam path of the lighting device 110 for scattering the light emitted by the lighting device 110 .

[0085] The image capture device 130 is configured to capture the image of the driver's head, especially the eyes, of the vehicle 10 so as to be able to determine whether the driver is tired or distracted based on the image. In this case, the lighting device 110 is configured to particularly illuminate the driver's head.

[0086] The storage unit 14 is configured to store information on whether the diffuser 120 is functional and properly arranged in the beam path of the lighting device 130 .

[0087] In the method according to the invention or with the aid of the driver monitoring device 100 according to the invention, communication is first carried out with the control unit 13 of the vehicle 10 in order to determine, based on the information stored in the storage unit 14 , whether the diffuser 120 is functional and appropriately arranged in the beam path of the lighting device 130 .

[0088] If it is determined that diffuser 120 is not functional and properly arranged in the beam path of lighting device 130, lighting device 110 is permanently deactivated and the method or the driver monitoring operation is terminated. On the other hand, if it is determined that diffuser 120 is functional and properly arranged in the beam path of lighting device 130, the method according to the invention continues with the following steps.

[0089] Figure 2 An image 20 of the interior 11 of a vehicle 10 which is configured for right-hand drive and which is captured by means of an image capture device 130 is shown, the image comprising a left-hand image segment 21 and a right-hand image segment 22 which is of interest in the present case since this contains an image of the driver's head when the driver is sitting in the vehicle 10 .

[0090] Figure 3 Shown based on Figure 2 The image 30 of the interior 11 generated from the image 20 shown, wherein the file size of the image 20 has been reduced, the right-hand image segment 22 has been extracted and the image is divided into six image segments, namely an upper left image segment 31, an upper right image segment 32, a left middle image segment 33, a right middle image segment 34, a lower left image segment 35 and a lower right image segment 36.

[0091] Figure 4 An image 30 of the interior 11 is shown, which image 30 is generated based on the image 20 captured by the image capture device 130 and which contains an image of the driver, wherein the generated image 30 is divided into six image segments 31-36 and when the image 20 was captured, the diffuser 120 was not active and properly arranged in the beam path of the lighting device. Figure 4 It can be seen that in this case the driver's head is brightly illuminated by the lighting device 110, so the driver's eyes may be damaged, while the rest of the image 30 is substantially dark.

[0092] Figure 5 An image 30 of the interior 11 is shown, which image 30 is generated based on the image 20 captured by the image capture device 130 and which contains an image of the driver, wherein the generated image 30 is divided into six image segments 31-36 and the diffuser 120 was active and appropriately arranged in the beam path of the lighting device when the image 20 was captured. Figure 5 It can be seen that, in this case, since the light from the lighting device 110 is scattered by means of the diffuser 120, the brightness of the generated image 30 is substantially uniform, which prevents damage to the driver's eyes.

[0093] In the method according to the invention, the maximum pixel intensity of a first group of image segments 33, 34 of the image 30, including the left middle image segment 33 and the right middle image segment 34, and the average intensity of a second group of image segments 31, 32, 36; 31, 32, 35 of the image 30 (including the upper left image segment 31, the upper right image segment 32 and the lower right image segment 36 or including the upper left image segment 31, the upper right image segment 32 and the lower left image segment 35) are determined. Since in the present case the vehicle 10 is a vehicle configured for right-hand drive, the second group includes the upper left image segment 31, the upper right image segment 32 and the lower right image segment 36.

[0094] Then, an illumination difference of the image 30 is determined by dividing the maximum intensity of the first group of image segments 33, 34 of the image 30 by the average intensity of the second group of image segments 31, 32, 36; 31, 32, 35 of the image 30. Then, based on the determined illumination difference, it is determined that the diffuser 120 is functional and properly arranged in the beam path of the lighting device 110, and if the illumination difference of the image 30 is less than a lower threshold, the lighting device 110 is enabled for driver monitoring, or when the illumination difference is greater than an upper threshold, it is determined that the diffuser 120 is not functional and properly arranged in the beam path of the lighting device 110, and the lighting device 110 is disabled.

[0095] If the illumination difference is greater than the lower threshold and less than the upper threshold, the image 30 is subjected to further processing steps as described below.

[0096] Figure 6 An image 30 of the interior 11 is shown which is generated based on an image 20 captured by an image capture device 130, which image 20 contains an image of the driver and in which the diffuser 120 is not functional and properly arranged in the beam path of the lighting device 110. In this case, the lighting difference is greater than a lower threshold and below an upper threshold.

[0097] Figure 7 Shown based on Figure 6 A modified image 30' of the image 30 is shown, in which a Laplacian filter has been applied to Figure 6 At least one edge 37 becomes visible in the image shown, thus in modified image 30 ′, which edge was captured by the method according to the invention or by driver monitoring device 100 .

[0098] Thereafter, the intensity values ​​of the pixels of the segment of the modified image 30' without the at least one edge 37 are set to the maximum intensity value, and the intensity values ​​of the pixels of the segment of the modified image 30' with the at least one edge 37 are set to the minimum intensity value, so as to obtain Figure 8 The modified image 30" is shown.

[0099] The median filter is then applied to Figure 8 The modified image 3" is shown to obtain Fig. 9 The filtered image 30'' is shown.

[0100] from Fig. 9 One or more segments are removed from the filtered image 30'' shown to obtain test image segments of the filtered image 30'', for example by setting the pixels of these segments to minimum intensity values, wherein at least one of the segments includes an image of at least one segment of the steering wheel area 12 of the vehicle 10, such as Fig.12 As shown, the driver's hands are usually located on the steering wheel area 12 , which results in / will result in pixels with high intensity due to the reflection of the light emitted by the lighting device 110 by the hand skin. Fig.11 Shows the Fig.10 an image segment of the generated image as shown; and Fig.12 The steering wheel area 12 is shown removed therefrom. Fig.11 Image shown.

[0101] Then, the number of pixels with maximum intensity in the test image section of the filtered image 30'" is determined. Based on the number of pixels with maximum intensity in the test image section of the filtered image 30'" and in particular based on the number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, it is then determined whether the diffuser 120 is functional and appropriately arranged in the beam path of the lighting device 110, and a corresponding piece of information is stored in the storage unit 14 of the control unit 13 of the vehicle 10.

[0102] Preferably, in the method according to the invention / by means of the driver monitoring device 110 according to the invention, a plurality of images, in particular four images, of the interior 11 are captured by means of the image capture device 130, wherein two of the images are captured with different exposure times while the interior 11 is illuminated by means of the lighting device 110 and two of the images are captured with different exposure times while the interior 11 is not illuminated by means of the lighting device 110. The four images are then processed as described above and it is determined whether the diffuser 120 is functional and appropriately arranged in the beam path of the lighting device 110 based on the number of pixels with maximum intensity in the respective test image segment.

[0103] Reference numerals list

[0104] 10 Vehicles

[0105] 11. Interior

[0106] 12 Steering wheel area

[0107] 13 Control unit

[0108] 14 Storage Unit

[0109] 20 Internal Image

[0110] 21 Left segment of internal image

[0111] 22 Right segment of internal image

[0112] 30 First Image

[0113] 37 Edge

[0114] 100 Driver Monitoring Device

[0115] 110 lighting fixtures

[0116] 120 Diffuser

[0117] 130 Image capture device

Claims

1. A method for determining a state of a driver monitoring device (100), the driver monitoring device comprising: A lighting device (110) for illuminating an interior (11) of a vehicle (10); a diffuser (120) arranged in a correct state of the driver monitoring device (100) in the light beam path of the lighting device (110) for scattering the light emitted by the lighting device (110); and an image capture device (130) for capturing an image (20) of the interior (11), wherein the method comprises the following steps: b) illuminating the interior (11) by means of an illumination device (110) and generating at least one first image (30) of at least one section of the interior (11), the first image being based on an image (20) of the interior (11) illuminated by the illumination device (110), the image being captured by the image capture device (130), c) dividing the at least one first image (30) into an upper left image segment (31), an upper right image segment (32), a left middle image segment (33), a right middle image segment (34), a lower left image segment (35) and a lower right image segment (36), d) determining the maximum intensity of pixels of a first group of image segments (33, 34) of the at least one first image (30), the group comprising the left middle image segment (33) and the right middle image segment (34), and determining the average intensity of a second group of image segments (31, 32, 36; 31, 33, 35) of the at least one first image (30), the group comprising the upper left image segment (31), the upper right image segment (32) and the lower right image segment (36) or comprising the upper left image segment (31), the upper right image segment (32) and the lower left image segment (35), e) determining an illumination difference of the at least one first image (30) by dividing a maximum intensity of a first set of image segments (33, 34) of the at least one first image (30) by an average intensity of a second set of image segments (31, 32, 36; 31, 32, 35) of the at least one first image (30), and f) if the illumination difference of the at least one first image (30) is less than a lower threshold, determining that the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110) and enabling the lighting device (110) for driver monitoring, or, if the illumination difference is greater than an upper threshold, determining that the diffuser (120) is not functional and properly arranged in the beam path of the lighting device (110) and deactivating the lighting device (110).

2. The method according to claim 1, further comprising the steps of: g) if the illumination difference is greater than the lower threshold and less than the upper threshold, applying a Laplacian filter to the at least one first image (30) so as to obtain a modified at least one first image (30'), and capturing at least one edge (37) in the modified at least one first image (30'), h) setting the intensity values ​​of the pixels of the section of the at least one modified first image (30') not having the at least one edge (37) to a maximum intensity value, and setting the intensity values ​​of the pixels of the section of the at least one modified first image (30') having the at least one edge (37) to a minimum intensity value, so as to obtain the at least one modified first image (30"), i) applying a median filter to the modified at least one first image (30") so as to obtain a filtered at least one first image (30'"), j) removing one or more sections of the at least one first filtered image (30'"), wherein at least one section comprises an image of at least one section of a steering wheel area (12) of the vehicle (10), in order to obtain a test image section of the at least one first filtered image (30'"), k) determining the number of pixels with maximum intensity in the test image section of the at least one first filtered image (30'") and, based on the number of pixels with maximum intensity in the test image section of the at least one first filtered image (30'"), in particular based on the number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value, determining whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110), and storing a corresponding piece of information in a storage unit (14) of a control unit (13) of the vehicle (10).

3. The method according to claim 2, further comprising the steps of: b1) generating at least one second image of the at least one section of the interior (11), the second image being based on an image captured by the image capture device (130) when the interior (11) is not illuminated by means of the illumination device (110), c1) dividing the at least one second image into an upper left image segment, an upper right image segment, a left middle image segment, a right middle image segment, a lower left image segment and a lower right image segment, d1) determining the maximum intensity of pixels of a first group of image segments of the at least one second image, the group comprising the lower left image segment and the lower right image segment, and determining the average intensity of a second group of image segments of the at least one second image, the group comprising the upper left image segment, the upper right image segment and the lower right image segment or comprising the upper left image segment, the upper right image segment and the lower left image segment, e1) determining an illumination difference of the at least one second image by dividing a maximum intensity of the first set of image segments of the at least one second image by an average intensity of the second set of image segments of the at least one second image, g1) if the illumination difference is greater than the lower threshold and less than the upper threshold, applying a Laplacian filter to the at least one second image to obtain a modified at least one second image, and capturing at least one edge in the modified at least one second image, h1) setting the intensity values ​​of pixels of the modified at least one second image that do not have the at least one edge to a maximum intensity value, and setting the intensity values ​​of pixels of the modified at least one second image that have the at least one edge to a minimum intensity value, so as to obtain the modified at least one second image, i1) applying a median filter to the modified at least one second image so as to obtain a filtered at least one second image, j1) removing one or more sections of the at least one second filtered image, wherein at least one section comprises an image of at least one section of a steering wheel area (12) of the vehicle (10), so as to obtain a test image section of the at least one second filtered image, and k1) determining the number of pixels having a maximum intensity in a test image section of the at least one second filtered image, Wherein, in step k), it is determined whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110) based on the number of pixels with maximum intensity in a test image segment of the at least one filtered first image (30) and the number of pixels with maximum intensity in the test image segment of the at least one filtered second image, in particular based on the number of pixels with maximum intensity scaled to the predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value.

4. The method according to claim 3, wherein: In step b), a first image (30) is generated based on an image (20) captured by the image capture device (130) with a first exposure time of preferably 36 μs, and another first image is generated based on an image captured by the image capture device with a second exposure time of preferably 640 μs, in step b1), a second image is generated based on the image captured by the image capture device (130) with the first exposure time, and another second image is generated based on the image captured by the image capture device (130) with the second exposure time, and in step k), based on the number of pixels with maximum intensity in a filtered test image section of one first image, the number of pixels with maximum intensity in a filtered test image section of another first image, the number of pixels with maximum intensity in a filtered test image section of a second image, and the number of pixels with maximum intensity in a filtered test image section of another second image, it is determined whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110).

5. A method according to any one of the preceding claims, wherein: In step b), the interior (11) is illuminated by means of the lighting device (110) with light having an intensity lower than the intensity of light illuminating the interior (11) during normal operation of the driver monitoring device (100).

6. A method according to any one of the preceding claims, wherein: The generation of the at least one first image (30) and / or the at least one second image includes extracting image segments (21, 22) of an image based on an image (20) of the interior (11) captured by the image capture device (130) and / or reducing the file size of an image based on an image of the interior captured by the image capture device (130).

7. A method according to any one of the preceding claims, wherein: In step b) and / or step b1), when the vehicle (10) is configured for right-hand drive, a left-hand image segment (22) is extracted, and when the vehicle (10) is configured for left-hand drive, a right-hand image segment (21) is extracted, and, in step d) and / or step d1), when the vehicle (10) is configured for right-hand drive, the second group includes an upper left image segment (31), an upper right image segment (32) and a lower right image segment (36), and when the vehicle (10) is configured for left-hand drive, the second group includes the upper left image segment (31), the upper right image segment (32) and the lower left image segment (35).

8. The method according to any one of the preceding claims and claim 2, further comprising the following steps performed before step b): a) communicating with the control unit (13) of the vehicle (10) in order to determine, based on information stored in the storage unit (14), whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (130), and if the information stored in the storage unit (14) is that the diffuser (120) is not functional and properly arranged in the beam path of the lighting device (110), permanently deactivating the lighting device (110) and terminating the method, or continuing the method with step b) when the information stored in the storage unit (14) is that the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110).

9. A driver monitoring device (100), comprising: A lighting device (110) for illuminating an interior (11) of a vehicle (10); a diffuser (120) arranged in a beam path of the lighting device (110) in a correct state of the driver monitoring device (100) for scattering light emitted by the lighting device (110); and an image capture device (130) for capturing an image (20) of the interior (11), wherein the driver monitoring device (100) is configured to: b) illuminating the interior (11) by means of an illumination device (110) and generating at least one first image (30) of at least one section of the interior (11), the first image being based on an image (20) of the interior (11) illuminated by means of the illumination device (110), the image being captured by the image capture device (130), c) dividing the at least one first image (30) into an upper left image segment (31), an upper right image segment (32), a left middle image segment (33), a right middle image segment (34), a lower left image segment (35) and a lower right image segment (36), d) determining the maximum intensity of pixels of a first group of image segments (33, 34) of the at least one first image (30), the group comprising the left middle image segment (33) and the right middle image segment (34), and determining the average intensity of pixels of a second group of image segments (31, 32, 36; 31, 33, 35) of the at least one first image (30), the group comprising the upper left image segment (31), the upper right image segment (32) and the lower right image segment (36) or comprising the upper left image segment (31), the upper right image segment (32) and the lower left image segment (35), e) determining an illumination difference of the at least one first image (30) by dividing a maximum intensity of a first set of image segments (33, 34) of the at least one first image (30) by an average intensity of the second set of image segments (31, 32, 36; 31, 32, 35) of the at least one first image (30), and f) if the illumination difference of the at least one first image (30) is less than a lower threshold, determining that the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110) and enabling the lighting device (110) for driver monitoring, or, if the illumination difference is greater than an upper threshold, determining that the diffuser (120) is not functional and properly arranged in the beam path of the lighting device (110) and deactivating the lighting device (110).

10. The driver monitoring device (100) according to claim 9, wherein the driver monitoring device (100) is further configured to: g) if the illumination difference is greater than the lower threshold and less than the upper threshold, applying a Laplacian filter to the at least one first image (30) to obtain a modified at least one first image (30'), and capturing at least one edge (37) in the modified at least one first image (30'), h) setting the intensity values ​​of the pixels of the section of the modified at least one first image (30') without the at least one edge (37) to a maximum intensity value, and setting the intensity values ​​of the pixels of the section of the modified at least one first image (30') with the at least one edge (37) to a minimum intensity value, so as to obtain the modified at least one first image (30"), i) applying a median filter to the modified at least one first image (30") so as to obtain a filtered at least one first image (30""), j) removing one or more sections of the at least one first filtered image (30'"), wherein at least one section comprises an image of at least one section of a steering wheel area (12) of the vehicle (10), in order to obtain a test image section of the at least one first filtered image (30'"), k) determining the number of pixels with maximum intensity in a test image section of the at least one first filtered image (30'") and, based on the number of pixels with maximum intensity in the test image section of the at least one first filtered image (30"), in particular based on the number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to a predetermined image size with a threshold value, determining whether a diffuser (120) is functional and properly arranged in the beam path of the lighting device (110), and storing a corresponding piece of information in a storage unit (14) of a control unit (13) of the vehicle (10).

11. The driver monitoring device (100) according to claim 10, wherein the driver monitoring device (100) is further configured as follows: b1) generating at least one second image of the at least one section of the interior (11), the second image being based on an image captured by the image capture device (130) when the interior (11) is not illuminated by means of the illumination device (110), c1) dividing the at least one second image into an upper left image segment, an upper right image segment, a left middle image segment, a right middle image segment, a lower left image segment and a lower right image segment, d1) determining a maximum intensity of pixels of a first group of image segments of the at least one second image, including the lower left image segment and the lower right image segment, and determining an average intensity of a second group of image segments of the at least one second image, the second group of image segments including the upper left image segment, the upper right image segment and the lower right image segment or including the upper left image segment, the upper right image segment and the lower left image segment, e1) determining the illumination difference of the at least one second image by dividing the maximum intensity of a first set of image segments of the at least one second image by the average intensity of a second set of image segments of the at least one second image, g1) if the illumination difference is greater than the lower threshold and less than the upper threshold, applying a Laplacian filter to the at least one second image to obtain a modified at least one second image, and capturing at least one edge in the modified at least one second image, h1) setting the intensity values ​​of pixels of the segment of the modified at least one second image without the at least one edge to a maximum intensity value, and setting the intensity values ​​of pixels of the segment of the modified at least one second image with the at least one edge to a minimum intensity value, so as to obtain the modified at least one second image, i1) applying a median filter to the modified at least one second image so as to obtain a filtered at least one second image, j1) removing one or more sections of the at least one second filtered image, wherein at least one section comprises an image of at least one section of a steering wheel area (12) of the vehicle (10), so as to obtain a test image section of the at least one second filtered image, and k1) determining the number of pixels having a maximum intensity in a test image section of the at least one second filtered image, The driver monitoring device (100) is configured to determine in step k) whether the diffuser (120) is functional and appropriately arranged in the beam path of the lighting device (110) based on the number of pixels with maximum intensity in a test image section of the at least one filtered first image (30) and the number of pixels with maximum intensity in a test image section of the at least one filtered second image, in particular based on the number of pixels with maximum intensity scaled to a predetermined image size and a comparison of the number of pixels scaled to the predetermined image size with a threshold value.

12. The driver monitoring device (100) according to claim 11, wherein the driver monitoring device (100) is configured to: in step b), generate a first image (30) based on an image (20) captured by the image capture device (130) with a first exposure time of preferably 36 μs, and generate another first image based on an image captured by the image capture device with a second exposure time of preferably 640 μs, and in step b1), generate a second image based on an image captured by the image capture device (130) with the first exposure time, and generate another first image based on an image captured by the image capture device (130) with a second exposure time of preferably 640 μs. Another second image is generated using the image captured by the second exposure time, and, in step k), it is determined whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110) based on the number of pixels with maximum intensity in a filtered test image section of the one first image, the number of pixels with maximum intensity in a filtered test image section of the other first image, the number of pixels with maximum intensity in a filtered test image section of the second image, and the number of pixels with maximum intensity in the filtered test image section of the other second image.

13. A driver monitoring device (100) according to any one of the preceding claims, wherein the driver monitoring device (100) is configured to: in b), illuminate the interior (11) with the aid of the lighting device (110) using light with an intensity lower than the intensity of light that illuminates the interior (11) during normal operation of the driver monitoring device (100).

14. A driver monitoring device (100) according to any one of the preceding claims, wherein the driver monitoring device (100) is configured to generate the at least one first image (30) and / or the at least one second image by extracting image segments (22, 22) of an image based on the image (20) of the interior (11) captured by the image capture device (130) and / or reducing the file size of an image based on the image of the interior captured by the image capture device (130).

15. A driver monitoring device (100) according to the preceding claim, wherein the driver monitoring device (100) is configured, in b) and / or b1), to extract a left-hand image segment (21) when the vehicle (10) is configured for right-hand drive and to extract a right-hand image segment (22) when the vehicle is configured for left-hand drive, and wherein, In d) and / or d1), when the vehicle (10) is configured for right-hand drive, the second group includes the upper left image segment (31), the upper right image segment (32) and the lower right image segment (36), and when the vehicle (10) is configured for left-hand drive, the second group includes the upper left image segment (31), the upper right image segment (32) and the lower left image segment (35).

16. The driver monitoring device (100) according to any one of the preceding claims and claim 10, wherein the driver monitoring device (100) is configured, before b), a) communicating with the control unit (13) of the vehicle (10) in order to determine, based on information stored in the storage unit (14), whether the diffuser (120) is functional and properly arranged in the beam path of the lighting device (130), and if the information stored in the storage unit (14) is that the diffuser (120) is not functional and properly arranged in the beam path of the lighting device (110), permanently deactivating the lighting device (100) and terminating the driver monitoring operation, or when the information stored in the storage unit (14) is that the diffuser (120) is functional and properly arranged in the beam path of the lighting device (110), continuing the driver monitoring operation with step b).

Images