Driver monitoring

Abstract

The present invention is in the field of driver monitoring for a vehicle. It relates to a driver monitoring system for a vehicle comprising: a) a projector configured to illuminate a driver of the vehicle with coherent light, b) a camera configured to record an image of the driver under illumination, c) a processor configured to determine a condition measure of the driver from the image and to generate a control signal for controlling a pre-save system, an emergency call system, or a vehicle automation system of the vehicle using the condition measure, and d) an output configured to output the control signal.

Description

[0001] Driver Monitoring

[0002] Description

[0003] The present invention is in the field of driver monitoring for a vehicle. In particular, it relates to a driver monitoring system for a vehicle, a vehicle containing the driver monitoring system, the use of the driver monitoring system controlling a vehicle, a method for controlling a vehicle and a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method for controlling a vehicle.

[0004] Background

[0005] The safety of vehicles has significantly improved over the years with various technical safety features. However, the human factor remains a major source for car accidents. Emotions, medical conditions or concentration problems regularly compromise the safety of vehicles. It is hence desirable to equip vehicles with systems which are able to avoid or at least mitigate the risk of human deficiencies.

[0006] US 2018 / 0186234 discloses an optical driver monitoring system which determines physical or psychophysiological changes of the driver. The determination result is translated into an alarm in case of a dangerous change. However, the safety is only increased if the driver properly reacts to the alarm.

[0007] It was therefore an object of the present invention to provide a system which actively increases the safety of a vehicle irrespective of the reaction of a person.

[0008] Summary

[0009] In one aspect it relates to a driver monitoring system for a vehicle comprising: a) a projector configured to illuminate a driver of the vehicle with coherent light, b) a camera configured to record an image of the driver under illumination, c) a processor configured to determine a condition measure of the driver from the image and to generate a control signal for controlling a functionality of the vehicle using the condition measure, and d) an output configured to output the control signal.

[0010] In one aspect it relates to a driver monitoring system for a vehicle comprising: a) a projector configured to illuminate a driver of the vehicle with coherent light, b) a camera configured to record an image of the driver under illumination, c) a processor configured to determine a condition measure of the driver from the image and to generate a control signal for controlling a pre-save system, an emergency call system, or a vehicle automation system of the vehicle using the condition measure, and d) an output configured to output the control signal.

[0011] In another aspect it relates to a vehicle containing the driver monitoring system according to the invention.

[0012] In another aspect it relates to a use of the driver monitoring system of any one of the preceding claims for controlling a vehicle.

[0013] In another aspect it relates to a method for controlling a vehicle comprising: a) illuminating a driver of the vehicle with coherent light, b) recording an image of the driver under illumination, c) determining a condition measure of the driver using the image, d) determining a control signal for controlling a functionality of the vehicle using the condition measure, and e) outputting the control signal.

[0014] In another aspect it relates to a method for controlling a vehicle comprising: a) illuminating a driver of the vehicle with coherent light, b) recording an image of the driver under illumination, c) determining a condition measure of the driver using the image, d) determining a control signal for controlling a pre-save system, an emergency call system, or a vehicle automation system of the vehicle using the condition measure, and e) outputting the control signal.

[0015] In another aspect it relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: a) illuminating a driver of the vehicle with coherent light, b) recording an image of the driver under illumination, c) determining a condition measure of the driver using the image, d) determining a control signal for controlling a functionality of the vehicle using the condition measure, and e) outputting the control signal.

[0016] In another aspect it relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: a) illuminating a driver of the vehicle with coherent light, b) recording an image of the driver under illumination, c) determining a condition measure of the driver using the image, d) determining a control signal for controlling a pre-save system, an emergency call system, or a vehicle automation system of the vehicle using the condition measure, and e) outputting the control signal. The invention has the advantage that it actively increases the safety of the vehicle and also the surrounding traffic rather than just signaling dangers which does not necessarily lead to action. In particular in cases in which the driver cannot take any action, for example in case of a heart attack, the system can actively avoid accidents or mitigate their consequences. The system has the further advantage that the same hardware can be used for other applications, for example face authentication to protect the car against theft.

[0017] The driver monitoring system is suitable for various vehicles including cars, motorcycles, buses, trucks, trains or even airplanes. It may be attached to a vehicle, or it may be integrated as component or as part of a component of a vehicle, for example as part of a display in the dashboard, an entertainment control system, or load speakers. It can be places at various places, for example it may be integrated into the steering wheel, besides a speed gauge, in the center of a dashboard, the A pillar, a side door, in a mirror or in a window.

[0018] The driver monitoring system comprises a projector to illuminate a driver with coherent light. The term “light” may refer to electromagnetic radiation in one or more of the infrared, the visible and the ultraviolet spectral range. Herein, the term “ultraviolet spectral range”, generally, refers to electromagnetic radiation having a wavelength of 1 nm to 380 nm, preferably of 100 nm to 380 nm. Further, in partial accordance with standard ISO-21348 in a valid version at the date of this document, the term “visible spectral range”, generally, refers to a spectral range of 380 nm to 760 nm. The term “infrared spectral range” (I R) generally refers to electromagnetic radiation of 760 nm to 1000 pm, wherein the range of 760 nm to 1 .5 pm is usually denominated as “near infrared spectral range” (NIR) while the range from 1.5 p to 15 pm is denoted as “mid infrared spectral range” (MidlR) and the range from 15 pm to 1000 pm as “far infrared spectral range” (FIR). Preferably, light used for the typical purposes of the present invention is light in the infrared (IR) spectral range, more preferred, in the near infrared (NIR) and / or the mid infrared spectral range (MidlR), especially the light having a wavelength of 1 pm to 5 pm, preferably of 1 pm to 3 pm.

[0019] The term “illuminate” may refer to the process of exposing at least one element to light. The term “projector” may refer to a device configured for generating or providing light in the sense of the above-mentioned definition. The projector may be a pattern projector, a floodlight projector or both either at the same time or the projector may repeatedly switch from illuminating patterned light to floodlight.

[0020] The term “pattern projector” may refer to a device configured for generating or providing at least one light pattern, in particular at least one infrared light pattern. The term “light pattern” may refer to at least one pattern comprising a plurality of light spots. The light spot may be at least partially spatially extended. At least one spot or any spot may have an arbitrary shape. In some cases a circular shape of at least one spot or any spot may be preferred. The spots may be arranged by considering a structure of a display comprised by a device that is further comprising the optoelectronic apparatus. Typically, an arrangement of an OLED-pixel-structure of the display may be considered. The term “infrared light pattern” may refer to a light pattern comprising spots in the infrared spectral range. The infrared light pattern may be a near infrared light pattern. The infrared light may be coherent. The infrared light pattern may be a coherent infrared light pattern.

[0021] The pattern projector may be configured for emitting light at a single wavelength, e.g. in the near infrared region. In other embodiments, the pattern projector may be adapted to emit light with a plurality of wavelengths, e.g. for allowing additional measurements in other wavelengths channels.

[0022] The infrared light pattern may comprise at least one regular and / or constant and / or periodic pattern such as a triangular pattern, a rectangular pattern, a hexagonal pattern or a pattern comprising further convex tilings. For example, the infrared light pattern is a hexagonal pattern, preferably a hexagonal infrared light pattern, preferably a 2 / 5 hexagonal infrared light pattern. Using a periodical 2 / 5 hexagonal pattern can allow distinguishing between artefacts and usable signal.

[0023] At least one of the infrared light spots may be associated with a beam divergence of 0.2° to 0.5°, preferably 0.1° to 0.3°. The term “beam divergence” may refer to at least one measure of an increase in at least one diameter and / or at least one diameter equivalent, such as a radius, with a distance from an optical aperture from which the beam emerges. The measure may be an angle or an angle equivalent. In the context of the present invention, typically, a beam divergence may be determined at 1 / e2.

[0024] The pattern projector may comprise at least one pattern projector configured for generating the infrared light pattern. The pattern projector may comprise at least one emitter, in particular a plurality of emitters. The term “emitter” may refer to at least one arbitrary device configured for providing at least one light beam. The light beam may generate the infrared light pattern. The emitter may comprise at least one element selected from the group consisting of at least one laser source such as at least one semi-conductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separate confinement heterostructure laser, at least one quantum cascade laser, at least one distributed Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one Indium Arsenide laser, at least one Gallium Arsenide laser, at least one transistor laser, at least 50 one diode pumped laser, at least one distributed feedback lasers, at least one quantum well laser, at least one interband cascade laser, at least one semiconductor ring laser, at least one vertical cavity surface emitting laser (VCSEL); at least one non-laser light source such as at least one LED or at least one light bulb. For example, the pattern projector comprises at least one least one VCSEL, preferably a plurality of VCSELs. The plurality of VCSELs may be arranged in at least one array, e.g. comprising a matrix of VCSELs. The VCSELs may be arranged on the same substrate, or on different substrates. The term “vertical-cavity surface-emitting laser” may refer to a semiconductor laser diode configured for laser beam emission perpendicular with respect to a top surface. Examples for VCSELs can be found e.g. in en.wikipedia.org / wiki / Verticalcav- ity_surface-emitting_laser. VCSELs are generally known to the skilled person such as from WO 2017 / 222618 A. Each of the VCSELs is configured for generating at least one light beam. The plurality of generated spots may be associated with the infrared light pattern. The VCSELs may be configured for emitting light beams at a wavelength range from 800 to 1000 nm. For example, the VCSELs may be configured for emitting light beams at 808 nm, 850 nm, 940 nm, and / or 980 nm. Preferably the VCSELs emit light at 940 nm, since terrestrial sun radiation has a local minimum in irradiance at this wavelength, e.g. as described in CIE 085-1989 „Solar spectral Irradiance”.

[0025] The pattern projector may comprise at least one optical element configured for increasing, e.g. duplicating, the number of spots generated by the pattern projector. The pattern projector, particularly the optical element, may comprises at least one diffractive optical element (DOE) and / or at least one metasurface element. The DOE and / or the metasurface element may be configured for generating multiple light beams from a single incoming light beam. Further arrangements, particularly comprising a different number of projecting VCSEL and / or at least one different optical element configured for increasing the number of spots may be possible. Other multiplication factors are possible. For example, a VCSEL or a plurality of VCSELs may be used and the generated laser spots may be duplicated by using at least one DOE.

[0026] The pattern projector may be configured to illuminated patterned light comprising less than 4000 light beams, preferably less than 3000 light beams, more preferably less than 2000 light beams, most preferably less than 1000 light beams. For example, the patterned light may comprise 100 to 4000 light beams or 200 to 3000 light beams or 300 to 2000 light beams or 500 to 1000 light beams.

[0027] The pattern projector may comprise at least one transfer device. The term “transfer device”, also denoted as “transfer system” may refer to one or more optical elements which are adapted to modify the light beam, particularly the light beam used for generating at least a portion of the infrared light pattern, such as by modifying one or more of a beam parameter of the light beam, a width of the light beam or a direction of the light beam. The transfer device may comprise at least one imaging optical device .The transfer device specifically may comprise one or more of: at least one lens, for example at least one lens selected from the group consisting of at least one focus-tunable lens, at least one aspheric lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multilens system; at least one holographic optical element; at least one meta optical element. Specifically, the transfer device comprises at least one refractive optical lens stack. Thus, the transfer device may comprise a multi-lens system having refractive properties. The pattern projector may be configured for emitting modulated or non-modulated light. In case a plurality of emitters is used, the different emitters may have different modulation frequencies, e.g. which can be used for distinguishing the light beams.

[0028] The light beam or light beams generated by the pattern projector may propagate parallel to an optical axis. The pattern projector may comprise at least one reflective element, preferably at least one prism, for deflecting the illuminating light beam onto the optical axis. As an example, the light beam or light beams, such as the laser light beam, and the optical axis may include an angle of less than 10°, preferably less than 5° or even less than 2°. Other embodiments, however, are feasible. Further, the light beam or light beams may be on the optical axis or off the optical axis. As an example, the light beam or light beams may be parallel to the optical axis having a distance of less 10 than 10 mm to the optical axis, preferably less than 5 mm to the optical axis or even less than 1 mm to the optical axis or may even coincide with the optical axis.

[0029] The term “flood projector” may refer to at least one device configured for providing substantially continuous spatial illumination. The flood projector may illuminate a measurement area, such as a user, a portion of the user and / or a face of the user, with a spatially constant or essentially constant illumination intensity. The term “flood light” may refer to substantially continuous spatial illumination, in particular diffuse and / or uniform illumination. The flood light has a wavelength in the infrared range, in particular in the near infrared range. The flood projector may comprise at least one least one VCSEL, preferably a plurality of VCSELs. The term “substantially continuous spatial illumination” may refer to uniform spatial illumination, wherein areas of non-uniform are possible.

[0030] A relative distance between the flood projector and the pattern projector may be below 3.0 mm. The relative distance between the flood projector and the pattern projector may be below 2.5 mm, preferably below 2.0 mm. The pattern projector and the flood projector may be combined into one module. For example, the pattern projector and the flood projector may be arranged on the same substrate, in particular having a minimum relative distance. The minimum relative distance may be defined by a physical extension of the flood projector and the pattern projector. Arranging the pattern projector and the flood projector having a relative distance below 3.0 mm can result in decreased space requirement of the two projectors. In particular, said projectors can even be combined into one module. Such a reduced space requirement can allow reducing the transparent area(s) in a display necessary for operation of the projector(s) behind the display.

[0031] In an embodiment, the pattern projector and the flood projector may comprise at least one VCSEL, preferably a plurality of VCSELs. The pattern projector may comprise a plurality of first VCSELs mounted on a first platform. The flood projector may comprise a plurality of second VCSELs mounted on a second platform. The second platform may be beside the first platform. The optoelectronic apparatus may comprise a heat sink. Above the heat sink a first increment comprising the first platform may be attached. Above the heat sink a second increment comprising the second platform may be attached. The second increment may be different from the first increment. Thus, the first platform may be more distant to the optical element configured for increasing, e.g. duplicating, the number of spots. The second platform may be closer to the optical element. The beam emitted from the second VCSEL may be defocused and thus, form overlapping spots. This leads to a substantially continuous illumination and, thus, to flood illumination.

[0032] The projector may be positioned such that it can illuminate light through the transparent display. Hence, light emitted by the projector crosses the transparent display before it impinges on the person. From the person’s view, the projector is placed behind the transparent display.

[0033] The driver monitoring system further comprises a camera. The term “camera” may refer to at least one unit of the optoelectronic apparatus configured for generating at least one image. The image may be generated via a hardware and / or a software interface, which may be considered as the camera. The term “image generation” may refer to capturing and / or generating and / or determining and / or recording at least one image by using the camera. The image generation may comprise imaging and / or recording the image. The image generation may comprise capturing a single image and / or a plurality of images such as a sequence of images. For generating an image via a hardware and / or a software interface, the capturing and / or generating and / or determining and / or recording of the image may be caused and / or initiated by the hardware and / or the software interface. For example, the image generation may comprise recording continuously a sequence of images such as a video or a movie. The image generation may be initiated by a user action or may automatically be initiated, e.g. once the presence of at least one object or user within a field of view and / or within a predetermined sector of the field of view of the camera is automatically detected.

[0034] The camera may comprise at least one optical sensor, in particular at least one pixelated optical sensor. The camera may comprise at least one CMOS sensor or at least one CCD chip. For example, the camera may comprise at least one CMOS sensor, which may be sensitive in the infrared spectral range. The term “image” may refer to data recorded by using the optical sensor, such as a plurality of electronic readings from the CMOS or CCD chip. The image may comprise raw image data or may be a pre-processed image. For example, the pre-processing may comprise applying at least one filter to the raw image data and / or at least one background correction and / or at least one background subtraction.

[0035] For example, the camera may comprise a color camera, e.g. comprising at least color pixels. The camera may comprise a color CMOS camera. For example, the camera may comprise black and white pixels and color pixels. The color pixels and the black and white pixels may be combined internally in the camera. The camera may comprise may comprise at least one color camera (e.g. RGB) and / or at least one black and white camera, such as a black and white CMOS. The camera may comprise at least one black and white CMOS chip. The camera generally may comprise a one-dimensional or two-dimensional array of image sensors, such as pixels. The color camera may be an internal and / or external camera of a device comprising the optoelectronic apparatus. The internal and / or external camera of the device may be accessed via a hardware and / or a software interface comprised by the optoelectronic apparatus, which is used as the camera. In case, the device is or comprises a smartphone the image generating unit may be a front camera, such as a selfie camera, and / or back camera of the smartphone.

[0036] The camera may have a field of view between 10°x10° and 75°x75°, preferably 55°x65°. The camera may have a resolution below 2 MP, preferably between 0.3 MP and 1.5 MP.

[0037] The camera may comprise further elements, such as one or more optical elements, e.g. one or more lenses. As an example, the optical sensor may be a fix-focus camera, having at least one lens which is fixedly adjusted with respect to the camera. Alternatively, however, the camera may also comprise one or more variable lenses which may be adjusted, automatically or manually. Other cameras, however, are feasible.

[0038] The term “pattern image” may refer to an image generated by the camera while illuminating the infrared light pattern, e.g. on an object and / or a user. The pattern image may comprise an image showing a user, in particular at least parts of the face of the user, while the user is being illuminated with the infrared light pattern, particularly on a respective area of interest comprised by the image. The pattern image may be generated by imaging and / or recording light reflected by an object and / or user which is illuminated by the infrared light pattern. The pattern image showing the user may comprise at least a portion of the illuminated infrared light pattern on at least a portion the user. For example, the illumination by the pattern illumination source and the imaging by using the optical sensor may be synchronized, e.g. by using at least one control unit of the optoelectronic apparatus.

[0039] The term “flood image” may refer to an image generated by the camera while illumination source is illuminating infrared flood light, e.g. on an object and / or a user. The flood image may comprise an image showing a user, in particular the face of the user, while the user is being illuminated with the flood light. The flood image may be generated by imaging and / or recording light reflected by an object and / or user which is illuminated by the flood light. The flood image showing the user may comprise at least a portion of the flood light on at least a portion the user. For example, the illumination by the flood illumination source and the imaging by using the optical sensor may be synchronized, e.g. by using at least one control unit of the optoelectronic apparatus.

[0040] The camera may be configured for imaging and / or recording the pattern image and the flood image at the same time or at different times. The camera may be configured for imaging and / or recording the pattern image and the flood image at at least partially overlapping measurement areas or equivalents of the measurement areas.

[0041] The driver monitoring system may contain or be placed behind a transparent display. The term “display” may refer to an arbitrary shaped device configured for displaying an item of information. The item of information may be arbitrary information such as at least one image, at least one diagram, at least one histogram, at least one graphic, text, numbers, at least one sign, or an operating menu. The display may be or may comprise at least one screen. The display may have an arbitrary shape, e.g. a rectangular shape. The display may be a front display of a device.

[0042] The display may be or may comprise at least one organic light-emitting diode (OLED) display. The term “organic light emitting diode” may refer to a light-emitting diode (LED) in which an emissive electroluminescent layer is a film of organic compound configured for emitting light in response to an electric current. The OLED display may be configured for emitting visible light. The display, particularly a display area, may be covered by glass. In particular, the display may comprise at least one glass cover.

[0043] The transparent display may be at least partially transparent. The term “at least partially transparent” may refer to a property of the display to allow light, in particular of a certain wavelength range, e.g. in the infrared spectral region, in particular in the near infrared spectral region, to pass at least partially through. For example, the display may be semitransparent in the near infrared region. For example, the display may have a transparency of 20 % to 50 % in the near infrared region. The display may have a different transparency for other wavelength ranges. For example, the display may have a transparency of > 80 % for the visible spectral range, preferably > 90 % for the visible spectral range. The transparent display may be at least partially transparent over the entire display area or only parts thereof. Typically, it is sufficient if only those parts of the display area are at least partially transparent trough which light needs to pass from the projector or to the camera.

[0044] The display may comprise a display area. The term “display area” may refer to an active area of the display, in particular an area which is activatable. The display may have additional areas such as recesses or cutouts. The display may have a first area associated with a first pixel per inch (PPI) value and a second area associated with a second PPI value. The first PPI value may be lower than the second PPI value, preferably first PPI value is equal to or below 400 PPI, more preferably the second PPI value may be equal to or higher than 300 PPI. The first PPI value may be associated with the at least one continuous area being at least partially transparent.

[0045] The camera may be positioned such that it can receive light from the driver through the transparent display. Light reflected or refracted from the person firstly crosses the transparent display before it impinges on the camera. From the driver’s view, the camera may be placed behind the transparent display.

[0046] The driver monitoring system further comprises a processor. The processor may be a logic circuitry configured for performing basic operations of a computer or system, and / or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor may be configured for processing basic instructions that drive the computer or system. As an example, the processor may comprise at least one arithmetic logic unit (ALU), at least one floating-point unit (FPU), such as a math co-processor or a numeric co-processor, a plurality of registers, specifically registers configured for supplying operands to the ALU and storing results of operations, and a memory, such as an L1 and L2 cache memory. In particular, the processor may be a multi-core processor. Specifically, the processor may be or may comprise a central processing unit (CPU). Additionally or alternatively, the processor may be or may comprise a microprocessor, thus specifically the processor’s elements may be contained in one single integrated circuitry (IC) chip. Additionally or alternatively, the processor may be or may comprise one or more application-specific integrated circuits (ASICs) and / or one or more field- programmable gate arrays (FPGAs) and / or one or more tensor processing unit (TPU) and / or one or more chip, such as a dedicated machine learning optimized chip, or the like. The processor specifically may be configured, such as by software programming, for performing one or more evaluation operations. At least one or any component of a computer program configured for performing the authentication process may be executed by the processing device. Alternatively or in addition, the processor may be or may comprise a connection interface. The connection interface may be configured to transfer data from the device to a remote device; or vice versa. At least one or any component of a computer program configured for performing the authentication process may be executed by the remote device.

[0047] The processor may be configured, such as by software programming, for performing one or more evaluation operations. At least one or any component of a computer program configured for performing the authentication process may be executed by the processing device. Alternatively or in addition, the processor may be or may comprise a connection interface. The connection interface may be configured to transfer data from the device to a remote device; or vice versa. At least one or any component of a computer program configured for performing the authentication process may be executed by the remote device.

[0048] The processor is configured to determine a condition measure of the driver from the image. The term “condition measure” may refer to a measure suitable for determining the condition of a driver. A condition of a driver may be a physical and / or mental condition. A physical condition may be associated with physical stress level, fatigue, excitation, suitability of performing a certain task of a driver or a medical condition. A mental condition may be associated with mental stress level, attentiveness, concentration level, excitation, suitability of performing a certain task of a driver or the like. Such a certain task may require concentration, attention, wakefulness, calming or similar characteristics of the driver. Condition measures may indicate a condition of a driver. Condition measures may be one or several of the following: heart rate, blood pressure, respiratory rate. In some embodiments, the condition of a driver may be critical corresponding to a high value of the condition measure and the condition of a driver may be non-critical corresponding to a low value of the condition measure. Followingly, the critical condition measure according to these embodiments may be equal or lower than a threshold and a non-critical condition measure may be lower than a threshold. In other embodiments, the condition of a driver may be critical corresponding to a low value of the condition measure and the condition of a driver may be non-critical corresponding to a high value of the condition measure. Followingly the critical condition measure according to these embodiments may be equal or higher than a threshold and a non-critical condition measure may be lower than a threshold. A critical condition measure may be associated with a high stress level, low attentiveness, low concentration level, high fatigue, high excitation.

[0049] The condition measure of a driver may be determined based on the motion of a body fluid, preferably blood, most preferably red blood cells. The motion of body fluids is not constant over time but changes due to activity of parts of the driver, e.g. the heart. Such a change in motion may be determined based on a change in feature contrast over time. A high difference between values of feature contrast at different points in time may be associated with a fast change in motion. A low difference between values of feature contrast at different points in time may be associated with a slow change in motion. The change in motion of a body fluid, preferably blood, may be periodically associated with a corresponding motion frequency. Accordingly, the feature contrast may change periodically with the corresponding motion frequency. The motion frequency may correspond to the length of a period associated with the periodic change in feature contrast. In some embodiments, half of a period may be comprised in the at least two reflection images. In other embodiments, one or several periods may be comprised in the at least two reflection images. Preferably, pattern feature associated with the same part of a driver may be used for determining the condition of a driver. This is advantageous due to the fact that the blood perfusion and thus, the feature contrast across different parts of the body varies. In some embodiments, at least one condition measure may be determined based on the feature contrast.

[0050] A feature contrast may represent a measure for a contrast of an intensity distribution within an area of a pattern, in particular within the area of a pattern feature. Additionally or alternatively, feature contrast may refer to a measure for a contrast associated with a pattern feature. The feature contrast may be determined of the at least two pattern features. The feature contrast may indicate at least two feature contrast values, wherein the first of the at least two feature contrast values may be associated with the first of the at least two reflection images and / or the first of the at least two pattern features and the second of the at least two feature contrast values may be associated with the second of the at least two reflection images and / or the second of the at least two pattern features. In particular, the first of the at least two pattern features, in particular the at least one first pattern feature may be associated with the first of the at least two reflection images, in particular the at least one first reflection image. In particular, the second of the at least two pattern features, in particular the at least one second pattern feature may be associated with the second of the at least two reflection images, in particular the at least one second reflection image. The feature contrast may be determined of the first pattern feature of the at least two reflection images and for the second pattern feature of the at least two reflection images. The feature contrast may be determined separately for the at least two pattern features of the at least two reflection images.

[0051] The feature contrast may indicate and / or may comprise determining at least two feature contrast values associated with the at least two pattern features. Determining the feature contrast of the at least two pattern features may include determining a first feature contrast value associated with the first pattern feature and determining a second feature contrast value associated with the second pattern feature. A feature contrast value may be determined by determining the ratio of a standard deviation of a pattern feature intensity and a mean of the pattern feature intensity. Pattern feature intensity may comprise intensity values associated with the corresponding pattern feature.

[0052] In particular, a feature contrast value K over an area of the pattern may be expressed as a ratio of standard deviation o to the mean pattern feature intensity <l>, i.e. ,

[0053] Feature contrast values are generally distributed between 0 and 1. Feature contrast may be determined based on at least one pattern feature. Followingly, at least two feature contrast values may be determined based on the at least two pattern features.

[0054] In some embodiments, for determining the feature contrast, the complete pattern of the reflection image may be used. Alternatively, for determining the feature contrast, a section of the pattern may be used. The section of the pattern, preferably, represents a smaller area of the pattern than an area of the pattern. The area may be of any shape. The section of the pattern may be obtained by cropping the reflection image. The feature contrast may be different for different parts of a driver. Different parts of the driver may correspond to different parts of the reflection image. Followingly, the feature contrast may be different for different parts of a reflection image.

[0055] An image set may comprise a set of reflection images. Preferably, the image set comprises at least two reflection images. The set of images may be generated at different points in time. In some embodiments, at least one or more than one feature contrast values may be determined of the at least one pattern feature. A feature contrast value may correspond to a numerical value of a feature contrast.

[0056] The image set may comprise a time series. The time series may comprise reflection images separated by a constant time interval associated with an imaging frequency or changing time intervals. Preferably, the time series is constituted such that the imaging frequency is at least twice the motion frequency. This is known as the Nyquist theorem. For higher resolution more reflection images than at least required by the Nyquist theorem may be received.

[0057] For the determination of the condition measure, an indication of an interval between the different points in time where the at least two reflection images are generated is received. The indication of the interval comprises measure(s) suitable for determining the time between the different points in time where the at least two reflection images are generated.

[0058] A frequency is a reciprocal value of the length of a period. The length of a period may be determined by the interval between two reflection images comprising a share of the period of the heart beating or the heart cycle. In a normal human at rest the heart beats between 60 and 80 times per minute corresponding to a resting heart rate of 60 to 80 beats per minute (bpm). The resting heart rate may be lower, e.g. if the human is sportive or suffers from bradycardia. In situation where the human is active, the heart rate may increase up to 230 bpm. Animals may have heart rates ranging from 6 to 1000 bpm. The reflection images may be generated depending on the expected heart rate of the driver examined. The interval between the reflection images may be chosen to be up to 10 seconds. In the case of a human, the interval may be chosen up to 2 seconds. Followingly, the imaging frequency may be chosen to be at least 12 reflection images per minute or at least 60 reflection images in the case of a human. In an example, the method may be used for determining the heart rate of a human. For this purpose, an imaging frequency of 60 images per minute may be chosen. As the feature contrast is determined, one may recognize that the imaging frequency may be too low. In such a case, the imaging frequency may be increased such that the condition measure may be determined. Alternatively, the imaging frequency for imaging a human may be chosen to be a high frequency such as 460 images per minute. A heart rate may be determined based on the at least two reflection images and an indication of the interval between the at least two different points in time indicating an interval of 0.13 seconds. In the example, the human may have a heart rate in the range of 60 to 80 bpm. Followingly, the imaging frequency may be adjusted according to expected and / or predetermined condition measures.

[0059] In an example, the interval may comprise a half, a full, double length of a period or the like. The interval may be between at least two reflection images. Followingly, the at least two reflection images may be separated by a half, a full, double length of a period or the like. In the exemplary case of three reflection images indication of one or two different intervals may be received. If more than two reflection images are received, the indication of the interval may comprise an indication of an interval between the first and the second reflection image and / or an interval between the first and the third reflection image (or every other reflection image if more than three reflection images may be received) and / or an interval between the second and the third reflection image (or every other reflection image if more than three reflection images may be received). This applies accordingly to other scenarios with a different amount of reflection images as the skilled person will recognize. Measures for the indication of the interval may be at least two points in time corresponding to the different points in time where the at least two reflection images are generated and / or the time that passed between the different points in time and / or an imaging frequency associated with the generation of the reflection images. The at least two points in time may be determined based on a timestamp of the at least two reflection images. The imaging frequency may comprise a selected value. The imaging frequency may be selected based on the expected condition measure, e.g. an expected heart rate. Alternatively, the image frequency of a video may be used to determine an imaging frequency. An expected heart rate may comprise a heart rate associated with the driver monitored. In some embodiments, estimation of condition measure may be used to select the imaging frequency. An estimation of condition measure may take the living species and its surrounding into account.

[0060] In an embodiment, the image data set may comprise at least one first reflection image and at least one second reflection image. The at least one first reflection image and the at least one second reflection image may be generated at different points in time, in particular at at least two different points in time. The at least one first reflection image and the at least one second reflection image may be generated while the driver is illuminated by patterned coherent electromagnetic radiation. The at least one first reflection image may show at least one first pattern feature, preferably at least two first pattern features, formed by illuminating at least a part of the driver by the patterned coherent electromagnetic radiation. The at least one second reflection image may show at least one second pattern feature, preferably at least two second pattern features, formed by illuminating at least a part of the driver by the patterned coherent electromagnetic radiation. In particular, the at least one first pattern feature and the at least one second pattern feature may be associated with the same body part of the driver.

[0061] A feature contrast may be determined of the at least one first pattern feature and the at least one second pattern feature. The feature contrast may indicate a first feature contrast value associated with the at least one first pattern feature and a second feature contrast value associated with the at least one second pattern feature. A condition measure may be determined based on the feature contrast and the indication of the at least one interval by providing the feature contrast of the at least one first pattern feature and the at least one second pattern feature and the indication of the at least one interval between the generation of the at least one first reflection image and the at least one second reflection image to a data-driven model, wherein the data-driven model is parametrized on a training data set including historical feature contrasts indicating a plurality of first feature contrast values associated with a plurality of first pattern features and a plurality of second feature contrast values associated with the a plurality of second pattern features, a plurality of historical indications of the at least one interval and a plurality of historical condition measures.

[0062] In an embodiment, the condition measure of the driver may be determined based on the feature contrast and the indication of the at least one interval by providing the feature contrast of the at least two pattern features and the indication of the at least one interval between the generation of the at least two pattern images to a data-driven model, wherein the data-driven model is parametrized on a training data set including historical feature contrasts, historical indications of the at least one interval and historical condition measures. The feature contrast may indicate at least two feature contrast values associated with the at least two pattern features.

[0063] In an embodiment, a condition measure of the driver based on the first feature contrast, the second feature contrast and the indication of the interval may be determined by providing the first feature contrast, the second feature contrast and the indication of the interval to a data-driven model, wherein the data-driven model may be parametrized based on a training data set including historical first feature contrasts, historical second feature contrasts, historical indications of the at least one interval and historical condition measures.

[0064] Providing the feature contrast may include providing a first feature contrast value and a second feature contrast value. The data-driven model may be parametrized based on the training data set to provide and / or output a condition measure based on being provided with the feature contrast of the at least two pattern features and the indication of the at least one interval between the generation of the at least two pattern images. The data-driven model may be trained based on the training data set including historical feature contrasts, historical indications of the at least one interval and historical condition measures to provide and / or output a condition measure based on being provided with the feature contrast of the at least two pattern features and the indication of the at least one interval between the generation of the at least two pattern images. The condition measure of the driver may be determined based on the feature contrast and the indication of the at least one interval by providing the feature contrast of the at least two pattern features and the indication of the at least one interval between the generation of the at least two pattern images to a data-driven model, wherein the data-driven model is trained on a training data set including historical feature contrasts, historical indications of the at least one interval and historical condition measures. The data-driven model may receive the feature contrast and the indication of the at least one interval at an input layer and / or may provide a condition measure based on having received the feature contrast and the indication of the at least one interval at an input layer. The data-driven model may comprise at least one machine learning architecture, in particular a deep learning architecture. For example, the data-driven model may be a neural network such as a CNN, in particular a 3D CNN, or a transformer. Further, the data-driven model may be a transformer network.

[0065] In an embodiment, the at least two pattern features may be associated with the at least two reflection images. Preferably, a first pattern feature of the at least two pattern features may be shown in the first reflection image of the at least two reflection images and a second pattern feature of the at least two pattern features may be shown in the second reflection image of the at least two reflection images. A feature contrast may be determined of the at least two pattern features including the first pattern feature and the second pattern feature. In particular, determining a feature contrast of the at least two pattern features including the first pattern feature and the second pattern feature may include determining a first feature contrast including a first feature contrast value of the first pattern feature of the first reflection image of the at least two reflection images and a second feature contrast value of the second pattern feature of second first reflection image of the at least two reflection images. Determining a condition measure of the driver based on the feature contrast and the indication of the at least one interval may include determining a condition measure based on a first feature contrast value of the first pattern feature of the first reflection image of the at least two reflection images and a second feature contrast value of the second pattern feature of second first reflection image of the at least two reflection images.

[0066] In some embodiments, the condition measure may be determined using an algorithm that may implement a mechanistic model or a data-driven model. The mechanistic model, preferably, reflects physical phenomena in mathematical form, e.g., including first-principle models. A mechanistic model may comprise a set of differential equations that describe an interaction between the object and the coherent electromagnetic radiation thereby resulting in a specific condition measure. In particular, flow of a fluid and / or the geometry of the object may be represented by the mechanistic model. The mechanistic model may comprise relations between the at least two reflection images, the indication of the point in time and the condition measure. For this purpose, the relations may be suitable for determining the time interval between the at least two reflection images and determining a motion frequency corresponding to the beats per time unit (heart rate). Based on the required input to the mechanistic model leading to a feature contrast as determined from the pattern of the at least two reflection images, an associated condition measure can be determined with the mechanistic model. Including a pulse wave analysis into the mechanistic model may be suitable for determining the blood pressure as another condition measure. To do so, the velocity of the pulse wave may be determined. This information may be comprised in the at least two reflection images and the indication about an interval. The absorption and reflection behaviour of the part of the driver may indicate the aspiration level comprised in the at least one pattern feature of the reflection images. Oxygen-rich blood absorbs and thus, reflects light differently than oxygen-poor blood. Other condition measures may be determined by deploying relations between pattern features and the condition measure.

[0067] Preferably, the data-driven model may a parametrized classification model. The classification model may comprise at least one machine-learning architecture and model parameters. For example, the machine-learning architecture may be or may comprise one or more of: linear regression, logistic regression, random forest, piecewise linear, nonlinear classifiers, support vector machines, naive Bayes classifications, nearest neighbours, neural networks, convolutional neural networks, generative adversarial networks, support vector machines, or gradient boosting algorithms or the like. In the case of a neural network, the model can be a multi-scale neural network or a recurrent neural network (RNN) such as, but not limited to, a gated recurrent unit (GRU) recurrent neural network or a long short-term memory (LSTM) recurrent neural network. The term “training”, also denoted learning, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, with-out limitation, to a process of building the classification model, in particular determining and / or updating parameters of the classification model. The classification model may be at least partially data-driven. For example, the classification model may be based on experimental data, such as data determined by illuminating a plurality of drivers such as humans and recording the reflection images. For example, the training may comprise using at least one training dataset, wherein the training data set comprises reflection images, e.g. of a plurality of humans with known condition measures. For example, if the neural network is a feedforward neural network such as a CNN, a backprop- agation-algorithm may be applied for training the neural network. In case of a RNN, a gradient descent algorithm or a backpropagation-through-time algorithm may be employed for training purposes.

[0068] The processor is configured to generate a control signal for controlling a functionality of the vehicle using the condition measure. Hence, the control signal may depend on the condition measure. The term “control signal” may refer to an electrical or electronic signal that is suitable to regulate or manipulate a specific functionality or operation of a vehicle. It may serve as a communication medium between the driver monitoring system and the targeted component or subsystem within the vehicle. The control signal may only be generated if the condition measure is determined to be critical, for example because it exceeds a certain threshold, for example the stress level exceeds a range which is considered normal. The control signal may be suitable for triggering the hardware of the vehicle to execute a function, either directly or indirectly by triggering a controller or processor to act on the hardware.

[0069] The term "functionality" of a vehicle may refer to a specific operation, capability, or feature that can be controlled or manipulated by an on-board computer or electronic control unit. It may encompass various tasks or actions that the vehicle is designed to perform to enhance its performance, safety, comfort, or user experience. The functionality of a vehicle may include engine management, transmission control, braking system, suspension system, infotainment system, climate control, driver assistance systems, headlight control, or wiper control. The functionality of the vehicle may be a security-related functionality, for example a pre-save system, an emergency call system, or a vehicle automation system.

[0070] The term “pre-save system” may refer to a system designed to anticipate potential collisions or accidents and take active measures to enhance safety, i.e. avoid collisions or if not possible to reduce the damage caused by the collision. In particular, a pre-save system may comprise an automated emergency braking system. Such system may provide the functionality to prepare the brakes for braking, for example by tightening the brake pads towards the brake disk. Hence, for example, if the condition measure indicates a critical stress level of the driver, the control signal may be directed to activate the tightening of the brake pads towards the brake disk. Another functionality may be an automatic brake pressure adjustment, i.e. adjusting the brake pressure given a certain pressure the driver exerts on the brake pedal. Hence, for example, if the condition measure indicates tiredness of the driver, a control signal may be generated triggering the break pressure to be adjusted to be higher at the same pressure the driver exerts on the brake pedal than normal. The pre-save system may comprise a seat belt tightener, i.e. a mechanism which when triggered activates and pulls the seat belt tighter against the occupant's body, helping to secure them firmly in their seat. Hence, for example, if the condition measure indicates a high stress level of the driver, a control signal may be generated triggering the seat belt tightener to act.

[0071] The term “emergency call system” may refer to a telecommunication system suitable to autonomously execute an emergency call, for example to an emergency central of the fire department, an ambulance dispatcher, or the police. For example, if the condition measure indicates a critical medical situation, for example no heartbeat as a consequence of a heart attack, the control signal may be directed to trigger the emergency call system to execute an emergency call. Such emergency call may be accompanied with information about the driver, the detected medical condition and the location, for example retrieved from the GPS system of the vehicle. An emergency call may be accompanied with the identity of the driver, for example retrieved from a face recognition system.

[0072] The term “vehicle automation system” may refer to a system which can at least partially and at least in certain situations drive the vehicle without input from the driver. The vehicle automation system may be a level 1 system or a driver assistance, i.e. the system can control either steering or speed autonomously in specific circumstances to assist the driver. The vehicle automation system may be a level 2 system or a partial automation system, i.e. the system can control both steering and speed autonomously in specific circumstances to assist the driver. The vehicle automation system may be a level 3 system or conditional automation system, i.e. the system can control both steering and speed autonomously under normal environmental conditions, but requires driver oversight. The vehicle automation system may be a level 4 or high automation system, i.e. the system can complete travel autonomously under normal environmental conditions and does not require driver oversight. The vehicle automation system may be a level 5 system or full autonomy system, i.e. the system can complete travel autonomously in any environmental conditions.

[0073] The vehicle automation system may be designed for autonomous vehicles for which human oversight is necessary, for example an autonomous vehicle of level 3 or 4. The condition measure may hence indicate the readiness of the driver to take over control of the vehicle, for example his attentiveness, fatigue or health. The condition measure may allow the prediction of the readiness of the driver to take over control of the vehicle within the near future, for example within the next ten minutes. The control signal may be output to the vehicle automation system. The control signal may be configured to trigger the vehicle automation system to bring the vehicle to a save state depending on the automation level of the autonomous vehicle. The control signal may be configured to trigger the vehicle automation system to prepare for bringing the vehicle to a save state. For example, the control signal may be configured to trigger the vehicle automation system to find a place where the vehicle can be safely stopped such as an emergency rest stop or a parking lot.

[0074] The condition measure may indicate a critical medical situation, for example no heartbeat as a consequence of a heart attack, the control signal may be directed to trigger the vehicle automation system to bring the vehicle into a save state, for example by driving on a break-down lane and stop the vehicle. Another example may be that the condition measure indicates a too low attention of the driver of a vehicle with a level 2 or level 3 system, the control signal may be directed to trigger the vehicle automation system to bring the vehicle into a save state, for example by driving on a break-down lane and stop the vehicle. The condition measure may indicate a fatigue level of the driver exceeding a preset threshold and the control signal may be directed to trigger the vehicle automation system to bring the vehicle to a save state.

[0075] The functionality of the vehicle may be the engine control. For example, if the condition measure indicates an increased stress level, the control signal may be directed to trigger the engine control to react smoother towards the driver’s acting on the accelerator pedal, for example cause a decreased or slightly delayed acceleration of the vehicle at the same accelerator pedal position.

[0076] The functionality of the vehicle may be passenger-comfort-related system, such as the air conditioning, the audio system, video system. For example, if the condition measure indicates an increased stress level, the control signal may be directed to trigger the audio and / or video system to decrease volume. This may enhance the driver’s ability to focus on the traffic. Additionally, or alternatively, the control signal may be directed to trigger the air conditioning to lower the temperature in the interior of the vehicle. This may decrease the stress level or at least increase the comfort level of the driver.

[0077] The processor may be configured for identifying the driver of the vehicle, for example based on the flood image. Particularly therefore, the processor may forward data to a remote device. Alternatively or in addition, the processor may perform the identification of the user based on the flood image, particularly by running an appropriate computer program having a respective functionality. The term “identifying” may refer to identity check and / or verifying an identity of the user. The identifying of the user may comprise analyzing the flood image. The analyzing of the flood image may comprise performing a face verification of the imaged face to be the user’s face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user’s face, with a template. Determining if the imaged face is the face of the user may comprise identifying the user, in particular determining if the imaged face corresponds to at least one image of the user’s face stored in at least one memory, e.g. of the device.

[0078] The analyzing may comprise one or more of the following: a filtering; a selection of at least one region of interest; a formation of a difference image between the flood image and at least one offset; an inversion of flood image; a background correction; a decomposition into color channels; a decomposition into hue; saturation; and brightness channels; a frequency decomposition; a singular value decomposition; applying a Canny edge detector; applying a Laplacian of Gaussian filter; applying a Difference of Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Scharr operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high-pass filter; applying a low-pass filter; applying a Fourier transformation; applying a Radon-transformation; applying a Hough-transformation; applying a wavelet-transformation; a thresholding; creating a binary image. The region of interest may be determined manually by a user or may be determined automatically, such as by recognizing the user within the image. In particular, the analyzing of the flood image may comprise using at least one image recognition technique, in particular a face recognition technique. An image recognition technique comprises at least one process of identifying the user in an image. The image recognition may comprise using at least one technique selected from the technique consisting of: color-based image recognition, e.g. using features such as hue, saturation, and value (HSV) or red, green, blue (RGB); template matching, for example as illustrated on https: / / www.mathworks.com / help / vision / ug / pattern-matching.html; image segment and / or blob analysis e.g. using size, color, or shape; machine learning and / or deep learning e.g. using at least one convolutional neural network. The neural network may be trained by the user, such as in a training procedure, in which the user is indicated to take at least one or a plurality of pictures showing himself.

[0079] The analyzing of the flood image may comprise determining a plurality of facial features. The analyzing may comprise comparing, in particular matching, the determined facial features with template features. The template features may be features extracted from at least one template. The template may be or may comprise at least one image generated in an enrollment process, e.g. when initializing the seat belt monitoring system. Template may be an image of an authorized user. The template features and / or the facial feature may comprise a vector. Matching of the features may comprise determining a distance between the vectors. The identifying of the user may comprise comparing the distance of the vectors to a least one predefined limit, wherein the user is successfully identified in case the distance is smaller than or equal to the predefined limit at least within tolerances. The user declining and / or rejected otherwise.

[0080] For example, the image recognition may comprise using at least one model, in particular a trained model comprising at least one face recognition model. The analyzing of the flood image may be performed by using a face recognition system, such as FaceNet, e.g. as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. The trained model may comprises at least one convolutional neural network. For example, the convolutional neural network may be designed as described in M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks”, CoRR, abs / 1311.2901, 2013, or C. Szegedy et al., “Going deeper with convolutions”, CoRR, abs / 1409.4842, 2014. For more details with respect to convolutional neural network for the face recognition system reference is made to Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. As training data labelled image data from an image database may be used. Specifically, labeled faces may be used from one or more of G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in unconstrained environments”, Technical Report 07-49, University of Massachusetts, Amherst, October 2007, the Youtube® Faces Database as described in L. Wolf, T. Hassner, and I. Maoz, “Face recognition in unconstrained videos with matched background similarity”, in IEEE Conf, on CVPR, 2011 , or Google® Facial Expression Comparison dataset. The training of the convolutional neural network may be performed as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832.

[0081] The processor may be configured to correct image artifacts caused by diffraction of the light when passing the transparent display. The term “correct” may mean partially or fully remove the artifacts or tag them so they can be excluded from further processing, in particular from determine if the imaged person is an authorized person. Correcting image artifacts may take into account the information about the transparent display, in particular the dimensions of the pixels or the distance of repeating features to each other. This information can facilitate identifying artifacts as diffraction patterns can be calculated and compared to the image. Correcting image artifacts may comprise identifying reflection features, sorting them by brightness and selecting the locally brightest features. For determining a distance around a feature in the image which qualifies as local, the information of the transparent display may be used, in particular a distance in the image by which a light beam may be displaced by diffraction on the transparent display may be calculated based on the information about the transparent display. This method can be particularly useful for pattern images. Further details are disclosed in WO 2021 / 105265 A1.

[0082] The processor may be configured to determine the quality of the image from the camera. Determination of the quality of the image can mean determining the brightness of the image, in particular determining if the brightness of the image is within a predetermined range. This predetermined range may be selected such that image recognition yields optimum results. The processor may generate a signal indicative of the brightness level of the image. Such signal may be use, for example by a controller of the projector, to adjust the illumination power of the projector. The signal may also be used by a controller of the camera to adjust the camera settings according to the signal indicative of the brightness level and / or trigger the camera to generate a new image. Determination of the quality of the image can also mean determining the head position of the person, in particular determining the angle of the face of the person relative to the camera. It may be determined if the angle of the face of the person relative to the camera is within a predetermined range. This predetermined range may be selected such that image recognition yields optimum results. The processor may generate a signal indicative of the head position of the person. Such signal may be used, for example by a controller of the camera to trigger the camera to generate a new image. The signal may also be used to inform the user to turn the head, for example by displaying such information on the transparent display.

[0083] The processor may be configured for outsourcing at least one step of the authentication process, such as the identification of the user, and / or at least one step of the validation of the authentication process, such as the consideration of the material data, to a remote device, specifically a server and / or a cloud server. The seat belt monitoring system and the remote device may be part of a computer network, particularly the internet. The seat belt monitoring system may transmit the generated data and / or data associated to an intermediate step of the authentication process and / or its validation to the remote device. In such a scenario, the processor may be and / or may comprise a connection interface configured for transmitting information to the remote device. Data generated by the remote device used in the authentication process and / or its validation may further be transmitted to the seat belt monitoring system. This data may be received by the connection interface comprised by the seat belt monitoring system. The connection interface may specifically be configured for transmitting or exchanging information. In particular, the connection interface may provide a data transfer connection, e.g. Bluetooth, NFC, or inductive coupling. As an example, the connection interface may be or may comprise at least one port comprising one or more of a network or internet port, a USB-port, and a disk drive.

[0084] The processor may be configured for using a facial recognition authentication process operating on the pattern image and / or extracted material data. The processor may be configured for extracting material data from the pattern image.

[0085] In an embodiment, a model may be suitable for determining an output based on an input. In particular, model may be suitable for determining material data based on an image as input. A model may be a deterministic model, a data-driven model or a hybrid model. The deterministic model, preferably, reflects physical phenomena in mathematical form, e.g., including first-principles models. A deterministic model may comprise a set of equations that describe an interaction between the material and the patterned electromagnetic radiation thereby resulting in a condition measure, a condition measure measure or the like. A data-driven model may be a classification model. A hybrid model may be a classification model comprising at least one machinelearning architecture with deterministic or statistical adaptations and model parameters. Statistical or deterministic adaptations may be introduced to improve the quality of the results since those provide a systematic relation between empiricism and theory. In an embodiment, the data-driven model may be a classification model. The classification model may comprise at least one machine-learning architecture and model parameters. For example, the machinelearning architecture may be or may comprise one or more of: linear regression, logistic regression, random forest, piecewise linear, nonlinear classifiers, support vector machines, naive Bayes classifications, nearest neighbors, neural networks, convolutional neural networks, generative adversarial networks, support vector machines, or gradient boosting algorithms or the like. In the case of a neural network, the model can be a multi-scale neural network or a recurrent neural network (RNN) such as, but not limited to, a gated recurrent unit (GRU) recurrent neural network or a long short-term memory (LSTM) recurrent neural network. The data-driven model may be parametrized according to a training data set. The data-driven model may be trained based on the training data set. Training the model may include parametrizing the model. The term training may also be denoted as learning. The term specifically may refer to a process of building the classification model, in particular determining and / or updating parameters of the classification model. Updating parameters of the classification model may also be referred to as retraining. Retraining may be included when referring to training herein. In an embodiment, the training data set may include at least one image and material information.

[0086] In an embodiment, extracting material data from the image with a data-driven model may comprise providing the image to a data-driven model. Additionally or alternatively, extracting material data from the image with a data-driven model may comprise may comprise generating an embedding associated with the image based on the data-driven model. An embedding may refer to a lower dimensional representation associated with the image such as a feature vector. Feature vector may be suitable for suppressing the background while maintaining the material signature indicating the material data. In this context, background may refer to information independent of the material signature and / or the material data. Further, background may refer to information related to biometric features such as facial features. Material data may be determined with the data-driven model based on the embedding associated with the image. Additionally or alternatively, extracting material data from the image by providing the image to a data-driven model may comprise transforming the image into material data, in particular a material feature vector indicating the material data. Hence, material data may comprise further the material feature vector and / or material feature vector may be used for determining material data.

[0087] In an embodiment, authentication process may be validated based on the extracted material data. In an embodiment, the validating based on the extracted material data may comprise determining if the extracted material data corresponds a desired material data. Determining if extracted material data matches the desired material data may be referred to as validating. Allowing or declining the user and / or object to perform at least one operation on the device that requires authentication based on the material data may comprise validating the authentication or authentication process. Validating may be based on material data and / or image. Determining if the extracted material data corresponds a desired material data may comprise determining a similarity of the extracted material data and the desired material data. Determining a similarity of the extracted material data and the desired material data may comprise comparing the extracted material data with the desired material data. Desired material data may refer to predetermined material data. In an example, desired material data may be skin. It may be determined if material data may correspond to the desired material data. In the example, material data may be nonskin material or silicon. Determining if material data corresponds to a desired material data may comprise comparing material data with desired material data. A comparison of material data with desired material data may result in a allowing and / or declining the user and / or object to perform at least one operation that requires authentication. In the example, skin as desired material data may be compared with non-skin material or silicon as material data and the result may be declination since silicon or non-skin material may be different from skin.

[0088] In an embodiment, the authentication process or its validation may include generating at least one feature vector from the material data and matching the material feature vector with associate reference template vector for material.

[0089] The authentication unit may be configured for authenticating the user in case the user can be identified and / or if the material data matches the desired material data. The device may comprise at least one authorization unit configured for allowing the user to perform at least one operation on the device, e.g. unlocking the device, in case of successful authentication of the user or declining the user to perform at least one operation on the device in case of non-successful authentication. Thereby, the user may become aware of the result of the authentication.

[0090] The processor may be configured to use the identification of the driver for the determination of the condition measure. Reference data for a particular driver may be stored in memory which may be used to determine the condition measure when this particular person has been identified. For example, the average blood perfusion rate or the average breath rate for a particular driver may be stored which may be used to determine a medical condition or the stress level of the particular driver when this driver has been identified.

[0091] The processor may be configured to use the identification of the driver for the determination of the control signal. Driver-specific personal data or preference settings may be stored in memory which may be used to generate the control signal when this particular driver has been identified. The preference settings may have been manually entered or they may have been gathered from previous events, for example feedback from the driver or an analysis of various sensors indicating the reaction of the driver to the action triggered by the control signal. For example, a person may suffer from a disease which may have an impact on the driver’s ability to drive a vehicle, for example epilepsy. If this driver is identified and the condition measure indicates an epileptic fit, the control signal may be directed to trigger the vehicle automation system to bring the vehicle into a save mode, for example by driving on a break-down lane and stopping the vehicle.

[0092] The driver monitoring system comprises an output to output the control signal. The control signal may be output to a storage device, for example a hard disc, a memory device, such as RAM or a flash memory. The control signal may be output to a computer system, for example the board computer of the vehicle or the controller of the functionality the control signal is designed to address.

[0093] All described method steps may be performed by hardware in the vehicle. Therefore, for determining the condition measure of a driver a processor may be configured to exclusively perform at least one computer program, in particular at least one line of computer program code configured to execute at least one algorithm, as used in at least one of the embodiments of the method according to the present invention. Herein, the computer program as executed on the single processing device may comprise all instructions causing the computer to carry out the method. Alternatively, or in addition, at least one method step may be performed by using at least one remote device, especially selected from at least one of a server or a cloud server, particularly when the device and the remote device may be part of a computer network. In this case, the computer program may comprise at least one remote component to be executed by the at least one remote processing device to carry out the at least one method step. The remote component may have the functionality of performing the identification of the user and / or the extraction of the material data. Further, the computer program may comprise at least one interface configured to forward to and / or receive data from the at least one remote component of the computer program.

[0094] The present invention further relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to the present invention. The term "computer-readable data medium" may refer to any suitable data storage device or computer readable memory on which is stored one or more sets of instructions (for example software) embodying any one or more of the methodologies or functions described herein. The instructions may also reside, completely or at least partially, within the main memory and / or within the processor during execution thereof by the computer, main memory, and processing device, which may constitute computer- readable storage media. The instructions may further be transmitted or received over a network via a network interface device. Computer-readable data medium include hard drives, for example on a server, USB storage device, CD, DVD or Blue-ray discs. The computer program may contain all functionalities and data required for execution of the method according to the present invention or it may provide interfaces to have parts of the method processed on remote systems, for example on a cloud system. Brief Description of the Figures

[0095] Figure 1 shows the hardware elements of the driver monitoring system.

[0096] Figure 2 shows possible placements of the driver monitoring system in the interior of a car.

[0097] Figure 3 shows an example for the driver monitoring system in a car.

[0098] Figure 4 illustrates an embodiment of the method of the invention.

[0099] Description of Embodiments

[0100] Figure 1 shows the hardware elements of the driver monitoring system. The driver monitoring system 100 a projector 101 , a camera 102 and a processor 103. These components may be mounted on a printed circuit board providing the communication lines and electricity from a battery or an interface to an electricity supply. The projector 101 may project light 120 to a person 110 and a seat belt 111. The projector 101 may comprise a VCSEL array and optics. The light may be patterned light, for example a periodic dot patter. The light 120 may impinge on driver 110. Light 130 may be reflected to the camera 102 which generates an image in the optical range matching the wavelength emitted by projector 102, for example in the infrared range. The image may be a grayscale image, i.e. each pixel contains only the total intensity information, or an RGB image, i.e. different pixels indicate the intensity in a particular wavelength.

[0101] The image may be passed to processor 103. The processor 103 may be a microcontroller, i.e. containing memory and IO controller functionalities or it may be a CPU which is connected to memory and IO controllers. The processor 103 may determine a condition measure, for example the pulse frequency or the blood pressure of the driver. The processor 103 may further generate a control signal using the condition measure, for example an activation signal for the presave system or a signal to the autonomous driving routine to bring the vehicle into a save state, for example to stop on a breakdown lane. The control signal may be output to controller 140. Controller 140 may control functionalities of the vehicle by using the control signal.

[0102] Figure 2 shows possible placements of the driver monitoring system in the interior of a car. The figure shows the dashboard, the middle console, the steering wheel and the windshield of a car as seen from the inside of the car. The driver monitoring system may be integrated into various places, for example behind a transparent display or as a separate device in parts of the car accessible to the driver. The driver monitoring system may be integrated in the center above the windscreen (201). Another option is to integrate the driver monitoring system into the interior mirror (202). This may be particularly useful if the mirror functionality is only mimicked by a display which displays the rear view recorded by a camera. The driver monitoring system may be integrated into the A column on the driver’s side (203). Another possibility is space behind the steering wheel (204) where the gauges such as the speed gauge are typically placed. The driver monitoring system may further be integrated into the steering wheel rim (205). Another possibility is to place the driver monitoring system into an engine start-stop button (206), a display in the center of the dashboard (207), the side door, for example just beneath the side window (208), the door handle (209), or the arm rest in the side door (210). Furthermore, the driver monitoring system may also be integrated into the center of the steering wheel (211), for example as part of a control system for the board computer or the entertainment system. The gearshift lever (212) or the center console, such as a display in the center console (213) or a button (214) such as the board computer control button or the park break button are further options. The latter may replace traditional controls with a display. The space behind the steering wheel (204), the center of the dashboard (207) and the center console (213) may be combined in a continuous display behind which the driver monitoring system may be placed.

[0103] Figure 3 shows an example for the driver monitoring system in a car. The driver monitoring system 301 may be placed on the center of the steering wheel. Light rays 302 are emitted onto as driver. The light reflected by the driver 403 may be recorded by a camera of the driver monitoring system which generates an image which is analyzed by a processor to determine the condition measure of the driver 303 and the control signal.

[0104] Figure 4 illustrates an embodiment of the method of the invention. A driver may be illuminated (401), for example with patterned infrared light using a VCSEL projector. An image of the driver may be recorded (402). The image may be used to determine a condition measure (403), for example the heart rate and / or the respiratory frequency. Alternatively, or additionally, the condition measure (403) may be a measure derived therefrom, for example a medical condition or a stress level. The condition measure may be used to generate a control signal (404) for controlling a functionality of the vehicle, for example triggering the pre-save system to prepare the brakes for braking. The image of the driver may be used to identify the driver (406), for example by face authentication. Personalized data of the driver may be retrieved from memory (407), for example average heart rate and respiratory frequency during driving. This personalized data may be used to generate the control signal (404), for example to set a driver-specific threshold from which stress level up the pre-save system should be activated. The control signal may be output (405), for example to the on-board computer or the controller for the system the control signal is designed for.

[0105] The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the claims.

[0106] Any steps presented herein can be performed in any order. The methods disclosed herein are not limited to a specific order of these steps. It is also not required that the different steps are per-formed at a certain place or in a certain computing node of a distributed system, i.e. each of the steps may be performed at different computing nodes using different equipment / data processing. As used herein ..determining" also includes ..initiating or causing to determine", “generating" also includes ..initiating and / or causing to generate" and “providing” also includes “initiating or causing to determine, generate, select, send and / or receive”. “Initiating or causing to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

[0107] In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation. In the claims as well as in the description the word “comprising” or “including” or similar wording does not exclude other elements or steps and shall not be construed limiting to the elements or steps lined out. The indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation or further elements may be included.

[0108] Providing in the scope of this disclosure may include any interface configured to provide data. This may include an application programming interface, a human-machine interface such as a display and / or a software module interface. Providing may include communication of data or sub-mission of data to the interface, in particular display to a user or use of the data by the receiving node, entity or interface.

[0109] Various units, circuits, entities, nodes or other computing components may be described as “con-figured to” perform a task or tasks. Configured to shall recite structure meaning “having circuitry that” performs the task or tasks on operation. The units, circuits, entities, nodes or other computing components can be configured to perform the task even when the unit / circuit / component is not operating. The units, circuits, entities, nodes or other computing components that form the structure corresponding to “configured to” may include hardware circuits and / or memory storing program instructions executable to implement the operation. The units, circuits, entities, nodes or other computing components may be described as performing a task or tasks, for convenience in the description. Such descriptions shall be interpreted as including the phrase “configured to.” Any recitation of “configured to” is expressly intended not to invoke 35 U.S.C. § 112(f) interpretation.

[0110] In general, the methods, apparatuses, systems, computer elements, nodes or other computing components described herein may include memory, software components and hardware components. The memory can include volatile memory such as static or dynamic randomaccess memory and / or nonvolatile memory such as optical or magnetic disk storage, flash memory, programmable read-only memories, etc. The hardware components may include any combination of combinatorial logic circuitry, clocked storage devices such as flops, registers, latches, etc., finite state machines, memory such as static random-access memory or embedded dynamic random-access memory, custom designed circuitry, programmable logic arrays, etc. Any disclosure and embodiments described herein relate to the methods, the systems, apparatuses, devices, chemicals, materials, computer program elements lined out above and vice versa. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples and vice versa. All terms and definitions used herein are understood broadly and have their general meaning.

Claims

Claims:

1. A driver monitoring system for a vehicle comprising: a) a projector configured to illuminate a driver of the vehicle with coherent light, b) a camera configured to record an image of the driver under illumination, c) a processor configured to determine a condition measure of the driver from the image and to generate a control signal for controlling a pre-save system, an emergency call system, or a vehicle automation system of the vehicle using the condition measure, and d) an output configured to output the control signal.

2. The driver monitoring system according to claim 1 , wherein the condition measure is associated with a stress level, fatigue, excitation, or a medical condition.

3. The driver monitoring system according to claim 1 or 2, wherein the control signal is configured to trigger the vehicle automation system to bring the vehicle to a save state.

4. The driver monitoring system according to any of the claims 1 to 3, wherein the processor is configured to identify the driver from the image and retrieve driver-specific data which is used to generate the control signal.

5. The driver monitoring system according to any of the claims 1 to 4, wherein the processor is configured to determine the condition measure from a feature contrast in the image.

6. The driver monitoring system according to any of the claims 1 to 5, wherein the camera is configured to record at least two images at different points in time and the processor is configured to determine the condition measure of the driver from the at least two images.

7. The driver monitoring system according to any of the claims 1 to 6, wherein the projector comprises a vertical cavity surface emitting laser (VCSEL) array.

8. The driver monitoring system according to any of the claims 1 to 7, wherein the projector is configured to illuminate coherent light at a wavelength range from 800 to 1000 nm.

9. The driver monitoring system according to any of the claims 1 to 8, wherein the projector is configured to illuminate a periodic light pattern.

10. The driver monitoring system according to any of the claims 1 to 9, wherein the driver monitoring system comprises a transparent display and wherein the camera is positioned such that it receives light from the driver through the transparent display.

11. A vehicle containing the driver monitoring system according to any of the preceding claims.

12. The vehicle according to claim 11 , wherein the driver monitoring system is integrated into the steering wheel, besides a speed gauge, in the center of a dashboard, the A pillar, a side door, in a mirror or in a window of the vehicle.

13. Use of the driver monitoring system of any one of the preceding claims for controlling a vehicle.

14. A method for controlling a vehicle comprising: a) illuminating a driver of the vehicle with coherent light, b) recording an image of the driver under illumination, c) determining a condition measure of the driver using the image, d) determining a control signal for controlling a pre-save system, an emergency call system, or a vehicle automation system of the vehicle using the condition measure, and e) outputting the control signal.

15. A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: a) illuminating a driver of the vehicle with coherent light, b) recording an image of the driver under illumination, c) determining a condition measure of the driver using the image, d) determining a control signal for controlling a pre-save system, an emergency call system, or a vehicle automation system of the vehicle using the condition measure, and e) outputting the control signal.

Images