Seat belt monitoring system

Abstract

The disclosure is in the field of seat belt monitoring system for a vehicle. It relates to seat belt monitoring system for a vehicle comprising: a) a projector configured to illuminate a person with patterned light, b) a camera configured to capture a pattern image of the person under pattern illumination, and c) a processor configured to receive an image from the camera and to determine the position of the seat belt in the image by determining the material of the seat belt, and d) an output configured to output the position of the seat belt.

Description

[0001] Seat Belt Monitoring System

[0002] The disclosure is in the field of seat belt monitoring system for a vehicle. It relates to a seat belt monitoring system for a vehicle, a vehicle containing the seat belt monitoring system, the use of the seat belt monitoring system for controlling a vehicle, a method for determining the position of a seat belt in a vehicle, and a non-transient computer-readable medium including instructions for a method for determining the position of a seat belt in a vehicle.

[0003] Background

[0004] Seat belts are a common and highly effective safety feature in vehicles like cars. However, seat belts only work as desired if they are correctly put in place, i.e. adjusted to the size of a person and tightened correctly. Presently, sensor systems in cars only make sure that the seat belt has been fastened but are not able to check if the seat belt fits correctly. Furthermore, such systems can easily be circumvented with plugs in the form of a seat belt plug.

[0005] US 2023 / 0182680 and DE 10 2020 208861 disclose optical seat belt detection systems to monitor the correct position of the seat belt. The method requires marking the seat belt with easily identifiable patterns. Such patterns require modified seat belts. Such modifications must be durable during the whole lifetime of the vehicle and it must not compromise the safety of the seatbelt. Also, such patterns may be esthetical ly less attractive. Hence, a system is desired which does not require any particular seat belts. https: / / www. youtube. com / watch?v=NvWqCWupcMQ discloses a method of seat belt detection using material detection. While this approach generally yields good results, some low reflective seat belt materials are challenging to be detected reliably.

[0006] Summary

[0007] It was therefore the object of the present disclosure to provide a seat belt monitoring system which does not have the disadvantages of the prior art.

[0008] In one aspect the disclosure relates to a seat belt monitoring system for a vehicle comprising: a) a projector configured to illuminate a person in a vehicle with light, b) a camera configured to record an image of the person under illumination, c) a processor configured to receive the image from the camera and to determine the position of the seat belt in the image by determining the material of the seat belt, and d) an output configured to output the position of the seat belt.

[0009] In another aspect the disclosure relates to a seat belt monitoring system for a vehicle comprising: a. a projector configured to illuminate a person in a vehicle with patterned light, b. a camera configured to capture a pattern image of the person under pattern illumination, c. a processor configured to receive the pattern image from the camera and to determine the position of the seat belt in the image by determining a material and a pattern shape from the pattern image, and d. an output configured to output the position of the seat belt. In another aspect the disclosure relates to a vehicle containing the seat belt monitoring system according to the disclosure.

[0010] In another aspect the disclosure relates to a use of the seat belt monitoring system of the disclosure for controlling a vehicle.

[0011] In another aspect the disclosure relates to a method for determining the position of a seat belt in a vehicle comprising: a) illuminating a person in a vehicle with light, b) recording an image of the person under illumination, c) determining the position of the seat belt in the image by determining the material of the seat belt, d) outputting the position of the seat belt.

[0012] In another aspect the disclosure relates to a method for determining the position of a seat belt in a vehicle comprising: a. illuminating a person in a vehicle with patterned light, b. capture a pattern image of the person under pattern illumination, c. determining the position of the seat belt in the image by determining a material and a pattern shape from the pattern image, d. outputting the position of the seat belt.

[0013] In another aspect the disclosure relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: a) illuminating a person in a vehicle with light, b) recording an image of the person under illumination, c) determining the position of the seat belt in the image by determining the material of the seat belt, d) outputting the position of the seat belt.

[0014] In another aspect the disclosure relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: a. illuminating a person in a vehicle with patterned light, b. capturing an image of the person under pattern illumination, c. determining the position of the seat belt in the image by determining a material and a pattern shape from the pattern image, d. outputting the position of the seat belt.

[0015] The material detection enables a reliable detection of the seat belt without requiring any markings on the seat belt. In addition, the pattern shape analysis uses the characteristic reflectance of woven seat belt materials adding further reliance and security. The combination of both material detection and pattern shape analysis yields a reliable result even for low reflectance materials. The system is difficult to circumvent with spoofing gadgets, for example a T-shirt with a printed seat belt, as the system can be easily recognized such gadgets from their material. The material detection further enables the use of seat belts of different colors which is difficult for other image recognition systems. Further, the system is robust to disturbances, for example if the seat belt is partially covered by an arm, as no markings or specific forms are necessary. Furthermore, the system can additionally determine the dimensions and the position of the person into account. In this way it is possible to make sure that the seat belt position fits to the person. Also, the system can be used for children to detect if they are correct placed in a children's seat and if the seat belt is correctly fastened without any modifications to the seat belt. The system can further be easily adjusted to other seat belts, for example if a different material is used. Also, the same hardware can be used for other functionalities, for example face recognition.

[0016] The seat belt monitoring system is suitable for various vehicles including cars, motorcycles, buses, trucks, trains or even airplanes. Seat belts may be made from polyamide, for example PA6 or PA6.6, or polyester, for example polyethylene terephthalate (PET), but other materials are conceivable. Seat belts may have a width of 40 mm to 50 mm, for example 46 mm or 48 mm. Seat belts may be woven in a herringbone twill weaving pattern or snag-proof selvedges, for example reinforced with strong threads, plain weave or Jacquard weave. The weaving pattern may have 4 to 8 warp weave features along the width of the seat belt, such as 4 to 6, 5 to 7, or 6 to 8. These features refer to the number of individual warp yarns that are interlaced with the weft yarns, which are the horizontal threads that run across the width of the fabric, to create the desired pattern and structure. A weaving feature may have 80 to 150 warp threads, for example 80 to 110, 90 to 120, 100 to 130, 110 to 140 or 120 to 150. Seat belts may have various colors, like black, grey, white or brown.

[0017] The seat belt monitoring system may be attached to a vehicle, or it may be integrated as component or as part of a component of a vehicle, for example as part of a display in the dashboard, an entertainment control system, or load speakers. It can be places at various places, for example it may be integrated into the steering wheel, besides a speed gauge, in the center of a dashboard, the A pillar, a side door, in a mirror or in a window.

[0018] The seat belt monitoring system comprises a projector to illuminate light to a person. The term "light” may refer to electromagnetic radiation in one or more of the infrared, the visible and the ultraviolet spectral range. Herein, the term "ultraviolet spectral range”, generally, refers to electromagnetic radiation having a wavelength of 1 nm to 380 nm, preferably of 100 nm to 380 nm. Further, in partial accordance with standard ISO-21348 in a valid version at the date of this document, the term "visible spectral range”, generally, refers to a spectral range of 380 nm to 760 nm. The term "infrared spectral range” (IR) generally refers to electromagnetic radiation of 760 nm to 1000 m, wherein the range of 760 nm to 1 .5 pm is usually denominated as "near infrared spectral range” (NIR) while the range from 1.5 p to 15 pm is denoted as "mid infrared spectral range” (MidlR) and the range from 15 pm to 1000 pm as "far infrared spectral range” (FIR). Preferably, light used for the typical purposes of the present disclosure is light in the infrared (IR) spectral range, more preferred, in the near infrared (NIR) and / or the mid infrared spectral range (MidlR), especially the light having a wavelength of 1 pm to 5 pm, preferably of 1 pm to 3 pm.

[0019] The term "illuminate” may refer to the process of exposing at least one element to light. The term "projector” may refer to a device configured for generating or providing light in the sense of the above-mentioned definition. The projector may be a pattern projector, a floodlight projector or both either at the same time or the projector may repeatedly switch from illuminating patterned light to floodlight.

[0020] The term "pattern projector” may refer to a device configured for generating or providing at least one light pattern, in particular at least one infrared light pattern. The term "light pattern” may refer to at least one pattern comprising a plurality of light spots. The light spot may be at least partially spatially extended. At least one spot or any spot may have an arbitrary shape. In some cases a circular shape of at least one spot or any spot may be preferred. The spots may be arranged by considering a structure of a display comprised by a device that is further comprising the optoelectronic apparatus. Typically, an arrangement of an OLED-pixel-structure of the display may be considered. The term "infrared light pattern” may refer to a light pattern comprising spots in the infrared spectral range. The infrared light pattern may be a near infrared light pattern. The infrared light may be coherent. The infrared light pattern may be a coherent infrared light pattern.

[0021] The pattern projector may be configured for emitting light at a single wavelength, e.g. in the near infrared region. In other embodiments, the pattern projector may be adapted to emit light with a plurality of wavelengths, e.g. for allowing additional measurements in other wavelengths channels.

[0022] The infrared light pattern may comprise at least one regular and / or constant and / or periodic pattern such as a triangular pattern, a rectangular pattern, a hexagonal pattern or a pattern comprising further convex tilings. For example, the infrared light pattern is a hexagonal pattern, preferably a hexagonal infrared light pattern, preferably a 2 / 5 hexagonal infrared light pattern.

[0023] Using a periodical 2 / 5 hexagonal pattern can allow distinguishing between artefacts and usable signal.

[0024] At least one of the infrared light spots may be associated with a beam divergence of 0.2° to 0.5°, preferably 0.1 ° to 0.3°. The term "beam divergence” may refer to at least one measure of an increase in at least one diameter and / or at least one diameter equivalent, such as a radius, with a distance from an optical aperture from which the beam emerges. The measure may be an angle or an angle equivalent. In the context of the present disclosure, typically, a beam divergence may be determined at 1 / e2.

[0025] The pattern projector may comprise at least one pattern projector configured for generating the infrared light pattern. The pattern projector may comprise at least one emitter, in particular a plurality of emitters. The term "emitter” may refer to at least one arbitrary device configured for providing at least one light beam. The light beam may generate the infrared light pattern. The emitter may comprise at least one element selected from the group consisting of at least one laser source such as at least one semi-conductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separate confinement heterostructure laser, at least one quantum cascade laser, at least one distributed Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one Indium Arsenide laser, at least one Gallium Arsenide laser, at least one transistor laser, at least 50 one diode pumped laser, at least one distributed feedback lasers, at least one quantum well laser, at least one interband cascade laser, at least one semiconductor ring laser, at least one vertical cavity surface emitting laser (VCSEL); at least one non-laser light source such as at least one LED or at least one light bulb. For example, the pattern projector comprises at least one least one VCSEL, preferably a plurality of VCSELs. The plurality of VCSELs may be arranged in at least one array, e.g. comprising a matrix of VCSELs. The VCSELs may be arranged on the same substrate, or on different substrates. The term "vertical-cavity surface-emitting laser” may refer to a semiconductor laser diode configured for laser beam emission perpendicular with respect to a top surface. Examples for VCSELs can be found e.g. in en.wikipedia.org / wikiA / erticalcavity_surface-emitting_laser. VCSELs are generally known to the skilled person such as from WO 2017 / 222618 A. Each of the VCSELs is configured for generating at least one light beam. The plurality of generated spots may be associated with the infrared light pattern. The VCSELs may be configured for emitting light beams at a wavelength range from 800 to 1000 nm. For example, the VCSELs may be configured for emitting light beams at 808 nm, 850 nm, 940 nm, and / or 980 nm. Preferably the VCSELs emit light at 940 nm, since terrestrial sun radiation has a local minimum in irradiance at this wavelength, e.g. as described in CIE 085-1989 „So- lar spectral Irradiance”.

[0026] The pattern projector may comprise at least one optical element configured for increasing, e.g. duplicating, the number of spots generated by the pattern projector. The pattern projector, particularly the optical element, may comprises at least one diffractive optical element (DOE) and / or at least one metasurface element. The DOE and / or the metasurface element may be configured for generating multiple light beams from a single incoming light beam. Further arrangements, particularly comprising a different number of projecting VCSEL and / or at least one different optical element configured for increasing the number of spots may be possible. Other multiplication factors are possible. For example, a VCSEL or a plurality of VCSELs may be used and the generated laser spots may be duplicated by using at least one DOE.

[0027] The pattern projector may comprise at least one transfer device. The term "transfer device”, also denoted as "transfer system” may refer to one or more optical elements which are adapted to modify the light beam, particularly the light beam used for generating at least a portion of the infrared light pattern, such as by modifying one or more of a beam parameter of the light beam, a width of the light beam or a direction of the light beam. The transfer device may comprise at least one imaging optical device .The transfer device specifically may comprise one or more of: at least one lens, for example at least one lens selected from the group consisting of at least one focus-tunable lens, at least one aspheric lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multilens system; at least one holographic optical element; at least one meta optical element. Specifically, the transfer device comprises at least one refractive optical lens stack. Thus, the transfer device may comprise a multi-lens system having refractive properties.

[0028] The pattern projector may be configured for emitting modulated or non-modulated light. In case a plurality of emitters is used, the different emitters may have different modulation frequencies, e.g. which can be used for distinguishing the light beams.

[0029] The light beam or light beams generated by the pattern projector may propagate parallel to an optical axis. The pattern projector may comprise at least one reflective element, preferably at least one prism, for deflecting the illuminating light beam onto the optical axis. As an example, the light beam or light beams, such as the laser light beam, and the optical axis may include an angle of less than 10°, preferably less than 5° or even less than 2°. Other embodiments, however, are feasible. Further, the light beam or light beams may be on the optical axis or off the optical axis. As an example, the light beam or light beams may be parallel to the optical axis having a distance of less 10 than 10 mm to the optical axis, preferably less than 5 mm to the optical axis or even less than 1 mm to the optical axis or may even coincide with the optical axis.

[0030] The term "flood projector” may refer to at least one device configured for providing substantially continuous spatial illumination. The flood projector may illuminate a measurement area, such as a user, a portion of the user and / or a face of the user, with a spatially constant or essentially constant illumination intensity. The term "flood light” may refer to substantially continuous spatial illumination, in particular diffuse and / or uniform illumination. The flood light has a wavelength in the infrared range, in particular in the near infrared range. The flood projector may comprise at least one least one VCSEL, preferably a plurality of VCSELs. The term "substantially continuous spatial illumination” may refer to uniform spatial illumination, wherein areas of non-uniform are possible.

[0031] A relative distance between the flood projector and the pattern projector may be below 3.0 mm. The relative distance between the flood projector and the pattern projector may be below 2.5 mm, preferably below 2.0 mm. The pattern projector and the flood projector may be combined into one module. For example, the pattern projector and the flood projector may be arranged on the same substrate, in particular having a minimum relative distance. The minimum relative distance may be defined by a physical extension of the flood projector and the pattern projector. Arranging the pattern projector and the flood projector having a relative distance below 3.0 mm can result in decreased space requirement of the two projectors. In particular, said projectors can even be combined into one module. Such a reduced space requirement can allow reducing the transparent area(s) in a display necessary for operation of the projectors) behind the display.

[0032] In an embodiment, the pattern projector and the flood projector may comprise at least one VCSEL, preferably a plurality of VCSELs. The pattern projector may comprise a plurality of first VCSELs mounted on a first platform. The flood projector may comprise a plurality of second VCSELs mounted on a second platform. The second platform may be beside the first platform. The optoelectronic apparatus may comprise a heat sink. Above the heat sink a first increment comprising the first platform may be attached. Above the heat sink a second increment comprising the second platform may be attached. The second increment may be different from the first increment. Thus, the first platform may be more distant to the optical element configured for increasing, e.g. duplicating, the number of spots. The second platform may be closer to the optical element. The beam emitted from the second VCSEL may be defocused and thus, form overlapping spots. This leads to a substantially continuous illumination and, thus, to flood illumination.

[0033] The projector may be positioned such that it can illuminate light through the transparent display. Hence, light emitted by the projector crosses the transparent display before it impinges on the person. From the person's view, the projector is placed behind the transparent display.

[0034] The seat belt monitoring system further comprises a camera. The term "camera” may refer to at least one unit of the optoelectronic apparatus configured for generating at least one image. The image may be generated via a hardware and / or a software interface, which may be considered as the camera. The term "image generation” may refer to capturing and / or generating and / or determining and / or recording at least one image by using the camera. The image generation may comprise imaging and / or recording the image. The image generation may comprise capturing a single image and / or a plurality of images such as a sequence of images. For generating an image via a hardware and / or a software interface, the capturing and / or generating and / or determining and / or recording of the image may be caused and / or initiated by the hardware and / or the software interface. For example, the image generation may comprise recording continuously a sequence of images such as a video or a movie. The image generation may be initiated by a user action or may automatically be initiated, e.g. once the presence of at least one object or user within a field of view and / or within a predetermined sector of the field of view of the camera is automatically detected.

[0035] The camera may comprise at least one optical sensor, in particular at least one pixelated optical sensor. The camera may comprise at least one CMOS sensor or at least one CCD chip. For example, the camera may comprise at least one CMOS sensor, which may be sensitive in the infrared spectral range. The term "image” may refer to data recorded by using the optical sensor, such as a plurality of electronic readings from the CMOS or CCD chip. The image may comprise raw image data or may be a pre-processed image. For example, the pre-processing may comprise applying at least one filter to the raw image data and / or at least one background correction and / or at least one background subtraction.

[0036] For example, the camera may comprise a color camera, e.g. comprising at least color pixels. The camera may comprise a color CMOS camera. For example, the camera may comprise black and white pixels and color pixels. The color pixels and the black and white pixels may be combined internally in the camera. The camera may comprise at least one color camera (e.g. RGB) and / or at least one black and white camera, such as a black and white CMOS. The camera may comprise at least one black and white CMOS chip. The camera generally may comprise a one-dimensional or two-dimensional array of image sensors, such as pixels.

[0037] The color camera may be an internal and / or external camera of a device comprising the optoelectronic apparatus. The internal and / or external camera of the device may be accessed via a hardware and / or a software interface comprised by the optoelectronic apparatus, which is used as the camera. In case, the device is or comprises a smartphone the image generating unit may be a front camera, such as a selfie camera, and / or back camera of the smartphone.

[0038] The camera may have a field of view between 10°x10° and 75°x75°, preferably 55°x65°. The camera may have a resolution below 2 MP, preferably between 0.3 MP and 1.5 MP.

[0039] The camera may comprise further elements, such as one or more optical elements, e.g. one or more lenses. As an example, the optical sensor may be a fix-focus camera, having at least one lens which is fixedly adjusted with respect to the camera. Alternatively, however, the camera may also comprise one or more variable lenses which may be adjusted, automatically or manually. Other cameras, however, are feasible.

[0040] The term "pattern image” may refer to an image generated by the camera while illuminating the infrared light pattern, e.g. on an object and / or a user. The pattern image may comprise an image showing a user, in particular at least parts of the face of the user, while the user is being illuminated with the infrared light pattern, particularly on a respective area of interest comprised by the image. The pattern image may be generated by imaging and / or recording light reflected by an object and / or user which is illuminated by the infrared light pattern. The pattern image showing the user may comprise at least a portion of the illuminated infrared light pattern on at least a portion the user. For example, the illumination by the pattern illumination source and the imaging by using the optical sensor may be synchronized, e.g. by using at least one control unit of the optoelectronic apparatus.

[0041] The term "flood image” may refer to an image generated by the camera while illumination source is illuminating infrared flood light, e.g. on an object and / or a user. The flood image may comprise an image showing a user, in particular the face of the user, while the user is being illuminated with the flood light. The flood image may be generated by imaging and / or recording light reflected by an object and / or user which is illuminated by the flood light. The flood image showing the user may comprise at least a portion of the flood light on at least a portion the user. For example, the illumination by the flood illumination source and the imaging by using the optical sensor may be synchronized, e.g. by using at least one control unit of the optoelectronic apparatus. The camera may be configured for imaging and / or recording the pattern image and the flood image at the same time or at different times. The camera may be configured for imaging and / or recording the pattern image and the flood image at at least partially overlapping measurement areas or equivalents of the measurement areas.

[0042] The seat belt monitoring system may contain or be placed behind a transparent display. The term "display” may refer to an arbitrary shaped device configured for displaying an item of information. The item of information may be arbitrary information such as at least one image, at least one diagram, at least one histogram, at least one graphic, text, numbers, at least one sign, or an operating menu. The display may be or may comprise at least one screen. The display may have an arbitrary shape, e.g. a rectangular shape. The display may be a front display of the device.

[0043] The display may be or may comprise at least one organic light-emitting diode (OLED) display. The term "organic light emitting diode” may refer to a light-emitting diode (LED) in which an emissive electroluminescent layer is a film of organic compound configured for emitting light in response to an electric current. The OLED display may be configured for emitting visible light. The display, particularly a display area, may be covered by glass. In particular, the display may comprise at least one glass cover.

[0044] The transparent display may be at least partially transparent. The term "at least partially transparent” may refer to a property of the display to allow light, in particular of a certain wavelength range, e.g. in the infrared spectral region, in particular in the near infrared spectral region, to pass at least partially through. For example, the display may be semitransparent in the near infrared region. For example, the display may have a transparency of 20 % to 50 % in the near infrared region. The display may have a different transparency for other wavelength ranges. For example, the display may have a transparency of > 80 % for the visible spectral range, preferably > 90 % for the visible spectral range. The transparent display may be at least partially transparent over the entire display area or only parts thereof. Typically, it is sufficient if only those parts of the display area are at least partially transparent trough which light needs to pass from the projector or to the camera.

[0045] The display comprises a display area. The term "display area” may refer to an active area of the display, in particular an area which is activatable. The display may have additional areas such as recesses or cutouts. The display may have a first area associated with a first pixel per inch (PPI) value and a second area associated with a second PPI value. The first PPI value may be lower than the second PPI value, preferably first PPI value is equal to or below 400 PPI, more preferably the second PPI value may be equal to or higher than 300 PPI. The first PPI value may be associated with the at least one continuous area being at least partially transparent.

[0046] The camera may be positioned such that it can receive light from the person through the transparent display. Light reflected or refracted from the person firstly crosses the transparent display before it impinges on the camera. From the person's view, the camera is placed behind the transparent display.

[0047] The seat belt monitoring system further comprises a processor. The processor may be a logic circuitry configured for performing basic operations of a computer or system, and / or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor may be configured for processing basic instructions that drive the computer or system. As an example, the processor may comprise at least one arithmetic logic unit (ALU), at least one floating-point unit (FPU), such as a math co-processor or a numeric co-processor, a plurality of registers, specifically registers configured for supplying operands to the ALU and storing results of operations, and a memory, such as an L1 and L2 cache memory. In particular, the processor may be a multi-core processor. Specifically, the processor may be or may comprise a central processing unit (CPU). Additionally or alternatively, the processor may be or may comprise a microprocessor, thus specifically the processor's elements may be contained in one single integrated circuitry (IC) chip. Additionally or alternatively, the processor may be or may comprise one or more application-specific integrated circuits (ASICs) and / or one or more field-programmable gate arrays (FPGAs) and / or one or more tensor processing unit (TPU) and / or one or more chip, such as a dedicated machine learning optimized chip, or the like. The processor specifically may be configured, such as by software programming, for performing one or more evaluation operations. At least one or any component of a computer program configured for performing the authentication process may be executed by the processing device. Alternatively or in addition, the processor may be or may comprise a connection interface. The connection interface may be configured to transfer data from the device to a remote device; or vice versa. At least one or any component of a computer program configured for performing the authentication process may be executed by the remote device.

[0048] The processor may be configured, such as by software programming, for performing one or more evaluation operations. At least one or any component of a computer program configured for performing the authentication process may be executed by the processing device. Alternatively or in addition, the processor may be or may comprise a connection interface. The connection interface may be configured to transfer data from the device to a remote device; or vice versa. At least one or any component of a computer program configured for performing the authentication process may be executed by the remote device.

[0049] The processor may be further configured to determining the position of the seat belt in the image by determining the material of the seat belt. The material of the seat belt may be stored on a storage device. The processor may identify in the image the material for each section, for example each pixel or a group of pixels, and classify this section according to the material of the seat belt. In other words, each section of the image can be classified if it contains seat belt material or not. In this way, the location of the seat belt in the image can be determined.

[0050] Material detection may be based on the pattern image. Particularly therefore, the processor may forward data to a remote device. Alternatively or in addition, the processor may perform the material determination based on the pattern image, particularly by running an appropriate computer program having a respective functionality.

[0051] The processor may be configured for determining the material from the pattern image by beam profile analysis of the light spots. With respect to beam profile analysis reference is made to WO 2018 / 091649 A1 , WO 2018 / 091638 A1 and WO 2018 / 091640 A1 , the full content of which is included by reference. Beam profile analysis can allow for providing a reliable classification of scenes based on a few light spots. Each of the light spots of the pattern image may comprise a beam profile. The term "beam profile” may generally refer to at least one intensity distribution of the light spot on the optical sensor as a function of the pixel. The beam profile may be selected from the group consisting of a trapezoid beam profile; a triangle beam profile; a conical beam profile and a linear combination of Gaussian beam profiles.

[0052] Determining the material from the pattern image may comprise generating the material type and / or data derived from the material type. Preferably, extracting material data may be based on the pattern image. Material data may be extracted by using at least one model. Extracting material data may include providing the pattern image to a model and / or receiving material data from the model. Providing the image to a model may comprise and may be followed by receiving the pattern image at an input layer of the model or via a model loss function. The model may be a data- driven model. Data-driven model may comprise a convolutional neural network and / or an encoder decoder structure such as an autoencoder. Other examples for generating a representation may be FFT, wavelets, deep learning, like CNNs, energy models, normalizing flows, GANs, vision transformers, or transformers used for natural language processing, Autoregressive Image Modeling, Normalizing Flows, Deep Autoencoders, Deep Energy-Based Models. Supervised or unsupervised schemes may be applicable to generate a representation, also embedding in e.g. cosine or Euclidian metric in ML language. The data-driven model may be parametrized according to a training data set including at least one image and material data, preferably at least one pattern image and material data. In another embodiment, extracting material data may include providing the image to a model and / or receiving material data from the model. In another embodiment, the data-driven model may be trained according to a training data set including at least one image and material data. In another embodiment, the data-driven model may be parametrized according to a training data set including at least one image and material data. The data-driven model may be parametrized according to a training data set to receive the image and provide material data based on the received image. The data- driven model may be trained according to a training data set to receive the image and provide material data as output based on the received image. The training data set may comprise at least one image and material data, preferably material data associated with the at least one image. The image may comprise a representation of the image. The representation may be a lower dimensional representation of the image. The representation may comprise at least a part of the data or the information associated with the image. The representation of an image may comprise a feature vector. In an embodiment, determining a representation, in particular a lower-dimensional representation may be based on principal component analysis (PCA) mapping or radial basis function (RBF) mapping. Determining a representation may also be referred to as generating a representation. Generating a representation based on PCA mapping may include clustering based on features in the pattern image and / or partial image. Additionally or alternatively, generating a representation may be based on neural network structures suitable for reducing dimensionality. Neural network structures suitable for reducing dimensionality may comprise encoder and / or decoder. In an example, neural network structure may be an autoencoder. In an example, neural network structure may comprise a convolutional neural network (CNN). The CNN may comprise at least one convolutional layer and / or at least one pooling layer. CNNs may reduce the dimensionality of a partial image and / or an image by applying a convolution, e.g. based on a convolutional layer, and / or by pooling. Applying a convolution may be suitable for selecting feature related to material information of the pattern image.

[0053] Determining the pattern shape from the pattern image may comprise a pattern shape model. A pattern shape model may comprise a segmentation algorithm. Segmentation algorithms may include thresholding, such as global thresholding or adaptive thresholding; edge detection, such as Canny edge detection or Sobel filtering; region-based segmentation, such as region growing, watershed algorithm; clustering-based segmentation such as K-means clustering; graph-based segmentation; template matching, wherein the illumination pattern may serve as template; or a convolutional neural network, for example U-Net or R-CNN. The pattern shape model may receive the pattern image as input and output pattern shape data. Pattern shape data may be associated with the shape of the pattern. The shape of the pattern may refer to the external contour or the internal structure of a pattern feature, for example a diffraction pattern within a spot. Pattern shape data may comprise the absolute shape of the pattern or a relationship to the shape of the illuminated pattern. For example, the illuminated pattern may comprise circular spots and the pattern shape data may comprise a value indicating the deviation from circularity, for example the spot asymmetry such as a geometric moment of the spot. The pattern shape may be influenced by anisotropic reflection of an object, for example due to a weaving pattern of a seat belt.

[0054] The seat belt position may be determined from the material data and the pattern shape data. A position in the image may be identified as seat belt if both material data and pattern shape data correspond to the seat belt. A position in the image may be identified as seat belt if at least one of material data and pattern shape data correspond to the seat belt.

[0055] Partial images may be generated from the pattern image, for example by cropping. A partial image may comprise one or more than on pattern feature. Hence, the pattern image may be divided into a plurality of partial images. Material data and pattern shape data may be determined for the partial images. For a partial image an indicator may be determined indicating whether the partial image shows the seat belt or not. The indicator may be determined by using the material data and the pattern shape data. A map may be generated comprising material data and pattern shape data for different parts of the pattern image, for example from the corresponding partial images. A map may be generated comprising an indicator for different parts of the pattern image, wherein the indicator indicates whether the part of the pattern image shows the seat belt or not. The seat belt position may be determined from the map.

[0056] The seat belt position may be determined from the flood image. Image analysis algorithms designed to recognize the shape or the color of the seat belt may be determined. The position of the seat belt may be verified by material detection. The position determined from the flood image may be verified by determining the material and / or the pattern shape from the pattern image at the corresponding positions.

[0057] The position and / or the size of the person in the vehicle may be detected from the image. For example, the position of the head, the shoulders, the hip or the legs may be determined. The position and / or size of the person may be determined from the flood image, for example by shape recognition. The position and / or size of the person may be determined from the pattern image, for example by determining a 3D model of the person. The position of the seat belt may be determined relative to the person, i.e. the position of the seat belt relative to body parts of the person. In this way, it may be determined if the seat belt is correctly positioned at the shoulder or the hip of the person.

[0058] An object in proximity to the seat belt may be recognized and its dimensions determined, for example its height, width, depth or thickness. A 3D model of the object may be determined. For example, clothing of the person like a jacket may be recognized. Another example may be a child seat. Such objects may have an influence on the correct position of the seat belt. In particular for a child seat, the seat belt should be correctly tightened to the child seat.

[0059] The processor may be configured for identifying the person in the vehicle, for example based on the flood image. Particularly therefore, the processor may forward data to a remote device. Alternatively or in addition, the processor may perform the identification of the user based on the flood image, particularly by running an appropriate computer program having a respective functionality. The term "identifying” may refer to identity check and / or verifying an identity of the user. The identifying of the user may comprise analyzing the flood image. The analyzing of the flood image may comprise performing a face verification of the imaged face to be the user's face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user's face, with a template. Determining if the imaged face is the face of the user may comprise identifying the user, in particular determining if the imaged face corresponds to at least one image of the user's face stored in at least one memory, e.g. of the device.

[0060] The analyzing may comprise one or more of the following: a filtering; a selection of at least one region of interest; a formation of a difference image between the flood image and at least one offset; an inversion of flood image; a background correction; a decomposition into color channels; a decomposition into hue; saturation; and brightness channels; a frequency decomposition; a singular value decomposition; applying a Canny edge detector; applying a Laplacian of Gaussian filter; applying a Difference of Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Scharr operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high-pass filter; applying a low-pass filter; applying a Fourier transformation; applying a Radon-transfor- mation; applying a Hough-transformation; applying a wavelet-transformation; a thresholding; creating a binary image. The region of interest may be determined manually by a user or may be determined automatically, such as by recognizing the user within the image. In particular, the analyzing of the flood image may comprise using at least one image recognition technique, in particular a face recognition technique. An image recognition technique comprises at least one process of identifying the user in an image. The image recognition may comprise using at least one technique selected from the technique consisting of: color-based image recognition, e.g. using features such as hue, saturation, and value (HSV) or red, green, blue (RGB); template matching, for example as illustrated on https: / / www.mathworks.com / help / vision / ug / pattern-matching.html; image segment and / or blob analysis e.g. using size, color, or shape; machine learning and / or deep learning e.g. using at least one convolutional neural network. The neural network may be trained by the user, such as in a training procedure, in which the user is indicated to take at least one or a plurality of pictures showing himself.

[0061] The analyzing of the flood image may comprise determining a plurality of facial features. The analyzing may comprise comparing, in particular matching, the determined facial features with template features. The template features may be features extracted from at least one template. The template may be or may comprise at least one image generated in an enrollment process, e.g. when initializing the seat belt monitoring system. Template may be an image of an authorized user. The template features and / or the facial feature may comprise a vector. Matching of the features may comprise determining a distance between the vectors. The identifying of the user may comprise comparing the distance of the vectors to a least one predefined limit, wherein the user is successfully identified in case the distance is smaller than or equal to the predefined limit at least within tolerances. The user declining and / or rejected otherwise.

[0062] For example, the image recognition may comprise using at least one model, in particular a trained model comprising at least one face recognition model. The analyzing of the flood image may be performed by using a face recognition system, such as FaceNet, e.g. as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. The trained model may comprises at least one convolutional neural network. For example, the convolutional neural network may be designed as described in M. D. Zeller and R. Fergus, "Visualizing and understanding convolutional networks”, CoRR, abs / 1311.2901 , 2013, or C. Szegedy et al., "Going deeper with convolutions”, CoRR, abs / 1409.4842, 2014. For more details with respect to convolutional neural network for the face recognition system reference is made to Florian Schroff, Dmitry Kalenichenko, James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. As training data labelled image data from an image database may be used. Specifically, labeled faces may be used from one or more of G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, "Labeled faces in the wild: A database for studying face recognition in unconstrained environments”, Technical Report 07-49, University of Massachusetts, Amherst, October 2007, the Youtube® Faces Database as described in L. Wolf, T. Hassner, and I. Maoz, "Face recognition in unconstrained videos with matched background similarity”, in IEEE Conf, on CVPR, 2011, or Google® Facial Expression Comparison dataset. The training of the convolutional neural network may be performed as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832.

[0063] The processor may be configured to correct image artifacts caused by diffraction of the light when passing the transparent display. The term "correct” may mean partially or fully remove the artifacts or tag them so they can be excluded from further processing, in particular from determine if the imaged person is an authorized person. Correcting image artifacts may take into account the information about the transparent display, in particular the dimensions of the pixels or the distance of repeating features to each other. This information can facilitate identifying artifacts as diffraction patterns can be calculated and compared to the image. Correcting image artifacts may comprise identifying reflection features, sorting them by brightness and selecting the locally brightest features. For determining a distance around a feature in the image which qualifies as local, the information of the transparent display may be used, in particular a distance in the image by which a light beam may be displaced by diffraction on the transparent display may be calculated based on the information about the transparent display. This method can be particularly useful for pattern images. Further details are disclosed in WO 2021 / 105265 A1.

[0064] The processor may be configured to determine the quality of the image from the camera. Determination of the quality of the image can mean determining the brightness of the image, in particular determining if the brightness of the image is within a predetermined range. This predetermined range may be selected such that image recognition yields optimum results. The processor may generate a signal indicative of the brightness level of the image. Such signal may be use, for example by a controller of the projector, to adjust the illumination power of the projector. The signal may also be used by a controller of the camera to adjust the camera settings according to the signal indicative of the brightness level and / or trigger the camera to generate a new image. Determination of the quality of the image can also mean determining the head position of the person, in particular determining the angle of the face of the person relative to the camera. It may be determined if the angle of the face of the person relative to the camera is within a predetermined range. This predetermined range may be selected such that image recognition yields optimum results. The processor may generate a signal indicative of the head position of the person. Such signal may be used, for example by a controller of the camera to trigger the camera to generate a new image. The signal may also be used to inform the user to turn the head, for example by displaying such information on the transparent display.

[0065] The processor may be configured for outsourcing at least one step of the authentication process, such as the identification of the user, and / or at least one step of the validation of the authentication process, such as the consideration of the material data, to a remote device, specifically a server and / or a cloud server. The seat belt monitoring system and the remote device may be part of a computer network, particularly the internet. The seat belt monitoring system may transmit the generated data and / or data associated to an intermediate step of the authentication process and / or its validation to the remote device. In such a scenario, the processor may be and / or may comprise a connection interface configured for transmitting information to the remote device. Data generated by the remote device used in the authentication process and / or its validation may further be transmitted to the seat belt monitoring system. This data may be received by the connection interface comprised by the seat belt monitoring system. The connection interface may specifically be configured for transmitting or exchanging information. In particular, the connection interface may provide a data transfer connection, e.g. Bluetooth, NFC, or inductive coupling. As an example, the connection interface may be or may comprise at least one port comprising one or more of a network or internet port, a USB-port, and a disk drive.

[0066] The processor may be configured for using a facial recognition authentication process operating on the pattern image and / or extracted material data. The processor may be configured for extracting material data from the pattern image.

[0067] In an embodiment, a model may be suitable for determining an output based on an input. In particular, model may be suitable for determining material data based on an image as input. A model may be a deterministic model, a data- driven model or a hybrid model. The deterministic model, preferably, reflects physical phenomena in mathematical form, e.g., including first-principles models. A deterministic model may comprise a set of equations that describe an interaction between the material and the patterned electromagnetic radiation thereby resulting in a condition measure, a vital sign measure or the like. A data-driven model may be a classification model. A hybrid model may be a classification model comprising at least one machine-learning architecture with deterministic or statistical adaptations and model parameters. Statistical or deterministic adaptations may be introduced to improve the quality of the results since those provide a systematic relation between empiricism and theory. In an embodiment, the data-driven model may be a classification model. The classification model may comprise at least one machine-learning architecture and model parameters. For example, the machine-learning architecture may be or may comprise one or more of: linear regression, logistic regression, random forest, piecewise linear, nonlinear classifiers, support vector machines, naive Bayes classifications, nearest neighbors, neural networks, convolutional neural networks, generative adversarial networks, support vector machines, or gradient boosting algorithms or the like. In the case of a neural network, the model can be a multi-scale neural network or a recurrent neural network (RNN) such as, but not limited to, a gated recurrent unit (GRU) recurrent neural network or a long short-term memory (LSTM) recurrent neural network. The data-driven model may be parametrized according to a training data set. The data-driven model may be trained based on the training data set. Training the model may include parametrizing the model. The term training may also be denoted as learning. The term specifically may refer to a process of building the classification model, in particular determining and / or updating parameters of the classification model. Updating parameters of the classification model may also be referred to as retraining. Retraining may be included when referring to training herein. In an embodiment, the training data set may include at least one image and material information.

[0068] In an embodiment, extracting material data from the image with a data-driven model may comprise providing the image to a data-driven model. Additionally or alternatively, extracting material data from the image with a data-driven model may comprise may comprise generating an embedding associated with the image based on the data-driven model. An embedding may refer to a lower dimensional representation associated with the image such as a feature vector. Feature vector may be suitable for suppressing the background while maintaining the material signature indicating the material data. In this context, background may refer to information independent of the material signature and / or the material data. Further, background may refer to information related to biometric features such as facial features. Material data may be determined with the data-driven model based on the embedding associated with the image. Additionally or alternatively, extracting material data from the image by providing the image to a data-driven model may comprise transforming the image into material data, in particular a material feature vector indicating the material data. Hence, material data may comprise further the material feature vector and / or material feature vector may be used for determining material data. In an embodiment, authentication process may be validated based on the extracted material data.

[0069] In an embodiment, the validating based on the extracted material data may comprise determining if the extracted material data corresponds a desired material data. Determining if extracted material data matches the desired material data may be referred to as validating. Allowing or declining the user and / or object to perform at least one operation on the device that requires authentication based on the material data may comprise validating the authentication or authentication process. Validating may be based on material data and / or image. Determining if the extracted material data corresponds a desired material data may comprise determining a similarity of the extracted material data and the desired material data. Determining a similarity of the extracted material data and the desired material data may comprise comparing the extracted material data with the desired material data. Desired material data may refer to predetermined material data. In an example, desired material data may be skin. It may be determined if material data may correspond to the desired material data. In the example, material data may be non-skin material or silicon. Determining if material data corresponds to a desired material data may comprise comparing material data with desired material data. A comparison of material data with desired material data may result in a allowing and / or declining the user and / or object to perform at least one operation that requires authentication. In the example, skin as desired material data may be compared with non-skin material or silicon as material data and the result may be declination since silicon or non-skin material may be different from skin.

[0070] In an embodiment, the authentication process or its validation may include generating at least one feature vector from the material data and matching the material feature vector with associate reference template vector for material.

[0071] The authentication unit may be configured for authenticating the user in case the user can be identified and / or if the material data matches the desired material data. The device may comprise at least one authorization unit configured for allowing the user to perform at least one operation on the device, e.g. unlocking the device, in case of successful authentication of the user or declining the user to perform at least one operation on the device in case of non-suc- cessful authentication. Thereby, the user may become aware of the result of the authentication.

[0072] The seat belt monitoring system comprises an output configured to output the position of the seat belt. The position of the seat belt may be output to a storage device, for example a hard disc, a memory device, such as RAM or a flash memory. The position of the seat belt may be output to a computer system, for example the board computer of the vehicle. The position of the seat belt may be used to determine if the seat belt is in the correct position. Such determination may be performed by the processor of the seat belt monitoring system or it may be performed by a different system, for example the board computer of the vehicle. The determination if the seat belt is in the correct position may be performed by using the position of the seat belt and the position and / or size of the person. The determination if the seat belt is in the correct position may be performed by using the position of the seat belt and recognized object in the proximity of the seat belt, for example a child seat. The determination if the seat belt is in the correct position may involve reference data, for example reference data indicating the correct position of the seat belt. Such reference data may be dependent on the size of the person or it may be specific for a certain person. A seat belt position which fits perfectly for an average sized person may be dangerous for a very small person or a very tall person. The seat belt position of thick person may fit well, but may be dangerous for thin persons. The selection of the reference data specific for a person may involve face recognition of the system. The determination if the seat belt is in the correct position may yield an indicator indicating if the seat belt is in the correct position. The indicator may be binary, for example 0 for an incorrect position and 1 for a correct position. The indicator may be a numeric value indicating the correctness of the position of the seat belt, for example a score such as 0 to 100, wherein 0 indicate completely incorrect position and 100 perfect position and any value in between a certain degree of correctness. If the indicator may be output to the same location as the position of the seat belt or to a different location. For example, the output of the seat belt may be output to a storage device and the indicator may be output to the board computer of the vehicle.

[0073] The position of the seat belt as determined by the system of the present disclosure may be used for controlling the vehicle. A warning to the driver may be displayed on a display if the position of the seat belt is incorrect for a person in the vehicle, for example for children in the back seat. The vehicle may be hindered to start or drive in case the seat belt of a person in the vehicle is incorrect. The seat belt tightener may be activated if a too loose seat belt is detected. Controlling the vehicle may include local regulation. For example, in one country the correct positioning of the seat belt may be required only for the persons sitting in the front, while in other countries this is required for all persons. The controlling may hence involve location information, for example from the GPS system, and local regulation data, for example received from a cloud server or from a local database.

[0074] The present disclosure further relates to a method for determining the position of a seat belt in a vehicle. Unless explicitly described differently in the following, the description including preferred embodiments described above apply to the method.

[0075] All described method steps may be performed by hardware in the vehicle. Therefore, for determining the position of the seat belt a processor may be configured to exclusively perform at least one computer program, in particular at least one line of computer program code configured to execute at least one algorithm, as used in at least one of the embodiments of the method according to the present disclosure. Herein, the computer program as executed on the single processing device may comprise all instructions causing the computer to carry out the method. Alternatively, or in addition, at least one method step may be performed by using at least one remote device, especially selected from at least one of a server or a cloud server, particularly when the device and the remote device may be part of a computer network. In this case, the computer program may comprise at least one remote component to be executed by the at least one remote processing device to carry out the at least one method step. The remote component may have the functionality of performing the identification of the user and / or the extraction of the material data. Further, the computer program may comprise at least one interface configured to forward to and / or receive data from the at least one remote component of the computer program.

[0076] The present disclosure further relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to the present disclosure. The term "computer-readable data medium" may refer to any suitable data storage device or computer readable memory on which is stored one or more sets of instructions (for example software) embodying any one or more of the methodologies or functions described herein. The instructions may also reside, completely or at least partially, within the main memory and / or within the processor during execution thereof by the computer, main memory, and processing device, which may constitute computer-readable storage media. The instructions may further be transmitted or received over a network via a network interface device. Computer-readable data medium include hard drives, for example on a server, USB storage device, CD, DVD or Blue-ray discs. The computer program may contain all functionalities and data required for execution of the method according to the present disclosure or it may provide interfaces to have parts of the method processed on remote systems, for example on a cloud system.

[0077] Brief Description of the Figures

[0078] Figure 1 shows the elements of the seat belt monitoring system.

[0079] Figure 2 shows possible placements of the seat belt monitoring system in the interior of a car.

[0080] Figure 3 shows an example for the seat belt monitoring system in a car.

[0081] Figure 4 illustrates an embodiment of the method of the disclosure.

[0082] Figure 5 illustrates an example of how the seat belt position may be determined.

[0083] Figure 6 illustrates an example of how the seat belt position may be determined from a pattern image.

[0084] Figure 7 illustrates an example for a seat belt indicator map.

[0085] Description of Embodiments

[0086] Figure 1 shows the elements of the seat belt monitoring system. The seat belt monitoring system 100 a projector 101, a camera 102 and a processor 103. These components may be mounted on a printed circuit board providing the communication lines and electricity from a battery or an interface to an electricity supply. The projector 101 may project light 120 to a person 110 and a seat belt 111. The projector 101 may comprise a VCSEL array and optics. The light may be diffuse light or it may be patterned light, for example a periodic dot pattern or it may be both, either simultaneously or alternating. The light 120 may impinge on the seat belt 111 and the person 110. Light 130 may be reflected to the camera 102 which generates an image in the optical range matching the wavelength emitted by projector 102, for example in the infrared range. The image may be a grayscale image, i.e. each pixel contains only the total intensity information, or an RGB image, i.e. different pixels indicate the intensity in a particular wavelength.

[0087] The image may be passed to processor 103. The processor 103 may be a microcontroller, i.e. containing memory and IO controller functionalities or it may be a CPU which is connected to memory and IO controllers. The processor 103 may determine the position of the seat belt 111. The processor 103 may in addition determine the position and / or size of the person 110. The processor 103 may output the position of the seat belt 111 to a controller 140, for example the board computer of a car. The processor 103 may generate an indicator indicating if the seat belt 111 is positioned correctly. The indicator may be determined using the position of the seat belt 111 and the size and / or position of the person 110. The indicator may be output to controller 140. Controller 140 may control functionalities of the vehicle, for example allow or deny the start of the engine or release of the breaks.

[0088] Figure 2 shows possible placements of the seat belt monitoring system in the interior of a car. The figure shows the dashboard, the middle console, the steering wheel and the windshield of a car as seen from the inside of the car. The seat belt monitoring system may be integrated into various places, for example behind a transparent display or a display with a notch for the projector and the camera. The seat belt monitoring system may be integrated into the interior mirror 201 . This may be particularly useful if the mirror functionality is only mimicked by a display which displays the rear view recorded by a camera. The seat belt monitoring system may e integrated into one or both of the A columns (202a, 202b). Another possibility is space behind the steering wheel 203 where the gauges such as the speed gauge are typically placed. Furthermore, the seat belt monitoring system may also be integrated into the steering wheel 204. The center console 205 is another option as replacing traditional controls with a display become more and more popular. In an embodiment, space behind the steering wheel 203 and the center console may be combined in a continuous display. Another possibility is to place the seat belt monitoring system into the side door, for example just beneath the side window (206), or in the center above the windscreen (207), or on the gearshift lever (208).

[0089] Figure 3 shows an example for the seat belt monitoring system in a car. The seat belt monitoring system 301 may be placed on the center of the steering wheel. Light rays 302 are emitted onto as driver wearing a seat belt 403. The light reflected by the seat belt 403 may be recorded by a camera of the seat belt monitoring system which generates an image which is analyzed by a processor to determine the position of the seat belt 303.

[0090] Figure 4 illustrates an embodiment of the method of the disclosure. A person may be illuminated (401), for example with patterned infrared light using a VCSEL projector. An image of the person may be recorded (402). The image may be used to determine the position of the seat belt (403). The determination may involve detection of the material the seat belt is made of, for example polyamide. The position of the seat belt may be output, for example to memory. The position of the seat belt may be used to determine an indicator (406) indicating if the seat belt is correctly positioned. Such determination may involve the size and / or the position of the person which may be determined from the image of the person (405). The determination of the indicator may further involve data about an object surrounding the seat belt which may be determined from the image of the person (404). The indicator may be used to control a vehicle functionality (407), for example allow or deny the start of the engine or release of the breaks.

[0091] Figure 5 illustrates an example of how the seat belt position may be determined. A projector may illuminate a person in a vehicle with patterned light 510, for example a hexagonal spot pattern of infrared light. A camera may capture a pattern image 520 of the person in the vehicle under pattern illumination.

[0092] The pattern image 520 may be subject to material detection, for example by using a material model comprising an encoder, such as a convolutional neural network, and a classifier, for example a fully connected neural network. The material model may receive the pattern image 520 as input and output material data 531, for example a material class or an array of material classes corresponding to different parts of the pattern image 520. A material class may correspond to the detected material, for example the material of the seat belt, for example polyamide. Other classes may refer to other expected material of the scene, for example skin or polyester.

[0093] The pattern image 520 may be subject to pattern shape analysis to yield pattern shape data 532. The pattern shape analysis may comprise a segmentation algorithm, for example an adaptive thresholding filter. The pattern shape analysis may yield pattern shape data 532, for example the asymmetry of the reflected illumination patterns. By using both material data 531 and pattern shape data 532, the position of the seat belt 540 may be detected. For example, all parts of the pattern image for which the material class corresponds to the material of the seat belt and the asymmetry of the reflected spots correspond to the weaving pattern of the seat belt may be attributed to show the seat belt. In addition to the pattern illumination 510, the scene may be illuminated with flood light 515. The camera may capture a flood image 525 of the scene under flood illumination 515. The flood image 525 may be used to generate object recognition data 535, for example comprising the position of the driver and his body parts. The object recognition data 535 may be used in addition to determine the seat belt position 540, for example by identifying incorrect determinations of material 531 or pattern shape data 532 in parts of the image where this would be geometrically impossible.

[0094] Figure 6 illustrates an example of how the seat belt position may be determined from a pattern image. From pattern image 610, partial images 611, 612, 619 may be generated, for example by cropping pattern image 610 around different positions, such as different pattern features. For the sake of space, only three partial images are shown in the figure. In practice, may more partial images may be generated, for example one for each pattern feature in the pattern image. A partial image 611, 612, 619 may show one or more than one pattern features. Partial images 611, 612, 619 may originate from different parts of the pattern image. They may, however, at least partially overlap. For the partial images 611, 612, 619, material data 621, 622, 629 may be generated, for example as described for figure 5. For the partial images 611, 612, 619, spot shape data 631, 632, 639 may be generated, for example as described for figure 5. For the partial images 611, 612, 619 a seat belt indicator 641, 642, 649 may be generated from the material data 621, 622, 629 and the spot shape data 631, 632, 639. For example, the seat belt indicator 641, 642, 649 may be a Boolean value indicating if the corresponding partial image 611, 612, 619 show the seat belt or not. The seat belt indicator 641, 642, 649 may be used to generate a seat belt map 650, for example by combining the seat belt indicators 641, 642, 649 with the respective position in the pattern image 610. The seat belt map 650 may be used to determine the seat belt position 660, for example by segmenting the seat belt map 650 according to the seat belt indicator.

[0095] Figure 7 illustrates an example for a seat belt indicator map. A person may be under pattern illumination 701, for example a hexagonal spot pattern of infrared light. A camera may capture a pattern image of the person under pattern illumination. The pattern image may be cropped into a plurality of partial images 702, for example each partial image 702 showing one pattern feature, for example one spot. The partial images 703 may be subject to material classification and spot shape analysis to determine if the spot is placed on a seat belt or not. This analysis may yield a Boolean indicator for each partial image 702. The indicator may be placed on a map corresponding to the position the partial image 702 originates in the pattern image. In this way, a seat belt indicator map 703 may be generated. In the image, an x may indicate that a seat belt is detected, an o may indicate that something else has been detected. The position of the seat belt may be determined by segmenting the seat belt indicator map 703. The seat belt position may be output to a vehicle board computer which controls the functionality of the vehicle.

[0096] The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the claims.

[0097] Any steps presented herein can be performed in any order. The methods disclosed herein are not limited to a specific order of these steps. It is also not required that the different steps are per-formed at a certain place or in a certain computing node of a distributed system, i.e. each of the steps may be performed at different computing nodes using different equipment / data processing. As used herein ..determining" also includes ..initiating or causing to determine", "generating" also includes ..initiating and / or causing to generate" and "providing” also includes "initiating or causing to determine, generate, select, send and / or receive”. "Initiating or causing to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

[0098] In the claims as well as in the description the word "comprising” does not exclude other elements or steps and the indefinite article "a” or "an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation. In the claims as well as in the description the word "comprising” or "including” or similar wording does not exclude other elements or steps and shall not be construed limiting to the elements or steps lined out. The indefinite article "a” or "an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation or further elements may be included.

[0099] Providing in the scope of this disclosure may include any interface configured to provide data. This may include an application programming interface, a human-machine interface such as a display and / or a software module interface. Providing may include communication of data or sub-mission of data to the interface, in particular display to a user or use of the data by the receiving node, entity or interface.

[0100] Various units, circuits, entities, nodes or other computing components may be described as "con-figured to” perform a task or tasks. Configured to shall recite structure meaning "having circuitry that” performs the task or tasks on operation. The units, circuits, entities, nodes or other computing components can be configured to perform the task even when the unit / circuit / component is not operating. The units, circuits, entities, nodes or other computing components that form the structure corresponding to "configured to” may include hardware circuits and / or memory storing program instructions executable to implement the operation. The units, circuits, entities, nodes or other computing components may be described as performing a task or tasks, for convenience in the description. Such descriptions shall be interpreted as including the phrase "configured to.” Any recitation of "configured to” is expressly intended not to invoke 35 U.S.C. § 112(f) interpretation.

[0101] In general, the methods, apparatuses, systems, computer elements, nodes or other computing components described herein may include memory, software components and hardware components. The memory can include volatile memory such as static or dynamic random-access memory and / or nonvolatile memory such as optical or magnetic disk storage, flash memory, programmable read-only memories, etc. The hardware components may include any combination of combinatorial logic circuitry, clocked storage devices such as flops, registers, latches, etc., finite state machines, memory such as static random-access memory or embedded dynamic random-access memory, custom designed circuitry, programmable logic arrays, etc.

[0102] Any disclosure and embodiments described herein relate to the methods, the systems, apparatuses, devices, chemicals, materials, computer program elements lined out above and vice versa. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples and vice versa. All terms and definitions used herein are understood broadly and have their general meaning.

Claims

Claims1 . A seat belt monitoring system for a vehicle comprising: a) a projector configured to illuminate a person in a vehicle with patterned light, b) a camera configured to capture a pattern image of the person under pattern illumination, c) a processor configured to receive the pattern image from the camera and to determine the position of the seat belt in the image by determining a material and a pattern shape from the pattern image, and d) an output configured to output the position of the seat belt.

2. The seat belt monitoring system according to claim 1 , wherein the processor is configured to determine if the seat belt is in the correct position using the position of the seat belt.

3. The seat belt monitoring system according to claim 2, wherein the processor is configured to determine the position and / or the size of the person in the vehicle from the image and wherein the processor is further configured to determine if the seat belt is in the correct position using the position and / or the size of the person in the vehicle.

4. The seat belt monitoring system according to claim 2 or 3, wherein the processor is configured to recognize an object in the proximity of the seat belt and determine its dimensions from the image and wherein the processor is further configured to determine if the seat belt is in the correct position using the dimensions of the recognized object in the proximity of the seat belt.

5. The seat belt monitoring system according to any of the claims 1 to 4, wherein the person in the vehicle is identified by face recognition.

6. The seat belt monitoring system according to claim 5, wherein the processor is configured to retrieve stored data of the identified person and to use the data for determination if the seat belt is in the correct position.

7. The seat belt monitoring system according to any of the claims 1 to 6, wherein seat belt monitoring system contains a transparent display, wherein the projector illuminates light through the transparent display and the camera receives light through the transparent display.

8. The seat belt monitoring system according to any of the claims 1 to 7, wherein the projector projects a periodic dot pattern.

9. The seat belt monitoring system according to any of the claims 1 to 8, wherein determining the pattern shape comprises determining the asymmetry of a pattern feature.

10. The seat belt monitoring system according to any of the claims 1 to 9, wherein the projector comprises a vertical cavity surface emitting laser (VCSEL) array.

11. A vehicle containing the seat belt monitoring system according to any of the preceding claims.

12. The vehicle according to claim 11 , wherein the vehicle is a car.

13. Use of the seat belt monitoring system of any one of the preceding claims for controlling a vehicle.

14. A method for determining the position of a seat belt in a vehicle comprising: a) illuminating a person in a vehicle with patterned light, b) capturing a pattern image of the person under pattern illumination, c) determining the position of the seat belt in the image by determining a material and a pattern shape from the pattern image, d) outputting the position of the seat belt.

15. A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: a) illuminating a person in a vehicle with patterned light, b) capturing an image of the person under pattern illumination, c) determining the position of the seat belt in the image by determining a material and a pattern shape from the pattern image, d) outputting the position of the seat belt.

Images