Imaging system

Abstract

An imaging system (111) is proposed, comprising - at least one pattern illumination source (112) configured for emitting at least one light pattern comprising a plurality of light beams having a wavelength in a short wavelength infrared spectral region, wherein the wavelength of the light is from 1000 nm to 1200 nm or from 1300 nm to 1500 nm; and - at least one image sensor (118) configured for generating at least one pattern image while the pattern illumination source (112) is emitting the light pattern, wherein the image sensor (118) is at least partially sensitive towards electromagnetic radiation in the short wavelength infrared spectral region.

Description

[0001] Imaging System

[0002] Technical Field

[0003] The invention relates to an imaging system, a device for authenticating a user of a device, a method for authenticating a user, computer programs, computer-readable storage media, nontransient computer-readable media and several uses. The devices, methods and uses according to the present invention specifically may be employed for example in various areas of daily life, security technology, gaming, traffic technology, production technology, photography such as digital photography or video photography for arts, documentation or technical purposes, safety technology, information technology, agriculture, crop protection, maintenance, cosmetics, medical technology or in the sciences. However, other applications are also possible.

[0004] Background art

[0005] Available authentication systems in mobile devices such as in smartphones, tablets and the like, include receivers such as at least one camera. Said mobile devices usually have a front display such as an organic light-emitting diode (OLED) area and / or a quantum-dot light emitting diode (QLED) area. The receiver may be positioned behind said front display. Moreover, in such devices for authentication a light emitter such as a projector is used, such as one or more light emitting diodes and / or laser, may be positioned behind the display. Usually, the light emitter projects a pattern such as a point pattern onto a target, e.g. a face, the receiver, e.g. the camera, captures an image of the projection onto the user and material data is determined by a processor. If the material is classified as skin, it is identified as human and, otherwise, as a spoof target. The pattern generated by the light projector can be designed for a 3D algorithm, i.e. the pattern may be designed to allow easily solving the so-called correspondence problem. For smartphone application, a resulting 3D depth map can be used for further face authentication. However, in case of using light in the NIR wavelength regime, interaction of light with the display electronics resulting in so-called flickering is a problem.

[0006] Steiner et aL, "Design of an Active Multispectral SWIR Camera System for Skin Detection and Face Verification", JOURNAL OF SENSORS, vol. 2016, 1 January 2016, pages 1-16, XP055570013, US ISSN: 1687-725X, DOI: 10.1155 / 2016 / 9682453 relates to biometric face recognition using spectral signatures of material surfaces in the short wave infrared range.

[0007] Problem to be solved

[0008] It is therefore an object of the present invention to provide devices and methods facing the above-mentioned technical challenges of known devices and methods. Specifically, it is an object of the present invention to provide methods and devices allowing reducing and / or preventing flickering. Summary

[0009] This problem is addressed by an imaging system, a device for authenticating a user of a device, a method for authenticating a user, computer programs, computer-readable storage media, non-transient computer-readable media and uses with the features of the independent claims. Advantageous embodiments which might be realized in an isolated fashion or in any arbitrary combinations are listed in the dependent claims as well as throughout the specification.

[0010] In a first aspect, an imaging system is disclosed. The imaging system comprises at least one pattern illumination source configured for emitting at least one light pattern comprising a plurality of light beams having a wavelength in a short wavelength infrared spectral region, wherein the wavelength of the light is from 1000 nm to 1200 nm or from 1300 nm to 1500 nm; and at least one image sensor configured for generating at least one pattern image while the pattern illumination source is emitting the light pattern, wherein the image sensor is at least partially sensitive towards electromagnetic radiation in the short wavelength infrared spectral region.

[0011] The term "system" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary set of interacting or interdependent components parts forming a whole. Specifically, the components may interact with each other in order to fulfill at least one common function. The at least two components may be handled independently or may be coupled or connectable.

[0012] The term “imaging system” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a system configured for imaging. The terms “imaging” and “generating at least one image” are used as synonyms herein. The term “imaging” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to capturing and / or determining and / or recording at least one image by using the image sensor. The imaging may comprise capturing a single image and / or a plurality of images such as a sequence of images. For generating an image via a hardware and / or a software interface, the capturing and / or determining and / or recording of the image may be caused and / or initiated by the hardware and / or the software interface. For example, the imaging may comprise recording continuously a sequence of images such as a video or a movie. The imaging may be initiated by a user action or may automatically be initiated, e.g. once the presence of at least one object or user within a field of view and / or within a predetermined sector of the field of view of the image sensor is automatically detected. The term “field of view” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an angular extent of the observable world and / or at least one scene that may be captured or viewed by an optical system, such as the image sensor. The field of view may, typically, be expressed in degrees and / or radians, and, exemplarily, may represent the total angle spanned by the image and / or viewable area.

[0013] The term “image sensor” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one unit of the system configured for generating at least one image. The image may be generated via a hardware and / or a software interface, which may be considered as the image sensor. The term “image” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to data recorded by using the image sensor, such as a plurality of electronic readings from the image sensor. The image may comprise raw image data or may be a pre-processed image. For example, the pre-processing may comprise applying at least one filter to the raw image data and / or at least one background correction and / or at least one background subtraction.

[0014] The term “illuminate”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to the process of exposing at least one element to light. The term “illumination source”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a device configured for generating at least one light beam for illumination of an object. The term “object” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary target, chosen from a living object and a non-living object. The object may be or may comprise one or more living beings and / or one or more parts thereof, such as one or more body parts of a human being, e.g. a user. The object may be a non-living object such as a silicon mask or a printed image of a human being. The term “living organism”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to any living body, in particular a living human. The term “living human”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an individual of the species homo sapiens, wherein the individual is currently alive.

[0015] The term “pattern illumination source”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an optical device configured for projecting at least one light pattern. The term “projecting”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to the process of providing at least one light beam, in particular a light pattern onto at least one surface.

[0016] The term “light pattern” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary pattern comprising a plurality of light spots. The light spot may be at least partially spatially extended. At least one spot or any spot may have an arbitrary shape. In some cases a circular shape of at least one spot or any spot may be preferred. The light pattern may comprise at least one point pattern. The light pattern may be a coherent light pattern. The light beams of the light pattern may have a single wavelength or have a plurality of wavelengths, e.g. for allowing additional measurements in other wavelengths channels. The light pattern may comprise at least one regular and / or constant and / or periodic pattern such as a triangular pattern, a rectangular pattern, a hexagonal pattern or a pattern comprising further convex tilings. For example, the light pattern is a hexagonal pattern, preferably a hexagonal light pattern, preferably a 2 / 5 hexagonal light pattern. Using a periodical 2 / 5 hexagonal pattern can allow distinguishing between artefacts and usable signal.

[0017] The emitted light pattern may illuminate the surface by a light pattern comprising a plurality of light spots. The light spots may be overlapping at least partially. For example, the number of light spots may be equal to the number of light beams associated with the emitted light pattern. The intensity associated with a light spot may be substantially similar. Substantially similar may refer to intensity values associated with the light spot may differ by less than 50%, preferably less than 30%, more preferably less than 20%. Using patterned light may be advantageous since it can enable the sparing of light-sensitive regions such as the eyes. The pattern may comprise at least one point pattern.

[0018] The pattern illumination source may comprise at least one least one emitter, in particular a plurality of emitters. The term “emitter” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary device configured for providing at least one light beam. The emitter may be selected from the group consisting of: at least one laser light source; at least one laser diode; at least one vertical cavity surface emitting laser (VCSEL); at least one edge emitting laser.

[0019] The pattern illumination source is configured for emitting light having a wavelength in a short wavelength infrared (SWIR) spectral region. The pattern illumination source may be configured for illuminating the object by using a light pattern having a central wavelength in the SWIR. The term “short wavelength infrared spectral region” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to light in the spectral region from 1000 nm to 3000 nm.

[0020] For example, the wavelength of the light is from 1000 nm to 1200 nm, preferably from 1050 nm to 1150 nm, e.g. 1100 nm. In these wavelength regions, a reflection behavior of skin is different from other materials.

[0021] For example, the wavelength of the light is from 1300 nm to 1500 nm, preferably from 1300 nm to 1400 nm. In these wavelength regions, skin has clearly distinguishable reflectance values.

[0022] Moreover, the spot structure of the light spots on the surface of the object is suitable for material detection for said wavelengths regions, in particular using beam profile analysis. The SWIR reflection and backscattering of materials relevant to secure face authentication (e.g., spoof targets and human skin) may show additional differentiating information in material comparison, compared to shorter (NIR) wavelengths. This can be used to better identify spoof targets. Moreover, using this wavelength can be advantageous in case of using the imaging system in combination with a display. For wavelengths in the SWIR the flickering effect, or sub-types of this effect, can be significantly reduced. Additionally, higher wavelengths have lower restrictions on eye safety, such that higher laser powers and / or longer illumination and exposure times can be used. This can give an advantage in compensating for lower display transmission if it is used under a display. It also gives more design freedom for the specifications of the projector. Moreover, for the wavelength regions mentioned above, the solar radiation is significantly lower compared to other wavelengths of the SWIR.

[0023] The imaging system may comprise at least one optical element configured for modifying light spots generated by the pattern illumination source. The optical element may be selected from the group consisting of: at least one lens; at least one Micro-lens-array (MLA); at least one diffractive optical element (DOE); and at least one meta surface element. For example, the emitters may be used in combination with at least one optical element like MLA, DOE, meta-surface, or lens. The pattern illumination source may comprise at least one optical element configured for modifying light spots generated by the emitter, wherein the optical element is selected from the group consisting of: at least one lens; at least one Micro-lens-array (MLA); at least one diffractive optical element (DOE); and at least one meta surface element. The optical element may be configured for generating multiple light beams from a single incoming light beam. For example, the emitters may project up to 2000 spots and the optical element, e.g. comprising a plurality of metasurface elements, may be used to duplicate the number of spots. Further arrangements, particularly comprising a different number of projecting emitters and / or at least one different optical element configured for increasing the number of spots may be possible. Other multiplication factors are possible.

[0024] The pattern illumination source comprise at least one transfer device. The term “transfer device”, also denoted as “transfer system”, as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to one or more optical elements which are adapted to modify the light beam, particularly the light beam used for generating at least a portion of the light pattern, such as by modifying one or more of a beam parameter of the light beam, a width of the light beam or a direction of the light beam. The transfer device may comprise at least one imaging optical device .The transfer device specifically may comprise one or more of: at least one lens, for example at least one lens selected from the group consisting of at least one focus-tunable lens, at least one aspheric lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multi-lens system; at least one holographic optical element; at least one meta optical element. Specifically, the transfer device comprises at least one refractive optical lens stack. The transfer device may comprise a multi-lens system having refractive properties.

[0025] The imaging system may further comprise at least one flood illumination source configured for emitting flood light. The term “flood illumination source” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary device configured for providing substantially continuous spatial illumination. The term “flood light” as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to substantially continuous spatial illumination, in particular diffuse and / or uniform illumination. The flood illumination source may comprise at least one least one emitter, in particular a plurality of emitters. The flood illumination source may comprise at least one LED or at least one VCSEL, preferably a plurality of VCSELs. The plurality of VCSELs may overlap to a uniform area. The term “substantially continuous spatial illumination” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to uniform spatial illumination, wherein areas of non-uniform are possible. The area, e.g. covering a user, a portion of the user and / or a face of the user, illuminated from the flood illumination source, may be contiguous. Power may be spread over a whole field of illumination. In contrast, illumination provided by the light pattern may comprise at least two contiguous areas, in particular a plurality of contiguous areas, and / or power may be concentrated in small (compared to the whole field of illumination) areas of the field of illumination. The flood illumination may be suitable for illuminating a contiguous area, in particular one contiguous area. The pattern illumination may be suitable for illuminating at least two contiguous areas.

[0026] The flood illumination source may illuminate a measurement area, such as the object, a portion of the object, with a substantially constant illumination intensity. The term “constant” as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a time aspect during an exposure time. Flood light may vary temporally and / or may be substantially constant over time. The term “substantially constant” as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a completely constant illumination and embodiments in which deviations from a constant illumination of < ± 10 %, preferably < ± 5 %, more preferably < ± 2 % are possible.

[0027] Central wavelengths of the flood light and the light of the light pattern may identical, e.g. within ± 100 nm. However, other embodiments are thinkable.

[0028] The flood light may have a wavelength in a short wavelength infrared spectral region. Specifically, the wavelength of the flood light may be from 1300 nm to 1500 nm, preferably from 1300 nm to 1400 nm.

[0029] Classical 2D spoofing targets are made such that their appearance is like a human face and mimic the reflection of human skin in the visible range of light. Using NIR can improve the secureness but there exists a bunch of usual materials having the same reflectivity property as skin. For said wavelengths in the SWIR region skin has clearly distinguishable reflectance values such that these wavelengths can be advantageous for distinguishing human skin from spoof target. Thus, these wavelengths may be advantageous for authentication processes. Moreover, using these wavelengths ranges may be advantageous since light of these wavelengths ranges is invisible to a human and the wavelength may be associated with a low intensity in the light spectrum of the sun. Hence, using the above-mentioned the wavelengths ranges can provide the advantage of being resistant to irradiance of the object by sun light.

[0030] The emitting of the flood light and the illumination of the light pattern may be performed subsequently or at at least partially overlapping times. For example, the flood light and the light pattern may be emitted at the same time. For example, one of the flood light or the light pattern may be emitted with a lower intensity compared to the other one.

[0031] The term “light” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to electromagnetic radiation in one or more of the infrared, the visible and the ultraviolet spectral range. Herein, the term “ultraviolet spectral range”, generally, refers to electromagnetic radiation having a wavelength of 1 nm to 380 nm, preferably of 100 nm to 380 nm. Further, in partial accordance with standard ISO- 21348 in a valid version at the date of this document, the term “visible spectral range”, generally, refers to a spectral range of 380 nm to 760 nm. The term “infrared spectral range” (IR) generally refers to electromagnetic radiation of 760 nm to 1000 pm, wherein the range of 760 nm to 1 .5 pm is usually denominated as “near infrared spectral range” (NIR) while the range from 1 .5 |j to 15 |jm is denoted as “mid infrared spectral range” (MidlR) and the range from 15 pm to 1000 pm as “far infrared spectral range” (FIR).

[0032] The term “ray” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a line that is perpendicular to wavefronts of light which points in a direction of energy flow. The term “light beam” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a collection of rays. In the following, the terms “ray” and “beam” will be used as synonyms. The term “light beam” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an amount of light, specifically an amount of light traveling essentially in the same direction, including the possibility of the light beam having a spreading angle or widening angle.

[0033] As outlined above, using wavelengths in the SWIR has several advantages. But, for these wavelength ranges, detectors commonly used for authentication systems become inefficient very quickly such that using higher wavelengths was problematic. The image sensor is at least partially sensitive towards electromagnetic radiation in the short wavelength infrared spectral region. The image sensor may comprise at least one quantum dot sensor. For example, the quantum dot sensor may be designed as described in US 2023 / 258498A1 . The image sensor may comprise a substrate, for example, made of semiconductor material, for example silicon. The substrate may be made of insulating layers, for example in silicon oxide, and conductive tracks and vias, for example in copper. The substrate may be covered by an insulation layer. The insulation layer may be in silicon oxide. Conductive vias may extend through the insulation layer. The vias may be made of copper or tungsten. The image sensor may comprise at least one via for each pixel. One via may be located in regard of the location of each pixel. The via of each pixel may correspond to an electrode of the pixel. The image sensor may further comprise a layer covering the insulation layer and all the vias corresponding to the pixels. Said layer may be continuous on the locations of all the pixels. Said layer may be in contact with the vias. Said layer may be made of quantum dots. The quantum dots may be fixed together and to the insulation layer by a resin, or a matrix. Preferably, said quantum dot layer may comprise only the quantum dots and the resin. All the quantum dots of said layer may be substantially identical. All the quantum dots of the quantum dot layer have substantially the same size and are made of the same components. For example, all the quantum dots of the quantum dot layer are in lead sulphide (PbS). A quantum dot, or semiconductor nanoparticle, is a nanoscopic material structure which produces electron-hole pairs given the incidence of photons onto the nanoscopic material structure. In this manner, it is possible to create detecting elements such as photodetectors for example, on the basis of semiconductor nanoparticles. A quantum dot comprises a semiconductor core. A quantum dot can also comprise a shell, preferably in a semiconductor material, surrounding the core in order to protect and passivate the core. A quantum dot further comprises ligands, organic aliphatics, organometallic, or inorganic molecules that extend from the shell and passivate, protect, and functionalize the semiconductor surface. The composition of a quantum dot can be chosen among the following materials. The core is, for example, made of a material among the following or an alloy of materials among the following: CdSe, CdS, CdTe, CdSeS, CdTeSe, AgS, ZnO, ZnS, ZnSe, CulnS, CulnSe, CulnGaS, CulnGaSe, PbS, PbSe, PbSeS, PbTe, InAsSb, InAs, InSb, InGaAs, InP, InGaP, InAlP, InGaAlP, InZnS, InZnSe, InZnSeS, HgTe, HgSe, HgSeTe, Ge, Si. The shell is, for example, made of a material among the following or an alloy of materials among the following: CdSe, CdS, CdTe, CdSeS, CdTeSe, AgS, ZnO, ZnS, ZnSe, CulnS, CulnSe, CulnGaS, CulnGaSe, PbS, PbSe, PbSeS, PbTe, InAsSb, InAs, InSb, InGaAs, InP, InGaP, InAlP, InGaAlP, InZnS, InZnSe, InZnSeS, HgTe, HgSe, HgSeTe, Ge, Si.

[0034] Using a quantum dot image sensor can allow efficient image detection even in the SWIR. As outlined above, for example, the wavelength of the light is from 1050 nm to 1150 nm, preferably 1100 nm, or the wavelength of the light is from 1300 nm to 1400 nm. A quantum efficiency of the image sensor may be > 10 %, preferably > 20 %, more preferably > 40 %, in the short wavelength infrared spectral region.

[0035] The term “pattern image” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an image generated by the image sensor while the object is being illuminated by the light pattern. The pattern image may comprise an image showing the object, in particular at least parts of the face of the user, while the user is being illuminated with the pattern, particularly on a respective area of interest comprised by the image. Thus, the image sensor may yield an intensity image of the object projected with the pattern. The pattern image may be generated by imaging and / or recording light reflected by an object, which is illuminated by the light pattern.

[0036] The imaging system may be comprised in a device. The device may comprise a display. The term “display” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary shaped device configured for displaying an item of information. The item of information may be arbitrary information such as at least one image, at least one diagram, at least one histogram, at least one graphic, text, numbers, at least one sign, an operating menu, and the like. The display may be or may comprise at least one display panel. The display may have an arbitrary shape, e.g. a rectangular shape. The display may be a front display of the device. The display may comprise at least one of a display panel, particularly comprising a plurality of pixels and / or a plurality of transistors, or a glass, specifically a cover glass, particularly configured for covering the display panel.

[0037] As used herein, the term “cover glass” is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an element configured for covering the display panel made from a glass material. The cover glass may also be denoted as protective glass or front glass. The cover glass may be configured for protecting the display panel, in particular from environmental influences such as mechanical influences. The cover glass may be made from a glass material having a refraction index from 1 .46 to 1 .9 (at 940 nm). For example, the glass material may be crown-glass having a refraction index of 1.5 (at 940 nm). For example, the glass material may be Borosilicate glass having a refraction index of 1 .51 (at 940 nm).

[0038] The display, specifically the display panel, may be or may comprise at least one organic lightemitting diode (OLED) display and / or at least one quantum-dot light emitting diode (QLED). As used herein, the term “organic light emitting diode” is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a lightemitting diode (LED) in which an emissive electroluminescent layer is a film of organic compound configured for emitting light in response to an electric current. The OLED display may be configured for emitting visible light. As used herein, the term “organic light emitting diode” is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a display technology that utilizes semiconductor particles called quantum dots in order to produce colors on a display. These quantum dots may emit a plurality of different colors of light depending on their size when excited by light. By using a combination of red, green and / or blue quantum dots, a QLED display may display a wide range of colors with high brightness and color accuracy.

[0039] The image sensor and / or the pattern illumination source may be arranged behind the display. The light may traverse the display while being illuminated from the illumination source. The display may be at least partially transparent. The display may be at least partially transparent in at least one continuous area covering the pattern illumination source and / or the flood illumination source and / or the image sensor. For example, the display may comprise a punch hole in the continuous area covering the pattern illumination source, the flood illumination source and / or the image sensor. For example, the display may have a transmission below or equal to 20 %, preferably below or equal to 15 %, more preferably below or equal to 10 %. For example, an intensity of a light beam after being projected through the display may correspond to < 10 % of the intensity associated with the light beam when being emitted.

[0040] The display may be at least partially transparent in at least one continuous area covering the image sensor. The display may be at least partially transparent in at least one continuous area in a manner that at least one of:

[0041] - the light pattern incident on the continuous areas traverses the display while being illuminated from the projector;

[0042] - the flood light incident on the continuous areas traverses the display while being illuminated from the flood illumination source; user light, generated by the light pattern and / or the flood light incident on a user, incident on the continuous areas traverses the display for impinging on the image sensor.

[0043] The term “at least partially transparent” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a property of the display to allow light, in particular of a certain wavelength range, e.g. in the infrared spectral region, in particular in the near infrared spectral region, to pass at least partially through. For example, the display may be semitransparent in the near infrared region. For example, the display may have a transparency of 20 % to 50 % in the near infrared region. The display may have a different transparency for differing wavelength ranges. The present invention may propose a device comprising the image sensor and the illumination source that can be placed behind the display of a device. The transparent area(s) of the display can allow for operation of the image sensor and the projector behind the display.

[0044] The display can be an at least partially transparent display, as described above. The partially transparent contiguous area of the display may be associated with a first pixel density value (Pixels per inch (PPI)), and a further area of the display may be associated with a second pixel density value. The first pixel density value may be lower than the second pixel density value. The transmission of light through the contiguous area may be higher compared to the transmission through the further area. The first pixel density value may be equal or below 450 PPI, preferably between 300 to 440 PPI, more preferably between 350 to 450 PPI. The first pixel density value may be constant over the entire contiguous area with a maximum deviation thereof of 20 %, or preferably 10 %. The second pixel density value may be between 400 to 500 PPI, preferably between 450 to 500 PPI.

[0045] The at least partially transparent continuous area of the display may comprise a first area and a second area. The first area may be associated with a first number of transistors configured for controlling at least one pixel and the second area may be associated with a second number of transistors configured for controlling at least one pixel, and wherein the first number of transistors may be smaller than the second number of transistors. The first number of transistors and / or the second number of transistors may refer to or be a density of the transistors.

[0046] The term “pixel” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a picture unit, particularly the smallest picture unit, that represents an addressable element. The entirety of the pixels may represent the display. A pixel may be manipulated by changing its color, brightness and / or contrast or the like. Particularly for manipulating the pixel, the pixel may be driven by at least one transistor, exemplarily a transistor the controls a current required for driving the pixel. Typically, a thin-film transistor may be used for driving the pixel. TFTs may preferably be used in a flatpanel display. For example, the device is selected from the group consisting of: a television device; a game console; a personal computer; a mobile device, particularly a cell phone, and / or a smart phone, and / or, and / or a tablet computer, and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch; or another type of portable computer.

[0047] In a further aspect, a use of the imaging system according to the present invention, such as described in one or more embodiments above or as described in more detail below, for authenticating a user of a device comprising the imaging system, is disclosed. With respect to embodiments and definitions reference is made to the description of the imaging system above and the devices and methods for authentication described in more detail below.

[0048] In a further aspect, a device for authenticating a user of a device to perform at least one operation on the device that requires authentication is disclosed.

[0049] The device for authentication comprises an imaging system according to the present invention, such as described in one or more embodiments above or as described in more detail below; at least one display, wherein the image sensor is arranged behind the display, wherein the display is at least partially transparent in at least one continuous area covering the image sensor, at least one authentication unit configured for performing at least one authentication process of a user using the pattern image.

[0050] With respect to embodiments and definitions reference is made to the description of the imaging system above and the devices and methods for authentication described in more detail below.

[0051] The term “user” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a person intended to and / or using the device.

[0052] The device may be selected from the group consisting of: a television device; a game console; a personal computer; a mobile device, particularly a cell phone, and / or a smart phone, and / or a tablet computer, and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch; or another type of portable computer.

[0053] The term “authenticating” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to verifying an identity of a user. Specifically, the authentication may comprise distinguishing between the user from other humans or objects, in particular between an authorized access from a non-authorized access. The authentication may comprise verifying identity of a respective user and / or assigning identity to a user. The authentication may comprise generating and / or providing identity information, e.g. to other devices or units such as to at least one authorization unit for authorization for providing access to the device. The identify information may be proofed by the authentication. For example, the identity information may be and / or may comprise at least one identity token. In case of successful authentication an image of a face recorded by an imaging detector such as of the imaging sensor described above may be verified to be an image of the user’s face and / or the identity of the user is verified.

[0054] The authentication may be performed using at least one authentication process. The authentication process may comprise a plurality of steps such as at least one face detection, e.g. on at least one flood image as will be described in more detail below, and at least one identification step in which an identity is assigned to the detected face and / or at least one identity check and / or verifying an identity of the user is performed.

[0055] The authentication may be and / or may comprise a biometric authentication. The term "biometric authentication" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to authentication using at least one biometric identifier such as a distinctive, measurable characteristics used to label and describe individuals. The biometric identifier may be a physiological characteristics.

[0056] The operation on the device that requires authentication may be an arbitrary operation requiring access to at least one resource associated with the device. The method may comprise receiving a request for accessing at least one resource associated with the device and executing at least one authentication process.

[0057] The term “access” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to entering and / or using the one or more functions associated with the device. The term “function associated with the device” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary function such as access to at least one element and / or at least one resource of the device or associated with the device. The functions that require authentication of the user may be pre-defined. The one or more functions associated with the device may comprise unlocking the device, and / or access to an application, preferably associated with the device and / or access to a part of an application, preferably associated with the device. For example, the function may comprise access to a content of the device, e.g. as stored in a database of the device, and / or retrievable by the device. In an embodiment, allowing the user to access a resource may include allowing the user to perform at least one operation with a device and / or system. The resource may be a device, a system, a function of a device, a function of a system and / or an entity. Additionally and / or alternatively, allowing the user to access a resource may include allowing the user to access an entity. The entity may be physical entity and / or virtual entity. The virtual entity may be a database for example. The physical entity may be an area with restricted access. The area with restricted access may be one of the following: security areas, rooms, apartments, vehicles, parts of the before mentioned examples, or the like. The device may be locked and may only be unlocked by authorized user.

[0058] The device may further comprise at least one communication interface, such as a user interface, configured for receiving a request for accessing at least one resource associated with the device, in particular to perform at least one operation on the device that requires authentication.

[0059] The term "communication interface" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an item or element forming a boundary configured for transferring information. In particular, the communication interface may be configured for transferring information from a computational device, e.g. a computer, such as to send or output information, e.g. onto an-other device. Additionally or alternatively, the communication interface may be configured for transferring information onto a computational device, e.g. onto a computer, such as to receive information. The communication interface may specifically provide means for transferring or exchanging information. In particular, the communication interface may provide a data transfer connection, e.g. Bluetooth, NFC, Ethernet, inductive coupling or the like. As an example, the communication interface may be or may comprise at least one port comprising one or more of a network or internet port, a USB-port and a disk drive. The communication interface may be at least one web interface. The term "user interface" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term may refer, without limitation, to a feature of the device which is configured for interacting with its environment, such as for the purpose of unidirectionally or bidirectionally exchanging information, such as for exchange of one or more of data or commands. For example, the user interface may be configured to share information with a user and to receive information by the user. The user interface may be a feature to interact visually with a user, such as a display, or a feature to interact acoustically with the user. The user interface, as an example, may comprise one or more of: a graphical user interface; a data interface, such as a wireless and / or a wire-bound data interface.

[0060] The term “request for accessing” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one act and / or instance of asking for access. The term “receiving a request” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a process of obtaining a request, e.g. from a data source and / or the user interface. The receiving may fully or partially take place automatically. The receiving of the request for accessing one or more functions associated with the device may be performed by using at least one communication interface. The receiving may comprise receiving at least one user input, e.g. via at least one user interface e.g. such as a display of the device, and / or a request from a remote device and / or cloud, e.g. via the communication of the device such as via the internet. For example, the request may be generated by or triggered by at least one user input, such as by inputting a security number or other unlocking action by the user, and / or may be send from a remote device and / or cloud such as via a connected account.

[0061] The authentication process may be performed using at least one authentication unit configured for performing at least one authentication process of a user. The authentication unit may comprise at least one processor. The execution of the authentication process may be triggered and / or started by receiving the request.

[0062] The authentication unit may comprise at least one processor. The term “processor”, also denoted as “processing unit”, as generally used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary logic circuitry configured for performing basic operations of a computer or system, and / or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor may be configured for processing basic instructions that drive the computer or system. As an example, the processor may comprise at least one arithmetic logic unit (ALU), at least one floating-point unit (FPU), such as a math co-processor or a numeric co-pro- cessor, a plurality of registers, specifically registers configured for supplying operands to the ALU and storing results of operations, and a memory, such as an L1 and L2 cache memory. In particular, the processor may be a multi-core processor. Specifically, the processor may be or may comprise a central processing unit (CPU). Additionally or alternatively, the processor may be or may comprise a microprocessor, thus specifically the processor’s elements may be contained in one single integrated circuitry (IC) chip. Additionally or alternatively, the processor may be or may comprise one or more application-specific integrated circuits (ASICs) and / or one or more field-programmable gate arrays (FPGAs) and / or one or more tensor processing unit (TPU) and / or one or more chip, such as a dedicated machine learning optimized chip, or the like. The processor specifically may be configured, such as by software programming, for performing one or more evaluation operations. At least one or any component of a computer program configured for performing the authentication process may be executed by the processing device. Alternatively or in addition, the authentication unit may be or may comprise a connection interface. The connection interface may be configured to transfer data from the device to a remote device; or vice versa. At least one or any component of a computer program configured for performing the authentication process may be executed by the remote device.

[0063] The authentication process may comprise a plurality of steps. For example, the authentication process may comprise performing at least one face detection step. The face detection step may comprise analyzing at least one image of the user, e.g. generated by the image sensor or a further camera. The image may be a flood image. The term “flood image” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an image generated by the image sensor while illumination source is emitting infrared flood light, e.g. on an object and / or a user. The flood image may comprise an image showing a user, in particular the face of the user, while the user is being illuminated with the flood light. The flood image may be generated by imaging and / or recording light reflected by an object and / or user which is illuminated by the flood light. The flood image showing the user may comprise at least a portion of the flood light on at least a portion the user. For example, the illumination by the flood illumination source and the imaging may be synchronized, e.g. by using at least one control unit.

[0064] The determining if the user may correspond to an authorized user may be performed by using an image of the user generated while the object is illuminated by SWIR. This can allow that the imaging is not detectable by human and comprised to a small degree within the light spectrum of the sun. The authentication process may comprise generating at least one flood image showing the user associated while the user is being illuminated by flood light and determining if the identity of the user corresponds to a verified identity based on the flood image. The authentication process may comprise allowing the user to access the resource in case the identity of the user corresponds to a verified identity and otherwise, in case the identity of the user does not correspond to a verified identity, denying the user to access the resource.

[0065] The authentication process may comprise: illuminating the user with flood light by using at least one the flood illumination source; capturing the at least one flood image by using at least one image sensor or a further camera of the device.

[0066] The face detection step may comprise analyzing the flood image. For example, the authentication process may comprise performing at least one face detection using the flood image. The face detection may be performed locally on the device. Face identification, i.e. assigning an identity to the detected face, however, may be performed remotely, e.g. in the cloud, e.g. especially when identification needs to be done and not only verification. User templates can be stored at the remote device, e.g. in the cloud, and would not need to be stored locally. This can be an advantage in view of storage space and security.

[0067] The authentication process may comprise identifying the user based on the flood image. The term “identifying” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user. Particularly therefore, the authentication unit may forward data to a remote device. Alternatively or in addition, the authentication unit may perform the identification of the user based on the flood image, particularly by running an appropriate computer program having a respective functionality. The identifying may comprise assigning an identity to a detected face and / or verifying an identity of the user. The identifying may comprise performing a face verification of the imaged face to be the user’s face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user’s face, with a template. For matching the flood image with a template, a similarity between at least one image feature vector obtained from the flood image and at least one template feature vector may be considered and / or evaluated. The template vector may be obtained from a template image.

[0068] The template image may be generated in an enrollment process. The term “enrollment process" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one step of registering, particularly to a service. In the enrollment process, the template image may be generated under secure conditions in a manner that it is guaranteed that the generated template image shows the user. The enrollment process may comprise at least one step of: capturing the template image; recording personal data and the like.

[0069] The face detection may comprise analyzing the flood image. In particular, the analyzing of the flood image may comprise using at least one image recognition technique, in particular a face recognition technique. An image recognition technique comprises at least one process of identifying the user in an image. The image recognition may comprise using at least one technique selected from the technique consisting of: color-based image recognition, e.g. using features such as template matching; segmentation and / or blob analysis e.g. using size, or shape; machine learning and / or deep learning e.g. using at least one convolutional neural network.

[0070] For example, the authentication may comprise identifying the user. The identifying may comprise assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user. The identifying may comprise performing a face verification of the imaged face to be the user’s face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user’s face, with a template, e.g. a template image generated within an enrollment process. The identifying of the user may comprise determining if the imaged face is the face of the user, in particular if the imaged face corresponds to at least one image of the user’s face stored in at least one memory, e.g. of the device. Authentication may be successful if the flood image can be matched with an image template. Authentication may be unsuccessful if the flood image cannot be matched with an image template.

[0071] The term “memory" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one electronic storage space configured for storing data, instructions, and programs. The stored data, instruction and / or programs may be forwarded for processing to a processor. The memory may be or may comprise at least one of: a Random Access Memory; a Read-Only Memory; a Cache Memory; a Hard Disk Drive; a Solid State Drive; a Virtual Memory.

[0072] For determining if the identity of the user corresponds to a verified identity based on the flood image, a similarity between at least one image feature vector obtained from the flood image and at least one template feature vector may be considered. The template vector may be obtained from a template image. The template image may be generated in an enrollment process.

[0073] For example, the identifying of the user may comprise determining a plurality of facial features. The analyzing may comprise comparing, in particular matching, the determined facial features with template features. The template features may be features extracted from at least one template. The template may be or may comprise at least one image generated in an enrollment process, e.g. when initializing the device. Template may be an image of an authorized user. The template features and / or the facial feature may comprise a vector. Matching of the features may comprise determining a distance between the vectors. The identifying of the user may comprise comparing the distance of the vectors to a least one predefined limit. The user may be successfully identified in case the distance is < the predefined limit at least within tolerances. The user may be declined and / or rejected otherwise.

[0074] The analyzing of the flood image may further comprise one or more of the following: a filtering; a selection of at least one region of interest; a formation of a difference image between the flood image and at least one offset; an inversion of flood image; a background correction; a decomposition into color channels; a decomposition into hue; saturation; and brightness channels; a frequency decomposition; a singular value decomposition; applying a Canny edge detector; applying a Laplacian of Gaussian filter; applying a Difference of Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Scharr operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high-pass filter; applying a low-pass filter; applying a Fourier transformation; applying a Radon-transformation; applying a Hough-transformation; applying a wavelet-transformation; a thresholding; creating a binary image. The region of interest may be determined manually by a user or may be determined automatically, such as by recognizing the user within the image.

[0075] For example, the image recognition may comprise using at least one model, in particular a trained model comprising at least one face recognition model. The analyzing of the flood image may be performed by using a face recognition system, such as FaceNet, e.g. as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. The trained model may comprise at least one convolutional neural network. For example, the convolutional neural network may be designed as described in M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks”, CoRR, abs / 1311.2901 , 2013, or C. Szegedy et al., “Going deeper with convolutions”, CoRR, abs / 1409.4842, 2014. For more details with respect to convolutional neural network for the face recognition system reference is made to Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. As training data labelled image data from an image database may be used. Specifically, labeled faces may be used from one or more of G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in unconstrained environments”, Technical Report 07-49, University of Massachusetts, Amherst, October 2007, the Youtube® Faces Database as described in L. Wolf, T. Hassner, and I. Maoz, “Face recognition in unconstrained videos with matched background similarity”, in IEEE Conf, on CVPR, 2011 , or Google® Facial Expression Comparison dataset. The training of the convolutional neural network may be performed as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832.

[0076] The determining if the user corresponds to an authorized user may comprise using an image of the user generated while the user is illuminated by RGB light, e.g. sun light and / or ambient light can be used and no further light needs to be emitted while already present hardware. For example, in smartphones the selfie camera can be utilized. The method may comprise illuminating the user with RGB light and generating at least one RGB image with the image sensor showing at least a part of the user while the user is being illuminated with the RGB light. The method further may comprise identifying the user using the RGB image. For identifying the user a similarity between at least one image feature vector obtained from the RGB image and at least one template feature vector may be considered. The template feature vector may be obtained from a template image. The template image may be generated in an enrollment process. Authenticating of the user may comprise a facial authentication. The feature vector is a facial feature vector. The template image one or more of shows, includes, or represent a face of an authorized user. With respect to analysis and identifying the user reference is made to the description of the analysis and identifying the user using the flood image.

[0077] The authentication unit is configured for performing at least one authentication process of a user using the pattern image. The authentication unit may be configured for extracting liveness data from the pattern image. The authentication unit may be configured for allowing or declining the user to perform at least one operation on the device that requires authentication using the liveness data.

[0078] The term “liveness data” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to data providing an indication that the object corresponds to a living organism. In particular, extracting liveness data comprises extracting material data and / or extracting blood perfusion data.

[0079] The term “material data”, as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to data providing information about a material. The material data may comprise an item of information on the type of material of the surface of the object under illumination, e.g. by the light pattern. Extracting material data may be or may comprise generating the material type and / or data derived from the material type. The material data may comprise an item of information on the type of material of the object. Advantageously, the material detection must only work on targets having the same reflectance as human skin.

[0080] Material data may be extracted from the pattern image. Material data may indicate the type of material. In particular, material data may indicate whether the object associated with the image comprises at least partially of skin. Material data may be associated with the object, in particular with the object shown in the image. Extracting material data from the pattern image may comprise generating the material type and / or data derived from the material type.

[0081] Extracting material data may be based on the pattern image, more preferably one or more partial images. For example, the image may be reduced to a predefined size, e.g. by applying one or more image processing techniques. The reducing may comprise selecting at least one area of interest and cutting the pattern image to the area of the pattern image of the predefined size. The area of the image of the predefined size may be associated with the object. The image processing technique may comprise at least one image recognition technique, in particular a face recognition technique. An image recognition technique comprises at least one process of identifying the user in an image. The image recognition may comprise using at least one technique selected from the technique consisting of: color-based image recognition, e.g. using features such as template matching; segmentation and / or blob analysis e.g. using size, or shape; machine learning and / or deep learning e.g. using at least one convolutional neural network. The part of the image other than the area of the image of the predefined size may be associated with background and / or may be independent of the object. The part of the image useful for the subsequent analysis may be selected.

[0082] The material data may comprise at least one reflectance measure.

[0083] The extracting of material data from the pattern image may comprise beam profile analysis of the light spots. With respect to beam profile analysis reference is made to WO 2018 / 091649 A1 , WO 2018 / 091638 A1 and WO 2018 / 091640 A1 , the full content of which is included by reference. Beam profile analysis can allow for providing a reliable classification of the object based on a few light spots. Each of the light spots of the pattern image may comprise a beam profile. As used herein, the term “beam profile” may generally refer to at least one intensity distribution of the light spot on the optical sensor as a function of the pixel. The beam profile may be selected from the group consisting of a trapezoid beam profile; a triangle beam profile; a conical beam profile and a linear combination of Gaussian beam profiles.

[0084] For extracting the material data, the complete pattern image may be used. Alternatively, partial images may be used. Extracting of material data may include generating one or more partial images from the pattern image. For example, different regions of the pattern image may be selected as partial images. The partial images may be different from each other. In particular, the partial images may be non-overlapping. Using partial images may allow having material data from different areas of the object (e.g. having different light conditions) and / or comparing the extracted material data and / or generating of a material map and / or reducing an uncertainty on the obtained material data.

[0085] The material data, e.g. reflectance, may be evaluated using at least one algorithm, wherein the algorithm is designed for checking if this object is a spoof target. Additionally or alternatively, an artificial neural network might be used to check if the image is a spoof target.

[0086] The material data may be determined by providing the image to a model and receiving the material from the model. Material data may be extracted by using at least one model. Extracting material data may comprise providing the image to at least one model and / or receiving material data from the model. Extracting material data may include providing the image to a model and / or receiving material data from the model. Providing the image to a model may comprise and may be followed by receiving the image at an input layer of the model or via a model loss function.

[0087] The model may be a data-driven model. The data-driven model may comprise a convolutional neural network and / or an encoder decoder structure such as an autoencoder. Other examples for generating a representation may be FFT, wavelets, deep learning, like CNNs, energy models, normalizing flows, gans, vision transformers, or transformers used for natural language processing, autoregressive image modelling. GANs, Autoregressive Image Modeling, Normalizing Flows, Deep Autoencoders, Deep Energy-Based Models, Vision Transformers. Supervised or unsupervised schemes may be applicable to generate representation, also embedding in e.g. cosine or Euclidian metric in in ML language.

[0088] The model may be a data-driven model. The data-driven model may comprise a convolutional neural network and / or an encoder decoder structure such as an autoencoder. Other examples for generating a representation may be FFT, wavelets, deep learning, like CNNs, energy models, normalizing flows, gans, vision transformers, or transformers used for natural language processing, autoregressive image modelling. GANs, Autoregressive Image Modeling, Normalizing Flows, Deep Autoencoders, Deep Energy-Based Models, Vision Transformers. Supervised or unsupervised schemes may be applicable to generate representation, also embedding in e.g. cosine or Euclidian metric in in ML language.

[0089] The comparing of the extracted material data to desired material data, in particular skin material data may comprise determining if the material data matches the desired material data at least within tolerances. In this case, the surface of the object is classified as skin. Otherwise, in case the material data does not match the desired material data, the object is determined to correspond to a non-living organism.

[0090] The determining if the surface is human skin may comprise comparing the extracted material data with material data relating to skin (skin material data). Comparing the material data with desired material data may comprise determining a similarity of the extracted material data and the skin material data. The skin material data may refer to predetermined material data of skin. For example, the material data may comprise reflectance values, wherein the method may comprise comparing the determined reflectance values for the surface with at least one range of reflectance values for skin. The method may comprise considering tolerances, e.g. of ±10 %, preferably of ±5 %, more preferably of ±1 %.

[0091] The skin material data may be stored in at least one database and / or may be retrieved from at least one database, e.g. the database may be at least partially cloud based, e.g. via at least one communication interface.

[0092] The authentication unit may be configured for human skin detection using the pattern image. The object is determined to correspond to a human in case the material data matches the material data of skin. This can allow determining if the object corresponds to a living organism. Otherwise, in case the material data does not match the material data of skin, the object is determined to correspond to a non-living organism. A comparison of material data with skin material data may result in a allowing and / or declining the user and / or object to perform at least one operation that requires authentication. The authentication process may comprise allowing the user to access the resource in case the pattern image of the user is determined to correspond to a living organism and otherwise, in case the pattern image of the user is determined not to correspond to a living organism, denying the user to access the resource. For example, the authentication process may be validated based on the extracted material data. The validating may comprise determining a similarity of the extracted material data and skin, e.g. comparing the extracted material data with the skin material data. A comparison of material data with skin may result in a allowing and / or declining the user and / or object to perform at least one operation that requires authentication. In the example, skin as desired material data may be compared with non-skin material or silicon as material data and the result may be declination since silicon or non-skin material may be different from skin.

[0093] The authentication unit may be further configured for considering additional security features, e.g. extracted from the pattern image. In particular, the authentication unit may be further configured for extracting liveness data such as a blood perfusion measure and / or considering the extracted liveness data from the pattern image. The term “blood perfusion measure" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a blood volume flow through a given volume or mass of tissue. Typically, the blood perfusion measure may be given in units of ml / ml / s or ml / 100 g / min. The blood perfusion measure may represent a local blood flow through the at least one capillary network and one or more extracellular spaces in a body tissue. Determining the at least one blood perfusion measure may comprise determining at least one speckle contrast of the pattern image. Alternatively or in addition, determining the at least one blood perfusion measure may comprise determining a blood perfusion measure based on the determined at least one speckle contrast. The term “speckle contrast " as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a degree of a variation in a speckle pattern generated by coherent light. The speckle pattern may be generated by the transmitter, particularly on the object. A speckle contrast may represent a measure for a mean contrast of an intensity distribution within an area of a speckle pattern. In particular, a speckle contrast K over an area of the speckle pattern may be expressed as a ratio of standard deviation o to the mean speckle intensity <l>, i.e. ,

[0094] Speckle contrast may comprise a speckle contrast value. Speckle contrast values may be distributed between 0 and 1. The blood perfusion measure may be determined based on the speckle contrast. The blood perfusion measure may depend on the determined speckle contrast. If the speckle contrast changes, the blood perfusion measure derived from the speckle contrast may change accordingly. A blood perfusion measure may be a single number or value that may represent a likelihood that the object is a living subject. For monitoring of speckle contrast changes a plurality of pattern images generated at different points in time may be used. For determining the speckle contrast, the complete pattern image may be used. Alternatively, for determining the speckle contrast, a section of the pattern image may be used. The section of the pattern image, preferably, represents a smaller area of the pattern image than an area of the complete pattern image. In an embodiment, a data-driven model may be used for determining a blood perfusion measure. Data-driven model be parametrized and / or trained based on a training data set. The training data set may comprise a pattern image and a blood perfusion measure. The data-driven model may be parametrized and / or trained based on the training data set to output a blood perfusion measure based on receiving a pattern image. In an embodiment, the determining if an object corresponds to a living organism based on the blood perfusion data may comprise determining if the blood perfusion measure corresponds to blood perfusion measure of a human being. Determining if the blood perfusion measure corresponds a human being may comprise comparing the blood perfusion measure to at least one pre-defined or predetermined range of values of blood perfusion measure, e.g. stored in at least one database. In case the extracted blood perfusion measure is at least within tolerances within the re-defined or pre-determined range of values of blood perfusion measure, the object is determined to correspond to a living organism, otherwise not.

[0095] The device may further comprise at least one distance sensing system configured for determining a distance between the object and the image sensor. For example, the imaging system may be configured for providing a distance estimate. Additionally or alternatively, the device may comprise an additional distance sensing system such as a triangulation system, at least one time-of-flight system and the like. The estimated distance can be considered for calculating the reflectance. The distance can be considered as a correction value for the reflectance measure. The distance can be used for determining if the distance of the object and the image sensor is within a working range of the model used for extraction of material data. Distances are further important for face authentication as well since the face recognition models are associated with a working range. The working range may specify a distance range between the object and the image sensor where the model works and / or is trained on the image of the user. In an embodiment, a working range may specify at least one upper and / or at least one lower boundary for a distance of an object from the image sensor and / or an illumination source. The working range may be associated with an authentication process. A working range may comprise at least one value. The value may be a numerical value, in particular a positive numerical value. An indication of a working range may be received, in particular prior to determining if the distance is within or outside of a working range of an authentication process. An indication of a working range may be suitable for determining if the distance is within or outside of a working range of an authentication process. An indication of a working range maybe suitable for comparing distance with a working range.

[0096] The distance may be calculated using at least one distance determination technique. For example, the distance may be determined using one or more of beam profile analysis, e.g. as described in WO 2018 / 091649 A1 , WO 2018 / 091638 A1 and WO 2018 / 091640 A1 , the full content of which is included by reference, time-of-flight, triangulation and the like. The distance may be calculated with a distance sensing system of a mobile device which comprises the illumination source and the image sensor.

[0097] The authentication process further may comprise determining a distance information of the user and determining if the user is within or outside of a working range of the authentication process by comparing the distance to the working range. The authentication process may comprise allowing the user to access the resource in case the user is determined to be within the working range and otherwise, in case the user is determined to be outside the working range, denying the user to access the resource.

[0098] The authentication unit may be further configured for determining a depth map. The determined depth map may be compared to a predetermined depth map of the user, e.g. determined during an enrollment process. The authentication unit may be configured for authenticating the user in case the determined depth map matches with the predetermined depth map of the user, in particular at least within tolerances. Otherwise, the user may be declined.

[0099] The authentication unit may forward data to a remote device. The authentication unit may be configured for outsourcing at least one step of the authentication process, such as the identifying of the user, and / or at least one step of the validation of the authentication process, such as the consideration of the material data, to a remote device, specifically a server and / or a cloud server. The device and the remote device may be part of a computer network, particularly the internet. Thereby, the device may be used as a field device that is used by the user for generating data required in the authentication process and / or its validation. The device may transmit the generated data and / or data associated to an intermediate step of the authentication process and / or its validation to the remote device. In such a scenario, the authentication unit may be and / or may comprise a connection interface configured for transmitting information to the remote device. Data generated by the remote device used in the authentication process and / or its validation may further be transmitted to the device. This data may be received by the connection interface comprised by the device. The connection interface may specifically be configured for transmitting or exchanging information. In particular, the connection interface may provide a data transfer connection. As an example, the connection interface may be or may comprise at least one port comprising one or more of a network or internet port, a USB-port, and a disk drive.

[0100] It is emphasized that data from the device may be transmitted to a specific remote device depending on at least one circumstance, such as a date, a day, a load of the specific remote device, and so on. The specific remote device may not be selected by the field device. Rather a further device may select to which specific remote device the data may be transmitted. The authentication process and and / or the generation of validation data may involve a use of several different entities of the remote device. At least one entity may generate intermediate data and transmit the intermediate data to at least one further entity.

[0101] The allowing the user to access the resource may comprise authorization of the user. The device may comprise at least one authorization unit configured for allowing the user to perform at least one operation on the device, e.g. unlocking the device, in case of successful authentication of the user or declining the user to perform at least one operation on the device in case of non-successful authentication. Thereby, the user may become aware of the result of the authentication. The authorization unit may be configured for allowing or declining the user to perform at least one operation on the device that requires authentication based on the material data and the identifying e.g. using the flood image. The authorization unit may be configured for allowing or declining the user to access one or more functions associated with the device depending on the authentication or denial. The allowing may comprise granting permission to access the one or more functions. The authorization unit may be configured for determining if the user correspond to an authorized user, wherein allowing or declining is further based on determining if the user corresponds to an authorized user. The term “authorization” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a process of assigning access rights to the user, in particular a selective permission or selective restriction of access to the device and / or at least one resource of the device. The authorization unit may be configured for access control. The term “authorization unit” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a unit such as a processor configured for authorization of a user. The authorization unit may comprise at least one processor or may be designed as software or application. The authorization unit and the authentication unit may be embodied integral, e.g. by using the same processor. The authorization unit may be configured for allowing the user to access the one or more functions, e.g. on the device, e.g. unlocking the device, in case of successful authentication of the user or declining the user to access the one or more functions, e.g. on the device, in case of non-successful authentication.

[0102] The device, e.g. by using a user interface such as a display of the device, may be configured for displaying a result of the authentication and / or the authorization.

[0103] In a further aspect, a use of the device for authenticating a user according to the present invention, such as described in one or more embodiments above or as described in more detail below, is disclosed, for one or more of: in-car payment, for vehicle access and / or for starting a vehicle; access control such as for at least one resource of a mobile device such as a mobile phone; in cabin sensing; for payment with a mobile phone and / or stationary devices e.g. ATM, for checkout. With respect to embodiments and definitions reference is made to the description of the device above and the devices and methods for authentication described in more detail below.

[0104] In a further aspect, a computer-implemented method for authenticating a user of a device to perform at least one operation on the device that requires authentication, is disclosed. The method uses a device for authenticating a user according to the present invention, such as described in one or more embodiments above or as described in more detail below. Thus, with respect to embodiments and definitions reference is made to the description of the device above and the devices and methods for authentication described in more detail below.

[0105] The method comprising:

[0106] - triggering illuminating the user with a light pattern from the pattern illumination source;

[0107] - triggering generating at least one pattern image with the image sensor showing at least a part of the user while the user is being illuminated with the light pattern;

[0108] - triggering extracting liveness data from the pattern image; allowing or declining the user to perform at least one operation on the device that requires authentication using the liveness data.

[0109] The method steps may be performed in the given order or may be performed in a different order. Further, one or more additional method steps may be present which are not listed. Further, one, more than one or even all of the method steps may be performed repeatedly.

[0110] The term "computer implemented" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a method involving at least one computer and / or at least one computer network. The computer and / or computer network may comprise at least one processor which is configured for performing at least one of the method steps of the method according to the present invention. Specifically, each of the method steps is performed by the computer and / or computer network. The method may be performed completely automatically, specifically without user interaction. For example, the illuminating and / or the generating of images may be triggered and / or executed by using at least one processor.

[0111] The term “triggering” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to executing and / or performing and / or one or more of causing, initiating, or actuating execution of the named action. The triggering may comprise executing at least one software, e.g. a control software, which when executed by a processing device causes at least one device or unit to perform the named steps.

[0112] Extracting liveness data may comprise extracting material data and / or extracting blood perfusion data, wherein extracting material data comprises providing the pattern image to a model and / or receiving material data from the model. Extracting blood perfusion data comprises determining a speckle contrast of the pattern image and determining a blood perfusion measure based on the determined speckle contrast. A speckle contrast represents a measure for a mean contrast of an intensity distribution within an area of a speckle pattern.

[0113] The method further may comprise illuminating the user with flood light from the flood illumination source, and generating at least one flood image with the image sensor showing at least a part of the user while the user is being illuminated with the flood light. The method further may comprise identifying the user using the flood image.

[0114] An illumination time and / or exposure time by the pattern illumination source may depend on the wavelength. For example, for wavelength around 1000 nm 200 to 2000 ps (with or without OLED) may be used. For higher wavelength longer exposure times may be used, e.g. 7 to 8 times higher.

[0115] The method may comprise receiving a request to perform at least one operation on the device that requires authentication.

[0116] The method may comprise determining if the object corresponds to a living human from the liveness data and allowing the user to access the one or more functions of the device in response to determining that the user corresponds to a living human.

[0117] A single processing device may be configured to exclusively perform at least one computer program, in particular at least one line of computer program code configured to execute at least one algorithm, as used in at least one of the embodiments of the method according to the present invention. Herein, the computer program as executed on the single processing device may comprise all instructions causing the computer to carry out the method. Alternatively, or in addition, at least one method step may be performed by using at least one remote device, especially selected from at least one of a server or a cloud server, particularly when the device and the remote device may be part of a computer network. In this case, the computer program may comprise at least one remote component to be executed by the at least one remote processing device to carry out the at least one method step. Further, the computer program may comprise at least one interface configured to forward to and / or receive data from the at least one remote component of the computer program.

[0118] Further disclosed and proposed herein is a computer program including computer-executable instructions for performing the method according to the present invention in one or more of the embodiments enclosed herein when the program is executed on a computer or computer network. Specifically, the computer program may be stored on a computer-readable data carrier and / or on a computer-readable storage medium.

[0119] As used herein, the terms “computer-readable data carrier” and “computer-readable storage medium” specifically may refer to non-transitory data storage means, such as a hardware storage medium having stored thereon computer-executable instructions. The computer-readable data carrier or storage medium specifically may be or may comprise a storage medium such as a random-access memory (RAM) and / or a read-only memory (ROM).

[0120] Thus, specifically, one, more than one or even all of method steps as indicated above may be performed by using a computer or a computer network, preferably by using a computer program.

[0121] Further disclosed and proposed herein is a computer program product having program code means, in order to perform the method according to the present invention in one or more of the embodiments enclosed herein when the program is executed on a computer or computer network. Specifically, the program code means may be stored on a computer-readable data carrier and / or on a computer-readable storage medium.

[0122] Further disclosed and proposed herein is a data carrier having a data structure stored thereon, which, after loading into a computer or computer network, such as into a working memory or main memory of the computer or computer network, may execute the method according to one or more of the embodiments disclosed herein.

[0123] Further disclosed and proposed herein is a computer program product with program code means stored on a machine-readable carrier, in order to perform the method according to one or more of the embodiments disclosed herein, when the program is executed on a computer or computer network. As used herein, a computer program product refers to the program as a tradable product. The product may generally exist in an arbitrary format, such as in a paper format, or on a computer-readable data carrier and / or on a computer-readable storage medium. Specifically, the computer program product may be distributed over a data network. Finally, disclosed and proposed herein is a modulated data signal which contains instructions readable by a computer system or computer network, for performing the method according to one or more of the embodiments disclosed herein.

[0124] Referring to the computer-implemented aspects of the invention, one or more of the method steps or even all of the method steps of the method according to one or more of the embodiments disclosed herein may be performed by using a computer or computer network. Thus, generally, any of the method steps including provision and / or manipulation of data may be performed by using a computer or computer network. Generally, these method steps may include any of the method steps, typically except for method steps requiring manual work, such as providing the samples and / or certain aspects of performing the actual measurements.

[0125] Specifically, further disclosed herein are: a computer or computer network comprising at least one processor, wherein the processor is adapted to perform the method according to one of the embodiments described in this description, a computer loadable data structure that is adapted to perform the method according to one of the embodiments described in this description while the data structure is being executed on a computer, a computer program, wherein the computer program is adapted to perform the method according to one of the embodiments described in this description while the program is being executed on a computer, a computer program comprising program means for performing the method according to one of the embodiments described in this description while the computer program is being executed on a computer or on a computer network, a computer program comprising program means according to the preceding embodiment, wherein the program means are stored on a storage medium readable to a computer, a storage medium, wherein a data structure is stored on the storage medium and wherein the data structure is adapted to perform the method according to one of the embodiments described in this description after having been loaded into a main and / or working storage of a computer or of a computer network, and a computer program product having program code means, wherein the program code means can be stored or are stored on a storage medium, for performing the method according to one of the embodiments described in this description, if the program code means are executed on a computer or on a computer network.

[0126] As used herein, the terms “have”, “comprise” or “include” or any arbitrary grammatical variations thereof are used in a non-exclusive way. Thus, these terms may both refer to a situation in which, besides the feature introduced by these terms, no further features are present in the entity described in this context and to a situation in which one or more further features are present. As an example, the expressions “A has B”, “A comprises B” and “A includes B” may both refer to a situation in which, besides B, no other element is present in A (i.e. a situation in which A solely and exclusively consists of B) and to a situation in which, besides B, one or more further elements are present in entity A, such as element C, elements C and D or even further elements.

[0127] Further, it shall be noted that the terms “at least one”, “one or more” or similar expressions indicating that a feature or element may be present once or more than once typically are used only once when introducing the respective feature or element. In most cases, when referring to the respective feature or element, the expressions “at least one” or “one or more” are not repeated, nonwithstanding the fact that the respective feature or element may be present once or more than once.

[0128] Further, as used herein, the terms "preferably", "more preferably", "particularly", "more particularly", "specifically", "more specifically" or similar terms are used in conjunction with optional features, without restricting alternative possibilities. Thus, features introduced by these terms are optional features and are not intended to restrict the scope of the claims in any way. The invention may, as the skilled person will recognize, be performed by using alternative features. Similarly, features introduced by "in an embodiment of the invention" or similar expressions are intended to be optional features, without any restriction regarding alternative embodiments of the invention, without any restrictions regarding the scope of the invention and without any restriction regarding the possibility of combining the features introduced in such way with other optional or non-optional features of the invention.

[0129] Summarizing and without excluding further possible embodiments, the following embodiments may be envisaged:

[0130] Embodiment 1 . An imaging system comprising at least one pattern illumination source configured for emitting at least one light pattern comprising a plurality of light beams having a wavelength in a short wavelength infrared spectral region, wherein the wavelength of the light is from 1000 nm to 1200 nm or from 1300 nm to 1500 nm; and at least one image sensor configured for generating at least one pattern image while the pattern illumination source is emitting the light pattern, wherein the image sensor is at least partially sensitive towards electromagnetic radiation in the short wavelength infrared spectral region.

[0131] Embodiment 2. The imaging system according to the preceding embodiment, wherein the wavelength of the light is from 1050 nm to 1150 nm, preferably 1100 nm, or wherein the wavelength of the light is from 1300 nm to 1400 nm.

[0132] Embodiment 3. The imaging system according to any one of the preceding embodiments, wherein a quantum efficiency of the image sensor is > 10 %, preferably > 20 %, more preferably > 40 % in the short wavelength infrared spectral region. Embodiment 4. The imaging system according to any one of the preceding embodiments, wherein the image sensor comprises at least one quantum dot sensor.

[0133] Embodiment 5. The imaging system according to any one of the preceding embodiments, wherein the imaging system is comprised in a device, wherein the device comprises a display, wherein the image sensor is arranged behind the display, wherein the display is at least partially transparent in at least one continuous area covering the image sensor.

[0134] Embodiment 6. The imaging system according to the preceding embodiment, wherein the display is or comprises at least one organic light-emitting diode (OLED) display and / or at least one quantum-dot light emitting diode (QLED) display.

[0135] Embodiment 7. The imaging system according to any one of the two preceding embodiments, wherein the light pattern traverses the display while being illuminated from the pattern illumination source, wherein the display is at least partially transparent in at least one continuous area covering the pattern illumination source.

[0136] Embodiment 8. The imaging system according to any one of the three preceding embodiments, wherein the imaging system further comprises at least one flood illumination source configured for emitting flood light.

[0137] Embodiment 9. The imaging system according to the preceding embodiment, wherein the flood light has wavelength in a short wavelength infrared spectral region.

[0138] Embodiment 10. The imaging system according to any one of the two preceding embodiments, wherein the flood light traverses the display while being illuminated from the flood illumination source, wherein the display of the device is at least partially transparent in at least one continuous area covering the flood illumination source.

[0139] Embodiment 11 . The imaging system according to any one of the six preceding embodiments, wherein device is selected from the group consisting of: a television device; a game console; a personal computer; a mobile device, particularly a cell phone, and / or a smart phone, and / or, and / or a tablet computer, and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch; or another type of portable computer.

[0140] Embodiment 12. The imaging system according to any one of the preceding embodiments, wherein the pattern illumination source comprises a plurality of light emitters selected from the group consisting of: at least one laser light source; at least one laser diode; at least one vertical cavity surface emitting laser (VCSEL); at least one edge emitting laser. Embodiment 13. The imaging system according to any one of the preceding embodiments, wherein the imaging system comprises at least one optical element configured for modifying light spots generated by the pattern illumination source, wherein the optical element is selected from the group consisting of: at least one lens; at least one Micro-lens-array (MLA); at least one diffractive optical element (DOE); and at least one meta surface element.

[0141] Embodiment 14. Use of an imaging system according to any of the preceding embodiments for authenticating a user of a device comprising the imaging system.

[0142] Embodiment 15. A device for authenticating a user of a device to perform at least one operation on the device that requires authentication, the device for authentication comprising: an imaging system according to any of the preceding embodiments referring to an imaging system; at least one display, wherein the image sensor is arranged behind the display, wherein the display is at least partially transparent in at least one continuous area covering the image sensor, at least one authentication unit configured for performing at least one authentication process of a user using the pattern image.

[0143] Embodiment 16. Use of a device according to embodiment 16 for authenticating a user for one or more of: in-car payment, for vehicle access and / or for starting a vehicle; access control such as for at least one resource of a mobile device such as a mobile phone; in cabin sensing; for payment with a mobile phone and / or stationary devices e.g. ATM, for checkout.

[0144] Embodiment 17. A computer-implemented method for authenticating a user of a device to perform at least one operation on the device that requires authentication, wherein the method uses a device for authenticating according to embodiment 15, and the method comprising:

[0145] - triggering illuminating the user with a light pattern from the pattern illumination source;

[0146] - triggering generating at least one pattern image with the image sensor showing at least a part of the user while the user is being illuminated with the light pattern;

[0147] - triggering extracting liveness data from the pattern image; allowing or declining the user to perform at least one operation on the device that requires authentication using the liveness data.

[0148] Embodiment 18. The method according to the preceding embodiment, wherein extracting liveness data comprises extracting material data and / or extracting blood perfusion data, wherein extracting material data comprises providing the pattern image to a model and / or receiving material data from the model, wherein extracting blood perfusion data comprises determining a speckle contrast of the pattern image and determining a blood perfusion measure based on the determined speckle contrast, wherein a speckle contrast represents a measure for a mean contrast of an intensity distribution within an area of a speckle pattern.

[0149] Embodiment 19. The method according to any one of the preceding embodiments referring to a method, wherein the method further comprises illuminating the user with flood light from the flood illumination source, and generating at least one flood image with the image sensor showing at least a part of the user while the user is being illuminated with the flood light, wherein the method further comprises identifying the user using the flood image.

[0150] Embodiment 20. The method according to any one of the preceding embodiments referring to a method, wherein an illumination time and / or exposure time by the pattern illumination source depends on the wavelength, wherein for wavelength around 1000 nm 200 to 2000 ps are used, wherein for higher wavelength longer exposure times are used, e.g. 7 to 8 times higher

[0151] Embodiment 21 . The method according to any one of the preceding embodiments referring to a method, wherein the method comprises receiving a request to perform at least one operation on the device that requires authentication.

[0152] Embodiment 22. The method according to any one of the preceding embodiments referring to a method, wherein the method comprises determining if the object corresponds to a living human from the liveness data and allowing the user to access the one or more functions of the device in response to determining that the user corresponds to a living human.

[0153] Embodiment 23. A computer program comprising instructions which, when the program is executed by the device according to embodiment 15, cause the device to perform the method according to any one of the preceding embodiments referring to a method.

[0154] Embodiment 24. A computer-readable storage medium comprising instructions which, when the instructions are executed by the device according to embodiment 15, cause the device to perform the method according to any one of the preceding embodiments referring to a method.

[0155] Embodiment 25. A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of the preceding embodiments referring to a method.

[0156] Short description of the Figures

[0157] Further optional features and embodiments will be disclosed in more detail in the subsequent description of embodiments, preferably in conjunction with the dependent claims. Therein, the respective optional features may be realized in an isolated fashion as well as in any arbitrary feasible combination, as the skilled person will realize. The scope of the invention is not restricted by the preferred embodiments. The embodiments are schematically depicted in the Figures. Therein, identical reference numbers in these Figures refer to identical or functionally comparable elements.

[0158] In the Figures:

[0159] Figure 1 shows an embodiment of a device for authenticating a user of a device to perform at least one operation on the device that requires authentication according to the present invention; and

[0160] Figure 2 shows a flowchart of an embodiment of a computer-implemented method for authenticating a user of a device.

[0161] Detailed description of the embodiments

[0162] Figure 1 shows an embodiment of a device 110 for authenticating a user of a device to perform at least one operation on the device that requires authentication in a highly schematic fashion. The device on which the user wants to perform the operation may be or may be comprised by the device 110 or a further device. For example, the device 110 may be selected from the group consisting of: a television device; a game console; a personal computer; a mobile device, particularly a cell phone, and / or a smart phone, and / or a tablet computer, and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch; or another type of portable computer.

[0163] The device 110 comprises at least one imaging system 111. The imaging system 111 comprises at least one pattern illumination source 112 configured for emitting light having a wavelength in a short wavelength infrared spectral region. The wavelength of the light is from 1000 nm to 1200 nm or from 1300 nm to 1500 nm.

[0164] The pattern illumination source 112 may be configured for illuminating an object, e.g. a user such as the user’s face, by a projected light pattern. The illuminating may comprise projecting a light pattern onto the surface of the object. The light pattern may comprise a plurality of light spots. The light spot may be at least partially spatially extended. At least one spot or any spot may have an arbitrary shape. In some cases a circular shape of at least one spot or any spot may be preferred. The light pattern may comprise at least one point pattern. The light pattern may be a coherent light pattern. The light beams of the light pattern may have a single wavelength or have a plurality of wavelengths, e.g. for allowing additional measurements in other wavelengths channels. The light pattern may comprise at least one regular and / or constant and / or periodic pattern such as a triangular pattern, a rectangular pattern, a hexagonal pattern or a pattern comprising further convex tilings. For example, the light pattern is a hexagonal pat- tern, preferably a hexagonal light pattern, preferably a 2 / 5 hexagonal infrared light pattern. Using a periodical 2 / 5 hexagonal pattern can allow distinguishing between artefacts and usable signal. The pattern illumination source 112 may comprise at least one least one emitter, in particular a plurality of emitters. For example, the emitter may be selected from the group consisting of: at least one laser light source; at least one laser diode; at least one vertical cavity surface emitting laser (VCSEL); at least one edge emitting laser. The emitters may be used in combination with at least one optical element like MLA, DOE, meta-surface, or lens.

[0165] For example, the imaging system 111 may comprise at least one flood illumination source 116 configured for emitting flood light. The flood illumination source may comprise at least one least one emitter, in particular a plurality of emitters. The flood illumination source may comprise at least one LED or at least one VCSEL, preferably a plurality of VCSELs.

[0166] The emitting of the flood light and the illumination of the light pattern may be performed subsequently or at at least partially overlapping times. For example, the flood light and the light pattern may be emitted at the same time. For example, one of the flood light or the light pattern may be emitted with a lower intensity compared to the other one.

[0167] Classical 2D spoofing targets are made such that their appearance is like a human face and mimic the reflection of human skin in the visible range of light. Using NIR can improve the secureness but there exists a bunch of usual materials having the same reflectivity property as skin. The present invention proposes to that the light has a wavelength in a short wavelength infrared (SWIR) spectral region. The illumination source 112 may be configured for illuminating the object by using flood light and / or the projected light pattern having a central wavelength in SWIR. Considering SWIR, can change the above-mentioned situation. Skin is a quite complex composition of different materials, especially water and lipids. Skin has different reflectance values in SWIR compared to common materials used for spoof targets, e.g. silicone. Thus, using light in the SWIR can allow improved distinguishing between human skin and spoof targets, e.g. spoof targets made of paper, silicone, resin.

[0168] For example, the pattern illumination source 112 may be configured for projecting the light pattern. The wavelength of the light is from 1000 nm to 1200 nm. For example, the wavelength of the light is from 1050 nm to 1150 nm, preferably 1100 nm. In these wavelength regions, a reflection behavior of skin is different from other materials. For example, the wavelength of the light is from 1300 nm to 1500 nm, preferably from 1300 nm to 1400 nm. In these wavelength regions, skin has clearly distinguishable reflectance values.

[0169] Moreover, the spot structure of the light spots on the surface if the object is suitable for material detection for said wavelengths regions, in particular using beam profile analysis.

[0170] For example, the flood illumination source 116 may be configured for illuminating the object by using flood light. Central wavelengths of the flood light and the light of the light pattern may identical, e.g. within ± 100 nm. However, other embodiments are thinkable. For example, the wavelength of the light is from 1300 nm to 1500 nm, preferably from 1300 nm to 1400 nm. For said wavelengths region skin has clearly distinguishable reflectance values. In particular, skin reflects due to water and lipids different to artificial materials, in particular less compared to artificial materials.

[0171] Thus, the present invention proposes to use short-wave infrared light for illuminating an object presented to a camera for verifying that the object is a living organism and not a spoofing object such as a mask. Using these wavelengths ranges is advantageous since light of these wavelengths ranges is invisible to a human and the wavelength may be associated with a low intensity in the light spectrum of the sun. Hence, using the above-mentioned the wavelengths ranges can provide the advantage of being resistant to irradiance of the object by sun light.

[0172] The imaging system 111 further comprises at least one image sensor 118 configured for generating at least one image while the pattern illumination source 112 is emitting the light. The image sensor 118 is at least partially sensitive towards electromagnetic radiation in the short wavelength infrared spectral region.

[0173] As outlined above, higher wavelengths has several advantages. But, for these wavelength ranges, detectors commonly used for authentication systems become inefficient very quickly such that using higher wavelengths was problematic. The image sensor 118 may comprise at least one quantum dot sensor. For example, the quantum dot sensor may be designed as described in US 2023 / 258498A1 . The image sensor may comprise a substrate, for example, made of semiconductor material, for example silicon. The substrate may be made of insulating layers, for example in silicon oxide, and conductive tracks and vias, for example in copper. The substrate may be covered by an insulation layer. The insulation layer may be in silicon oxide. Conductive vias may extend through the insulation layer. The vias may be made of copper or tungsten. The image sensor 118 may comprise at least one via for each pixel. One via may be located in regard of the location of each pixel. The via of each pixel may correspond to an electrode of the pixel. The image sensor 118 may further comprise a layer covering the insulation layer and all the vias corresponding to the pixels. Said layer may be continuous on the locations of all the pixels. Said layer may be in contact with the vias. Said layer may be made of quantum dots. The quantum dots may be fixed together and to the insulation layer by a resin, or a matrix. Preferably, said quantum dot layer may comprise only the quantum dots and the resin. All the quantum dots of said layer may be substantially identical. All the quantum dots of the quantum dot layer have substantially the same size and are made of the same components. For example, all the quantum dots of the quantum dot layer are in lead sulphide (PbS). A quantum dot, or semiconductor nanoparticle, is a nanoscopic material structure which produces electron-hole pairs given the incidence of photons onto the nanoscopic material structure. In this manner, it is possible to create detecting elements such as photodetectors for example, on the basis of semiconductor nanoparticles. A quantum dot comprises a semiconductor core. A quantum dot can also comprise a shell, preferably in a semiconductor material, surrounding the core in order to protect and passivate the core. A quantum dot further comprises ligands, organic aliphatics, or- ganometallic, or inorganic molecules that extend from the shell and passivate, protect, and functionalize the semiconductor surface. The composition of a quantum dot can be chosen among the following materials. The core is, for example, made of a material among the following or an alloy of materials among the following: CdSe, CdS, CdTe, CdSeS, CdTeSe, AgS, ZnO, ZnS, ZnSe, CulnS, CulnSe, CulnGaS, CulnGaSe, PbS, PbSe, PbSeS, PbTe, InAsSb, InAs, InSb, In- GaAs, InP, InGaP, InAlP, InGaAlP, InZnS, InZnSe, InZnSeS, HgTe, HgSe, HgSeTe, Ge, Si. The shell is, for example, made of a material among the following or an alloy of materials among the following: CdSe, CdS, CdTe, CdSeS, CdTeSe, AgS, ZnO, ZnS, ZnSe, CulnS, CulnSe, CulnGaS, CulnGaSe, PbS, PbSe, PbSeS, PbTe, InAsSb, InAs, InSb, InGaAs, InP, InGaP, InAlP, InGaAlP, InZnS, InZnSe, InZnSeS, HgTe, HgSe, HgSeTe, Ge, Si.

[0174] Using a quantum dot image sensor can allow efficient image detection even in the SWIR. A quantum efficiency of the image sensor may be > 10 %, preferably > 20 %, more preferably > 40 %, in the short wavelength infrared spectral region.

[0175] The device 110 further comprises at least one display 120, wherein the image sensor is arranged behind the display 120. The display 120 may be configured for displaying an item of information. The item of information may be arbitrary information such as at least one image, at least one diagram, at least one histogram, at least one graphic, text, numbers, at least one sign, an operating menu, and the like. The display 120 may be or may comprise at least one display panel. The display 120 may have an arbitrary shape, e.g. a rectangular shape. The display may be a front display of the device. The display 120 may comprise at least one of a display panel, particularly comprising a plurality of pixels and / or a plurality of transistors, or a glass, specifically a cover glass, particularly configured for covering the display panel. The display 120, specifically the display panel, may be or may comprise at least one organic light-emitting diode (OLED) display and / or at least one quantum-dot light emitting diode (QLED).

[0176] The pattern illumination source 112 and / or the image sensor 118 may be arranged behind the display 120. The light may traverse the display 120 while being illuminated from the pattern illumination source 112. The display 120 may be at least partially transparent. The display 120 may be at least partially transparent in at least one continuous area covering the pattern illumination source 112, the flood illumination source 116 and / or the image sensor 118. For example, the display 120 may comprise a punch hole in the continuous area covering the pattern illumination source 112, the flood illumination source 116 and / or the image sensor 118. For example, the display 120 may have a transmission below or equal to 20 %, preferably below or equal to 15 %, more preferably below or equal to 10 %. For example, an intensity of a light beam after being projected through the display 120 may correspond to < 10 % of the intensity associated with the light beam when being emitted.

[0177] The device 110 further comprises at least one authentication unit 122 configured for performing at least one authenticating process 124, in particular a computer-implemented method for authenticating a user according to the present invention such as shown in the embodiment of Figure 2. The authentication unit 122 may comprise at least one processor. The method steps may be performed in the given order or may be performed in a different order. Further, one or more additional method steps may be present which are not listed. Further, one, more than one or even all of the method steps may be performed repeatedly

[0178] The method comprising:

[0179] (128) triggering illuminating the user with a light pattern from the pattern illumination source 112;

[0180] (130) triggering generating at least one pattern image with the image sensor showing at least a part of the user while the user is being illuminated with the light pattern;

[0181] (132) triggering extracting liveness data from the pattern image;

[0182] (126) allowing or declining the user to perform at least one operation on the device that requires authentication using the liveness data.

[0183] The authentication unit 122 may be configured for extracting liveness data from the pattern image. The authentication unit 122 may be configured for allowing or declining the user to perform at least one operation on the device that requires authentication using the liveness data. In particular, extracting liveness data comprises extracting material data and / or extracting blood perfusion data.

[0184] The material data may comprise an item of information on the type of material of the surface of the object under illumination, e.g. by the light patter. Extracting material data may be or may comprise generating the material type and / or data derived from the material type. The material data may comprise an item of information on the type of material of the object. Advantageously, the material detection must only work on targets having the same reflectance as human skin.

[0185] Material data may be extracted from the pattern image. Material data may indicate the type of material. In particular, material data may indicate whether the object associated with the image comprises at least partially of skin. Material data may be associated with the object, in particular with the object shown in the image. Extracting material data from the pattern image may comprise generating the material type and / or data derived from the material type.

[0186] Extracting material data may be based on the pattern image, more preferably one or more partial images. For example, the image may be reduced to a predefined size, e.g. by applying one or more image processing techniques. The reducing may comprise selecting at least one area of interest and cutting the pattern image to the area of the pattern image of the predefined size. The area of the image of the predefined size may be associated with the object. The image processing technique may comprise at least one image recognition technique, in particular a face recognition technique. An image recognition technique comprises at least one process of identifying the user in an image. The image recognition may comprise using at least one technique selected from the technique consisting of: color-based image recognition, e.g. using features such as template matching; segmentation and / or blob analysis e.g. using size, or shape; machine learning and / or deep learning e.g. using at least one convolutional neural network. The part of the image other than the area of the image of the predefined size may be associated with background and / or may be independent of the object. The part of the image useful for the subsequent analysis may be selected.

[0187] The material data may comprise at least one reflectance measure. The material data, e.g. reflectance, may be evaluated using at least one algorithm, wherein the algorithm is designed for checking if this object is a spoof target. Additionally or alternatively, an artificial neural network might be used to check if the image is a spoof target. With respect to extraction of material data reference is made to the description above.

[0188] The determining if the surface is human skin may comprise comparing the extracted material data with material data relating to skin (skin material data). Comparing the material data with desired material data may comprise determining a similarity of the extracted material data and the skin material data. The skin material data may refer to predetermined material data of skin. For example, the material data may comprise reflectance values, wherein the method may comprise comparing the determined reflectance values for the surface with at least one range of reflectance values for skin. The method may comprise considering tolerances, e.g. of ±10 %, preferably of ±5 %, more preferably of ±1 %.

[0189] The skin material data may be stored in at least one database and / or may be retrieved from at least one database, e.g. the database may be at least partially cloud based, e.g. via at least one communication interface 134.

[0190] The object is determined to correspond to a human in case the material data matches the material data of skin. Otherwise, in case the material data does not match the material data of skin, the object is determined to correspond to a non-living organism. A comparison of material data with skin material data may result in a allowing and / or declining the user and / or object to perform at least one operation that requires authentication, as will be described in more detail below.

[0191] For example, the extracting of material data from the pattern image may comprise beam profile analysis of the light spots. With respect to beam profile analysis reference is made to WO 2018 / 091649 A1 , WO 2018 / 091638 A1 and WO 2018 / 091640 A1 , the full content of which is included by reference.

[0192] The method may comprise determining a distance between the object and the image sensor 118. The distance can be considered for calculating the reflectance. The distance can be considered as a correction value for the reflectance measure. The distance can be used for determining if the distance of the object and the image sensor 118 is within a working range of the model used for extraction of material data. Distances are further important for face authentication as well since the face recognition models are associated with a working range. The working range may specify a distance range between the object and the image sensor where the model works and / or is trained on the image of the user. In an embodiment, a working range may specify at least one upper and / or at least one lower boundary for a distance of an object from the image sensor and / or an illumination source. The working range may be associated with an authentication process. A working range may comprise at least one value. The value may be a numerical value, in particular a positive numerical value. An indication of a working range may be received, in particular prior to determining if the distance is within or outside of a working range of an authentication process. An indication of a working range may be suitable for determining if the distance is within or outside of a working range of an authentication process. An indication of a working range maybe suitable for comparing distance with a working range. The distance may be calculated using at least one distance determination technique. For example, the distance may be determined using one or more of beam profile analysis, e.g. as described in WO 2018 / 091649 A1 , WO 2018 / 091638 A1 and WO 2018 / 091640 A1 , the full content of which is included by reference, time-of-flight, triangulation and the like. The distance may be calculated with a distance sensing system of a mobile device which comprises the illumination source and the image sensor.

[0193] The operation on the device that requires authentication may be an arbitrary operation requiring access to at least one resource associated with the device. The method may comprise receiving a request for accessing at least one resource associated with the device and executing at least one authentication process. The communication interface 134, such as a user interface, may be configured for receiving a request for accessing at least one resource associated with the device, in particular to perform at least one operation on the device that requires authentication.

[0194] For example, the authentication process may comprise performing at least one face detection step. The face detection step may comprise analyzing at least one image of the user, e.g. generated by the image sensor or a further camera. The image may be a flood image. The determining if the user may correspond to an authorized user may be performed by using an image of the user generated while the object is illuminated by SWIR. This can allow that the imaging is not detectable by human and comprised to a small degree within the light spectrum of the sun. The authentication process may comprise generating at least one flood image showing the user associated while the user is being illuminated by flood light and determining if the identity of the user corresponds to a verified identity based on the flood image. The authentication process may comprise allowing the user to access the resource in case the identity of the user corresponds to a verified identity and otherwise, in case the identity of the user does not correspond to a verified identity, denying the user to access the resource. The method may comprise: illuminating the user with flood light by using the flood illumination source 116; capturing the at least one flood image by using the image sensor 118 or a further camera of the device 110.

[0195] The face detection step may comprise analyzing the flood image. For example, the authentication process may comprise performing at least one face detection using the flood image. The face detection may be performed locally on the device. Face identification, i.e. assigning an identity to the detected face, however, may be performed remotely, e.g. in the cloud, e.g. especially when identification needs to be done and not only verification. User templates can be stored at the remote device, e.g. in the cloud, and would not need to be stored locally. This can be an advantage in view of storage space and security. The authentication process may comprise identifying the user based on the flood image. The identifying may comprise assigning an identity to a detected face and / or verifying an identity of the user. The identifying may comprise performing a face verification of the imaged face to be the user’s face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user’s face, with a template. For matching the flood image with a template, a similarity between at least one image feature vector obtained from the flood image and at least one template feature vector may be considered and / or evaluated. The template vector may be obtained from a template image. The template image may be generated in an enrollment process. The face detection may comprise analyzing the flood image. In particular, the analyzing of the flood image may comprise using at least one image recognition technique, in particular a face recognition technique. An image recognition technique comprises at least one process of identifying the user in an image. The image recognition may comprise using at least one technique selected from the technique consisting of: color-based image recognition, e.g. using features such as template matching; segmentation and / or blob analysis e.g. using size, or shape; machine learning and / or deep learning e.g. using at least one convolutional neural network.

[0196] For example, the authentication may comprise identifying the user. The identifying may comprise assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user. The identifying may comprise performing a face verification of the imaged face to be the user’s face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user’s face, with a template, e.g. a template image generated within an enrollment process. The identifying of the user may comprise determining if the imaged face is the face of the user, in particular if the imaged face corresponds to at least one image of the user’s face stored in at least one memory, e.g. of the device. Authentication may be successful if the flood image can be matched with an image template. Authentication may be unsuccessful if the flood image cannot be matched with an image template.

[0197] The performing of the method according to the present invention can allow determining if the object corresponds to a living organism. The authentication process may comprise allowing the user to access the resource in case the pattern image of the user is determined to correspond to a living organism and otherwise, in case the pattern image of the user is determined not to correspond to a living organism, denying the user to access the resource. For example, the authentication process may be validated based on the extracted material data. The validating may comprise determining a similarity of the extracted material data and skin, e.g. comparing the extracted material data with the skin material data. A comparison of material data with skin may result in a allowing and / or declining the user and / or object to perform at least one operation that requires authentication. In the example, skin as desired material data may be compared with non-skin material or silicon as material data and the result may be declination since silicon or non-skin material may be different from skin.

[0198] The authentication unit may be further configured for determining a depth map. The determined depth map may be compared to a predetermined depth map of the user, e.g. determined during an enrollment process. The authentication unit may be configured for authenticating the user in case the determined depth map matches with the predetermined depth map of the user, in particular at least within tolerances. Otherwise, the user may be declined.

[0199] The allowing the user to access the resource may comprise authorization of the user. The device 110 may comprise at least one authorization unit 136 configured for allowing the user to perform at least one operation on the device, e.g. unlocking the device, in case of successful authentication of the user or declining the user to perform at least one operation on the device in case of non-successful authentication. Thereby, the user may become aware of the result of the authentication. The authorization unit 136 may be configured for allowing or declining the user to perform at least one operation on the device that requires authentication based on the extracted material data and the identifying e.g. using the flood image. The authorization unit 136 may be configured for allowing or declining the user to access one or more functions associated with the device depending on the authentication or denial. The allowing may comprise granting permission to access the one or more functions. The authorization unit 136 may be configured for determining if the user correspond to an authorized user, wherein allowing or declining is further based on determining if the user corresponds to an authorized user. The authorization unit 136 may comprise at least one processor or may be designed as software or application. The authorization unit 136 and the authentication unit 122 may be embodied integral, e.g. by using the same processor.

[0200] List of reference numbers device for authenticating a user imaging system pattern illumination source flood illumination source image sensor display authentication unit authenticating process allowing or declining the user to perform at least one operation on the device triggering illuminating triggering generating at least one image triggering extracting liveness data communication interface authorization unit

Claims

Claims1 . An imaging system (111) comprising at least one pattern illumination source (112) configured for emitting at least one light pattern comprising a plurality of light beams having a wavelength in a short wavelength infrared spectral region, wherein the wavelength of the light is from 1000 nm to 1200 nm or from 1300 nm to 1500 nm; and at least one image sensor (118) configured for generating at least one pattern image while the pattern illumination source (112) is emitting the light pattern, wherein the image sensor (118) is at least partially sensitive towards electromagnetic radiation in the short wavelength infrared spectral region.

2. The imaging system (111) according to the preceding claim, wherein the wavelength of the light is from 1050 nm to 1150 nm, preferably 1100 nm, or wherein the wavelength of the light is from 1300 nm to 1400 nm.

3. The imaging system (111) according to any one of the preceding claims, wherein a quantum efficiency of the image sensor (118) is > 10 %, preferably > 20 %, more preferably > 40 % in the short wavelength infrared spectral region.

4. The imaging system (111) according to any one of the preceding claims, wherein the image sensor (118) comprises at least one quantum dot sensor.

5. The imaging system (111) according to any one of the preceding claims, wherein the imaging system (111) further comprises at least one flood illumination source (116) configured for emitting flood light, wherein the flood light has wavelength in a short wavelength infrared spectral region.

6. Use of an imaging system (111) according to any of the preceding claims for authenticating a user of a device comprising the imaging system.

7. A device (110) for authenticating a user of a device to perform at least one operation on the device that requires authentication, the device (110) for authentication comprising: an imaging system (111) according to any of the preceding claims referring to an imaging system (111); at least one display (120), wherein the image sensor (118) is arranged behind the display (120), wherein the display (120) is at least partially transparent in at least one continuous area covering the image sensor (118), at least one authentication unit (122) configured for performing at least one authentication process of a user using the pattern image.

8. Use of a device (110) according to claim 7 for authenticating a user for one or more of: in- car payment, for vehicle access and / or for starting a vehicle; access control such as for at least one resource of a mobile device such as a mobile phone; in cabin sensing; for payment with a mobile phone and / or stationary devices e.g. ATM, for checkout.

9. A computer-implemented method for authenticating a user of a device to perform at least one operation on the device that requires authentication, wherein the method uses a device (110) for authenticating according to claim 7, and the method comprising:(128) triggering illuminating the user with a light pattern from the pattern illumination source;(130) triggering generating at least one pattern image with the image sensor showing at least a part of the user while the user is being illuminated with the light pattern;(132) triggering extracting liveness data from the pattern image;(128) allowing or declining the user to perform at least one operation on the device that requires authentication using the liveness data.

10. The method according to the preceding claim, wherein extracting liveness data comprises extracting material data and / or extracting blood perfusion data, wherein extracting material data comprises providing the pattern image to a model and / or receiving material data from the model, wherein extracting blood perfusion data comprises determining a speckle contrast of the pattern image and determining a blood perfusion measure based on the determined speckle contrast, wherein a speckle contrast represents a measure for a mean contrast of an intensity distribution within an area of a speckle pattern.11 . The method according to the preceding claim, wherein the material data is data comprising information about a material of the illuminated user, and / or wherein the material data comprises an item of information on the type of material of the surface of the user under illumination.

12. The method according to any one of the preceding claims referring to a method, wherein the method further comprises illuminating the user with flood light from the flood illumination source, and generating at least one flood image with the image sensor showing at least a part of the user while the user is being illuminated with the flood light, wherein the method further comprises identifying the user using the flood image.

13. A computer program comprising instructions which, when the program is executed by the device (110) according to claim 7, cause the device (110) to perform the method according to any one of the preceding claims referring to a method.

14. A computer-readable storage medium comprising instructions which, when the instructions are executed by the device (110) according to claim 7, cause the device(110) to perform the method according to any one of the preceding claims referring to a method.

15. A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of the preceding claims referring to a method.

Images